BisonRouter L2TP Performance Lab: Design, Methodology, and Results
Overview
This document describes an L2TPv2 LAC/LNS performance test lab built around BisonRouter.
The goals of the lab are:
- Validate L2TP LAC switching performance with 40k PPPoE subscribers.
- Validate L2TP LNS termination performance with 40k L2TP subscribers.
- Measure bidirectional subscriber traffic throughput through a full PPPoE-to-L2TP-to-IP forwarding chain.
- Compare a near-lossless run with a maximum-load run.
The lab emulates an ISP-style access topology in which PPPoE subscribers are terminated by a LAC and forwarded through an L2TP tunnel to a LNS.
Topology
Physical layout
The testbed consists of two hosts connected with 40G links:
- ger1
- Traffic generator running BNG Blaster DPDK edition.
- ger1beast
- Device under test. The host runs two independent BisonRouter instances: one L2TP LAC instance and one L2TP LNS instance. Each instance uses a dedicated Intel XL710 40G NIC.
Topology diagram:
BNG Blaster (PPPoE) <-> BisonRouter LAC <-> BisonRouter LNS <-> BNG Blaster (network)
ger1 ger1beast ger1beast ger1
Traffic path
Access-to-network direction:
BNG Blaster PPPoE access -> BR LAC -> L2TP tunnel -> BR LNS -> BNG Blaster network
Network-to-access direction:
BNG Blaster network -> BR LNS -> L2TP tunnel -> BR LAC -> BNG Blaster PPPoE access
In this setup each end-to-end stream packet is processed by both BisonRouter
instances. The LAC handles PPPoE-to-L2TP switching, while the LNS terminates
the L2TP session and forwards subscriber IP traffic. Therefore the BNG Blaster
end-to-end stream rate is only one view of the test. The total packet
processing work on ger1beast is higher because the same traffic crosses a
BisonRouter LAC instance and a BisonRouter LNS instance.
Hardware
BisonRouter host (ger1beast)
- CPU: AMD Ryzen 9 9950X3D 16-Core Processor.
- LAC NIC: Intel XL710 40G.
- LNS NIC: Intel XL710 40G.
CPU allocation on ger1beast
The two BisonRouter instances were intentionally split across the host CPU resources:
- LAC instance: 7 cores assigned.
- LNS instance: 7 cores assigned.
- In each instance, 6 out of 7 cores were forwarding traffic.
This is important when interpreting the result. The test does not show a single BisonRouter instance using all available CPU resources. It shows two BisonRouter instances running at the same time, each limited to roughly one half of the available host resources.
Generator host (ger1)
- CPU: AMD Ryzen 9 5900XT 16-Core Processor.
- NIC: Intel XL710 40G.
Software
- BisonRouter L2TP LAC instance on ger1beast.
- BisonRouter L2TP LNS instance on ger1beast.
- BNG Blaster DPDK edition on ger1.
Test methodology
Each test run consists of two stages:
- Session establishment.
- Bidirectional stream traffic forwarding.
BNG Blaster establishes 40,000 PPPoE subscribers. The LAC forwards each PPP subscriber through L2TP to the LNS, resulting in 40,000 LAC sessions and 40,000 LNS sessions on a single L2TP tunnel.
After all sessions are established, BNG Blaster verifies 80,000 traffic flows and starts bidirectional subscriber traffic.
Two load levels were tested:
- Test 1
- 4,000,000 total stream packets per second. This run represents the near-lossless operating point.
- Test 2
- 8,000,000 total stream packets per second. This run pushes the lab to the maximum observed throughput and accepts measurable packet loss.
Result summary
| Test | Offered stream rate | Sessions | Streams | Access loss | Network loss |
|---|---|---|---|---|---|
| Test 1 | 4.0 Mpps | 40,000 | 80,000 | 2,336 packets, 0.001% | 1,838 packets, 0.000% |
| Test 2 | 8.0 Mpps | 40,000 | 80,000 | 13,365,361 packets, 0.745% | 2,249,658 packets, 0.125% |
Test 1 is the clean performance result. It keeps all 40,000 subscribers established and forwards approximately 4 Mpps of end-to-end stream traffic with almost zero packet loss.
Test 2 is the overload result. It reaches approximately 8 Mpps of end-to-end stream traffic, but packet loss becomes visible and BNG Blaster reports 43 flapped sessions.
Host workload summary
Because both the LAC and LNS run on ger1beast, the host processes the LAC
datapath and the LNS datapath at the same time. A 4 Mpps end-to-end test means
approximately 4 Mpps through the LAC and another 4 Mpps through the LNS.
| Test | End-to-end stream rate | Approximate per-instance forwarding | Approximate total on ger1beast |
|---|---|---|---|
| Test 1 | 4.0 Mpps | 4.0 Mpps on LAC and 4.0 Mpps on LNS | 8.0 Mpps ingress and 8.0 Mpps egress |
| Test 2 | 8.0 Mpps | 8.0 Mpps on LAC and 8.0 Mpps on LNS | 16.0 Mpps ingress and 16.0 Mpps egress |
The Test 1 result is therefore stronger than a simple 4 Mpps subscriber traffic number. It shows that the machine can forward at least the combined LAC and LNS workload shown above while using only 6 forwarding cores per BisonRouter instance.
Test 1 run
BNG Blaster summary
F1: Select View F7/F8: Start/Stop Traffic F9: Terminate Sessions May 22 16:23:05.444925 Total PPS of all streams: 4000000.00
F2: Network Interface Left/Right: Access Interface May 22 16:23:05.552461 Resolve network interfaces
May 22 16:23:05.552493 All network interfaces resolved
____ __ ____ _ __ ,/ May 22 16:24:12.197948 ALL SESSIONS ESTABLISHED
/ __ \ / /_ / __ ) _____ (_)_____ / /__ ,'/ May 22 16:24:12.447146 ALL STREAM TRAFFIC FLOWS VERIFIED
/ /_/ // __// __ |/ ___// // ___// //_/ ,' /
/ _, _// /_ / /_/ // / / // /__ / ,< ,' /_____,
/_/ |_| \__//_____//_/ /_/ \___//_/|_| .'____ ,'
____ _ _ ______ ____ _ _ / ,'
/ __ ) / | / // ____/ / __ ) / /____ _ _____ / /_ ___ ____ / ,'
/ __ |/ |/ // / __ / __ |/ // __ `// ___// __// _ \ / ___/ /,'
/ /_/ // /| // /_/ / / /_/ // // /_/ /(__ )/ /_ / __// / /
/_____//_/ |_/ \____/ /_____//_/ \__,_//____/ \__/ \___//_/
Test Duration: 244s All Sessions Established: 177s
Sessions 40000 (40000 PPPoE / 0 IPoE)
Established 40000 [############################################################]
Outstanding 0 [ ]
Terminated 0 [ ]
DHCPv6 0/40000 [ ]
Setup Time 66646 ms
Setup Rate 600.19 CPS (MIN: 260.87 AVG: 601.66 MAX: 663.51)
Flapped 0
Traffic Flows Verified
Stream 80000/80000 [############################################################]
Network Interface ( 0000:06:00.1 )
TX Packets 421544766 |2000191 PPS 4320411 Kbps 4.320 Gbps
RX Packets 421542783 |2000244 PPS 4320528 Kbps 4.321 Gbps
TX Stream Packets 421544728 |2000190 PPS
RX Stream Packets 421542769 |2000244 PPS 1838 Loss (0.000%)
TX Multicast Packets 0 | 0 PPS
Access Interface ( 0000:06:00.0 )
TX Packets 424501849 |2011212 PPS 4587396 Kbps 4.587 Gbps
RX Packets 422790890 |2003305 PPS 4578035 Kbps 4.578 Gbps
TX Stream Packets 421544797 |2000187 PPS
RX Stream Packets 421542229 |2000220 PPS 2336 Loss (0.001%)
RX Multicast Packets 0 | 0 PPS 0 Loss
BisonRouter LAC port rate
Representative samples from the LAC instance:
Port 0: RX 2,010,898.0 pps, TX 2,002,889.9 pps, Missed 0.0/sec
Port 1: RX 2,002,885.7 pps, TX 2,010,895.8 pps, Missed 13.3/sec
Port 0: RX 2,010,584.3 pps, TX 2,002,575.3 pps, Missed 0.0/sec
Port 1: RX 2,002,569.8 pps, TX 2,010,580.4 pps, Missed 5.7/sec
BisonRouter LNS port rate
Representative samples from the LNS instance:
Port 0: RX 2,010,796.8 pps, TX 2,002,965.4 pps, Missed 0.0/sec
Port 1: RX 2,000,005.5 pps, TX 2,000,023.0 pps, Missed 0.0/sec
Port 0: RX 2,010,749.5 pps, TX 2,003,023.5 pps, Missed 15.2/sec
Port 1: RX 1,999,994.4 pps, TX 1,999,980.2 pps, Missed 0.0/sec
Tunnel and session validation
LAC tunnel:
root@alex-gerbeast:~# rcli sh l2tp tunnels
id our addr:port peer addr:port sessions uptime
1 1.0.0.1:1701 1.0.0.2:1701 40000 0 day(s), 0 hour(s), 4 min(s), 45 sec(s)
LNS tunnel:
root@alex-gerbeast:~# rcli2 sh l2tp tunnels
id our addr:port peer addr:port sessions uptime
1 1.0.0.2:1701 1.0.0.1:1701 40000 0 day(s), 0 hour(s), 4 min(s), 48 sec(s)
Subscriber counters:
root@alex-gerbeast:~# rcli sh l2tp lac | wc -l
40001
root@alex-gerbeast:~# rcli2 sh l2tp lns | wc -l
40001
The output includes the header line, so both the LAC and LNS have 40,000 online subscriber sessions.
Test 1 observations
At 4 Mpps of bidirectional stream traffic, both BisonRouter instances forward traffic steadily. The BNG Blaster loss counters remain effectively zero, and the BisonRouter missed counters are either zero or only a few packets per second on isolated samples.
This is the near-lossless operating point for this lab configuration.
Test 2 run
BNG Blaster summary
F1: Select View F7/F8: Start/Stop Traffic F9: Terminate Sessions May 22 16:33:47.804927 Total PPS of all streams: 8000000.00
F2: Network Interface Left/Right: Access Interface May 22 16:33:47.912367 Resolve network interfaces
May 22 16:33:47.912400 All network interfaces resolved
____ __ ____ _ __ ,/ May 22 16:35:09.472641 ALL SESSIONS ESTABLISHED
/ __ \ / /_ / __ ) _____ (_)_____ / /__ ,'/ May 22 16:35:09.808527 ALL STREAM TRAFFIC FLOWS VERIFIED
/ /_/ // __// __ |/ ___// // ___// //_/ ,' /
/ _, _// /_ / /_/ // / / // /__ / ,< ,' /_____,
/_/ |_| \__//_____//_/ /_/ \___//_/|_| .'____ ,'
____ _ _ ______ ____ _ _ / ,'
/ __ ) / | / // ____/ / __ ) / /____ _ _____ / /_ ___ ____ / ,'
/ __ |/ |/ // / __ / __ |/ // __ `// ___// __// _ \ / ___/ /,'
/ /_/ // /| // /_/ / / /_/ // // /_/ /(__ )/ /_ / __// / /
/_____//_/ |_/ \____/ /_____//_/ \__,_//____/ \__/ \___//_/
Test Duration: 482s All Sessions Established: 400s
Sessions 40000 (40000 PPPoE / 0 IPoE)
Established 40000 [############################################################]
Outstanding 0 [ ]
Terminated 0 [ ]
DHCPv6 0/40000 [ ]
Setup Time 81561 ms
Setup Rate 490.43 CPS (MIN: 490.43 AVG: 605.06 MAX: 1935.48)
Flapped 43
Traffic Flows Verified
Stream 80000/80000 [############################################################]
Network Interface ( 0000:06:00.1 )
TX Packets 1793201556 |4000355 PPS 8640767 Kbps 8.641 Gbps
RX Packets 1790951789 |3994777 PPS 8628718 Kbps 8.629 Gbps
TX Stream Packets 1793201486 |4000355 PPS
RX Stream Packets 1790951756 |3994777 PPS 2249658 Loss (0.125%)
TX Multicast Packets 0 | 0 PPS
Access Interface ( 0000:06:00.0 )
TX Packets 1798694099 |4011127 PPS 9163643 Kbps 9.164 Gbps
RX Packets 1781730865 |3975553 PPS 9090853 Kbps 9.091 Gbps
TX Stream Packets 1793201508 |4000324 PPS
RX Stream Packets 1779828087 |3972645 PPS 13365361 Loss (0.745%)
RX Multicast Packets 0 | 0 PPS 0 Loss
BisonRouter LNS port rate
Representative samples from the LNS instance:
Port 0: RX 4,005,859.5 pps, TX 4,002,665.9 pps, Missed 4,204.6/sec
Port 1: RX 3,999,978.5 pps, TX 3,995,345.5 pps, Missed 77.8/sec
Port 0: RX 4,006,908.6 pps, TX 4,002,928.6 pps, Missed 3,536.7/sec
Port 1: RX 3,999,906.4 pps, TX 3,995,989.2 pps, Missed 57.4/sec
BisonRouter LAC port rate
Representative samples from the LAC instance:
Port 0: RX 4,010,657.6 pps, TX 3,999,073.0 pps, Missed 288.2/sec
Port 1: RX 3,999,175.7 pps, TX 4,010,664.4 pps, Missed 3,771.8/sec
Port 0: RX 4,010,094.7 pps, TX 3,997,657.7 pps, Missed 814.5/sec
Port 1: RX 3,997,555.3 pps, TX 4,010,097.5 pps, Missed 5,377.3/sec
Tunnel validation
After the maximum-load run, the LNS still has 40,000 sessions on the L2TP tunnel:
root@alex-gerbeast:~# rcli2 sh l2tp tunnels
id our addr:port peer addr:port sessions uptime
1 1.0.0.2:1701 1.0.0.1:1701 40000 0 day(s), 0 hour(s), 19 min(s), 20 sec(s)
Test 2 observations
At 8 Mpps of bidirectional stream traffic, the lab reaches its maximum observed throughput. Both BisonRouter instances continue forwarding at approximately 4 Mpps per active port, but missed counters increase to thousands of packets per second on the busiest ingress ports.
BNG Blaster reports measurable packet loss:
- 0.125% on the network-side stream direction.
- 0.745% on the access-side stream direction.
BNG Blaster also reports 43 flapped sessions. This run should therefore be treated as an overload test, not as a lossless performance point.
Conclusion
With 40,000 PPPoE subscribers forwarded through L2TP, this lab reached a stable near-lossless point at 4 Mpps total bidirectional stream traffic. At that rate, both the LAC and LNS maintained 40,000 sessions on one L2TP tunnel and packet loss stayed effectively zero.
Since the LAC and LNS were both running on ger1beast, this 4 Mpps
end-to-end result corresponds to approximately 8 Mpps of ingress packet
processing and 8 Mpps of egress packet processing across the host. Each
BisonRouter instance used only 6 forwarding cores, so the result should be read
as a lower-bound capacity result for this split-instance lab, not as a
full-machine ceiling.
The maximum-load run at 8 Mpps demonstrated that the same topology can forward substantially more traffic, but it crosses into overload. Packet loss becomes visible, missed counters rise on both BisonRouter instances, and session flaps appear. In terms of host workload, this run corresponds to approximately 16 Mpps ingress and 16 Mpps egress across the two BisonRouter instances.
For this hardware and configuration, the practical clean result is the 4 Mpps run. The 8 Mpps run is useful as a stress boundary and shows where the lab starts to drop packets.