bngblaster

BNG Blaster 1.0.2

This release improves the AF_XDP performance and robustness, allows to run many BNG Blaster instances in parallel on one host, and fixes the LACP negotiation.

New Features

• Startup status in test-info: The control socket is now opened before the configuration is loaded, and test-info reports the status (starting, running, stopping), the current task (e.g. wait-startup-lock, init-interfaces, running, teardown, cleanup) and the pid. This allows a controller to follow the startup and shutdown of an instance.
• New threads command: Shows the main thread and all I/O threads with the CPU they are pinned to, whether the CPU was selected automatically or is shared with another thread, and the Linux thread ID.
• Abort the startup: The startup can be aborted with test-stop, terminate or the signals SIGINT, SIGTERM and SIGHUP, also while waiting for other instances. The interface configuration is restored and BNG Blaster exits with status 3.
• AF_XDP mode: The new link option af-xdp-mode (auto, zero-copy, copy, generic) selects the AF_XDP socket mode explicitly. Copy mode helps if the zero-copy mode of a driver performs badly, e.g. the Intel ice driver with an IOMMU in translation mode.
• L2TP connect speeds: The new l2tp-client (LAC) options tx-connect-speed and rx-connect-speed set the TX (AVP 24) and RX (AVP 38) Connect Speed sent in ICCN (#407).

Improvements

• RX and TX performance: RX and TX threads prefetch packets and stream state, so that the cache misses of multiple packets overlap. With AF_XDP on an Intel X710 (PPPoE QinQ), a single RX thread now receives 10G line rate (9.7 Mpps) with up to 1M flows instead of 4.2 to 6.6 Mpps, and a single TX thread sends 8.0 Mpps with 1M flows instead of 6.5 Mpps.
• Automatic CPU placement: The automatic CPU placement (rx-auto-cpuset, tx-auto-cpuset) now avoids CPUs handling the interrupts of the used NICs, uses only CPUs allowed for the process (taskset, systemd AllowedCPUs, container cpusets) and pins the main thread to a free CPU as well. A warning is logged for manually pinned threads running on CPUs handling NIC interrupts.
• Parallel instances: Instances running in parallel (e.g. one per SR-IOV virtual function) no longer select the same CPUs. Each instance locks its CPUs with /run/lock/bngblaster_cpu<N>.<PID>.lock, and CPUs are shared evenly if no CPU is free. Instances started at the same time are initialized one after another (/run/lock/bngblaster_startup.lock), which is logged while waiting.
• Lock files: Interface and CPU lock files use kernel locks (flock), which are released if an instance is killed or crashes. Stale lock files no longer block further tests.
• AF_XDP NIC configuration: Channels, RSS indirection table, NIC RX ring and RX VLAN offload are restored on exit, also if the startup fails. The NIC RX ring is increased to 4096 descriptors (a larger ring is kept), and the RX frames (io-slots-rx) are sized automatically to at least twice the NIC RX ring, unless configured explicitly. Based on the work of Andreas Zimber in #408.
• AF_XDP diagnostics: An interface MTU larger than the AF_XDP frame size is now accepted with a warning. A warning is logged if an XDP program is already attached. If the driver supports AF_XDP on fewer queues than required by the threads (e.g. i40e only on as many queues as CPUs), this is reported clearly.
• LACP: The LACP negotiation follows IEEE 802.1AX. A member collects once the partner is in sync and distributes once the partner collects as well, and changes of the own state are sent immediately. Standby members (lacp-max-active-links) are not in sync, so that the partner does not use them either. Two BNG Blaster instances can now be connected back to back with LACP.

Bug Fixes

• AF_XDP: Fixed RX stalling forever if the NIC RX ring ran empty, which happened e.g.

BNG Blaster 1.0.1

This is a maintenance release with fixes for BGP and OSPF, new LCP echo statistics, and an official container image.

New Features

• Container image on GitHub Container Registry: The BNG Blaster image (amd64) is now published to ghcr.io/rtbrick/bngblaster. It includes the BNG Blaster, the tools bngblaster-cli, lspgen, bgpupdate and ldpupdate, and the BNG Blaster controller with its REST API and web UI.

  • latest and <version>: release builds
  • dev: builds from the dev branch

  docker run -d --network host --cap-add NET_ADMIN --cap-add NET_RAW -v $PWD/instances:/var/bngblaster ghcr.io/rtbrick/bngblaster:latest

  See the installation guide for details.

• LCP echo expired counter: A new lcp-echo-expired counter on each access interface counts sessions that were terminated because none of their LCP echo requests got a response. The existing counter for each unanswered echo request is still there.

• New access-protocol-stats command: Shows the protocol packet and timeout statistics for each access interface (the same data as the final report) while the test is still running.

Bug Fixes

• BGP: Fixed sessions that stayed established after the peer closed the connection. A TCP FIN from the peer was ignored, so the session stayed up on a half-closed connection and new connections from the peer were rejected. This could delay the reconnect by up to one keepalive interval. A pending active connect is now also aborted once the session is established, which avoids the connection collision that caused this.
• OSPF: Fixed re-origination of self-originated LSAs when a neighbor holds a newer instance, for example from a previous run with the same router-id. Before, only the router LSA was re-originated, so other LSAs such as the router information LSA could show an old hostname until they reached MaxAge. All affected LSAs are now re-originated with a higher sequence number (RFC 2328 section 13.4).
• Reports: icmpv6-rs-timeout in the JSON report showed the DHCPv6 timeout counter instead of the ICMPv6 RS timeouts.
• Reports / interactive UI: The session summary is back to its pre-1.0.0 layout. The PPPoL2TP session count is only shown if PPPoL2TP sessions are configured.

Full Changelog: https://github.com/rtbrick/bngblaster/compare/1.0.0...1.0.1

BNG Blaster 1.0.0

This is the first major release of BNG Blaster. It adds BGP EVPN and authentication, L2TP LAC emulation, IS-IS Flex-Algo, AF_XDP and loopback I/O, and large performance improvements. It also ships a container image and a Nix flake.

Highlights

Routing protocols

• BGP authentication: TCP-AO (RFC 5925/5926) with hmac-sha-1-96, hmac-sha-256-128 or aes-128-cmac-96, and the legacy TCP MD5 signature (RFC 2385).
• BGP EVPN:
  • Learns EVPN routes (address family evpn, Adj-RIB-In per session); the new bgp-evpn-routes command shows them.
  • EVPN streams resolve their labels from MAC/IP advertisement routes (type 2) and IP prefix routes (type 5).
  • EVPN VPWS (E-LINE) verification.
  • bgpupdate can generate EVPN updates.
• IS-IS Segment Routing and Flex-Algo (#278):
  • Multiple prefix SIDs per algorithm (sr-node-sids).
  • The SR-Algorithm sub-TLV.
  • Flexible Algorithm Definitions (flex-algo).
  • Node MSD (node-msd).
  • Per-link application-specific TE metric, delay and admin group (ASLA).
• lspgen:
  • SR algorithms (-A), Flex-Algo (-F) with FAD and FAPM.
  • Link delay (-D), TE metric (-E), Node MSD (-B) and the ELC flag (-O).
  • New JSON keys for all of them.
• IS-IS and OSPF overload: the IS-IS overload bit and OSPF max-metric (RFC 6987) are now implemented.
• OSPF: fixes to the DR/BDR election.

Access protocols

• L2TP LAC: the new access type PPPoL2TP lets BNG Blaster act as an L2TPv2 LAC, tunneling PPP sessions to an LNS (incl. LCP/IPCP/IPv6CP, DHCPv6 and session traffic). Special thanks to Olivier Matz, the original author of this feature.
• LNS:
  • LCP keepalive.
  • CHAP authentication.
  • The optional LCP Conf-Request lcp-conf-request, for LACs without proxy LCP.
  • A StopCCN reply with a result code.
• DHCPv6 IA_NA for PPPoE (#272): the new access interface option dhcpv6-ia-na.
• Fixes:
  • IPoE sessions stuck in DHCP init after a release.
  • Decoding of data after the DHCP END option.
  • The LCP CHAP option encoding.

I/O and performance

• AF_XDP I/O mode, including the RSS hash and batched TX descriptors. It also sends fewer TX kicks in zero-copy mode.
• Loopback I/O mode: connects two links of the same instance back to back through in-memory rings. It needs no NIC or privileges, which makes it useful for measuring BNG Blaster's own I/O performance.
• DPDK:
  • Locked memory pages and port stats.
  • Sequence fixes for flows dropped on TX.
• General performance:
  • Higher throughput and IPC, fewer cache misses and less stream memory.
  • Cache-line-aligned TXQ.
  • The quadratic stream group lookup at startup is fixed.
• I/O and control threads are named, and BNG Blaster warns when I/O threads exceed the available CPUs or the AF_XDP NIC RX ring is too small.
• Stream rates:
  • Average rates are computed from the elapsed time, so late rate jobs and partial samples no longer skew them.
  • Rates are shown with two decimals (#303).
• Stream fixes:
  • Stream RX in RX threads on LAG member interfaces.
  • The rx-polled interface stats.

Controller, container and packaging

• Web UI: the BNG Blaster controller now provides a web UI and file upload. See the controller documentation.
• Container image (docker/Dockerfile): BNG Blaster and the latest BNG Blaster controller on Debian trixie. The instance directory /var/bngblaster can be mounted.

  docker build -f docker/Dockerfile -t bngblaster .
  docker run -d -p 8001:8001 -v $PWD/instances:/var/bngblaster bngblaster
  

• Nix flake: fixed and updated, with a dev shell and apps for bngblaster and lspgen.
• Debian package:
  • A post-install script.
  • Bash completion.
  • The JSON configuration schema is installed.
• bngblaster-cli commands output now includes the type a

RtBrick BNG Blaster Version 0.9.39

Fixes

• fix DHCPv6 T2 timer

Full Changelog

https://github.com/rtbrick/bngblaster/compare/0.9.38..0.9.39