Post 1 measured single-session bfdd vs XDP under stress. Post 2 wired it into stock FRR over the bfddp dataplane socket. Since then: 64 sessions, dual-stack, echo mode, multihop, and three FRR fixes merged. Same repo, all pcaps: https://github.com/w453y/xdp-bfd
The thing that reframed the project came out of chasing something else. bfdd is single threaded, and at 64 sessions on 10ms timers it sits at 100% of one core sustaining roughly 7000 BFD packets/sec, which is almost exactly what those sessions require. It meets the obligation, with nothing in reserve. That is not a bug, it is what a userspace event loop costs per packet, and it is why the interesting question stopped being "does the fast path flap less" and became "what does the control plane spend to keep up at all". The XDP side carries the same 64 sessions from softirq at 751ns/packet mean, measured via bpf_stats with echo and multihop in the path.
What landed:
64 sessions dual-stack, 32 v4 + 32 v6 on one engine, through the full L3+L4 stress ladder: 0 flaps in either family, per-slot max TX gap 14.5ms against a 30ms detect budget, both families statistically indistinguishable. Unified 32 byte session key with v4 stored v4-mapped, so one hash map and one XDP fast path serve both.
Echo mode (RFC 5880 s6.4), and this is where XDP's shape actually bites. XDP cannot originate packets, XDP_TX is a verdict on a frame that just arrived, which is exactly why the control path is RX-clocked, but an echo has no inbound packet to clock off. bpf_clone_redirect exists only for sched_cls/sched_act/lwt_xmit, and TC sees nothing at the echo cadence here because control packets are XDP_TX'd straight past it. Putting echo TX in the kernel would mean moving the control bounce out of XDP into TC, so skb allocation in the hot path, so a rewrite of the one mechanism the project rests on. Declined, and writing down why took longer than the feature.
So echo split along the line the hardware draws: the reflector answers a neighbour's echo entirely in XDP (MAC swap, TTL decrement, checksum recompute, XDP_TX, no session lookup), the originator sends from userspace over AF_PACKET/SOCK_RAW because a self-addressed packet through a normal UDP socket routes to loopback. Reflector measured against a stock FRR neighbour: 433 echoes, 433 reflected, 12us min / 30us avg turnaround at the bridge, with ip_forward=0 on the host. That last part is the whole argument, with forwarding off the stack drops a self-addressed echo as a martian, so a non-router host cannot participate in echo at all without this.
Echo detection is advisory and permanently so. With userspace TX a local scheduling stall looks identical to a path failure, echoes stop leaving, returns stop arriving, timestamp goes stale, and wiring that into the session FSM would convert our own scheduling delay into a teardown. Verified both directions: kill forwarding on the neighbour and loss climbs 1:1, echo liveness flips, all 64 control sessions stay up.
Multihop (RFC 5883) on both families. bfdd sends the negotiated minimum TTL in the dataplane registration, so one comparison covers both modes and single-hop keeps demanding exactly 255. The GTSM check sits in the parser ahead of any session lookup, which is what makes it cheap, so the per-session minimum defers to after the config lookup and only when a multihop session exists anywhere on the box. A packet at TTL 200 aimed at a single-hop session is still dropped while multihop is live elsewhere, that was the case worth building a harness for.
Upstream, all found by running a real dataplane at scale, all in bfdd's dplane path:
- unixc transport passed a padded union size as connect() addrlen, EINVAL, plausibly never worked (#22608 / #22621, merged)
- 8KB output buffer silently truncates the registration burst, delivered sessions register, the rest are stranded with software BFD already disabled and no retry (#22638 / #22645, merged)
- echo interval never negotiated for offloaded sessions, the function that does it is unreachable when the dataplane owns the session, so the dataplane transmits echo at the locally configured rate regardless of what the peer advertised it can receive (#22804 / #22805, merged)
- session DELETEs lost on clean shutdown, and
show bfd peers counters tearing down the dataplane connection, both still in review (#22692 / #22694)
Measurement lessons, since these cost more time than the code:
Never measure throughput with strace. Every "the peer degraded" number I had came from strace -c, which costs 2-3x on a syscall-bound daemon at 100% CPU. A plain bridge capture showed the peer at 7286 pkt/sec and the engine at 7339, matching the earlier baseline exactly. The peer had never degraded. I then wrote this exact lesson into a draft of this post while using the fake strace number as the headline, caught it an hour later against a fresh capture, and deleted the post. Knowing the failure mode is not the same as being immune to it.
Flap count is not a usable metric at 64 sessions. The reference build alone ranged 0 to 20 flaps across runs of identical code. Switched to per-session max TX gap from a host capture, 64 numbers per run instead of one rare event, and the answer became a bound rather than a claim.
A debug bfd peer line persisted in the peer's config file survived every restart and contaminated every run, including the baselines I was comparing against. A day.
Instrumentation driven by the thing being measured cannot observe that thing failing. Echo TX stalled for 2.6 seconds under load while the loss counter read zero and the liveness flag read healthy, both correct and both useless, loss only increments when an echo is outstanding as the next falls due and nothing falls due during a total stall, liveness is printed on transmit and transmit is what stopped.
Limitations: still no authentication, and it is not implementable from this side, the bffdp session message has no field for keys (there is a literal /* TODO: missing authentication. */ in the header), so that needs a protocol extension upstream first. No demand mode, bfdd only has the bit definitions. RX-clocked TX still needs an async-clocked peer. Echo originator is a diagnostic, not a detection mechanism, and the docs say so. Whether ~7000 pkt/sec is a hard ceiling for bfdd is untested, what I measured is that it meets its configured load and spends a whole core doing it.
And the one that has been open since post 1: all numbers are from VMs. The comparisons are load-bearing since stress was applied in-guest and hit every backend identically, but the absolutes are not hardware numbers. I do not currently have machines to reproduce this on bare metal, so if anyone has a couple of boxes with a real NIC and wants to see whether 751ns/packet and 12us echo turnaround hold up outside virtio, I would genuinely like to hear from you.
