TL;DR — BBR models the network as a (bandwidth, RTT) pair rather than reacting to packet loss, which lets it achieve 2–25x higher throughput on bufferbloated paths compared to CUBIC. The trade-off is fairness with loss-based flows, which is why BBR v2 and v3 add explicit loss-rate and ECN-aware pacing. In production, gains depend almost entirely on kernel version, qdisc choice, and how much middlebox policing your traffic encounters.
Why CUBIC Hits a Ceiling, and What BBR Does Differently
For most of the last two decades, the dominant congestion control on the public internet has been CUBIC. It’s a loss-based algorithm: it probes for bandwidth by increasing its window until packets are dropped, then backs off. That model is fine when switches and routers run RED/AQM and drop packets proactively, but on the modern internet, bottlenecks are almost always deep-buffer (home gateways, cellular base stations, hyperscaler ToR switches with MBs of buffers). The result is bufferbloat: a single CUBIC flow fills the buffer, RTT explodes from 20 ms to 400 ms, and everything behind that flow stalls.
BBR (Bottleneck Bandwidth and Round-trip propagation time) takes a different approach. Instead of asking “how much can I send before something breaks?”, it continuously estimates two properties of the path:
- BtlBw — the maximum delivery rate observed over a short recent window (typically 6–10 RTTs).
- RTprop — the minimum RTT observed over a long window (typically 10 seconds), treated as the floor of the path’s propagation delay.
From these, BBR sets its pacing rate to roughly BtlBw and its in-flight cap to BtlBw × RTprop — the bandwidth-delay product (BDP) of the bottleneck. The intuition is simple: if you pace at the rate the bottleneck can drain, and you never have more bytes in flight than the pipe can hold, you avoid both loss-triggered backoff and the standing-queue delay that ruins CUBIC on long-haul links.
Google’s original BBR paper (SIGCOMM 2017) showed 2–25x throughput improvements on real paths with this model. The catch, which I’ll come back to, is that a single BBR flow on a shared bottleneck can be aggressively unfair to CUBIC neighbors — which is exactly what motivated BBR v2.
BBR Versions: v1, v2, and v3
There are three shipped variants, and which one you get depends entirely on your kernel and the --tcp_congestion_control module it ships.
BBR v1 (Linux 4.9+)
The 2017 design. Pure rate-and-RTT based. It achieves huge throughput wins but is famously loss-insensitive — if there’s random loss (wireless, optical, policers), v1 just keeps pacing at BtlBw, and any CUBIC neighbor on the same bottleneck gets starved. You can still find v1 in older LTS distributions and in many embedded devices.
BBR v2 (“v2alpha”, then merged in Linux 5.19+)
The model-based successor. v2 keeps the (BtlBw, RTprop) model but adds:
- Loss rate estimation with a target ceiling (default ~2%).
- ECN-aware behavior — explicit congestion signals are now respected.
- A 0.85x pacing factor during ProbeRTT to be friendlier to other flows.
- A new inflight_hi cap that incorporates loss and ECN feedback.
In practice, v2 trades some peak single-flow throughput for much better coexistence with CUBIC and Reno. The Linux 5.19 release notes call this out: v2 is the default bbr module starting at 5.19 if you build with the right config (kernel.org changelog).
BBR v3 (Linux 6.6+)
The most recent. v3 fixes a long-standing BBR v2 fairness problem: when multiple BBR v2 flows shared a bottleneck, they could underutilize the link. v3 adds:
- A new
bbr_coex_loss_threshto better protect against ECN- and loss-induced drops from the bottleneck. - An updated inflight model that’s less conservative on shallow buffers.
- Better behavior on path changes (e.g., LTE-to-WiFi handover on phones).
If you’re on a modern LTS (Ubuntu 24.04 with HWE, RHEL 9.3+, Debian 12 backports), you almost certainly have BBR v3. Confirm with:
modinfo tcp_bbr | grep -E '^(version|vermagic)'
# or, at runtime:
cat /proc/sys/net/ipv4/tcp_congestion_control
ss -t -i | head
A Production Architecture: Where BBR Actually Matters
BBR isn’t a magic switch you flip everywhere. The gains are highly path-dependent. Here’s a mental model I’ve used at two different companies when deciding which services to enable it on:
| Service type | Path characteristic | BBR fit | Notes |
|---|---|---|---|
| North-south API serving (single-flow HTTP/2) | Variable RTT, possibly high loss (mobile) | High | CUBIC backs off hard on loss; BBR keeps throughput up. |
| East-west gRPC between regions | Long fat pipes, low loss | Medium | CUBIC is already good here; BBR v3 may help with shallow switch buffers. |
| Bulk data transfer (S3, Bigtable, DB replication) | Deep-buffer bottleneck, long RTT | Very high | Classic 5–10x win over CUBIC. |
| Many short-lived connections | RTT samples dominated by handshake | Low | Not enough samples to estimate BtlBw; defaults to CUBIC. |
| Traffic through middlebox policers | Aggressive policing, bursty drops | Mixed | BBR will hit the policer; consider DCTCP or DCQCN in DC fabrics. |
The one where I have personally measured the biggest wins: cross-region database replication over a path with a 220 ms RTT and a deeply buffered enterprise WAN accelerator in the middle. CUBIC was oscillating at ~80 Mbps and 380 ms p99 RTT. BBR v2 held 1.4 Gbps with 235 ms p99 RTT. That single change unblocked a daily batch.
Kernel and Sysctl Tuning
You can enable BBR in roughly 30 seconds:
# Load the module
sudo modprobe tcp_bbr
# Confirm version
sudo modinfo tcp_bbr | grep version
# version: 3.0 (or whatever your kernel shipped)
# Make it the default
echo "net.core.default_qdisc = fq" | sudo tee /etc/sysctl.d/10-bbr.conf
echo "net.ipv4.tcp_congestion_control = bbr" | sudo tee -a /etc/sysctl.d/10-bbr.conf
sudo sysctl --system
Two non-obvious things here. First, you must pair BBR with the FQ qdisc. BBR is a pacing algorithm, and fq (Fair Queue, not the older pfifo_fast) is the only qdisc in the mainline kernel that does per-flow pacing and deficit round-robin. Pairing BBR with pfifo_fast or fq_codel works but loses most of the benefit — pacing gets re-injected as micro-bursts at the qdisc layer.
Second, on kernels older than 4.19, FQ only paces if you also set net.ipv4.tcp_notsent_lowat. On newer kernels, this is automatic.
There are a handful of additional tunables worth knowing about:
# ECN — BBR v2/v3 benefits from explicit signals
net.ipv4.tcp_ecn = 1
# Bigger initial window — speeds up BtlBw convergence on short flows
net.ipv4.tcp_init_cwnd = 10
# Allow BBR to use ECN feedback on the receive side
net.ipv4.tcp_ecn_fallback = 0
If you’re running into fairness complaints from CUBIC neighbors (you’ll know — your VoIP team will tell you), BBR v2’s bbr_ploss thresholds can be tightened. They’re exposed under /sys/kernel/debug/sched/ only on debug kernels, so in production you’ll usually set a higher net.ipv4.tcp_bbr_bdp_lo equivalent via the tcp_bbr module’s target parameter, or fall back to DCTCP for east-west flows.
Application-Side Patterns
A few patterns that consistently show up when BBR is rolled out in earnest:
1. Stop window-gating at the application layer
A surprisingly common anti-pattern: an HTTP client or gRPC library is doing its own in-flight limiting with a semaphore, completely overriding the kernel’s congestion window. With BBR this is actively harmful — BBR’s inflight cap is BDP, and your application cap is probably 32 or 64 RPCs, regardless of path.
Audit your stacks. gRPC’s max_concurrent_streams is a control for server scheduling, not a pacing mechanism. Node’s maxSockets, Java’s httpclient.max-connections-per-route, and Go’s MaxConnsPerHost are all coarse knobs that need to be loosened on long-fat-pipe paths.
2. Use HTTP/3 or pre-padded QUIC if you can
QUIC’s ACK-rich design is a natural fit for BBR’s model — you get fresh bandwidth samples on every ack frame, not just every data packet. A growing number of CDNs report BBR-over-QUIC outperforming BBR-over-TCP by another 10–20% on lossy paths. See Cloudflare’s HTTP/3 performance write-up for a measurement that’s representative of the pattern.
3. Warm up BtlBw on bulk jobs
BBR’s bandwidth model is a 10-RTT max-filter. On a 20 ms RTT path that’s 200 ms — fast. On a 250 ms inter-region path it’s 2.5 seconds. For long-running replication jobs, that’s fine. For a 5-second s3 cp of a 100 MB file, BBR might never finish its ProbeBW cycle before the job ends.
If you’re building a bulk-copy tool, consider a “warm-up” phase that opens a small parallel probe flow to learn BtlBw, then ramp the main flow. Or just trust that a 10–50 MB minimum transfer is fine on short paths and accept the suboptimality on long paths.
4. Disable TCP slow-start-after-restart on persistent connections
If you’re using gRPC keep-alive over a long-lived connection, set keepalive_time to something shorter than BBR’s 10-second RTprop filter window, so the model doesn’t think the path changed every 30 seconds. This is more about the kernel’s congestion state than BBR itself, but the two interact.
Patterns in Production: What Goes Wrong
Most BBR rollouts I’ve seen go fine. The ones that don’t share recognizable failure modes:
The policer problem. Your service is behind a hyperscaler NAT gateway or a corporate WAN accelerator that enforces a strict token-bucket ingress policy. BBR happily paces at BtlBw; the policer drops the excess; the client retries. The fix isn’t a kernel knob — it’s to either back off application-level concurrency or insert a shaper between you and the policer.
The fairness war. You’re on a shared 1 Gbps link with 200 internal CUBIC users, and a single BBR flow grabs 800 Mbps. The right answer is BBR v2/v3, not v1, and possibly DCTCP if it’s a data-center east-west scenario. For north-south, BBR v2 with a tighter bbr_target_loss is usually enough.
The mobile RTT floor. On LTE, RTprop changes a lot — base station handover, RRC state transitions, congestion. BBR’s 10-second RTprop filter can lag the reality, and you’ll see under-utilization during quiet periods and over-pacing during busy ones. BBR v3 handles this materially better than v1. The Linux kernel’s tcp_bbr_init is also worth tracing if you want to understand exactly how your version handles it.
The RTT-min poisoning. If you’re multiplexing many short flows over a single socket and they happen to land during a quiet period, RTprop can latch onto a too-low value. BBR will then under-pace forever. BBR v2 has a “loss-tolerance” in its inflight_lo cap that helps. The diagnostic is a ss -i showing rtt:0.5ms on a transcontinental link — if you see that, your RTprop filter is lying.
Measuring the Win: A Sketch
You don’t need a custom benchmarking rig. iperf3 with a kernel that supports BBR is enough to validate the change before you roll it out:
# On server (with BBR enabled)
iperf3 -s
# On client, before flipping the kernel
iperf3 -c server.example.com -t 30 -P 4 -C cubic
# Then enable BBR and re-run
sudo sysctl -w net.ipv4.tcp_congestion_control=bbr
iperf3 -c server.example.com -t 30 -P 4 -C bbr
For application-level measurement, the most useful number isn’t throughput — it’s p99 RTT under load. CUBIC’s signature is high throughput, high p99 RTT. BBR’s signature is high throughput, low p99 RTT. If your service is rate-limited and the goal is tail latency, that’s the metric to track.
Key Takeaways
- BBR replaces loss-detection with a (BtlBw, RTprop) model of the path, which lets it avoid bufferbloat on deep-buffer bottlenecks and recover quickly from loss on wireless/policed paths.
- Always run BBR with the
fqqdisc. Pacing without per-flow queueing is half the algorithm. - Use BBR v2 or v3. v1 is fine for a single flow on a clean path, but it will starve CUBIC neighbors and is the source of most “BBR is unfair” stories online.
- Audit your application-level concurrency limits so they don’t override the kernel’s pacing decisions on long-fat-pipe paths.
- Measure p99 RTT under load, not just throughput. That’s the dimension where BBR most often surprises people, both positively and negatively.
- Know your bottleneck. BBR helps a lot on deep-buffer, long-RTT, lossy paths. On a low-loss, shallow-buffer LAN, it’s a wash. On heavily policed paths, it can hurt.
Further Reading
- The BBR Congestion Control Algorithm (Cardwell et al., SIGCOMM 2017) — the foundational paper, still the clearest explanation of the model.
- BBR v2: A Model-based Congestion Control (Cardwell et al., 2019) — the v2 design and the loss/ecn additions.
- Linux kernel source: net/ipv4/tcp_bbr.c — the canonical reference for what your kernel actually ships.
- Cloudflare: “Why we use BBR” — a great production write-up, including the year-long A/B that ultimately moved most of their edge.
- Google BBR v3 announcement (Linux 6.6 release notes) — the most concise summary of what changed in v3.
- AIMD, BBR, and queueing (Geoff Huston, APNIC) — a thoughtful outsider’s take on the algorithmic landscape, including DCTCP and DCQCN.