Low Latency Low Loss Scalable throughput (L4S) – a new default service for the Internet
We are so used to the unpredictability of queuing delay, we don’t know how good the Internet would feel without it. The RITE project has developed simple technology to make queuing delay a thing of the past—not just for a select few apps, but for all. With the resources on this page you can get a feel for the difference, understand how the solution works and help make it happen.
- Queuing Delay—the Problem
- 4-minute video demonstrating and explaining Ultra-Low Queuing Delay for All
- 30-minute video demonstrating, explaining and evaluating “Ultra-Low Queuing Delay for All”
- Videos used in IETF dispatch WG presentation of “Ultra-Low Queuing Delay for All Apps”
- Standards specifications
- Presentations
- Papers and magazine articles
- Implementations
- Source code
- Evaluations
- Getting Involved
Queuing Delay? What’s the Problem?
The large majority of Internet traffic is capacity-seeking, usually by using TCP. Until each flow completes it will fill available capacity and the buffers intended to absorb bursts. This adds queuing delay for the flow itself and for other applications running at the same time.
People think voice and gaming need low delay, because they don’t work well otherwise. But, nowadays the performance of nearly every application depends at least as much on delay as on bandwidth, e.g. instant messaging, conversational video, interactive video, remote presence, jumping around streaming video like YouTube, and all those little interactive transfers over the web, web services, finance apps, as well as cloud-based apps, cloud-rendered video, remote desktop, video-assisted control of remote machinery and industrial process control. That’s pretty much everything.
In the presence of other long-running traffic flow(s) or when the application is itself a long-running flow, even with a well-tuned network, the base delay typically doubles, halving performance. Sometimes it can be a lot worse. So queuing delay is one of the main causes of the Internet’s unpredictability. Some delay-sensitive applications are capacity-seeking themselves, e.g. interactive video. For them, queuing delay is ever-present, because they inflict it on themselves.
“Ultra-Low Queuing Delay for All” in 4 minutes
Video of cloud-rendered Panoramic Interactive Video (PIV) compiled during the IETF Bits-N-Bites Exhibition, Prague, July 2015.
“Ultra-Low Queuing Delay for All” in 30 minutes

Click the image for a video presentation, live demo and questions about the DualQ Coupled AQM, The slot starts 33mins into the session.
Presented by Koen De Schepper of Bell Labs at the IETF-93 Active Queue Management (AQM) working group, Prague, Jul 2015.
The live demo is by remote login to Bell Labs’ testbed in Antwerp, which uses real network equipment: data centre, core, backhaul, DSL access and home networks.
Videos used in IETF dispatch WG “Ultra-Low Queuing Delay for All Apps” slot
(Note, these videos have no soundtrack, because they were intended to complement remote attendance at the meeting)
Video extract showing ultra-low delay finger-gesture Interaction with a cloud-rendered Panoramic Interactive Video (PIV) of a Football Stadium. The network paths for both the L4S (DCTCP) and the Classic (TCP Cubic) clients use the same broadband residential, DSL access and core networks built using production equipment. They both have a base delay of 7ms; downstream capacity of 40Mb/s; and 2-3 other background TCP flows in progress. The PIV application uses one TCP flow for the media, which consumes no more than about 4Mb/s, adapting if less is available.
Standards specifications
As the IETF L4S architecture explains, there are three main parts to IETF standardisation: the identifier (#1), the network algorithms (#2) and the host algorithms (#3):
- IETF: Low Latency, Low Loss, Scalable Throughput (L4S) Internet Service: Architecture and Problem Statement
<draft-ietf-tsvwg-l4s-arch>
- An identifier for L4S packets
- The IETF’s proposed new identifier for Low Latency, Low Loss, Scalable throughput (L4S) packets
<draft-ietf-tsvwg-ecn-l4s-id>
and the IETF’s enabling standards track RFC to make way for this
<RFC8311> - The IETF is also in the process of assigning new standards track Diffserv codepoint to identify non-queue-building (NQB) traffic that could share the L4S queue with L4S traffic without all the usual bandwidth prioritization baggage of Diffserv:
<draft-ietf-tsvwg-nqb>
- The IETF’s proposed new identifier for Low Latency, Low Loss, Scalable throughput (L4S) packets
- Then network operators can ensure L4S flows are isolated from existing Internet traffic in terms of latency, but they coexist in terms of throughput.
- The IETF’s simple Dual-Queue Coupled AQM framework achieves this without inspecting deeper than the IP header:
<draft-ietf-tsvwg-aqm-dualq-coupled>
Example algorithms are given in an appendices. - L4S is deliberately separate from bandwidth allocation, which is not typically necessary, but could be added by network operators – see <draft-briscoe-tsvwg-l4s-diffserv>
- Cable modems and the cable head-end are required to support Low Latency DOCSIS (LLD), which supports L4S through the IETF’s DualQ Coupled AQM framework. The following versions of the DOCSIS 3.1 spec’s have defined LLD since Jan 2019:
<DOCSIS 3.1 MAC and Upper Layer Protocols Interface (MULPI) Spec (i17+)>
<Cable Modem Operations Support System Interface Spec (i14+)>
<CCAP Operations Support System Interface Specification (i14+)> - An Internet Draft outlining the Low Latency DOCSIS Queue Protection function:
Queue Protection to Preserve Low Latency
- The IETF’s simple Dual-Queue Coupled AQM framework achieves this without inspecting deeper than the IP header:
- And host developers can deploy new scalable TCP algorithms, e.g.
- the Prague Congestion Control
<draft-briscoe-iccrg-prague-congestion-control> - The L4S variant of SCReAM for real-time media;
(L4S support is currently in the code:
<https://github.com/EricssonResearch/scream/blob/master/README.md>
but not yet described in the spec:
<RFC8298>) - TCP Alpha/Preview Release of BBRv2;
(L4S support is currently in the code:
<https://github.com/google/bbr/blob/v2alpha/README.md>
but not yet described in the draft spec:
<draft-cardwell-iccrg-bbr-congestion-control-02>) - Prague and the L4S Support in BBRv2 are based on Data Centre TCP (DCTCP):
<RFC8257>
but DCTCP no longer works well over the Internet – it has not been maintained for this purpose. - Without steps #2 & #3, scalable TCPs are too aggressive to coexist with classic TCPs like Reno and Cubic, so DCTCP was initially confined to controlled networks like Data Centres, where all classic traffic sources could be upgraded (or isolated). To be safe over the public Internet, scalable congestion control algorithms will need to comply with the list of requirements agreed at an informally convened meeting in Prague that have become know as the TCP Prague requirements, even though they apply whatever the transport protocol, not just TCP (e.g. QUIC, RTP). The IETF has since defined draft safety requirements for scalable congestion controls here: <draft-ietf-tsvwg-ecn-l4s-id#section-4>. This in turn refers to the following IETF specifications that address specific TCP Prague requirements:
- Accurate ECN feedback, which the IETF has recognised will need an update to the TCP protocol, and it has stated the requirements a solution should comply with:
<RFC7560>
A candidate protocol to satisfy these requirements has been developed and adopted by the IETF:
<draft-ietf-tcpm-accurate-ecn> - Adding ECN to TCP control packets:
<draft-ietf-tcpm-generalized-ecn> - Recent ACKnowledgements: “RACK: a time-based fast loss detection algorithm for TCP”
<draft-ietf-tcpm-rack>
- Accurate ECN feedback, which the IETF has recognised will need an update to the TCP protocol, and it has stated the requirements a solution should comply with:
- the Prague Congestion Control
Presentations
This list of presentations is not currently being maintained.
- Ultra Low Queuing Delay for All; Koen De Schepper, Bob Briscoe, Olga Bondarenko & Inton Tsang, IETF ad hoc meeting (‘Bar BoF’) (Apr 2016) [odp|pdf]
- L4S: A gradually deployable simplifying clean-slate opportunity, Bob Briscoe & Koen De Schepper, IETF ad hoc meeting (‘Bar BoF’) (Apr 2016) [ppt|pdf]
- PI2 AQM for classic and scalable congestion control, Koen De Schepper (Bell Labs), Bob Briscoe, Olga Bondarenko (Simula), Ing-Jyh (Inton) Tsang (Bell Labs), Internet Research Task Force (IRTF) Internet congestion control research group (ICCRG), (Apr 2016) [pdf]
- L4S Birds of a Feather (‘BoF’) held at the IETF in Berlin (Jul 2016)
- L4S: Ultra-Low Queuing Delay for All; Bob Briscoe, Koen De Schepper, Olga Bondarenko & Inton Tsang, Stock presentation for network operators (Jul 2017) [odp|pdf]
- “Flow-start: Faster and Less Overshoot with Paced Chirping” Joakim Misund and Bob Briscoe, Internet Congestion Control Research Group (Jul 2018), a proposed flow-start algorithm for TCP Prague [pdf]
Papers and Magazine Articles
- Article in the IETF Journal describing the Demo in Bits-N-Bites at the IETF in Prague, July 2015.
Bob Briscoe, “Ultra-Low Delay for All” IETF Journal, Nov 2015 [ Pre-print (with pictures) | Journal archive (no pictures) ]. - Description of an ambitious demo:
Olga Bondarenko, Koen De Schepper, Ing-Jyh Tsang, Bob Briscoe, Andreas Petlund and Carsten Griwodz “Ultra-Low Delay for All: Live Experience, Live Analysis“, Proc. ACM Multimedia Systems; Demo Session (May 2016). - A white paper from CableLabs “Low Latency DOCSIS: Technology Overview” (Feb 2019). Low Latency DOCSIS supports L4S as well as other low latency features, and it is now required for DOCSIS 3.1.
- Draft of a journal paper explaining why scalable TCP algorithms solve the problem (in plain english and maths), how the Dual Queue Coupled AQM algorithm works, and recording the results of extensive testbed experiments.
“`Data Centre to the Home’: Deployable Ultra-Low Latency for All” De Schepper, K., Albisser, O., Tilmans, O and Briscoe, B. (2019)
- “PI2: A Linearized AQM for both Classic and Scalable TCP,” Koen De Schepper (Nokia Bell Labs), Olga Bondarenko (Simula), Ing-Jyh Tsang (Nokia Bell Labs), Bob Briscoe (Simula), Proc. ACM CONEXT 2016 (Dec 2016).
- “DualPI2 – Low Latency, Low Loss and Scalable (L4S) AQM,” Olga Albisser (Simula), Koen De Schepper (Nokia Bell-Labs), Bob Briscoe (Independent), Olivier Tilmans (Nokia Bell-Labs) and Henrik Steen (Simula), in Proc. Netdev 0x13 (Mar 2019).
- “Enabling time-critical applications over 5G with rate adaptation,” Per Willars, Emma Wittenmark, Henrik Ronkainen, Christer Östberg, Ingemar Johansson, Johan Strand (Ericsson), Petr Lédl, Dominik Schnieders (Deutsche Telekom), Ericsson – Deutsche Telekom White Paper BNEW-21:025455 Uen (May 2021)
- “Implementing the `TCP Prague’ Requirements for Low Latency Low Loss Scalable Throughput (L4S),” Bob Briscoe (Independent), Koen De Schepper (Nokia Bell-Labs), Olga Albisser (Simula), Joakim Misund (Uni Oslo), Olivier Tilmans (Nokia Bell-Labs), Mirja Kühlewind (ETH Zurich) and Asad Sajjad Ahmed (Simula), in Proc. Netdev 0x13 (Mar 2019).
- Discussion paper: “Resolving Tensions Between Congestion Control Scaling Requirements“, Bob Briscoe (Simula) and Koen De Schepper (Nokia Bell Labs), Simula Technical Report TR-CS-2016-001 (July 2017).
- Report (RITE project deliverable D3.3) including:
- QoS Simplification Objectives from a network operator (BT) (Section 3.1)
- HTTP Adaptive Streaming (HAS) and Web tests of DualQ Coupled (Section 3)
- Panoramic Interactive Video and Web tests of DualQ Coupled (Section 4)
- Research to address the more challenging ‘Prague Requirements’:
- Faster Convergence than DCTP at Flow Start still with ultra-low queueing delay:
- “Paced Chirping: Rapid flow start with very low queuing delay,” Joakim Misund (Uni Oslo) and Bob Briscoe (Independent), in Proc. IEEE Global Internet Symposium (May 2019).
- “Paced Chirping – Rethinking TCP start-up,” Joakim Misund (Uni Oslo) and Bob Briscoe (Independent), in Proc. Netdev 0x13 (Mar 2019).
- TCP congestion control working with a sub-packet window rather overriding the AQM on a low delay path: “Extending TCP for Low Round Trip Delay“, Masters Thesis of Asad Sajjad Ahmed (University of Oslo) (Aug 2019).
- Faster Convergence than DCTP at Flow Start still with ultra-low queueing delay:
Implementations
Source Code
- Dual Queue Coupled AQM
- with PI2: Linux repo
- With Curvy RED (TBA)
- TCP Prague
- QUIC Prague
- General repo (should work for Linux, FreeBSD, Windows)
- SCReAM (Self-Clocked Rate Adaptation for Multimedia) a mobile optimised congestion control algorithm for real-time interactive media, with support for L4S
- BBRv2
- ns3 network simulator
- l4s-evaluation: Programs and scripts for evaluation of the L4S architecture in ns-3.
Component Parts – Source Code
- Accurate ECN TCP Feedback (included in TCP Prague above)
- Linux repo without AccECN TCP Option
- FreeBSD repo without AccECN TCP Option (TBA)
- Paced Chirping
- for Linux (initial proof-of-concept research code)
- Working with a sub-packet window
- modification to Linux TCP stack (research prototype)
- Data Centre TCP (DCTCP) for
- Linux (in the mainline kernel)
- FreeBSD (in the mainline kernel)
- ns2 patch.
- Testing GUI for Congestion Controls and AQMs (incl. L4S)
- L4S team repo
Implementations – Closed Source
- Dual Queue Coupled AQM
- Nokia WiFi Beacon 1
using DualPI2 - Broadcom BCM88800 packet processor and traffic manager single chip device
Seventh generation of the StrataDNX™ switching product line
Supports Curvy RED
- Nokia WiFi Beacon 1
Evaluations
- Draft of a journal paper explaining why scalable TCP algorithms solve the problem (in plain english and maths), how the Dual Queue Coupled AQM algorithm works, and recording the results of extensive testbed experiments.
“`Data Centre to the Home’: Deployable Ultra-Low Latency for All” De Schepper, K., Albisser, O., Tilmans, O and Briscoe, B. (2019)
- An independent validation of the above performance results using TCP Prague (as opposed to the experiments with DCTCP, which has not been maintained for use over the Internet)
“Validating the Sharing Behavior and Latency Characteristics of the L4S Architecture“, ACM CCR 50(2):37–44 (May 2020). - “Destruction Testing: Ultra-Low Delay using Dual Queue Coupled Active Queue Management“, Masters Thesis of Henrik Steen (University of Oslo) (May 2017).
- Online interactive site to navigate the results of an extensive Evaluation of the Classic ECN AQM Fallback algorithm as implemented in TCP Prague.
- A summary of the initial results is included in this technical report about the algorithm design: “TCP Prague Fall-back on Detection of a Classic ECN AQM” Bob Briscoe & Asad Sajjad, Technical Report TR-BB-2019-002, arXiv:1911.00710 [cs.NI] (Apr 2020)
Getting Involved
- TCP Prague mailing list
- IETF TCP Maintenance and Minor Extensions working group mailing list
- IETF Transport Area working group mailing list
- IETF Active Queue Management working group mailing list (the AQM WG has been folded into the above Transport Area WG, but the mailing list continues)
- Contacts for the authors are included within each of the above specifications and papers