This section discusses how the congestion window is updated during the different stages of the CUBIC congestion controller.
The unit of all window sizes in this document is segments of the SMSS, and the unit of all times is seconds. Implementations can use bytes to express window sizes, which would require factoring in the SMSS wherever necessary and replacing
segments_acked (
Figure 4) with the number of acknowledged bytes.
-
βcubic: CUBIC multiplicative decrease factor as described in Section 4.6.
-
αcubic: CUBIC additive increase factor used in the Reno-friendly region as described in Section 4.3.
-
C: Constant that determines the aggressiveness of CUBIC in competing with other congestion control algorithms in high-BDP networks. Please see Section 5 for more explanation on how it is set. The unit for C is
This section defines the variables required to implement CUBIC:
-
RTT:Smoothed round-trip time in seconds, calculated as described in[RFC 6298].
-
cwnd:Current congestion window in segments.
-
ssthresh:Current slow start threshold in segments.
-
cwndprior:Size of cwnd in segments at the time of setting ssthreshmost recently, either upon exiting the first slow start orjust before cwnd was reduced in the last congestion event.
-
Wmax:Size of cwnd in segments just before cwnd was reduced in the lastcongestion event when fast convergence is disabled (same ascwndprior on a congestion event). However, if fastconvergence is enabled, Wmax may be further reducedbased on the current saturation point.
-
K:The time period in seconds it takes to increase the congestion windowsize at the beginning of the current congestion avoidance stage toWmax.
-
tcurrent:Current time of the system in seconds.
-
tepoch:The time in seconds at which the current congestion avoidance stagestarted.
-
cwndepoch:The cwnd at the beginning of the current congestion avoidance stage,i.e., at time tepoch.
-
Wcubic(t): The congestion window in segments at time t in seconds based on the cubic increase function, as described in Section 4.2.
-
target:Target value of the congestion window in segments after the next RTT --that is, Wcubic(t + RTT), as described in Section 4.2.
-
West:An estimate for the congestion window in segments in the Reno-friendlyregion -- that is, an estimate for the congestion window of Reno.
-
segments_acked:Number of SMSS-sized segments acked when a "new ACK" is received, i.e., anACK that cumulatively acknowledges the delivery of previously unacknowledged data. Thisnumber will be a decimal value when a new ACK acknowledges an amount ofdata that is not SMSS-sized. Specifically, it can be less than 1 when anew ACK acknowledges a segment smaller than the SMSS.
CUBIC maintains the ACK clocking of Reno by increasing the congestion window only at the reception of a new ACK. It does not make any changes to the TCP Fast Recovery and Fast Retransmit algorithms [
RFC 6582] [
RFC 6675].
During congestion avoidance, after a congestion event is detected as described in
Section 3.1, CUBIC uses a window increase function different from Reno.
CUBIC uses the following window increase function:
3
W (t) = C * (t - K) + W
cubic max
where
t is the elapsed time in seconds from the beginning of the current congestion avoidance stage -- that is,
and where
tepoch is the time at which the current congestion avoidance stage starts.
K is the time period that the above function takes to increase the congestion window size at the beginning of the current congestion avoidance stage to
Wmax if there are no further congestion events.
K is calculated using the following equation:
┌────────────────┐
3 │W - cwnd
╲ │ max epoch
K = ╲ │────────────────
╲│ C
where
cwndepoch is the congestion window at the beginning of the current congestion avoidance stage.
Upon receiving a new ACK during congestion avoidance, CUBIC computes the
target congestion window size after the next
RTT using
Figure 1 as follows, where
RTT is the smoothed round-trip time. The lower and upper bounds below ensure that CUBIC's congestion window increase rate is non-decreasing and is less than the increase rate of slow start [
SXEZ19].
⎧
⎪cwnd if W (t + RTT) < cwnd
⎪ cubic
⎨1.5 * cwnd if W (t + RTT) > 1.5 * cwnd
target = ⎪ cubic
⎪W (t + RTT) otherwise
⎩ cubic
The elapsed time
t in
Figure 1 MUST NOT include periods during which
cwnd has not been updated due to application-limited behavior (see
Section 5.8).
Depending on the value of the current congestion window size
cwnd, CUBIC runs in three different regions:
-
The Reno-friendly region, which ensures that CUBIC achieves at least the same throughput as Reno.
-
The concave region, if CUBIC is not in the Reno-friendly region and cwnd is less than Wmax.
-
The convex region, if CUBIC is not in the Reno-friendly region and cwnd is greater than Wmax.
To summarize, CUBIC computes both W
cubic(
t) and
West (see
Section 4.3) on receiving a new ACK in congestion avoidance and chooses the larger of the two values.
The next sections describe the exact actions taken by CUBIC in each region.
Reno performs well in certain types of networks -- for example, under short RTTs and small bandwidths (or small BDPs). In these networks, CUBIC remains in the Reno-friendly region to achieve at least the same throughput as Reno.
The Reno-friendly region is designed according to the analysis discussed in [
FHP00], which studies the performance of an AIMD algorithm with an additive factor of α (segments per
RTT) and a multiplicative factor of β, denoted by AIMD(α, β).
p is the packet loss rate. Specifically, the average congestion window size of AIMD(α, β) can be calculated using
Figure 3.
┌───────────────┐
│ α * (1 + β)
AVG_AIMD(α, β) = ╲ │───────────────
╲│2 * (1 - β) * p
By the same analysis, to achieve an average window size similar to Reno that uses AIMD(1, 0.5), α must be equal to
Thus, CUBIC uses
Figure 4 to estimate the window size
West in the Reno-friendly region with
1 - β
cubic
α = 3 * ──────────
cubic 1 + β
cubic
which achieves approximately the same average window size as Reno in many cases. The model used to calculate α
cubic is not absolutely precise, but analysis and simulation as discussed in [
AIMD-friendliness], as well as over a decade of experience with CUBIC in the public Internet, show that this approach produces acceptable levels of rate fairness between CUBIC and Reno flows. Also, no significant drawbacks of the model have been reported. However, continued detailed analysis of this approach would be beneficial. When receiving a new ACK in congestion avoidance (where
cwnd could be greater than or less than
Wmax), CUBIC checks whether W
cubic(
t) is less than
West. If so, CUBIC is in the Reno-friendly region and
cwnd SHOULD be set to
West at each reception of a new ACK.
West is set equal to
cwndepoch at the start of the congestion avoidance stage. After that, on every new ACK,
West is updated using
Figure 4. Note that this equation uses
segments_acked and
cwnd is measured in segments. An implementation that measures
cwnd in bytes should adjust the equation accordingly using the number of acknowledged bytes and the SMSS. Also note that this equation works for connections with enabled or disabled delayed ACKs [
RFC 5681], as
segments_acked will be different based on the segments actually acknowledged by a new ACK.
segments_acked
W = W + α * ──────────────
est est cubic cwnd
Once
West has grown to reach the
cwnd at the time of most recently setting
ssthresh -- that is,
West >=
cwndprior -- the sender
SHOULD set α
cubic to 1 to ensure that it can achieve the same congestion window increment rate as Reno, which uses AIMD(1, 0.5).
The next two sections assume that CUBIC is not in the Reno-friendly region and uses the window increase function described in
Section 4.2. Although
cwnd is incremented in the same way for both concave and convex regions, they are discussed separately to analyze and understand the difference between the two regions.
When receiving a new ACK in congestion avoidance, if CUBIC is not in the Reno-friendly region and
cwnd is less than
Wmax, then CUBIC is in the concave region. In this region,
cwnd MUST be incremented by
target - cwnd
─────────────
cwnd
for each received new ACK, where
target is calculated as described in
Section 4.2.
When receiving a new ACK in congestion avoidance, if CUBIC is not in the Reno-friendly region and
cwnd is larger than or equal to
Wmax, then CUBIC is in the convex region.
The convex region indicates that the network conditions might have changed since the last congestion event, possibly implying more available bandwidth after some flow departures. Since the Internet is highly asynchronous, some amount of perturbation is always possible without causing a major change in available bandwidth.
Unless the cwnd is overridden by the AIMD window increase, CUBIC will behave cautiously when operating in this region. The convex profile aims to increase the window very slowly at the beginning when
cwnd is around
Wmax and then gradually increases its rate of increase. This region is also called the "maximum probing phase", since CUBIC is searching for a new
Wmax. In this region,
cwnd MUST be incremented by
target - cwnd
─────────────
cwnd
for each received new ACK, where
target is calculated as described in
Section 4.2.
When a congestion event is detected by the mechanisms described in
Section 3.1, CUBIC updates
Wmax and reduces
cwnd and
ssthresh immediately, as described below. In the case of packet loss, the sender
MUST reduce
cwnd and
ssthresh immediately upon entering loss recovery, similar to [
RFC 5681] (and [
RFC 6675]). Note that other mechanisms, such as Proportional Rate Reduction [
RFC 6937], can be used to reduce the sending rate during loss recovery more gradually. The parameter β
cubic SHOULD be set to 0.7, which is different from the multiplicative decrease factor used in [
RFC 5681] (and [
RFC 6675]) during fast recovery.
In
Figure 5,
flight_size is the amount of outstanding (unacknowledged) data in the network, as defined in [
RFC 5681]. Note that a rate-limited application with idle periods or periods when unable to send at the full rate permitted by
cwnd could easily encounter notable variations in the volume of data sent from one RTT to another, resulting in
flight_size that is significantly less than
cwnd when there is a congestion event. The congestion response would therefore decrease
cwnd to a much lower value than necessary. To avoid such suboptimal performance, the mechanisms described in [
RFC 7661] can be used. [
RFC 7661] describes how to manage and use
cwnd and
ssthresh during a rate-limited interval, and how to update
cwnd and
ssthresh after congestion has been detected. The mechanisms defined in [
RFC 7661] are safe to use even when
cwnd is greater than the receive window, because they validate
cwnd based on the amount of data acknowledged by the network in an RTT, which implicitly accounts for the allowed receive window.
Some implementations of CUBIC currently use
cwnd instead of
flight_size when calculating a new
ssthresh. Implementations that use
cwnd MUST use other measures to prevent
cwnd from growing when the volume of bytes in flight is smaller than
cwnd. This also effectively prevents
cwnd from growing beyond the receive window. Such measures are important for preventing a CUBIC sender from using an arbitrarily high cwnd
value when calculating new values for
ssthresh and
cwnd when congestion is detected. This might not be as robust as the mechanisms described in [
RFC 7661].
A QUIC sender that uses a
cwnd value to calculate new values for
cwnd and
ssthresh after detecting a congestion event is
REQUIRED to apply similar mechanisms [
RFC 9002].
ssthresh = flight_size * β new ssthresh
cubic
cwnd = cwnd save cwnd
prior
⎧max(ssthresh, 2) reduction on loss, cwnd >= 2 SMSS
cwnd = ⎨max(ssthresh, 1) reduction on ECE, cwnd >= 1 SMSS
⎩
ssthresh = max(ssthresh, 2) ssthresh >= 2 SMSS
A side effect of setting β
cubic to a value bigger than 0.5 is that packet loss can happen for more than one RTT in certain cases, but it can work efficiently in other cases -- for example, when HyStart++ [
RFC 9406] is used along with CUBIC or when the sending rate is limited by the application. While a more adaptive setting of β
cubic could help limit packet loss to a single round, it would require detailed analyses and large-scale evaluations to validate such algorithms.
Note that CUBIC
MUST continue to reduce
cwnd in response to congestion events detected by ECN-Echo ACKs until it reaches a value of 1 SMSS. If congestion events indicated by ECN-Echo ACKs persist, a sender with a
cwnd of 1 SMSS
MUST reduce its sending rate even further. This can be achieved by using a retransmission timer with exponential backoff, as described in [
RFC 3168].
To improve convergence speed, CUBIC uses a heuristic. When a new flow joins the network, existing flows need to give up some of their bandwidth to allow the new flow some room for growth if the existing flows have been using all the network bandwidth. To speed up this bandwidth release by existing flows, the following fast convergence mechanism
SHOULD be implemented.
With fast convergence, when a congestion event occurs,
Wmax is updated as follows, before the window reduction described in
Section 4.6.
⎧ 1 + β
⎪ cubic
⎪cwnd * ────────── if cwnd < W and fast convergence enabled,
W = ⎨ 2 max
max ⎪ further reduce W
⎪ max
⎩cwnd otherwise, remember cwnd before reduction
During a congestion event, if the current
cwnd is less than
Wmax, this indicates that the saturation point experienced by this flow is getting reduced because of a change in available bandwidth. This flow can then release more bandwidth by reducing
Wmax further. This action effectively lengthens the time for this flow to increase its congestion window, because the reduced
Wmax forces the flow to plateau earlier. This allows more time for the new flow to catch up to its congestion window size.
Fast convergence is designed for network environments with multiple CUBIC flows. In network environments with only a single CUBIC flow and without any other traffic, fast convergence
SHOULD be disabled.
In the case of a timeout, CUBIC follows Reno to reduce
cwnd [
RFC 5681] but sets
ssthresh using β
cubic (same as in
Section 4.6) in a way that is different from Reno TCP [
RFC 5681].
During the first congestion avoidance stage after a timeout, CUBIC increases its congestion window size using
Figure 1, where
t is the elapsed time since the beginning of the current congestion avoidance stage,
K is set to 0, and
Wmax is set to the congestion window size at the beginning of the current congestion avoidance stage. In addition, for the Reno-friendly region,
West SHOULD be set to the congestion window size at the beginning of the current congestion avoidance stage.
In cases where CUBIC reduces its congestion window in response to having detected packet loss via duplicate ACKs or timeouts, it is possible that the missing ACK could arrive after the congestion window reduction and a corresponding packet retransmission. For example, packet reordering could trigger this behavior. A high degree of packet reordering could cause multiple congestion window reduction events, where spurious losses are incorrectly interpreted as congestion signals, thus degrading CUBIC's performance significantly.
For TCP, there are two types of spurious events: spurious timeouts and spurious fast retransmits. In the case of QUIC, there are no spurious timeouts, as the loss is only detected after receiving an ACK.
An implementation
MAY detect spurious timeouts based on the mechanisms described in Forward RTO-Recovery [
RFC 5682]. Experimental alternatives include the Eifel detection algorithm [
RFC 3522]. When a spurious timeout is detected, a TCP implementation
MAY follow the response algorithm described in [
RFC 4015] to restore the congestion control state and adapt the retransmission timer to avoid further spurious timeouts.
Upon receiving an ACK, a TCP implementation
MAY detect spurious fast retransmits either using TCP Timestamps or via D-SACK [
RFC 2883]. As noted above, experimental alternatives include the Eifel detection algorithm [
RFC 3522], which uses TCP Timestamps; and DSACK-based detection [
RFC 3708], which uses DSACK information. A QUIC implementation can easily determine a spurious fast retransmit if a QUIC packet is acknowledged after it has been marked as lost and the original data has been retransmitted with a new QUIC packet.
This section specifies a simple response algorithm when a spurious fast retransmit is detected by acknowledgments. Implementations would need to carefully evaluate the impact of using this algorithm in different environments that may experience a sudden change in available capacity (e.g., due to variable radio capacity, a routing change, or a mobility event).
When packet loss is detected via acknowledgments, a CUBIC implementation
MAY save the current value of the following variables before the congestion window is reduced.
undo_cwnd = cwnd
undo_cwnd = cwnd
prior prior
undo_ssthresh = ssthresh
undo_W = W
max max
undo_K = K
undo_t = t
epoch epoch
undo_W = W
est est
Once the previously declared packet loss is confirmed to be spurious, CUBIC
MAY restore the original values of the above-mentioned variables as follows if the current
cwnd is lower than
cwndprior. Restoring the original values ensures that CUBIC's performance is similar to what it would be without spurious losses.
cwnd = undo_cwnd ⎫
cwnd = undo_cwnd ⎮
prior prior ⎮
ssthresh = undo_ssthresh ⎮
W = undo_W ⎮
max max ⎬if cwnd < cwnd
K = undo_K ⎮ prior
t = undo_t ⎮
epoch epoch ⎮
W = undo_W ⎮
est est ⎭
In rare cases, when the detection happens long after a spurious fast retransmit event and the current
cwnd is already higher than
cwndprior, CUBIC
SHOULD continue to use the current and the most recent values of these variables.
When
cwnd is no more than
ssthresh, CUBIC
MUST employ a slow start algorithm. In general, CUBIC
SHOULD use the HyStart++ slow start algorithm [
RFC 9406] or
MAY use the Reno TCP slow start algorithm [
RFC 5681] in the rare cases when HyStart++ is not suitable. Experimental alternatives include hybrid slow start [
HR11], a predecessor to HyStart++ that some CUBIC implementations have used as the default for the last decade, and limited slow start [
RFC 3742]. Whichever startup algorithm is used, work might be needed to ensure that the end of slow start and the first multiplicative decrease of congestion avoidance work well together.
When CUBIC uses HyStart++ [
RFC 9406], it may exit the first slow start without incurring any packet loss and thus
Wmax is undefined. In this special case, CUBIC sets
cwndprior = cwnd and switches to congestion avoidance. It then increases its congestion window size using
Figure 1, where
t is the elapsed time since the beginning of the current congestion avoidance stage,
K is set to 0, and
Wmax is set to the congestion window size at the beginning of the current congestion avoidance stage.