RFC 3449
TCP Performance Implications of Network Path Asymmetry
Pages: 41
Best Current Practice: 69
Best Current Practice: 69
Part 3 of 3 – Pages 29 to 41
6. Security Considerations
The recommendations contained in this document do not impact the integrity of TCP, introduce new security implications to the TCP protocol, or applications using TCP. Some security considerations in the context of this document arise from the implications of using IPSec by the end hosts or routers operating along the return path. Use of IPSec prevents, or complicates, some of the mitigations. For example: (i) When IPSec ESP [RFC2406] is used to encrypt the IP payload, the TCP header can neither be read nor modified by intermediate entities. This rules out header compression, ACK Filtering, ACK Reconstruction, and the ACK Compaction. (ii) The TCP header information may be visible, when some forms of network layer security are used. For example, using IPSec AH [RFC2402], the TCP header may be read, but not modified, by intermediaries. This may in future allow extensions to support ACK Filtering, but rules out the generation of new
packets by intermediaries (e.g., ACK Reconstruction). The
enhanced header compression scheme discussed in [RFC2507] would
also work with IPSec AH.
There are potential Denial-of-Service (DoS) implications when using
Type 2 schemes. Unless additional security mechanisms are used, a
Reconstructor/expander could be exploited as a packet amplifier. A
third party may inject unauthorized Stretch ACKs into the reverse
path, triggering the generation of additional ACKs. These ACKs would
consume capacity on the return path and processing resources at the
systems along the path, including the destination host. This
provides a potential platform for a DoS attack. The usual
precautions must be taken to verify the correct tunnel end point, and
to ensure that applications cannot falsely inject packets that expand
to generate unwanted traffic. Imposing a rate limit and bound on the
delayed ACK factor(d) would also lessen the impact of any undetected
exploitation.
7. Summary
This document considers several TCP performance constraints that
arise from asymmetry in the properties of the forward and reverse
paths across an IP network. Such performance constraints arise,
e.g., as a result of both bandwidth (capacity) asymmetry, asymmetric
shared media in the reverse direction, and interactions with Media
Access Control (MAC) protocols. Asymmetric capacity may cause TCP
Acknowledgments (ACKs) to be lost or become inordinately delayed
(e.g., when a bottleneck link is shared between many flows, or when
there is bidirectional traffic). This effect may be exacerbated with
media-access delays (e.g., in certain multi-hop radio subnetworks,
satellite Bandwidth on Demand access). Asymmetry, and particular
high asymmetry, raises a set of TCP performance issues.
A set of techniques providing performance improvement is surveyed.
These include techniques to alleviate ACK Congestion and techniques
that enable a TCP sender to cope with infrequent ACKs without
destroying TCP self-clocking. These techniques include both end-to-
end, local link-layer, and subnetwork schemes. Many of these
techniques have been evaluated in detail via analysis, simulation,
and/or implementation on asymmetric subnetworks forming part of the
Internet. There is however as yet insufficient operational
experience for some techniques, and these therefore currently remain
items of on-going research and experimentation.
The following table summarizes the current recommendations.
Mechanisms are classified as recommended (REC), not recommended (NOT
REC) or experimental (EXP). Experimental techniques may not be well
specified. These techniques will require further operational
experience before they can be recommended for use in the public
Internet.
The recommendations for end-to-end host modifications are summarized
in table 1. This lists each technique, the section in which each
technique is discussed, and where it is applied (S denotes the host
sending TCP data packets in the forward direction, R denotes the host
which receives these data packets).
+------------------------+-------------+------------+--------+
| Technique | Use | Section | Where |
+------------------------+-------------+------------+--------+
| Modified Delayed ACKs | NOT REC | 4.1 | TCP R |
| Large MSS & NO FRAG | REC | 4.2 | TCP SR |
| Large MSS & IP FRAG | NOT REC | 4.2 | TCP SR |
| ACK Congestion Control | EXP | 4.3 | TCP SR |
| Window Pred. Mech (WPM)| NOT REC | 4.4 | TCP R |
| Window Cwnd. Est. (ACE)| NOT REC | 4.5 | TCP R |
| TCP Sender Pacing | EXP *1 | 4.6 | TCP S |
| Byte Counting | NOT REC *2 | 4.7 | TCP S |
| Backpressure | EXP *1 | 4.8 | TCP R |
+------------------------+-------------+------------+--------+
Table 1: Recommendations concerning host modifications.
*1 Implementation of the technique may require changes to the
internal design of the protocol stack in end hosts.
*2 Dependent on a scheme for preventing excessive TCP transmission
burst.
The recommendations for techniques that do not require the TCP sender
and receiver to be aware of their existence (i.e., transparent
techniques) are summarized in table 2. Each technique is listed
along with the section in which each mechanism is discussed, and
where the technique is applied (S denotes the sending interface prior
to the upstream bottleneck link, R denotes receiving interface
following the upstream bottleneck link).
+------------------------+-------------+------------+--------+
| Mechanism | Use | Section | Type |
+------------------------+-------------+------------+--------+
| Header Compr. (V-J) | REC *1 | 5.1.1 | 0 SR |
| Header Compr. (ROHC) | REC *1 *2 | 5.1.2 | 0 SR |
+------------------------+-------------+------------+--------+
| ACK Filtering (AF) | EXP *3 | 5.2.1 | 1 S |
| ACK Decimation | EXP *3 | 5.2.2 | 1 S |
+------------------------+-------------+------------+--------+
| ACK Reconstruction (AR)| NOT REC | 5.3.1 | 2 *4 |
| ACK Compaction/Compand.| EXP | 5.3.2 | 2 S *4 |
| Gen. Traff. Shap. (GTS)| REC | 5.3.3 | 2 *5 |
+------------------------+-------------+------------+--------+
| Fair Queueing (FQ) | REC | 5.4.1 | 3 S |
| ACKs-First Scheduling | NOT REC | 5.4.2 | 3 S |
+------------------------+-------------+------------+--------+
Table 2: Recommendations concerning transparent modifications.
*1 At high asymmetry these schemes may degrade TCP performance, but
are not considered harmful to the Internet.
*2 Standardisation of new TCP compression protocols is the subject of
ongoing work within the ROHC WG, refer to other IETF RFCs on the
use of these techniques.
*3 Use in the Internet is dependent on a scheme for preventing
excessive TCP transmission burst.
*4 Performed at a point along the reverse path after the upstream
bottleneck link.
*5 Performed at a point along the forward path.
8. Acknowledgments
This document has benefited from comments from the members of the
Performance Implications of Links (PILC) Working Group. In
particular, the authors would like to thank John Border, Spencer
Dawkins, Aaron Falk, Dan Grossman, Randy Katz, Jeff Mandin, Rod
Ragland, Ramon Segura, Joe Touch, and Lloyd Wood for their useful
comments. They also acknowledge the data provided by Metricom Inc.,
concerning operation of their packet data network.
9. References
References of the form RFCnnnn are Internet Request for Comments
(RFC) documents available online at http://www.rfc-editor.org/.
9.1 Normative References
[RFC793] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, September 1981. [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - Communication Layers", STD 3, RFC 1122, October 1989. [RFC1144] Jacobson, V., "Compressing TCP/IP Headers for Low-Speed Serial Links", RFC 1144, February 1990. [RFC1191] Mogul, J. and S. Deering, "Path MTU Discovery", RFC 1191, November 1990. [RFC2581] Allman, M., Paxson, V. and W. Stevens, "TCP Congestion Control", RFC 2581, April 1999. [RFC2784] Farinacci, D., Li, T., Hanks, S., Meyer, D. and P. Traina, "Generic Routing Encapsulation (GRE)", RFC 2784, March 2000. [RFC3135] Border, J., Kojo, M., Griner, J., Montenegro, G. and Z. Shelby, "Performance Enhancing Proxies Intended to Mitigate Link-Related Degradations", RFC 3135, June 2001.9.2 Informative References
[abc-ID] Allman, M., "TCP Congestion Control with Appropriate Byte Counting", Work in Progress. [All97b] Allman, M., "Fixing Two BSD TCP Bugs", Technical Report CR-204151, NASA Lewis Research Center, October 1997. [ANS01] ANSI Standard T1.413, "Network to Customer Installation Interfaces - Asymmetric Digital Subscriber Lines (ADSL) Metallic Interface", November 1998. [ASB96] Arora, V., Suphasindhu, N., Baras, J.S. and D. Dillon, "Asymmetric Internet Access over Satellite-Terrestrial Networks", Proc. AIAA: 16th International Communications Satellite Systems Conference and Exhibit, Part 1, Washington, D.C., February 25-29, 1996, pp.476-482. [AST00] Aggarwal, A., Savage, S., and T. Anderson, "Understanding the Performance of TCP Pacing", Proc. IEEE INFOCOM, Tel- Aviv, Israel, V.3, March 2000, pp. 1157-1165.
[Bal98] Balakrishnan, H., "Challenges to Reliable Data Transport over Heterogeneous Wireless Networks", Ph.D. Thesis, University of California at Berkeley, USA, August 1998. http://nms.lcs.mit.edu/papers/hari-phd/ [BPK99] Balakrishnan, H., Padmanabhan, V. N., and R. H. Katz, "The Effects of Asymmetry on TCP Performance", ACM Mobile Networks and Applications (MONET), Vol.4, No.3, 1999, pp. 219-241. An expanded version of a paper published at Proc. ACM/IEEE Mobile Communications Conference (MOBICOM), 1997. [BPS00] Bennett, J. C., Partridge, C., and N. Schectman, "Packet Reordering is Not Pathological Network Behaviour", IEEE/ACM Transactions on Networking, Vol. 7, Issue. 6, 2000, pp.789-798. [Cla88] Clark, D.D, "The Design Philosophy of the DARPA Internet Protocols", ACM Computer Communications Review (CCR), Vol. 18, Issue 4, 1988, pp.106-114. [CLC99] Clausen, H., Linder, H., and B. Collini-Nocker, "Internet over Broadcast Satellites", IEEE Communications Magazine, Vol. 37, Issue. 6, 1999, pp.146-151. [CLP98] Calveras, A., Linares, J., and J. Paradells, "Window Prediction Mechanism for Improving TCP in Wireless Asymmetric Links". Proc. IEEE Global Communications Conference (GLOBECOM), Sydney Australia, November 1998, pp.533-538. [CR98] Cohen, R., and Ramanathan, S., "Tuning TCP for High Performance in Hybrid Fiber Coaxial Broad-Band Access Networks", IEEE/ACM Transactions on Networking, Vol.6, No.1, 1998, pp.15-29. [DS00] Cable Television Laboratories, Inc., Data-Over-Cable Service Interface Specifications---Radio Frequency Interface Specification SP-RFIv1.1-I04-00407, 2000 [DS01] Data-Over-Cable Service Interface Specifications, Radio Frequency Interface Specification 1.0, SP-RFI-I05-991105, Cable Television Laboratories, Inc., November 1999. [DMT96] Durst, R., Miller, G., and E. Travis, "TCP Extensions for Space Communications", ACM/IEEE Mobile Communications Conference (MOBICOM), New York, USA, November 1996, pp.15- 26.
[EN97] "Digital Video Broadcasting (DVB); DVB Specification for Data Broadcasting", European Standard (Telecommunications series) EN 301 192, 1997. [EN00] "Digital Video Broadcasting (DVB); Interaction Channel for Satellite Distribution Systems", Draft European Standard (Telecommunications series) ETSI, Draft EN 301 790, v.1.2.1 [FJ93] Floyd, S., and V. Jacobson, "Random Early Detection gateways for Congestion Avoidance", IEEE/ACM Transactions on Networking, Vol.1, No.4, 1993, pp.397-413. [FSS01] Fairhurst, G., Samaraweera, N.K.G, Sooriyabandara, M., Harun, H., Hodson, K., and R. Donardio, "Performance Issues in Asymmetric Service Provision using Broadband Satellite", IEE Proceedings on Communication, Vol.148, No.2, 2001, pp.95-99. [ITU01] ITU-T Recommendation E.681, "Traffic Engineering Methods For IP Access Networks Based on Hybrid Fiber/Coax System", September 2001. [ITU02] ITU-T Recommendation G.992.1, "Asymmetrical Digital Subscriber Line (ADSL) Transceivers", July 1999. [Jac88] Jacobson, V., "Congestion Avoidance and Control", Proc. ACM SIGCOMM, Stanford, CA, ACM Computer Communications Review (CCR), Vol.18, No.4, 1988, pp.314-329. [Ken87] Kent C.A., and J. C. Mogul, "Fragmentation Considered Harmful", Proc. ACM SIGCOMM, USA, ACM Computer Communications Review (CCR), Vol.17, No.5, 1988, pp.390- 401. [KSG98] Krout, T., Solsman, M., and J. Goldstein, "The Effects of Asymmetric Satellite Networks on Protocols", Proc. IEEE Military Communications Conference (MILCOM), Bradford, MA, USA, Vol.3, 1998, pp.1072-1076. [KVR98] Kalampoukas, L., Varma, A., and Ramakrishnan, K.K., "Improving TCP Throughput over Two-Way Asymmetric Links: Analysis and Solutions", Proc. ACM SIGMETRICS, Medison, USA, 1998, pp.78-89. [LM97] Lin, D., and R. Morris, "Dynamics of Random Early Detection", Proc. ACM SIGCOMM, Cannes, France, ACM Computer Communications Review (CCR), Vol.27, No.4, 1997, pp.78-89.
[LMS97] Lakshman, T.V., Madhow, U., and B. Suter, "Window-based Error Recovery and Flow Control with a Slow Acknowledgement Channel: A Study of TCP/IP Performance", Proc. IEEE INFOCOM, Vol.3, Kobe, Japan, 1997, pp.1199-1209. [MJW00] Ming-Chit, I.T., Jinsong, D., and W. Wang,"Improving TCP Performance Over Asymmetric Networks", ACM SIGCOMM, ACM Computer Communications Review (CCR), Vol.30, No.3, 2000. [Pad98] Padmanabhan, V.N., "Addressing the Challenges of Web Data Transport", Ph.D. Thesis, University of California at Berkeley, USA, September 1998 (also Tech Report UCB/CSD- 98-1016). http://www.cs.berkeley.edu/~padmanab/phd- thesis.html [RFC1323] Jacobson, V., Braden, R. and D. Borman, "TCP Extensions for High Performance", RFC 1323, May 1992. [RFC2018] Mathis, B., Mahdavi, J., Floyd, S. and A. Romanow, "TCP Selective Acknowledgment Options", RFC 2018, October 1996. [RFC2402] Kent, S. and R. Atkinson, "IP Authentication Header", RFC 2402, November 1998. [RFC2406] Kent, S. and R. Atkinson, "IP Encapsulating Security Payload (ESP)", RFC 2406, November 1998. [RFC2507] Degermark, M., Nordgren, B. and S. Pink, "IP Header Compression", RFC 2507, February 1999. [RFC2525] Paxson, V., Allman, M., Dawson, S., Heavens, I. and B. Volz, "Known TCP Implementation Problems", RFC 2525, March 1999. [RFC2686] Bormann, C., "The Multi-Class Extension to Multi-Link PPP", RFC 2686, September 1999. [RFC2760] Allman, M., Dawkins, S., Glover, D., Griner, J., Henderson, T., Heidemann, J., Kruse, H., Ostermann, S., Scott, K., Semke, J., Touch, J. and D. Tran, "Ongoing TCP Research Related to Satellites", RFC 2760, February 2000. [RFC2988] Paxson, V. and M. Allman, "Computing TCP's Retransmission Timer", RFC 2988, November 2000. [RFC3077] Duros, E., Dabbous, W., Izumiyama, H., Fujii, N. and Y. Zhang, "A link Layer tunneling mechanism for unidirectional links", RFC 3077, March 2001.
[RFC3095] Bormann, C., Burmeister, C., Degermark, M., Fukushima, H., Hannu, H., Jonsson, E., Hakenberg, R., Koren, T., Le, K., Liu, Z., Martensson, A., Miyazaki, A., Svanbro, K., Wiebke, T., Yoshimura, T. and H. Zheng, "RObust Header Compression (ROHC): Framework and four profiles: RTP, UDP ESP and uncompressed", RFC 3095, July 2001. [RFC3150] Dawkins, S., Montenegro, G., Kojo, M. and V. Magret, "End- to-end Performance Implications of Slow Links", BCP 48, RFC 3150, July 2001. [RFC3168] Ramakrishnan K., Floyd, S. and D. Black, "A Proposal to add Explicit Congestion Notification (ECN) to IP", RFC 3168, September 2001. [Sam99] Samaraweera, N.K.G, "Return Link Optimization for Internet Service Provision Using DVB-S Networks", ACM Computer Communications Review (CCR), Vol.29, No.3, 1999, pp.4-19. [Seg00] Segura R., "Asymmetric Networking Techniques For Hybrid Satellite Communications", NC3A, The Hague, Netherlands, NATO Technical Note 810, August 2000, pp.32-37. [SF98] Samaraweera, N.K.G., and G. Fairhurst. "High Speed Internet Access using Satellite-based DVB Networks", Proc. IEEE International Networks Conference (INC98), Plymouth, UK, 1998, pp.23-28. [ZSC91] Zhang, L., Shenker, S., and D. D. Clark, "Observations and Dynamics of a Congestion Control Algorithm: The Effects of Two-Way Traffic", Proc. ACM SIGCOMM, ACM Computer Communications Review (CCR), Vol 21, No 4, 1991, pp.133- 147.10. IANA Considerations
There are no IANA considerations associated with this document.
Appendix A. Examples of Subnetworks Exhibiting Network Path Asymmetry
This appendix provides a list of some subnetworks which are known to experience network path asymmetry. The asymmetry in capacity of these network paths can require mitigations to provide acceptable overall performance. Examples include the following: - IP service over some wide area and local area wireless networks. In such networks, the predominant network path asymmetry arises from the hub-and-spokes architecture of the network (e.g., a single base station that communicates with multiple mobile stations), this requires a Ready To Send / Clear To Send (RTS/CTS) protocol and a Medium Access Control (MAC) protocol which needs to accommodate the significant turn-around time for the radios. A high per-packet transmission overhead may lead to significant network path asymmetry. - IP service over a forward satellite link utilizing Digital Video Broadcast (DVB) transmission [EN97] (e.g., 38-45 Mbps), and a slower upstream link using terrestrial network technology (e.g., dial-up modem, line of sight microwave, cellular radio) [CLC99]. Network path asymmetry arises from a difference in the upstream and downstream link capacities. - Certain military networks [KSG98] providing Internet access to in-transit or isolated hosts [Seg00] using a high capacity downstream satellite link (e.g., 2-3 Mbps) with a narrowband upstream link (e.g., 2.4-9.6 kbps) using either Demand Assigned Multiple Access (DAMA) or fixed rate satellite links. The main factor contributing to network path asymmetry is the difference in the upstream and downstream link capacities. Some differences between forward and reverse paths may arise from the way in which upstream link capacity is allocated. - Most data over cable TV networks (e.g., DOCSIS [ITU01, DS00]), where the analogue channels assigned for upstream communication (i.e., in the reverse direction) are narrower and may be more noisy than those assigned for the downstream link. As a consequence, the upstream and downstream links differ in their transmission rate. For example, in DOCSIS 1.0 [DS00], the downstream transmission rate is either 27 or 52 Mbps. Upstream transmission rates may be dynamically selected to be one of a series of rates which range between 166 kbps to 9 Mbps. Operators may assign multiple upstream channels per downstream channel. Physical layer (PHY) overhead (which accompanies upstream transmissions, but is not present in the downstream link) can also increase the network path asymmetry. The Best Effort service, which is typically used to carry TCP, uses a
contention/reservation MAC protocol. A cable modem (CM) sending
an isolated packet (such as a TCP ACK) on the upstream link must
contend with other CMs to request capacity from the central cable
modem termination system (CMTS). The CMTS then grants timeslots
to a CM for the upstream transmission. The CM may "piggyback"
subsequent requests onto upstream packets, avoiding contention
cycles; as a result, spacing of TCP ACKs can be dramatically
altered due to minor variations in load of the cable data network
and inter-arrival times of TCP DATA packets. Numerous other
complexities may add to, or mitigate, the asymmetry in rate and
access latency experienced by packets sent on the upstream link
relative to downstream packets in DOCSIS. The asymmetry
experienced by end hosts may also change dynamically (e.g., with
network load), and when best effort services share capacity with
services that have symmetric reserved capacity (e.g., IP telephony
over the Unsolicited Grant service) [ITU01].
- Asymmetric Digital Subscriber Line (ADSL), by definition, offers a
downstream link transmission rate that is higher than that of the
upstream link. The available rates depend upon channel quality
and system configuration. For example, one widely deployed ADSL
technology [ITU02, ANS01] operates at rates that are multiples of
32 kbps (up to 6.144 Mbps) in the downstream link, and up to 640
kbps for the upstream link. The network path asymmetry
experienced by end hosts may be further increased when best effort
services, e.g., Internet access over ADSL, share the available
upstream capacity with reserved services (e.g., constant bit rate
voice telephony).
Authors' Addresses
Hari Balakrishnan Laboratory for Computer Science 200 Technology Square Massachusetts Institute of Technology Cambridge, MA 02139 USA Phone: +1-617-253-8713 EMail: hari@lcs.mit.edu Web: http://nms.lcs.mit.edu/~hari/ Venkata N. Padmanabhan Microsoft Research One Microsoft Way Redmond, WA 98052 USA Phone: +1-425-705-2790 EMail: padmanab@microsoft.com Web: http://www.research.microsoft.com/~padmanab/ Godred Fairhurst Department of Engineering Fraser Noble Building University of Aberdeen Aberdeen AB24 3UE UK EMail: gorry@erg.abdn.ac.uk Web: http://www.erg.abdn.ac.uk/users/gorry Mahesh Sooriyabandara Department of Engineering Fraser Noble Building University of Aberdeen Aberdeen AB24 3UE UK EMail: mahesh@erg.abdn.ac.uk Web: http://www.erg.abdn.ac.uk/users/mahesh
Full Copyright Statement Copyright (C) The Internet Society (2002). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society.