RFC 8041
Use Cases and Operational Experience with Multipath TCP
Pages: 30
Informational
Informational
Part 2 of 2 – Pages 16 to 30
3.6. Closing Subflows
client server | | MPTCP: ESTABLISHED | | MPTCP: ESTABLISHED Sub: ESTABLISHED | | Sub: ESTABLISHED | | | DATA_FIN | MPTCP: CLOSE-WAIT | <------------------------ | close() (step 1) Sub: ESTABLISHED | DATA_ACK | | ------------------------> | MPTCP: FIN-WAIT-2 | | Sub: ESTABLISHED | | | DATA_FIN + subflow-FIN | close()/shutdown() | ------------------------> | MPTCP: TIME-WAIT (step 2) | DATA_ACK | Sub: CLOSE-WAIT MPTCP: CLOSED | <------------------------ | Sub: FIN-WAIT-2 | | | | | subflow-FIN | MPTCP: CLOSED | <------------------------ | subflow-close() Sub: TIME-WAIT | subflow-ACK | (step 3) | ------------------------> | MPTCP: TIME-WAIT | | Sub: CLOSED | | Figure 5: Multipath TCP may not be able to avoid time-wait state on the subflow (indicated as Sub in the drawing), even if enforced by the application on the client-side. Figure 5 shows a very particular issue within Multipath TCP. Many high-performance applications try to avoid TIME-WAIT state by deferring the closure of the connection until the peer has sent a FIN. That way, the client on the left of Figure 5 does a passive closure of the connection, transitioning from CLOSE-WAIT to Last-ACK and finally freeing the resources after reception of the ACK of the FIN. An application running on top of an MPTCP-enabled Linux kernel might also use this approach. The difference here is that the close() of the connection (step 1 in Figure 5) only triggers the
sending of a DATA_FIN. Nothing guarantees that the kernel is ready to combine the DATA_FIN with a subflow-FIN. The reception of the DATA_FIN will make the application trigger the closure of the connection (step 2), trying to avoid TIME-WAIT state with this late closure. This time, the kernel might decide to combine the DATA_FIN with a subflow-FIN. This decision will be fatal, as the subflow's state machine will not transition from CLOSE_WAIT to Last-ACK, but rather go through FIN_WAIT-2 into TIME-WAIT state. The TIME-WAIT state will consume resources on the host for at least 2 MSL (Maximum Segment Lifetime). Thus, a smart application that tries to avoid TIME-WAIT state by doing late closure of the connection actually ends up with one of its subflows in TIME-WAIT state. A high-performance Multipath TCP kernel implementation should honor the desire of the application to do passive closure of the connection and successfully avoid TIME-WAIT state -- even on the subflows. The solution to this problem lies in an optimistic assumption that a host doing active-closure of a Multipath TCP connection by sending a DATA_FIN will soon also send a FIN on all its subflows. Thus, the passive closer of the connection can simply wait for the peer to send exactly this FIN -- enforcing passive closure even on the subflows. Of course, to avoid consuming resources indefinitely, a timer must limit the time our implementation waits for the FIN.3.7. Packet Schedulers
In a Multipath TCP implementation, the packet scheduler is the algorithm that is executed when transmitting each packet to decide on which subflow it needs to be transmitted. The packet scheduler itself does not have any impact on the interoperability of Multipath TCP implementations. However, it may clearly impact the performance of Multipath TCP sessions. The Multipath TCP implementation in the Linux kernel supports a pluggable architecture for the packet scheduler [PaaschPhD]. As of this writing, two schedulers have been implemented: round-robin and lowest-rtt-first. The second scheduler relies on the round-trip time (rtt) measured on each TCP subflow and sends first segments over the subflow having the lowest round-trip time. They are compared in [CSWS14]. The experiments and measurements described in [CSWS14] show that the lowest-rtt-first scheduler appears to be the best compromise from a performance viewpoint. Another study of the packet schedulers is presented in [PAMS2014]. This study relies on simulations with the Multipath TCP implementation in the Linux kernel. They compare the lowest-rtt- first with the round-robin and a random scheduler. They show some situations where the lowest-rtt-first scheduler does not perform as well as the other schedulers, but there are many scenarios where the
opposite is true. [PAMS2014] notes that "it is highly likely that the optimal scheduling strategy depends on the characteristics of the paths being used."3.8. Segment Size Selection
When an application performs a write/send system call, the kernel allocates a packet buffer (sk_buff in Linux) to store the data the application wants to send. The kernel will store at most one MSS (Maximum Segment Size) of data per buffer. As the MSS can differ amongst subflows, an MPTCP implementation must select carefully the MSS used to generate application data. The Linux kernel implementation had various ways of selecting the MSS: minimum or maximum amongst the different subflows. However, these heuristics of MSS selection can cause significant performance issues in some environments. Consider the following example. An MPTCP connection has two established subflows that respectively use an MSS of 1420 and 1428 bytes. If MPTCP selects the maximum, then the application will generate segments of 1428 bytes of data. An MPTCP implementation will have to split the segment in two (1420-byte and 8-byte) segments when pushing on the subflow with the smallest MSS. The latter segment will introduce a large overhead as this single data segment will use 2 slots in the congestion window (in packets) therefore reducing by roughly twice the potential throughput (in bytes/s) of this subflow. Taking the smallest MSS does not solve the issue as there might be a case where the subflow with the smallest MSS only sends a few packets, therefore reducing the potential throughput of the other subflows. The Linux implementation recently took another approach [DetalMSS]. Instead of selecting the minimum and maximum values, it now dynamically adapts the MSS based on the contribution of all the subflows to the connection's throughput. For each subflow, it computes the potential throughput achieved by selecting each MSS value and by taking into account the lost space in the congestion window. It then selects the MSS that allows to achieve the highest potential throughput. Given the prevalence of middleboxes that clamp the MSS, Multipath TCP implementations must be able to efficiently support subflows with different MSS values. The strategy described above is a possible solution to this problem.
3.9. Interactions with the Domain Name System
Multihomed clients such as smartphones can send DNS queries over any of their interfaces. When a single-homed client performs a DNS query, it receives from its local resolver the best answer for its request. If the client is multihomed, the answer in response to the DNS query may vary with the interface over which it has been sent. cdn1 | client -- cellular -- internet -- cdn3 | | +----- wifi --------+ | cdn2 Figure 6: Simple Network Topology If the client sends a DNS query over the WiFi interface, the answer will point to the cdn2 server while the same request sent over the cellular interface will point to the cdn1 server. This might cause problems for CDN providers that locate their servers inside ISP networks and have contracts that specify that the CDN server will only be accessed from within this particular ISP. Assume now that both the client and the CDN servers support Multipath TCP. In this case, a Multipath TCP session from cdn1 or cdn2 would potentially use both the cellular network and the WiFi network. Serving the client from cdn2 over the cellular interface could violate the contract between the CDN provider and the network operators. A similar problem occurs with regular TCP if the client caches DNS replies. For example, the client obtains a DNS answer over the cellular interface and then stops this interface and starts to use its WiFi interface. If the client retrieves data from cdn1 over its WiFi interface, this may also violate the contract between the CDN and the network operators. A possible solution to prevent this problem would be to modify the DNS resolution on the client. The client subnet Extension Mechanisms for DNS (EDNS) defined in [RFC7871] could be used for this purpose. When the client sends a DNS query from its WiFi interface, it should also send the client subnet corresponding to the cellular interface in this request. This would indicate to the resolver that the answer should be valid for both the WiFi and the cellular interfaces (e.g., the cdn3 server).
3.10. Captive Portals
Multipath TCP enables a host to use different interfaces to reach a server. In theory, this should ensure connectivity when at least one of the interfaces is active. However, in practice, there are some particular scenarios with captive portals that may cause operational problems. The reference environment is shown in Figure 7. client ----- network1 | +------- internet ------------- server Figure 7: Issue with Captive Portal The client is attached to two networks: network1 that provides limited connectivity and the entire Internet through the second network interface. In practice, this scenario corresponds to an open WiFi network with a captive portal for network1 and a cellular service for the second interface. On many smartphones, the WiFi interface is preferred over the cellular interface. If the smartphone learns a default route via both interfaces, it will typically prefer to use the WiFi interface to send its DNS request and create the first subflow. This is not optimal with Multipath TCP. A better approach would probably be to try a few attempts on the WiFi interface and then, upon failure of these attempts, try to use the second interface for the initial subflow as well.3.11. Stateless Webservers
MPTCP has been designed to interoperate with webservers that benefit from SYN-cookies to protect against SYN-flooding attacks [RFC4987]. MPTCP achieves this by echoing the keys negotiated during the MP_CAPABLE handshake in the third ACK of the three-way handshake. Reception of this third ACK then allows the server to reconstruct the state specific to MPTCP. However, one caveat to this mechanism is the unreliable nature of the third ACK. Indeed, when the third ACK gets lost, the server will not be able to reconstruct the MPTCP state. MPTCP will fall back to regular TCP in this case. This is in contrast to regular TCP. When the client starts sending data, the first data segment also includes the SYN-cookie, which allows the server to reconstruct the TCP-state. Further, this data segment will be retransmitted by the client in case it gets lost and thus is resilient against loss. MPTCP does not include the keys in this data segment and thus the server cannot reconstruct the MPTCP state.
This issue might be considered as a minor one for MPTCP. Losing the third ACK should only happen when packet loss is high; in this case, MPTCP provides a lot of benefits as it can move traffic away from the lossy link. It is undesirable that MPTCP has a higher chance to fall back to regular TCP in those lossy environments. [MPTCP-DEPLOY] discusses this issue and suggests a modified handshake mechanism that ensures reliable delivery of the MP_CAPABLE, following the three-way handshake. This modification will make MPTCP reliable, even in lossy environments when servers need to use SYN-cookies to protect against SYN-flooding attacks.3.12. Load-Balanced Server Farms
Large-scale server farms typically deploy thousands of servers behind a single virtual IP (VIP). Steering traffic to these servers is done through Layer 4 load-balancers that ensure that a TCP-flow will always be routed to the same server [Presto08]. As Multipath TCP uses multiple different TCP subflows to steer the traffic across the different paths, load-balancers need to ensure that all these subflows are routed to the same server. This implies that the load-balancers need to track the MPTCP-related state, allowing them to parse the token in the MP_JOIN and assign those subflows to the appropriate server. However, server farms typically deploy several load-balancers for reliability and capacity reasons. As a TCP subflow might get routed to any of these load-balancers, they would need to synchronize the MPTCP-related state -- a solution that is not feasible on a large scale. The token (carried in the MP_JOIN) contains the information indicating to which MPTCP-session the subflow belongs. As the token is a hash of the key, servers are not able to generate the token in such a way that the token can provide the necessary information to the load-balancers, which would allow them to route TCP subflows to the appropriate server. [MPTCP-LOAD] discusses this issue in detail and suggests two alternative MP_CAPABLE handshakes to overcome these.4. Security Considerations
This informational document discusses use cases and operational experience with Multipath TCP. An extensive analysis of the remaining security issues in the Multipath TCP specification has been published in [RFC7430], together with suggestions for possible solutions.
From a security viewpoint, it is important to note that Multipath TCP, like other multipath solutions such as SCTP, has the ability to send packets belonging to a single connection over different paths. This design feature of Multipath TCP implies that middleboxes that have been deployed on-path assuming that they would observe all the packets exchanged for a given connection in both directions may not function correctly anymore. A typical example are firewalls, Intrusion Detection System (IDS) or deep packet inspections (DPIs) deployed in enterprise networks. Those devices expect to observe all the packets from all TCP connections. With Multipath TCP, those middleboxes may not observe anymore all packets since some of them may follow a different path. The two examples below illustrate typical deployments of such middleboxes. The first example, Figure 8, shows an MPTCP-enabled smartphone attached to both an enterprise and a cellular network. If a Multipath TCP connection is established by the smartphone towards a server, some of the packets sent by the smartphone or the server may be transmitted over the cellular network and thus be invisible for the enterprise middlebox. smartphone +----- enterprise net --- MBox----+------ server | | +----- cellular net -------------+ Figure 8: Enterprise Middlebox May Not Observe All Packets from Multihomed Host The second example, Figure 9, shows a possible issue when multiple middleboxes are deployed inside a network. For simplicity, we assume that network1 is the default IPv4 path while network2 is the default IPv6 path. A similar issue could occur with per-flow load-balancing such as ECMP [RFC2992]. With regular TCP, all packets from each connection would either pass through Mbox1 or Mbox2. With Multipath TCP, the client can easily establish a subflow over network1 and another over network2 and each middlebox would only observe a part of the traffic of the end-to-end Multipath TCP connection. client ----R-- network1 --- MBox1 -----R------------- server | | +-- network2 --- MBox2 -----+ Figure 9: Interactions between Load-Balancing and Security Middleboxes In these two cases, it is possible for an attacker to evade some security measures operating on the TCP byte stream and implemented on the middleboxes by controlling the bytes that are actually sent over each subflow and there are tools that ease those kinds of evasion [PZ15] [PT14]. This is not a security issue for Multipath TCP itself
since Multipath TCP behaves correctly. However, this demonstrates the difficulty of enforcing security policies by relying only on on-path middleboxes instead of enforcing them directly on the endpoints.5. References
5.1. Normative References
[RFC6182] Ford, A., Raiciu, C., Handley, M., Barre, S., and J. Iyengar, "Architectural Guidelines for Multipath TCP Development", RFC 6182, DOI 10.17487/RFC6182, March 2011, <http://www.rfc-editor.org/info/rfc6182>. [RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure, "TCP Extensions for Multipath Operation with Multiple Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013, <http://www.rfc-editor.org/info/rfc6824>.5.2. Informative References
[BALIA] Peng, Q., Walid, A., Hwang, J., and S. Low, "Multipath TCP: analysis, design, and implementation", IEEE/ACM Trans. on Networking (TON), Volume 24, Issue 1, February 2016. [CACM14] Paasch, C. and O. Bonaventure, "Multipath TCP", Communications of the ACM, 57(4):51-57, April 2014, <http://inl.info.ucl.ac.be/publications/multipath-tcp>. [Cellnet12] Paasch, C., Detal, G., Duchene, F., Raiciu, C., and O. Bonaventure, "Exploring Mobile/WiFi Handover with Multipath TCP", ACM SIGCOMM workshop on Cellular Networks (Cellnet12), August 2012, <http://inl.info.ucl.ac.be/publications/ exploring-mobilewifi-handover-multipath-tcp>. [COMCOM2016] Tran, V., De Coninck, Q., Hesmans, B., Sadre, R., and O. Bonaventure, "Observing real Multipath TCP traffic", Computer Communications, DOI 10.1016/j.comcom.2016.01.014, April 2016, <http://inl.info.ucl.ac.be/publications/ observing-real-multipath-tcp-traffic>.
[COMMAG2016] De Coninck, Q., Baerts, M., Hesmans, B., and O. Bonaventure, "Observing Real Smartphone Applications over Multipath TCP", IEEE Communications Magazine Network Testing Series, 54(3), March 2016, <http://inl.info.ucl.ac.be/publications/observing-real- smartphone-applications-over-multipath-tcp>. [CONEXT12] Khalili, R., Gast, N., Popovic, M., Upadhyay, U., and J. Leboudec, "MPTCP is not Pareto-Optimal: Performance Issues and a Possible Solution", CoNEXT '12: Proceedings of the 8th international conference on Emerging networking experiments and technologies, DOI 10.1145/2413176.2413178, December 2012. [CONEXT13] Paasch, C., Khalili, R., and O. Bonaventure, "On the Benefits of Applying Experimental Design to Improve Multipath TCP", Conference on emerging Networking EXperiments and Technologies (CoNEXT), DOI 10.1145/2535372.2535403, December 2013, <http://inl.info.ucl.ac.be/publications/benefits-applying- experimental-design-improve-multipath-tcp>. [CONEXT15] Hesmans, B., Detal, G., Barre, S., Bauduin, R., and O. Bonaventure, "SMAPP: Towards Smart Multipath TCP-enabled APPlications", Proc. Conext 2015, Heidelberg, Germany, December 2015, <http://inl.info.ucl.ac.be/publications/ smapp-towards-smart-multipath-tcp-enabled-applications>. [CSWS14] Paasch, C., Ferlin, S., Alay, O., and O. Bonaventure, "Experimental evaluation of multipath TCP schedulers", CSWS '14: Proceedings of the 2014 ACM SIGCOMM workshop on Capacity sharing workshop, DOI 10.1145/2630088.2631977, August 2014. [DetalMSS] Detal, G., "dynamically adapt mss value", Post on the mptcp-dev mailing list, September 2014, <https://listes-2.sipr.ucl.ac.be/sympa/arc/mptcp-dev/ 2014-09/msg00130.html>. [FreeBSD-MPTCP] Williams, N., "Multipath TCP For FreeBSD Kernel Patch v0.5", <http://caia.swin.edu.au/urp/newtcp/mptcp>.
[GRE-NOTIFY] Leymann, N., Heidemann, C., Wasserman, M., Xue, L., and M. Zhang, "GRE Notifications for Hybrid Access", Work in Progress, draft-lhwxz-gre-notifications-hybrid-access-01, January 2015. [HAMPEL] Hampel, G., Rana, A., and T. Klein, "Seamless TCP mobility using lightweight MPTCP proxy", MobiWac '13: Proceedings of the 11th ACM international symposium on Mobility management and wireless access, DOI 10.1145/2508222.2508226, November 2013. [HotMiddlebox13] Hesmans, B., Duchene, F., Paasch, C., Detal, G., and O. Bonaventure, "Are TCP Extensions Middlebox-proof?", CoNEXT workshop Hot Middlebox, December 2013, <http://inl.info.ucl.ac.be/publications/ are-tcp-extensions-middlebox-proof>. [HotMiddlebox13b] Detal, G., Paasch, C., and O. Bonaventure, "Multipath in the Middle(Box)", HotMiddlebox '13, December 2013, <http://inl.info.ucl.ac.be/publications/ multipath-middlebox>. [HotNets] Raiciu, C., Pluntke, C., Barre, S., Greenhalgh, A., Wischik, D., and M. Handley, "Data center networking with multipath TCP", Hotnetx-IX: Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks Article No. 10, DOI 10.1145/1868447.1868457, October 2010, <http://doi.acm.org/10.1145/1868447.1868457>. [HYA-ARCH] Leymann, N., Heidemann, C., Wasserman, M., Xue, L., and M. Zhang, "Hybrid Access Network Architecture", Work in Progress, draft-lhwxz-hybrid-access-network- architecture-02, January 2015. [ICNP12] Cao, Y., Xu, M., and X. Fu, "Delay-based congestion control for multipath TCP", 20th IEEE International Conference on Network Protocols (INCP), DOI 10.1109/ICNP.2012.6459978, October 2012. [IETF88] Stewart, L., "IETF 88 Meeting minutes of the MPTCP working group", November 2013, <https://www.ietf.org/proceedings/ 88/minutes/minutes-88-mptcp>.
[IETFJ] Bonaventure, O. and S. Seo, "Multipath TCP Deployments", IETF Journal, Vol. 12, Issue 2, November 2016. [IMC11] Honda, M., Nishida, Y., Raiciu, C., Greenhalgh, A., Handley, M., and H. Tokuda, "Is it still possible to extend TCP?", IMC '11: Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference, DOI 10.1145/2068816.2068834, November 2011, <http://doi.acm.org/10.1145/2068816.2068834>. [IMC13a] Detal, G., Hesmans, B., Bonaventure, O., Vanaubel, Y., and B. Donnet, "Revealing Middlebox Interference with Tracebox", Proceedings of the 2013 ACM SIGCOMM conference on Internet measurement conference, DOI 10.1145/2504730.2504757, October 2013, <http://inl.info.ucl.ac.be/publications/ revealing-middlebox-interference-tracebox>. [IMC13b] Chen, Y., Lim, Y., Gibbens, R., Nahum, E., Khalili, R., and D. Towsley, "A measurement-based study of MultiPath TCP performance over wireless network", ICM '13: Proceedings of the 2013 conference on Internet measurement conference, DOI 10.1145/2504730.2504751, October 2013, <http://doi.acm.org/10.1145/2504730.2504751>. [IMC13c] Pelsser, C., Cittadini, L., Vissicchio, S., and R. Bush, "From Paris to Tokyo: on the suitability of ping to measure latency", IMC '13: Proceedings of the 2013 conference on Internet measurement Conference, DOI 10.1145/2504730.2504765, October 2013, <http://doi.acm.org/10.1145/2504730.2504765>. [INFOCOM14] Lim, Y., Chen, Y., Nahum, E., Towsley, D., and K. Lee, "Cross-layer path management in multi-path transport protocol for mobile devices", IEEE INFOCOM'14, DOI 10.1109/INFOCOM.2014.6848120, April 2014. [KT] Seo, S., "KT's GiGA LTE", July 2015, <https://www.ietf.org/proceedings/93/slides/ slides-93-mptcp-3.pdf>. [MBTest] Hesmans, B., "MBTest", October 2013, <https://bitbucket.org/bhesmans/mbtest>.
[Mobicom15] De Coninck, Q., Baerts, M., Hesmans, B., and O. Bonaventure, "Poster - Evaluating Android Applications with Multipath TCP", Mobicom 2015 (Poster), DOI 10.1145/2789168.2795165, September 2015. [MPTCP-DEPLOY] Paasch, C., Biswas, A., and D. Haas, "Making Multipath TCP robust for stateless webservers", Work in Progress, draft-paasch-mptcp-syncookies-02, October 2015. [MPTCP-LOAD] Paasch, C., Greenway, G., and A. Ford, "Multipath TCP behind Layer-4 loadbalancers", Work in Progress, draft-paasch-mptcp-loadbalancer-00, September 2015. [MPTCP-MAX-SUB] Boucadair, M. and C. Jacquenet, "Negotiating the Maximum Number of Multipath TCP (MPTCP) Subflows", Work in Progress draft-boucadair-mptcp-max-subflow-02, May 2016. [MPTCPBIB] Bonaventure, O., "Multipath TCP - Annotated bibliography", Technical report, April 2015, <https://github.com/obonaventure/mptcp-bib>. [MultipathTCP-Linux] Paasch, C., Barre, S., and . et al, "Multipath TCP - Linux Kernel implementation", <http://www.multipath-tcp.org>. [NSDI11] Wischik, D., Raiciu, C., Greenhalgh, A., and M. Handley, "Design, implementation and evaluation of congestion control for multipath TCP", NSDI11: In Proceedings of the 8th USENIX conference on Networked systems design and implementation, 2011. [NSDI12] Raiciu, C., Paasch, C., Barre, S., Ford, A., Honda, M., Duchene, F., Bonaventure, O., and M. Handley, "How Hard Can It Be? Designing and Implementing a Deployable Multipath TCP", NSDI '12: USENIX Symposium of Networked Systems Design and implementation, April 2012, <http://inl.info.ucl.ac.be/publications/how-hard-can-it- be-designing-and-implementing-deployable-multipath-tcp>. [PaaschPhD] Paasch, C., "Improving Multipath TCP", Ph.D. Thesis , November 2014, <http://inl.info.ucl.ac.be/publications/ improving-multipath-tcp>.
[PAM2016] De Coninck, Q., Baerts, M., Hesmans, B., and O. Bonaventure, "A First Analysis of Multipath TCP on Smartphones", 17th International Passive and Active Measurements Conference (PAM2016) volume 17, March 2016, <http://inl.info.ucl.ac.be/publications/ first-analysis-multipath-tcp-smartphones>. [PAMS2014] Arzani, B., Gurney, A., Cheng, S., Guerin, R., and B. Loo, "Impact of Path Selection and Scheduling Policies on MPTCP Performance", PAMS2014, DOI 10.1109/WAINA.2014.121, May 2014. [Presto08] Greenberg, A., Lahiri, P., Maltz, D., Patel, P., and S. Sengupta, "Towards a next generation data center architecture: scalability and commoditization", ACM PRESTO 2008, DOI 10.1145/1397718.1397732, August 2008, <http://dl.acm.org/citation.cfm?id=1397732>. [PT14] Pearce, C. and P. Thomas, "Multipath TCP Breaking Today's Networks with Tomorrow's Protocols", Proc. Blackhat Briefings, 2014, <http://www.blackhat.com/docs/ us-14/materials/us-14-Pearce-Multipath-TCP-Breaking- Todays-Networks-With-Tomorrows-Protocols-WP.pdf>. [PZ15] Pearce, C. and S. Zeadally, "Ancillary Impacts of Multipath TCP on Current and Future Network Security", IEEE Internet Computing, vol. 19, no. 5, pp. 58-65, DOI 10.1109/MIC.2015.70, September 2015. [RFC1812] Baker, F., Ed., "Requirements for IP Version 4 Routers", RFC 1812, DOI 10.17487/RFC1812, June 1995, <http://www.rfc-editor.org/info/rfc1812>. [RFC1928] Leech, M., Ganis, M., Lee, Y., Kuris, R., Koblas, D., and L. Jones, "SOCKS Protocol Version 5", RFC 1928, DOI 10.17487/RFC1928, March 1996, <http://www.rfc-editor.org/info/rfc1928>. [RFC2992] Hopps, C., "Analysis of an Equal-Cost Multi-Path Algorithm", RFC 2992, DOI 10.17487/RFC2992, November 2000, <http://www.rfc-editor.org/info/rfc2992>. [RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common Mitigations", RFC 4987, DOI 10.17487/RFC4987, August 2007, <http://www.rfc-editor.org/info/rfc4987>.
[RFC6356] Raiciu, C., Handley, M., and D. Wischik, "Coupled Congestion Control for Multipath Transport Protocols", RFC 6356, DOI 10.17487/RFC6356, October 2011, <http://www.rfc-editor.org/info/rfc6356>. [RFC7430] Bagnulo, M., Paasch, C., Gont, F., Bonaventure, O., and C. Raiciu, "Analysis of Residual Threats and Possible Fixes for Multipath TCP (MPTCP)", RFC 7430, DOI 10.17487/RFC7430, July 2015, <http://www.rfc-editor.org/info/rfc7430>. [RFC7871] Contavalli, C., van der Gaast, W., Lawrence, D., and W. Kumari, "Client Subnet in DNS Queries", RFC 7871, DOI 10.17487/RFC7871, May 2016, <http://www.rfc-editor.org/info/rfc7871>. [SIGCOMM11] Raiciu, C., Barre, S., Pluntke, C., Greenhalgh, A., Wischik, D., and M. Handley, "Improving datacenter performance and robustness with multipath TCP", SIGCOMM '11: Proceedings of the ACM SIGCOMM 2011 conference, DOI 10.1145/2018436.2018467, August 2011, <http://doi.acm.org/10.1145/2018436.2018467>. [SOCKET] Hesmans, B. and O. Bonaventure, "An enhanced socket API for Multipath TCP", Proceedings of the 2016 Applied Networking Research Workshop, DOI 10.1145/2959424.2959433, July 2016, <http://doi.acm.org/10.1145/2959424.2959433>. [StrangeMbox] Bonaventure, O., "Multipath TCP through a strange middlebox", Blog post, January 2015, <http://blog.multipath-tcp.org/blog/html/2015/01/30/ multipath_tcp_through_a_strange_middlebox.html>. [TMA2015] Hesmans, B., Tran Viet, H., Sadre, R., and O. Bonaventure, "A First Look at Real Multipath TCP Traffic", Traffic Monitoring and Analysis, 2015, <http://inl.info.ucl.ac.be/publications/ first-look-real-multipath-tcp-traffic>. [TR-348] Broadband Forum, ., "TR 348 - Hybrid Access Broadband Network Architecture", Issue: 1, July 2016, <https://www.broadband-forum.org/technical/download/ TR-348.pdf>.
[tracebox] Detal, G. and O. Tilmans, "Tracebox: A Middlebox Detection Tool", 2013, <http://www.tracebox.org>.Acknowledgements
This work was partially supported by the FP7-Trilogy2 project. We would like to thank all the implementers and users of the Multipath TCP implementation in the Linux kernel. This document has benefited from the comments of John Ronan, Yoshifumi Nishida, Phil Eardley, Jaehyun Hwang, Mirja Kuehlewind, Benoit Claise, Jari Arkko, Qin Wu, Spencer Dawkins, and Ben Campbell.Authors' Addresses
Olivier Bonaventure UCLouvain Email: Olivier.Bonaventure@uclouvain.be Christoph Paasch Apple, Inc. Email: cpaasch@apple.com Gregory Detal Tessares Email: Gregory.Detal@tessares.net