5.  Troubleshooting and Diagnostics

   Any service provider supporting multicast delivery of content should
   be able to collect diagnostics as part of multicast troubleshooting
   practices and resolve network issues accordingly.  Issues may become
   apparent or identifiable through either (1) network monitoring
   functions or (2) problems reported by customers, as described in
   Section 4.4.

   It is recommended that multicast diagnostics be performed, leveraging
   established operational practices such as those documented in
   [MDH-05].  However, given that inter-domain multicast creates a
   significant interdependence of proper networking functionality
   between providers, there exists a need for providers to be able to
   signal (or otherwise alert) each other if there are any issues noted
   by either one.

   For troubleshooting purposes, service providers may also wish to
   allow limited read-only administrative access to their routers to
   their AD peers.  Access to active troubleshooting tools -- especially
   [Traceroute] and the tools discussed in [Mtrace-v2] -- is of specific
   interest.

   Another option is to include this functionality in the IP multicast
   receiver application on the EU device and allow these diagnostics to
   be remotely used by support operations.  Note, though, that AMT
   does not allow the passing of traceroute or mtrace requests;
   therefore, troubleshooting in the presence of AMT does not work as
   well end to end as it can with native (or even GRE-encapsulated) IP
   multicast, especially with regard to traceroute and mtrace.  Instead,
   troubleshooting directly on the actual network devices is then more
   likely necessary.

   The specifics of notifications and alerts are beyond the scope of
   this document, but general guidelines are similar to those described
   in Section 4.4.  Some general communications issues are as follows.

   o  Appropriate communications channels will be established between
      the customer service and operations groups from both ADs to
      facilitate information-sharing related to diagnostic
      troubleshooting.

   o  A default resolution period may be considered to resolve open
      issues.  Alternately, mutually acceptable resolution periods could
      be established, depending on the severity of the identified
      trouble.

6.  Security Considerations

6.1.  DoS Attacks (against State and Bandwidth)

   Reliable IP multicast operations require some basic protection
   against DoS (Denial of Service) attacks.

   SSM IP multicast is self-protecting against attacks from illicit
   sources; such traffic will not be forwarded beyond the first-hop
   router, because that would require (S,G) membership reports from the
   receiver.  Only valid traffic from sources will be forwarded, because
   RPF ("Reverse Path Forwarding") is part of the protocols.  One can
   say that protection against spoofed source traffic performed in the
   style of [BCP38] is therefore built into PIM-SM / PIM-SSM.

   Receivers can attack SSM IP multicast by originating such (S,G)
   membership reports.  This can result in a DoS attack against state
   through the creation of a large number of (S,G) states that create
   high control-plane load or even inhibit the later creation of a valid
   (S,G).  In conjunction with collaborating illicit sources, it can
   also result in the forwarding of traffic from illicit sources.

   Today, these types of attacks are usually mitigated by explicitly
   defining the set of permissible (S,G) on, for example, the last-hop
   routers in replicating IP multicast to EUs (e.g., via (S,G) access
   control lists applied to IGMP/MLD membership state creation).  Each
   AD (say, "ADi") is expected to know what sources located in ADi are
   permitted to send and what their valid (S,G)s are.  ADi can therefore
   also filter invalid (S,G)s for any "S" located inside ADi, but not
   sources located in another AD.

   In the peering case, without further information, AD-2 is not aware
   of the set of valid (S,G) from AD-1, so this set needs to be
   communicated via operational procedures from AD-1 to AD-2 to provide
   protection against this type of DoS attack.  Future work could signal
   this information in an automated way: BGP extensions, DNS resource
   records, or backend automation between AD-1 and AD-2.  Backend
   automation is, in the short term, the most viable solution: unlike
   BGP extensions or DNS resource records, backend automation does not
   require router software extensions.  Observation of traffic flowing
   via (S,G) state could also be used to automate the recognition of
   invalid (S,G) state created by receivers in the absence of explicit
   information from AD-1.

   The second type of DoS attack through (S,G) membership reports exists
   when the attacking receiver creates too much valid (S,G) state and
   the traffic carried by these (S,G)s congests bandwidth on links
   shared with other EUs.  Consider the uplink to a last-hop router
   connecting to 100 EUs.  If one EU joins to more multicast content
   than what fits into this link, then this would also impact the
   quality of the same content for the other 99 EUs.  If traffic is not
   rate adaptive, the effects are even worse.

   The mitigation technique is the same as what is often employed for
   unicast: policing of the per-EU total amount of traffic.  Unlike
   unicast, though, this cannot be done anywhere along the path (e.g.,
   on an arbitrary bottleneck link); it has to happen at the point of
   last replication to the different EU.  Simple solutions such as
   limiting the maximum number of joined (S,G)s per EU are readily
   available; solutions that take consumed bandwidth into account are
   available as vendor-specific features in routers.  Note that this is
   primarily a non-peering issue in AD-2; it only becomes a peering
   issue if the peering link itself is not big enough to carry all
   possible content from AD-1 or, as in Use Case 3.4, when the AMT relay
   in AD-1 is that last replication point.

   Limiting the amount of (S,G) state per EU is also a good first
   measure to prohibit too much undesired "empty" state from being built
   (state not carrying traffic), but it would not suffice in the case of
   DDoS attacks, e.g., viruses that impact a large number of EU devices.

6.2.  Content Security

   Content confidentiality, DRM (Digital Rights Management),
   authentication, and authorization are optional, based on the content
   delivered.  For content that is "FTA" (Free To Air), the following
   considerations can be ignored, and content can be sent unencrypted
   and without EU authentication and authorization.  Note, though, that
   the mechanisms described here may also be desirable for the
   application source to better track users even if the content itself
   would not require it.

   For inter-domain content, there are at least two models for content
   confidentiality, including (1) DRM authentication and authorization
   and (2) EU authentication and authorization:

   o  In the classical (IP)TV model, responsibility is per domain, and
      content is and can be passed on unencrypted.  AD-1 delivers
      content to AD-2; AD-2 can further process the content, including
      features like ad insertion, and AD-2 is the sole point of contact
      regarding the contact for its EUs.  In this document, we do not
      consider this case because it typically involves service aspects
      operated by AD-2 that are higher than the network layer; this
      document focuses on the network-layer AD-1/AD-2 peering case but
      not the application-layer peering case.  Nevertheless, this model
      can be derived through additional work beyond what is described
      here.

   o  The other model is the one in which content confidentiality, DRM,
      EU authentication, and EU authorization are end to end:
      responsibilities of the multicast application source provider and
      receiver application.  This is the model assumed here.  It is also
      the model used in Internet "Over the Top" (OTT) video delivery.
      Below, we discuss the threats incurred in this model due to the
      use of IP multicast in AD-1 or AD-2 and across the peering point.

   End-to-end encryption enables end-to-end EU authentication and
   authorization: the EU may be able to join (via IGMP/MLD) and receive
   the content, but it can only decrypt it when it receives the
   decryption key from the content source in AD-1.  The key is the
   authorization.  Keeping that key to itself and prohibiting playout of
   the decrypted content to non-copy-protected interfaces are typical
   DRM features in that receiver application or EU device operating
   system.

   End-to-end encryption is continuously attacked.  Keys may be subject
   to brute-force attacks so that content can potentially be decrypted
   later, or keys are extracted from the EU application/device and
   shared with other unauthenticated receivers.  One important class of

   content is where the value is in live consumption, such as sports or
   other event (e.g., concert) streaming.  Extraction of keying material
   from compromised authenticated EUs and sharing with unauthenticated
   EUs are not sufficient.  It is also necessary for those
   unauthenticated EUs to get a streaming copy of the content itself.
   In unicast streaming, they cannot get such a copy from the content
   source (because they cannot authenticate), and, because of asymmetric
   bandwidths, it is often impossible to get the content from
   compromised EUs to a large number of unauthenticated EUs.  EUs behind
   classical "16 Mbps down, 1 Mbps up" ADSL links are the best example.
   With increasing broadband access speeds, unicast peer-to-peer copying
   of content becomes easier, but it likely will always be easily
   detectable by the ADs because of its traffic patterns and volume.

   When IP multicast is being used without additional security, AD-2 is
   not aware of which EU is authenticated for which content.  Any
   unauthenticated EU in AD-2 could therefore get a copy of the
   encrypted content without triggering suspicion on the part of AD-2 or
   AD-1 and then either (1) live-decode it, in the presence of the
   compromised authenticated EU and key-sharing or (2) decrypt it later,
   in the presence of federated brute-force key-cracking.

   To mitigate this issue, the last replication point that is creating
   (S,G) copies to EUs would need to permit those copies only after
   authentication of the EUs.  This would establish the same
   authenticated "EU only" copy that is used in unicast.

   Schemes for per-EU IP multicast authentication/authorization (and, as
   a result, non-delivery or copying of per-content IP multicast
   traffic) have been built in the past and are deployed in service
   providers for intra-domain IPTV services, but no standards exist for
   this.  For example, there is no standardized RADIUS attribute for
   authenticating the IGMP/MLD filter set, but such implementations
   exist.  The authors of this document are specifically also not aware
   of schemes where the same authentication credentials used to get the
   encryption key from the content source could also be used to
   authenticate and authorize the network-layer IP multicast replication
   for the content.  Such schemes are technically not difficult to build
   and would avoid creating and maintaining a separate network
   traffic-forwarding authentication/authorization scheme decoupled from
   the end-to-end authentication/authorization system of the
   application.

   If delivery of such high-value content in conjunction with the
   peering described here is desired, the short-term recommendations are
   for sources to clearly isolate the source and group addresses used
   for different content bundles, communicate those (S,G) patterns from
   AD-1 to AD-2, and let AD-2 leverage existing per-EU authentication/
   authorization mechanisms in network devices to establish filters for
   (S,G) sets to each EU.

6.3.  Peering Encryption

   Encryption at peering points for multicast delivery may be used per
   agreement between AD-1 and AD-2.

   In the case of a private peering link, IP multicast does not have
   attack vectors on a peering link different from those of IP unicast,
   but the content owner may have defined strict constraints against
   unauthenticated copying of even the end-to-end encrypted content; in
   this case, AD-1 and AD-2 can agree on additional transport encryption
   across that peering link.  In the case of a broadcast peering
   connection (e.g., IXP), transport encryption is again the easiest way
   to prohibit unauthenticated copies by other ADs on the same peering
   point.

   If peering is across a tunnel that spans intermittent transit ADs
   (not discussed in detail in this document), then encryption of that
   tunnel traffic is recommended.  It not only prohibits possible
   "leakage" of content but also protects the information regarding what
   content is being consumed in AD-2 (aggregated privacy protection).

   See Section 6.4 for reasons why the peering point may also need to be
   encrypted for operational reasons.

6.4.  Operational Aspects

   Section 4.3.3 discusses the exchange of log information, and
   Section 7 discusses the exchange of program information.  All these
   operational pieces of data should by default be exchanged via
   authenticated and encrypted peer-to-peer communication protocols
   between AD-1 and AD-2 so that only the intended recipients in the
   peers' AD have access to it.  Even exposure of the least sensitive
   information to third parties opens up attack vectors.  Putting valid
   (S,G) information, for example, into DNS (as opposed to passing it
   via secured channels from AD-1 to AD-2) to allow easier filtering of
   invalid (S,G) information would also allow attackers to more easily
   identify valid (S,G) information and change their attack vector.

   From the perspective of the ADs, security is most critical for log
   information, as it provides operational insight into the originating
   AD but also contains sensitive user data.

   Sensitive user data exported from AD-2 to AD-1 as part of logs could
   be as much as the equivalent of 5-tuple unicast traffic flow
   accounting (but not more, e.g., no application-level information).
   As mentioned in Section 7, in unicast, AD-1 could capture these
   traffic statistics itself because this is all about traffic flows
   (originated by AD-1) to EU receivers in AD-2, and operationally
   passing it from AD-2 to AD-1 may be necessary when IP multicast is
   used because of the replication taking place in AD-2.

   Nevertheless, passing such traffic statistics inside AD-1 from a
   capturing router to a backend system is likely less subject to
   third-party attacks than passing it "inter-domain" from AD-2 to AD-1,
   so more diligence needs to be applied to secure it.

   If any protocols used for the operational exchange of information are
   not easily secured at the transport layer or higher (because of the
   use of legacy products or protocols in the network), then AD-1 and
   AD-2 can also consider ensuring that all operational data exchanges
   go across the same peering point as the traffic and use network-layer
   encryption of the peering point (as discussed previously) to
   protect it.

   End-to-end authentication and authorization of EUs may involve some
   kind of token authentication and are done at the application layer,
   independently of the two ADs.  If there are problems related to the
   failure of token authentication when EUs are supported by AD-2, then
   some means of validating proper operation of the token authentication
   process (e.g., validating that backend servers querying the multicast
   application source provider's token authentication server are
   communicating properly) should be considered.  Implementation details
   are beyond the scope of this document.

   In the event of a security breach, the two ADs are expected to have a
   mitigation plan for shutting down the peering point and directing
   multicast traffic over alternative peering points.  It is also
   expected that appropriate information will be shared for the purpose
   of securing the identified breach.

7.  Privacy Considerations

   The described flow of information about content and EUs as described
   in this document aims to maintain privacy:

   AD-1 is operating on behalf of (or owns) the content source and is
   therefore part of the content-consumption relationship with the EU.
   The privacy considerations between the EU and AD-1 are therefore
   generally the same (with one exception; see below) as they would be
   if no IP multicast was used, especially because end-to-end encryption
   can and should be used for any privacy-conscious content.

   Information related to inter-domain multicast transport service is
   provided to AD-1 by the AD-2 operators.  AD-2 is not required to gain
   additional insight into the user's behavior through this process
   other than what it would already have without service collaboration
   with AD-1, unless AD-1 and AD-2 agree on it and get approval from
   the EU.

   For example, if it is deemed beneficial for the EU to get support
   directly from AD-2, then it would generally be necessary for AD-2 to
   be aware of the mapping between content and network (S,G) state so
   that AD-2 knows which (S,G) to troubleshoot when the EU complains
   about problems with specific content.  The degree to which this
   dissemination is done by AD-1 explicitly to meet privacy expectations
   of EUs is typically easy to assess by AD-1.  Two simple examples are
   as follows:

   o  For a sports content bundle, every EU will happily click on the
      "I approve that the content program information is shared with
      your service provider" button, to ensure best service reliability,
      because service-conscious AD-2 would likely also try to ensure
      that high-value content, such as the (S,G) for the Super Bowl,
      would be the first to receive care in the case of network issues.

   o  If the content in question was content for which the EU expected
      more privacy, the EU should prefer a content bundle that included
      this content in a large variety of other content, have all content
      end-to-end encrypted, and not share programming information with
      AD-2, to maximize privacy.  Nevertheless, the privacy of the EU
      against AD-2 observing traffic would still be lower than in the
      equivalent setup using unicast, because in unicast, AD-2 could not
      correlate which EUs are watching the same content and use that to
      deduce the content.  Note that even the setup in Section 3.4,
      where AD-2 is not involved in IP multicast at all, does not
      provide privacy against this level of analysis by AD-2, because
      there is no transport-layer encryption in AMT; therefore, AD-2 can
      correlate by on-path traffic analysis who is consuming the same

      content from an AMT relay from both the (S,G) join messages in AMT
      and the identical content segments (that were replicated at the
      AMT relay).

   In summary, because only content to be consumed by multiple EUs is
   carried via IP multicast here and all of that content can be
   end-to-end encrypted, the only privacy consideration specific to IP
   multicast is for AD-2 to know or reconstruct what content an EU is
   consuming.  For content for which this is undesirable, some form of
   protections as explained above are possible, but ideally, the model
   described in Section 3.4 could be used in conjunction with future
   work, e.g., adding Datagram Transport Layer Security (DTLS)
   encryption [RFC6347] between the AMT relay and the EU.

   Note that IP multicast by nature would permit the EU's privacy
   against the content source operator because, unlike unicast, the
   content source does not natively know which EU is consuming which
   content: in all cases where AD-2 provides replication, only AD-2
   knows this directly.  This document does not attempt to describe a
   model that maintains such a level of privacy against the content
   source; rather, we describe a model that only protects against
   exposure to intermediate parties -- in this case, AD-2.

8.  IANA Considerations

   This document does not require any IANA actions.

9.  References

9.1.  Normative References

   [RFC2784]  Farinacci, D., Li, T., Hanks, S., Meyer, D., and P.
              Traina, "Generic Routing Encapsulation (GRE)", RFC 2784,
              DOI 10.17487/RFC2784, March 2000,
              <https://www.rfc-editor.org/info/rfc2784>.

   [RFC3376]  Cain, B., Deering, S., Kouvelas, I., Fenner, B., and A.
              Thyagarajan, "Internet Group Management Protocol,
              Version 3", RFC 3376, DOI 10.17487/RFC3376, October 2002,
              <https://www.rfc-editor.org/info/rfc3376>.

   [RFC3810]  Vida, R., Ed., and L. Costa, Ed., "Multicast Listener
              Discovery Version 2 (MLDv2) for IPv6", RFC 3810,
              DOI 10.17487/RFC3810, June 2004,
              <https://www.rfc-editor.org/info/rfc3810>.

   [RFC4760]  Bates, T., Chandra, R., Katz, D., and Y. Rekhter,
              "Multiprotocol Extensions for BGP-4", RFC 4760,
              DOI 10.17487/RFC4760, January 2007,
              <https://www.rfc-editor.org/info/rfc4760>.

   [RFC4604]  Holbrook, H., Cain, B., and B. Haberman, "Using Internet
              Group Management Protocol Version 3 (IGMPv3) and Multicast
              Listener Discovery Protocol Version 2 (MLDv2) for
              Source-Specific Multicast", RFC 4604,
              DOI 10.17487/RFC4604, August 2006,
              <https://www.rfc-editor.org/info/rfc4604>.

   [RFC4609]  Savola, P., Lehtonen, R., and D. Meyer, "Protocol
              Independent Multicast - Sparse Mode (PIM-SM) Multicast
              Routing Security Issues and Enhancements", RFC 4609,
              DOI 10.17487/RFC4609, October 2006,
              <https://www.rfc-editor.org/info/rfc4609>.

   [RFC7450]  Bumgardner, G., "Automatic Multicast Tunneling", RFC 7450,
              DOI 10.17487/RFC7450, February 2015,
              <https://www.rfc-editor.org/info/rfc7450>.

   [RFC7761]  Fenner, B., Handley, M., Holbrook, H., Kouvelas, I.,
              Parekh, R., Zhang, Z., and L. Zheng, "Protocol Independent
              Multicast - Sparse Mode (PIM-SM): Protocol Specification
              (Revised)", STD 83, RFC 7761, DOI 10.17487/RFC7761,
              March 2016, <https://www.rfc-editor.org/info/rfc7761>.

   [BCP38]    Ferguson, P. and D. Senie, "Network Ingress Filtering:
              Defeating Denial of Service Attacks which employ IP Source
              Address Spoofing", BCP 38, RFC 2827, May 2000,
              <https://www.rfc-editor.org/info/rfc2827>.

   [BCP41]    Floyd, S., "Congestion Control Principles", BCP 41,
              RFC 2914, September 2000,
              <https://www.rfc-editor.org/info/rfc2914>.

   [BCP145]   Eggert, L., Fairhurst, G., and G. Shepherd, "UDP Usage
              Guidelines", BCP 145, RFC 8085, March 2017,
              <https://www.rfc-editor.org/info/rfc8085>.

9.2.  Informative References

   [RFC4786]  Abley, J. and K. Lindqvist, "Operation of Anycast
              Services", BCP 126, RFC 4786, DOI 10.17487/RFC4786,
              December 2006, <https://www.rfc-editor.org/info/rfc4786>.

   [RFC6347]  Rescorla, E. and N. Modadugu, "Datagram Transport Layer
              Security Version 1.2", RFC 6347, DOI 10.17487/RFC6347,
              January 2012, <https://www.rfc-editor.org/info/rfc6347>.

   [INF_ATIS_10]
              "CDN Interconnection Use Cases and Requirements in a
              Multi-Party Federation Environment", ATIS Standard
              A-0200010, December 2012.

   [MDH-05]   Thaler, D. and B. Aboba, "Multicast Debugging Handbook",
              Work in Progress, draft-ietf-mboned-mdh-05, November 2000.

   [Traceroute]
              "traceroute.org", <http://traceroute.org/#source%20code>.

   [Mtrace-v2]
              Asaeda, H., Meyer, K., and W. Lee, Ed., "Mtrace Version 2:
              Traceroute Facility for IP Multicast", Work in Progress,
              draft-ietf-mboned-mtrace-v2-22, December 2017.

Acknowledgments

   The authors would like to thank the following individuals for their
   suggestions, comments, and corrections:

      Mikael Abrahamsson

      Hitoshi Asaeda

      Dale Carder

      Tim Chown

      Leonard Giuliano

      Jake Holland

      Joel Jaeggli

      Henrik Levkowetz

      Albert Manfredi

      Stig Venaas

Authors' Addresses

   Percy S. Tarapore (editor)
   AT&T

   Phone: 1-732-420-4172
   Email: tarapore@att.com


   Robert Sayko
   AT&T

   Phone: 1-732-420-3292
   Email: rs1983@att.com


   Greg Shepherd
   Cisco

   Email: shep@cisco.com


   Toerless Eckert (editor)
   Huawei USA - Futurewei Technologies Inc.

   Email: tte+ietf@cs.fau.de, toerless.eckert@huawei.com


   Ram Krishnan
   SupportVectors

   Email: ramkri123@gmail.com