RFC 6726
FLUTE - File Delivery over Unidirectional Transport
Internet Engineering Task Force (IETF) T. Paila Request for Comments: 6726 Nokia Obsoletes: 3926 R. Walsh Category: Standards Track Nokia/TUT ISSN: 2070-1721 M. Luby Qualcomm Technologies, Inc. V. Roca INRIA R. Lehtonen TeliaSonera November 2012 FLUTE - File Delivery over Unidirectional TransportAbstract
This document defines File Delivery over Unidirectional Transport (FLUTE), a protocol for the unidirectional delivery of files over the Internet, which is particularly suited to multicast networks. The specification builds on Asynchronous Layered Coding, the base protocol designed for massively scalable multicast distribution. This document obsoletes RFC 3926. Status of This Memo This is an Internet Standards Track document. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 5741. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc6726.
Copyright Notice Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English.Table of Contents
1. Introduction ....................................................3 1.1. Applicability Statement ....................................5 1.1.1. The Target Application Space ........................5 1.1.2. The Target Scale ....................................5 1.1.3. Intended Environments ...............................5 1.1.4. Weaknesses ..........................................6 2. Conventions Used in This Document ...............................6 3. File Delivery ...................................................7 3.1. File Delivery Session ......................................8 3.2. File Delivery Table .......................................10 3.3. Dynamics of FDT Instances within a File Delivery Session ..12 3.4. Structure of FDT Instance Packets .........................15 3.4.1. Format of FDT Instance Header ......................16 3.4.2. Syntax of FDT Instance .............................17 3.4.3. Content Encoding of FDT Instance ...................21 3.5. Multiplexing of Files within a File Delivery Session ......22 4. Channels, Congestion Control, and Timing .......................23 5. Delivering FEC Object Transmission Information .................24 6. Describing File Delivery Sessions ..............................26
7. Security Considerations ........................................27 7.1. Problem Statement .........................................27 7.2. Attacks against the Data Flow .............................28 7.2.1. Access to Confidential Files .......................28 7.2.2. File Corruption ....................................28 7.3. Attacks against the Session Control Parameters and Associated Building Blocks ................................30 7.3.1. Attacks against the Session Description ............30 7.3.2. Attacks against the FDT Instances ..................31 7.3.3. Attacks against the ALC/LCT Parameters .............31 7.3.4. Attacks against the Associated Building Blocks .....32 7.4. Other Security Considerations .............................32 7.5. Minimum Security Recommendations ..........................33 8. IANA Considerations ............................................34 8.1. Registration of the FDT Instance XML Namespace ............34 8.2. Registration of the FDT Instance XML Schema ...............34 8.3. Registration of the application/fdt+xml Media Type ........35 8.4. Creation of the FLUTE Content Encoding Algorithms Registry ..................................................36 8.5. Registration of LCT Header Extension Types ................36 9. Acknowledgments ................................................36 10. Contributors ..................................................37 11. Change Log ....................................................37 11.1. RFC 3926 to This Document ................................37 12. References ....................................................40 12.1. Normative References .....................................40 12.2. Informative References ...................................41 Appendix A. Receiver Operation (Informative) ......................44 Appendix B. Example of FDT Instance (Informative) .................451. Introduction
This document defines FLUTE version 2, a protocol for unidirectional delivery of files over the Internet. This specification is not backwards compatible with the previous experimental version defined in [RFC3926] (see Section 11 for details). The specification builds on Asynchronous Layered Coding (ALC), version 1 [RFC5775], the base protocol designed for massively scalable multicast distribution. ALC defines transport of arbitrary binary objects. For file delivery applications, mere transport of objects is not enough, however. The end systems need to know what the objects actually represent. This document specifies a technique called FLUTE -- a mechanism for signaling and mapping the properties of files to concepts of ALC in a way that allows receivers to assign those parameters for received objects. Consequently, throughout this document the term 'file' relates to an 'object' as discussed in ALC. Although this
specification frequently makes use of multicast addressing as an
example, the techniques are similarly applicable for use with unicast
addressing.
This document defines a specific transport application of ALC, adding
the following specifications:
- Definition of a file delivery session built on top of ALC,
including transport details and timing constraints.
- In-band signaling of the transport parameters of the ALC session.
- In-band signaling of the properties of delivered files.
- Details associated with the multiplexing of multiple files within
a session.
This specification is structured as follows. Section 3 begins by
defining the concept of the file delivery session. Following that,
it introduces the File Delivery Table, which forms the core part of
this specification. Further, it discusses multiplexing issues of
transmission objects within a file delivery session. Section 4
describes the use of congestion control and channels with FLUTE.
Section 5 defines how the Forward Error Correction (FEC) Object
Transmission Information is to be delivered within a file delivery
session. Section 6 defines the required parameters for describing
file delivery sessions in a general case. Section 7 outlines
security considerations regarding file delivery with FLUTE. Last,
there are two informative appendices. Appendix A describes an
envisioned receiver operation for the receiver of the file delivery
session. Readers who want to see a simple example of FLUTE in
operation should refer to Appendix A right away. Appendix B gives an
example of a File Delivery Table.
This specification contains part of the definitions necessary to
fully specify a Reliable Multicast Transport (RMT) protocol in
accordance with [RFC2357].
This document obsoletes [RFC3926], which contained a previous version
of this specification and was published in the "Experimental"
category. This Proposed Standard specification is thus based on
[RFC3926] and has been updated according to accumulated experience
and growing protocol maturity since the publication of [RFC3926].
Said experience applies both to this specification itself and to
congestion control strategies related to the use of this
specification.
The differences between [RFC3926] and this document are listed in Section 11. This document updates ALC [RFC5775] and Layered Coding Transport (LCT) [RFC5651] in the sense that it defines two new header extensions, EXT_FDT and EXT_CENC.1.1. Applicability Statement
1.1.1. The Target Application Space
FLUTE is applicable to the delivery of large and small files to many hosts, using delivery sessions of several seconds or more. For instance, FLUTE could be used for the delivery of large software updates to many hosts simultaneously. It could also be used for continuous, but segmented, data such as time-lined text for subtitling -- potentially leveraging its layering inheritance from ALC and LCT to scale the richness of the session to the congestion status of the network. It is also suitable for the basic transport of metadata, for example, Session Description Protocol (SDP) [RFC4566] files that enable user applications to access multimedia sessions.1.1.2. The Target Scale
Massive scalability is a primary design goal for FLUTE. IP multicast is inherently massively scalable, but the best-effort service that it provides does not provide session management functionality, congestion control, or reliability. FLUTE provides all of this by using ALC and IP multicast without sacrificing any of the inherent scalability of IP multicast.1.1.3. Intended Environments
All of the environmental requirements and considerations that apply to the RMT building blocks used by FLUTE shall also apply to FLUTE. These are the ALC protocol instantiation [RFC5775], the LCT building block [RFC5651], and the FEC building block [RFC5052]. FLUTE can be used with both multicast and unicast delivery, but its primary application is for unidirectional multicast file delivery. FLUTE requires connectivity between a sender and receivers but does not require connectivity from receivers to a sender. Because of its low expectations, FLUTE works with most types of networks, including LANs, WANs, Intranets, the Internet, asymmetric networks, wireless networks, and satellite networks.
FLUTE is compatible with both IPv4 and IPv6, as no part of the packet is IP version specific. FLUTE works with both multicast models: Any-Source Multicast (ASM) [RFC1112] and Source-Specific Multicast (SSM) [PAPER.SSM]. FLUTE is applicable for both shared networks, such as the Internet, with a suitable congestion control building block; and provisioned/ controlled networks, such as wireless broadcast radio systems, with a traffic-shaping building block.1.1.4. Weaknesses
FLUTE congestion control protocols depend on the ability of a receiver to change multicast subscriptions between multicast groups supporting different rates and/or layered codings. If the network does not support this, then the FLUTE congestion control protocols may not be amenable to such a network. FLUTE can also be used for point-to-point (unicast) communications. At a minimum, implementations of ALC MUST support the Wave and Equation Based Rate Control (WEBRC) [RFC3738] multiple-rate congestion control scheme [RFC5775]. However, since WEBRC has been designed for massively scalable multicast flows, it is not clear how appropriate it is to the particular case of unicast flows. Using a separate point-to-point congestion control scheme is another alternative. How to do that is outside the scope of the present document. FLUTE provides reliability using the FEC building block. This will reduce the error rate as seen by applications. However, FLUTE does not provide a method for senders to verify the reception success of receivers, and the specification of such a method is outside the scope of this document.2. Conventions Used in This Document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. The terms "object" and "transmission object" are consistent with the definitions in ALC [RFC5775] and LCT [RFC5651]. The terms "file" and "source object" are pseudonyms for "object".
3. File Delivery
Asynchronous Layered Coding [RFC5775] is a protocol designed for delivery of arbitrary binary objects. It is especially suitable for massively scalable, unidirectional multicast distribution. ALC provides the basic transport for FLUTE, and thus FLUTE inherits the requirements of ALC. This specification is designed for the delivery of files. The core of this specification is to define how the properties of the files are carried in-band together with the delivered files. As an example, let us consider a 5200-byte file referred to by "http://www.example.com/docs/file.txt". Using the example, the following properties describe the properties that need to be conveyed by the file delivery protocol. * Identifier of the file, expressed as a URI [RFC3986]. The identifier MAY provide a location for the file. In the above example: "http://www.example.com/docs/file.txt". * File name (usually, this can be concluded from the URI). In the above example: "file.txt". * File type, expressed as Internet Media Types (often referred to as "Media Types"). In the above example: "text/plain". * File size, expressed in octets. In the above example: "5200". If the file is content encoded, then this is the file size before content encoding. * Content encoding of the file, within transport. In the above example, the file could be encoded using ZLIB [RFC1950]. In this case, the size of the transmission object carrying the file would probably differ from the file size. The transmission object size is delivered to receivers as part of the FLUTE protocol. * Security properties of the file, such as digital signatures, message digests, etc. For example, one could use S/MIME [RFC5751] as the content encoding type for files with this authentication wrapper, and one could use XML Digital Signatures (XML-DSIG) [RFC3275] to digitally sign the file. XML-DSIG can also be used to provide tamper prevention, e.g., in the Content-Location field. Content encoding is applied to file data before FEC protection.
For each unique file, FLUTE encodes the attributes listed above and other attributes as children of an XML file element. A table of XML file elements is transmitted as a special file called a 'File Delivery Table' (FDT), which is further described in the next subsection and in Section 3.2.3.1. File Delivery Session
ALC is a protocol instantiation of the Layered Coding Transport (LCT) building block [RFC5651]. Thus, ALC inherits the session concept of LCT. In this document, we will use the concept of the ALC/LCT session to collectively denote the interchangeable terms "ALC session" and "LCT session". An ALC/LCT session consists of a set of logically grouped ALC/LCT channels associated with a single sender sending ALC/LCT packets for one or more objects. An ALC/LCT channel is defined by the combination of a sender and an address associated with the channel by the sender. A receiver joins a channel to start receiving the data packets sent to the channel by the sender, and a receiver leaves a channel to stop receiving data packets from the channel. One of the fields carried in the ALC/LCT header is the Transport Session Identifier (TSI), an integer carried in a field of size 16, 32, or 48 bits (note that the TSI may be carried by other means, in which case it is absent from the LCT header [RFC5651]). The (source IP address, TSI) pair uniquely identifies a session. Note that the TSI is scoped by the IP address, so the same TSI may be used by several source IP addresses at once. Thus, the receiver uses the (source IP address, TSI) pair from each packet to uniquely identify the session sending each packet. When a session carries multiple objects, the Transmission Object Identifier (TOI) field within the ALC/LCT header names the object used to generate each packet. Note that each object is associated with a unique TOI within the scope of a session. A FLUTE session consistent with this specification MUST use FLUTE version 2 as specified in this document. Thus, all sessions consistent with this specification MUST set the FLUTE version to 2. The FLUTE version is carried within the EXT_FDT Header Extension (defined in Section 3.4.1) in the ALC/LCT layer. A FLUTE session consistent with this specification MUST use ALC version 1 as specified in [RFC5775], and LCT version 1 as specified in [RFC5651]. If multiple FLUTE sessions are sent to a channel, then receivers MUST determine the FLUTE protocol version, based on version fields and the (source IP address, TSI) pair carried in the ALC/LCT header of the packet. Note that when a receiver first begins receiving packets, it
might not know the FLUTE protocol version, as not every LCT packet carries the EXT_FDT header (containing the FLUTE protocol version). A new receiver MAY keep an open binding in the LCT protocol layer between the TSI and the FLUTE protocol version, until the EXT_FDT header arrives. Alternatively, a new receiver MAY discover a binding between TSI and FLUTE protocol version via a session discovery protocol that is out of scope of this document. If the sender's IP address is not accessible to receivers, then packets that can be received by receivers contain an intermediate IP address. In this case, the TSI is scoped by this intermediate IP address of the sender for the duration of the session. As an example, the sender may be behind a Network Address Translation (NAT) device that temporarily assigns an IP address for the sender. In this case, the TSI is scoped by the intermediate IP address assigned by the NAT. As another example, the sender may send its original packets using IPv6, but some portions of the network may not be IPv6 capable. Thus, there may be an IPv6-to-IPv4 translator that changes the IP address of the packets to a different IPv4 address. In this case, receivers in the IPv4 portion of the network will receive packets containing the IPv4 address, and thus the TSI for them is scoped by the IPv4 address. How the IP address of the sender to be used to scope the session by receivers is delivered to receivers, whether it is the sender's IP address or an intermediate IP address, is outside the scope of this document. When FLUTE is used for file delivery over ALC, the ALC/LCT session is called a file delivery session, and the ALC/LCT concept of 'object' denotes either a 'file' or a 'File Delivery Table Instance' (Section 3.2). Additionally, the following rules apply: * The TOI field MUST be included in ALC packets sent within a FLUTE session, with the exception that ALC packets sent in a FLUTE session with the Close Session (A) flag set to 1 (signaling the end of the session) and that contain no payload (carrying no information for any file or FDT) SHALL NOT carry the TOI. See Section 5.1 of [RFC5651] for the LCT definition of the Close Session flag, and see Section 4.2 of [RFC5775] for an example of the use of a TOI within an ALC packet. * The TOI value '0' is reserved for the delivery of File Delivery Table Instances. Each non-expired File Delivery Table Instance is uniquely identified by an FDT Instance ID within the EXT_FDT header defined in Section 3.4.1.
* Each file in a file delivery session MUST be associated with a TOI
(>0) in the scope of that session.
* Information carried in the headers and the payload of a packet is
scoped by the source IP address and the TSI. Information
particular to the object carried in the headers and the payload of
a packet is further scoped by the TOI for file objects, and is
further scoped by both the TOI and the FDT Instance ID for FDT
Instance objects.
3.2. File Delivery Table
The File Delivery Table (FDT) provides a means to describe various
attributes associated with files that are to be delivered within the
file delivery session. The following lists are examples of such
attributes and are not intended to be mutually exclusive or
exhaustive.
Attributes related to the delivery of a file:
- TOI value that represents the file
- FEC Object Transmission Information (including the FEC Encoding ID
and, if relevant, the FEC Instance ID)
- Size of the transmission object carrying the file
- Aggregate rate of sending packets to all channels
Attributes related to the file itself:
- Name, Identification, and Location of file (specified by the URI)
- Media type of file
- Size of file
- Encoding of file
- Message digest of file
Some of these attributes MUST be included in the file description
entry for a file; others are optional, as defined in Section 3.4.2.
Logically, the FDT is a set of file description entries for files to
be delivered in the session. Each file description entry MUST
include the TOI for the file that it describes and the URI
identifying the file. The TOI carried in each file description entry
is how FLUTE names the ALC/LCT data packets used for delivery of the file. Each file description entry may also contain one or more descriptors that map the above-mentioned attributes to the file. Each file delivery session MUST have an FDT that is local to the given session. The FDT MUST provide a file description entry mapped to a TOI for each file appearing within the session. An object that is delivered within the ALC session, but not described in the FDT, other than the FDT itself, is not considered a 'file' belonging to the file delivery session. This object received with an unmapped TOI (non-zero TOI that is not resolved by the FDT) SHOULD in general be ignored by a FLUTE receiver. The details of how to do that are out of scope of this specification. Note that a client that joins an active file delivery session MAY receive data packets for a TOI > 0 before receiving any FDT Instance (see Section 3.3 for recommendations on how to limit the probability that this situation will occur). Even if the TOI is not mapped to any file description entry, this is hopefully a transient situation. When this happens, system performance might be improved by caching such packets within a reasonable time window and storage size. Such optimizations are use-case and implementation specific, and further details are beyond the scope of this document. Within the file delivery session, the FDT is delivered as FDT Instances. An FDT Instance contains one or more file description entries of the FDT. Any FDT Instance can be equal to, be a subset of, be a superset of, overlap with, or complement any other FDT Instance. A certain FDT Instance may be repeated multiple times during a session, even after subsequent FDT Instances (with higher FDT Instance ID numbers) have been transmitted. Each FDT Instance contains at least a single file description entry and at most the exhaustive set of file description entries of the files being delivered in the file delivery session. A receiver of the file delivery session keeps an FDT database for received file description entries. The receiver maintains the database, for example, upon reception of FDT Instances. Thus, at any given time the contents of the FDT database represent the receiver's current view of the FDT of the file delivery session. Since each receiver behaves independently of other receivers, it SHOULD NOT be assumed that the contents of the FDT database are the same for all the receivers of a given file delivery session. Since the FDT database is an abstract concept, the structure and the maintenance of the FDT database are left to individual implementations and are thus out of scope of this specification.
3.3. Dynamics of FDT Instances within a File Delivery Session
The following rules define the dynamics of the FDT Instances within a file delivery session: * For every file delivered within a file delivery session, there MUST be a file description entry included in at least one FDT Instance sent within the session. A file description entry contains at a minimum the mapping between the TOI and the URI. * An FDT Instance MAY appear in any part of the file delivery session, and packets for an FDT Instance MAY be interleaved with packets for other files or other FDT Instances within a session. * The TOI value of '0' MUST be reserved for delivery of FDT Instances. The use of other TOI values (i.e., an integer > 0) for FDT Instances is outside the scope of this specification. * The FDT Instance is identified by the use of a new fixed-length LCT Header Extension, EXT_FDT (defined later in this section). Each non-expired FDT Instance is uniquely identified within the file delivery session by its FDT Instance ID, carried by the EXT_FDT Header Extension. Any ALC/LCT packet carrying an FDT Instance MUST include EXT_FDT. * It is RECOMMENDED that an FDT Instance that contains the file description entry for a file be sent at least once before sending the described file within a file delivery session. This recommendation is intended to minimize the amount of file data that may be received by receivers in advance of the FDT Instance containing the entry for a file (such data must either be speculatively buffered or discarded). Note that this possibility cannot be completely eliminated, since the first transmission of FDT data might be lost. * Within a file delivery session, any TOI > 0 MAY be described more than once. For example, a previous FDT Instance 0 describes a TOI of value '3'. Now, subsequent FDT Instances can either keep TOI '3' unmodified in the table, not include it, or augment the description. However, subsequent FDT Instances MUST NOT change the parameters already described for a specific TOI. * An FDT Instance is valid until its expiration time. The expiration time is expressed within the FDT Instance payload as a UTF-8 decimal representation of a 32-bit unsigned integer. The value of this integer represents the 32 most significant bits of a 64-bit Network Time Protocol (NTP) [RFC5905] time value. These 32 bits provide an unsigned integer representing the time in
seconds relative to 0 hours 1 January 1900 in the case of the
prime epoch (era 0) [RFC5905]. The handling of time wraparound
(to happen in 2036) requires that the associated epoch be
considered. In any case, both a sender and a receiver easily
determine to which (136-year) epoch the FDT Instance expiration
time value pertains by choosing the epoch for which the expiration
time is closest in time to the current time.
Here is an example. Let us imagine that a new FLUTE session is
started on February 7th, 2036, 0h, i.e., at NTP time
4,294,944,000, a few hours before the end of epoch 0. In order to
define an FDT Instance valid for the next 48 hours, The FLUTE
sender sets an expiry time of 149,504. This FDT Instance will
expire exactly on February 9th, 2036, 0h. A client that receives
this FDT Instance on the 7th, 0h, just after it has been sent,
immediately understands that this value corresponds to epoch 1. A
client that joins the session on February 8th, 0h, i.e., at NTP
time 63,104, epoch 1, immediately understands that the 149,504 NTP
timestamp corresponds to epoch 1.
* The space of FDT Instance IDs is limited by the associated field
size (i.e., 20 bits) in the EXT_FDT Header Extension
(Section 3.4.1). Therefore, senders should take care to always
have a large enough supply of available FDT Instance IDs when
specifying FDT expiration times.
* The receiver MUST NOT use a received FDT Instance to interpret
packets received beyond the expiration time of the FDT Instance.
* A sender MUST use an expiration time in the future upon creation
of an FDT Instance relative to its Sender Current Time (SCT).
* Any FEC Encoding ID MAY be used for the sending of FDT Instances.
The default is to use the Compact No-Code FEC Encoding ID 0
[RFC5445] for the sending of FDT Instances. (Note that since FEC
Encoding ID 0 is the default for FLUTE, this implies that Source
Block Number and Encoding Symbol ID lengths both default to
16 bits each.)
* If the receiver does not support the FEC Scheme indicated by the
FEC Encoding ID, the receiver MUST NOT decode the associated FDT.
* It is RECOMMENDED that the mechanisms used for file attribute
delivery SHOULD achieve a delivery probability that is higher than
the file recovery probability and the file attributes SHOULD be
delivered at this higher priority before the delivery of the
associated files begins.
Generally, a receiver needs to receive an FDT Instance describing a file before it is able to recover the file itself. In this sense, FDT Instances are of higher priority than files. Additionally, a FLUTE sender SHOULD assume that receivers will not receive all packets pertaining to FDT Instances. The way FDT Instances are transmitted has a large impact on satisfying the recommendation above. When there is a single file transmitted in the session, one way to satisfy the recommendation above is to repeatedly transmit on a regular enough basis FDT Instances describing the file while the file is being transmitted. If an FDT Instance is longer than one packet payload in length, it is RECOMMENDED that an FEC code that provides protection against loss be used for delivering this FDT Instance. When there are multiple files in a session concurrently being transmitted to receivers, the way the FDT Instances are structured and transmitted also has a large impact. As an example, a way to satisfy the recommendation above is to transmit an FDT Instance that describes all files currently being transmitted, and to transmit this FDT Instance reliably, using the same techniques as explained for the case when there is a single file transmitted in a session. If instead the concurrently transmitted files are described in separate FDT Instances, another way to satisfy this recommendation is to transmit all the relevant FDT Instances reliably, using the same techniques as explained for the case when there is a single file transmitted in a session. In any case, how often the description of a file is sent in an FDT Instance, how often an FDT Instance is sent, and how much FEC protection is provided for an FDT Instance (if longer than one packet payload) are dependent on the particular application and are outside the scope of this document. Sometimes the various attributes associated with files that are to be delivered within the file delivery session are sent out-of-band. The details of how this is done are out of the scope of this document. However, it is still RECOMMENDED that any out-of-band transmission be managed in such a way that a receiver will be able to recover the attributes associated with a file at least as reliably as the receiver is able to receive enough packets containing encoding symbols to recover the file. For example, the probability of a randomly chosen receiver being able to recover a given file can often be estimated based on a statistical model of reception conditions, the amount of data transmitted, and the properties of any Forward Error Correction in use. The recommendation above suggests that mechanisms used for file attribute delivery should achieve a higher delivery probability than the file recovery probability. The sender MAY also continue sending the various file attributes in-band, in addition to the out-of-band transmission.
3.4. Structure of FDT Instance Packets
FDT Instances are carried in ALC packets with TOI = 0 and with an additional REQUIRED LCT Header extension called the FDT Instance Header. The FDT Instance Header (EXT_FDT) contains the FDT Instance ID that uniquely identifies FDT Instances within a file delivery session. Placement of the FDT Instance Header is the same as that of any other LCT Header Extension. There MAY be other LCT Header Extensions in use. The FDT Instance is encoded for transmission, like any other object, using an FEC Scheme (which MAY be the Compact No-Code FEC Scheme). The LCT Header Extensions are followed by the FEC Payload ID, and finally the Encoding Symbols for the FDT Instance, which contains one or more file description entries. An FDT Instance MAY span several ALC packets -- the number of ALC packets is a function of the file attributes associated with the FDT Instance. The FDT Instance Header is carried in each ALC packet carrying the FDT Instance. The FDT Instance Header is identical for all ALC/LCT packets for a particular FDT Instance. The overall format of ALC/LCT packets carrying an FDT Instance is depicted in Figure 1 below. All integer fields are carried in "big-endian" or "network order" format (i.e., most significant byte (octet) first). As defined in [RFC5775], all ALC/LCT packets are sent using UDP. +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | UDP header | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Default LCT header (with TOI = 0) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | LCT Header Extensions (EXT_FDT, EXT_FTI, etc.) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FEC Payload ID | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ FLUTE Payload: Encoding Symbol(s) ~ (for FDT Instance in an FDT packet) ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 1: Overall FDT Packet
3.4.1. Format of FDT Instance Header
The FDT Instance Header (EXT_FDT) is a new fixed-length, ALC Protocol-Instantiation-specific LCT Header Extension [RFC5651]. The Header Extension Type (HET) for the extension is 192. Its format is defined below: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | HET = 192 | V | FDT Instance ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 2: EXT_FDT Format Version of FLUTE (V), 4 bits: This document specifies FLUTE version 2. Hence, in any ALC packet that carries an FDT Instance and that belongs to the file delivery session as specified in this specification MUST set this field to '2'. FDT Instance ID, 20 bits: For each file delivery session, the numbering of FDT Instances starts from '0' and is incremented by one for each subsequent FDT Instance. After reaching the maximum value (2^20-1), the numbering starts from the smallest FDT Instance ID value assigned to an expired FDT Instance. When wraparound from a greater FDT Instance ID value to a smaller FDT Instance ID value occurs, the smaller FDT Instance ID value is considered logically higher than the greater FDT Instance ID value. Then, the subsequent FDT Instances are assigned the next available smallest FDT Instance ID value, in order to always keep the FDT Instance ID values logically increasing. Senders MUST NOT reuse an FDT Instance ID value that is already in use for a non-expired FDT Instance. Sender behavior when all the FDT Instance IDs are used by non-expired FEC Instances is outside the scope of this specification and left to individual implementations of FLUTE. Receipt of an FDT Instance that reuses an FDT Instance ID value that is currently used by a non-expired FDT Instance MUST be considered an error case. Receiver behavior in this case (e.g., leave the session or ignore the new FDT Instance) is outside the scope of this specification and left to individual implementations of FLUTE. Receivers MUST be ready to handle FDT Instance ID wraparound and situations where missing FDT Instance IDs result in increments larger than one.
3.4.2. Syntax of FDT Instance
The FDT Instance contains file description entries that provide the mapping functionality described in Section 3.2 above. The FDT Instance is an Extensible Markup Language (XML) structure that has a single root element "FDT-Instance". The "FDT-Instance" element MUST contain the "Expires" attribute, which provides the expiration time of the FDT Instance. In addition, the "FDT-Instance" element MAY contain the "Complete" attribute, a boolean that can be either set to '1' or 'true' for TRUE, or '0' or 'false' for FALSE. When TRUE, the "Complete" attribute signals that this "FDT Instance" includes the set of "File" entries that exhausts both the set of files delivered so far and the set of files to be delivered in the session. This implies that no new data will be provided in future FDT Instances within this session (i.e., that either FDT Instances with higher ID numbers will not be used or, if they are used, will only provide file parameters identical to those already given in this and previous FDT Instances). The "Complete" attribute is therefore used to provide a complete list of files in an entire FLUTE session (a "complete FDT"). Note that when all the FDT Instances received so far have no "Complete" attribute, the receiver MUST consider that the session is not complete and that new data MAY be provided in future FDT Instances. This is equivalent to receiving FDT Instances having the "Complete" attribute set to FALSE. The "FDT-Instance" element MAY contain attributes that give common parameters for all files of an FDT Instance. These attributes MAY also be provided for individual files in the "File" element. Where the same attribute appears in both the "FDT-Instance" and the "File" elements, the value of the attribute provided in the "File" element takes precedence. For each file to be declared in the given FDT Instance, there is a single file description entry in the FDT Instance. Each entry is represented by element "File", which is a child element of the FDT Instance structure. The attributes of the "File" element in the XML structure represent the attributes given to the file that is delivered in the file delivery session. The value of the XML attribute name corresponds to the MIME field name, and the XML attribute value corresponds to the value of the MIME field body [RFC2045]. Each "File" element MUST contain at least two attributes: "TOI" and "Content-Location". "TOI" MUST be assigned a valid TOI value as described in Section 3.3. "Content-Location" [RFC2616] MUST be assigned a syntactically valid URI, as defined in [RFC3986], which identifies the file to be delivered. For example, it can be a URI with the "http" or "file"
URI scheme. Only one "Content-Location" attribute is allowed for
each file. The "Content-Location" field MUST be considered a string
that identifies a file (i.e., two different strings are two different
identifiers). Any use of the "Content-Location" field for anything
else other than to identify the object is out of scope of this
specification. The semantics for any two "File" elements declaring
the same "Content-Location" but differing "TOI" is that the element
appearing in the FDT Instance with the greater FDT Instance ID is
considered to declare a newer instance (e.g., version) of the same
"File".
In addition to mandatory attributes, the "FDT-Instance" element and
the "File" element MAY contain other attributes, of which the
following are specifically pointed out:
* The attribute "Content-Type" SHOULD be included and, when present,
MUST be used for the purpose defined in [RFC2616].
* Where the length is described, the attribute "Content-Length" MUST
be used for the purpose defined in [RFC2616]. The transfer length
is defined to be the length of the object transported in octets.
It is often important to convey the transfer length to receivers,
because the source block structure needs to be known for the FEC
decoder to be applied to recover source blocks of the file, and
the transfer length is often needed to properly determine the
source block structure of the file. There generally will be a
difference between the length of the original file and the
transfer length if content encoding is applied to the file before
transport, and thus the "Content-Encoding" attribute is used. If
the file is not content encoded before transport (and thus the
"Content-Encoding" attribute is not used), then the transfer
length is the length of the original file, and in this case the
"Content-Length" is also the transfer length. However, if the
file is content encoded before transport (and thus the
"Content-Encoding" attribute is used), e.g., if compression is
applied before transport to reduce the number of octets that need
to be transferred, then the transfer length is generally different
than the length of the original file, and in this case the
attribute "Transfer-Length" MAY be used to carry the transfer
length.
* Whenever content encoding is applied, the attribute
"Content-Encoding" MUST be included. Whenever the attribute
"Content-Encoding" is included, it MUST be used as described in
[RFC2616].
* Where the MD5 message digest is described, the attribute
"Content-MD5" MUST be used for the purpose defined in [RFC2616].
Note that the goal is to provide a decoded object integrity
service in cases where transmission and/or FLUTE/ALC processing
errors may occur (the probability of collision is in that case
negligible). It MUST NOT be regarded as a security mechanism (see
Section 7 for information regarding security measures).
* The FEC Object Transmission Information attributes are described
in Section 5.
The following specifies the XML Schema [XML-Schema-Part-1]
[XML-Schema-Part-2] for the FDT Instance:
BEGIN
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns="urn:ietf:params:xml:ns:fdt"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
targetNamespace="urn:ietf:params:xml:ns:fdt"
elementFormDefault="qualified">
<xs:element name="FDT-Instance" type="FDT-InstanceType"/>
<xs:complexType name="FDT-InstanceType">
<xs:sequence>
<xs:element name="File" type="FileType" maxOccurs="unbounded"/>
<xs:any namespace="##other" processContents="skip"
minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>
<xs:attribute name="Expires"
type="xs:string"
use="required"/>
<xs:attribute name="Complete"
type="xs:boolean"
use="optional"/>
<xs:attribute name="Content-Type"
type="xs:string"
use="optional"/>
<xs:attribute name="Content-Encoding"
type="xs:string"
use="optional"/>
<xs:attribute name="FEC-OTI-FEC-Encoding-ID"
type="xs:unsignedByte"
use="optional"/>
<xs:attribute name="FEC-OTI-FEC-Instance-ID"
type="xs:unsignedLong"
use="optional"/>
<xs:attribute name="FEC-OTI-Maximum-Source-Block-Length"
type="xs:unsignedLong"
use="optional"/>
<xs:attribute name="FEC-OTI-Encoding-Symbol-Length"
type="xs:unsignedLong"
use="optional"/>
<xs:attribute name="FEC-OTI-Max-Number-of-Encoding-Symbols"
type="xs:unsignedLong"
use="optional"/>
<xs:attribute name="FEC-OTI-Scheme-Specific-Info"
type="xs:base64Binary"
use="optional"/>
<xs:anyAttribute processContents="skip"/>
</xs:complexType>
<xs:complexType name="FileType">
<xs:sequence>
<xs:any namespace="##other" processContents="skip"
minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>
<xs:attribute name="Content-Location"
type="xs:anyURI"
use="required"/>
<xs:attribute name="TOI"
type="xs:positiveInteger"
use="required"/>
<xs:attribute name="Content-Length"
type="xs:unsignedLong"
use="optional"/>
<xs:attribute name="Transfer-Length"
type="xs:unsignedLong"
use="optional"/>
<xs:attribute name="Content-Type"
type="xs:string"
use="optional"/>
<xs:attribute name="Content-Encoding"
type="xs:string"
use="optional"/>
<xs:attribute name="Content-MD5"
type="xs:base64Binary"
use="optional"/>
<xs:attribute name="FEC-OTI-FEC-Encoding-ID"
type="xs:unsignedByte"
use="optional"/>
<xs:attribute name="FEC-OTI-FEC-Instance-ID"
type="xs:unsignedLong"
use="optional"/>
<xs:attribute name="FEC-OTI-Maximum-Source-Block-Length"
type="xs:unsignedLong"
use="optional"/>
<xs:attribute name="FEC-OTI-Encoding-Symbol-Length" type="xs:unsignedLong" use="optional"/> <xs:attribute name="FEC-OTI-Max-Number-of-Encoding-Symbols" type="xs:unsignedLong" use="optional"/> <xs:attribute name="FEC-OTI-Scheme-Specific-Info" type="xs:base64Binary" use="optional"/> <xs:anyAttribute processContents="skip"/> </xs:complexType> </xs:schema> END Figure 3: XML Schema for the FDT Instance Any valid FDT Instance MUST use the above XML Schema. This way, FDT provides extensibility to support private elements and private attributes within the file description entries. Those could be, for example, the attributes related to the delivery of the file (timing, packet transmission rate, etc.). Unsupported private elements and attributes SHOULD be silently ignored by a FLUTE receiver. In case the basic FDT XML Schema is extended in terms of new descriptors (attributes or elements), for descriptors applying to a single file, those MUST be placed within the element "File". For descriptors applying to all files described by the current FDT Instance, those MUST be placed within the element "FDT-Instance". It is RECOMMENDED that the new attributes applied in the FDT be in the format of message header fields and be either defined in the HTTP/1.1 specification [RFC2616] or another well-known specification, or in an IANA registry [IANAheaderfields]. However, this specification doesn't prohibit the use of other formats to allow private attributes to be used when interoperability is not a concern.3.4.3. Content Encoding of FDT Instance
The FDT Instance itself MAY be content encoded (e.g., compressed). This specification defines the FDT Instance Content Encoding Header (EXT_CENC). EXT_CENC is a new fixed-length LCT Header Extension [RFC5651]. The Header Extension Type (HET) for the extension is 193. If the FDT Instance is content encoded, EXT_CENC MUST be used to signal the content encoding type. In that case, the EXT_CENC Header Extension MUST be used in all ALC packets carrying the same FDT Instance ID. Consequently, when the EXT_CENC header is used, it MUST be used together with a proper FDT Instance Header (EXT_FDT). Within a file delivery session, FDT Instances that are not content encoded and FDT Instances that are content encoded MAY both appear. If
content encoding is not used for a given FDT Instance, EXT_CENC MUST NOT be used in any packet carrying the FDT Instance. The format of EXT_CENC is defined below: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | HET = 193 | CENC | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 4: EXT_CENC Format Content Encoding Algorithm (CENC), 8 bits: This field signals the content encoding algorithm used in the FDT Instance payload. This subsection reserves the Content Encoding Algorithm values 0, 1, 2, and 3 for null, ZLIB [RFC1950], DEFLATE [RFC1951], and GZIP [RFC1952], respectively. Reserved, 16 bits: This field MUST be set to all '0's. This field MUST be ignored on reception.3.5. Multiplexing of Files within a File Delivery Session
The delivered files are carried as transmission objects (identified with TOIs) in the file delivery session. All these objects, including the FDT Instances, MAY be multiplexed in any order and in parallel with each other within a session; i.e., packets for one file may be interleaved with packets for other files or other FDT Instances within a session. Multiple FDT Instances MAY be delivered in a single session using TOI = 0. In this case, it is RECOMMENDED that the sending of a previous FDT Instance SHOULD end before the sending of the next FDT Instance starts. However, due to unexpected network conditions, packets for the FDT Instances might be interleaved. A receiver can determine which FDT Instance a packet contains information about, since the FDT Instances are uniquely identified by their FDT Instance ID carried in the EXT_FDT headers.
(next page on part 2)
