RFC 7011
Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information
Internet Engineering Task Force (IETF) B. Claise, Ed. Request for Comments: 7011 Cisco Systems, Inc. STD: 77 B. Trammell, Ed. Obsoletes: 5101 ETH Zurich Category: Standards Track P. Aitken ISSN: 2070-1721 Cisco Systems, Inc. September 2013 Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow InformationAbstract
This document specifies the IP Flow Information Export (IPFIX) protocol, which serves as a means for transmitting Traffic Flow information over the network. In order to transmit Traffic Flow information from an Exporting Process to a Collecting Process, a common representation of flow data and a standard means of communicating them are required. This document describes how the IPFIX Data and Template Records are carried over a number of transport protocols from an IPFIX Exporting Process to an IPFIX Collecting Process. This document obsoletes RFC 5101. Status of This Memo This is an Internet Standards Track document. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 5741. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc7011.
Copyright Notice Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.Table of Contents
1. Introduction ....................................................5 1.1. Changes since RFC 5101 .....................................5 1.2. IPFIX Documents Overview ...................................6 2. Terminology .....................................................7 2.1. Terminology Summary Table .................................13 3. IPFIX Message Format ...........................................13 3.1. Message Header Format .....................................15 3.2. Field Specifier Format ....................................16 3.3. Set and Set Header Format .................................18 3.3.1. Set Format .........................................18 3.3.2. Set Header Format ..................................19 3.4. Record Format .............................................20 3.4.1. Template Record Format .............................20 3.4.2. Options Template Record Format .....................23 3.4.2.1. Scope .....................................23 3.4.2.2. Options Template Record Format ............24 3.4.3. Data Record Format .................................27 4. Specific Reporting Requirements ................................28 4.1. The Metering Process Statistics Options Template ..........29 4.2. The Metering Process Reliability Statistics Options Template ..........................................29 4.3. The Exporting Process Reliability Statistics Options Template ..........................................31 4.4. The Flow Keys Options Template ............................32 5. Timing Considerations ..........................................32 5.1. IPFIX Message Header Export Time and Flow Record Time .....32 5.2. Supporting Timestamp Wraparound ...........................33
6. Linkage with the Information Model .............................34 6.1. Encoding of IPFIX Data Types ..............................34 6.1.1. Integral Data Types ................................34 6.1.2. Address Types ......................................34 6.1.3. float32 ............................................34 6.1.4. float64 ............................................34 6.1.5. boolean ............................................35 6.1.6. string and octetArray ..............................35 6.1.7. dateTimeSeconds ....................................35 6.1.8. dateTimeMilliseconds ...............................35 6.1.9. dateTimeMicroseconds ...............................35 6.1.10. dateTimeNanoseconds ...............................36 6.2. Reduced-Size Encoding .....................................36 7. Variable-Length Information Element ............................37 8. Template Management ............................................38 8.1. Template Withdrawal and Redefinition ......................40 8.2. Sequencing Template Management Actions ....................42 8.3. Additional Considerations for Template Management over SCTP .................................................43 8.4. Additional Considerations for Template Management over UDP ..................................................44 9. The Collecting Process's Side ..................................45 9.1. Collecting Process Handling of Malformed IPFIX Messages ...46 9.2. Additional Considerations for SCTP Collecting Processes ...46 9.3. Additional Considerations for UDP Collecting Processes ....46 10. Transport Protocol ............................................47 10.1. Transport Compliance and Transport Usage .................47 10.2. SCTP .....................................................48 10.2.1. Congestion Avoidance ..............................48 10.2.2. Reliability .......................................49 10.2.3. MTU ...............................................49 10.2.4. Association Establishment and Shutdown ............49 10.2.5. Failover ..........................................50 10.2.6. Streams ...........................................50 10.3. UDP ......................................................50 10.3.1. Congestion Avoidance ..............................50 10.3.2. Reliability .......................................51 10.3.3. MTU ...............................................51 10.3.4. Session Establishment and Shutdown ................51 10.3.5. Failover and Session Duplication ..................51 10.4. TCP ......................................................52 10.4.1. Congestion Avoidance ..............................52 10.4.2. Reliability .......................................52 10.4.3. MTU ...............................................52 10.4.4. Connection Establishment and Shutdown .............53 10.4.5. Failover ..........................................53
11. Security Considerations .......................................54 11.1. Applicability of TLS and DTLS ............................55 11.2. Usage ....................................................56 11.3. Mutual Authentication ....................................56 11.4. Protection against DoS Attacks ...........................57 11.5. When DTLS or TLS Is Not an Option ........................58 11.6. Logging an IPFIX Attack ..................................58 11.7. Securing the Collector ...................................59 11.8. Privacy Considerations for Collected Data ................59 12. Management Considerations .....................................60 13. IANA Considerations ...........................................61 Appendix A. IPFIX Encoding Examples ...............................62 A.1. Message Header Example ....................................62 A.2. Template Set Examples .....................................63 A.2.1. Template Set Using IANA Information Elements ..........63 A.2.2. Template Set Using Enterprise-Specific Information Elements ..............................................64 A.3. Data Set Example ..........................................65 A.4. Options Template Set Examples .............................66 A.4.1. Options Template Set Using IANA Information Elements ..66 A.4.2. Options Template Set Using Enterprise-Specific Information Elements ..................................66 A.4.3. Options Template Set Using an Enterprise-Specific Scope .................................................67 A.4.4. Data Set Using an Enterprise-Specific Scope ...........68 A.5. Variable-Length Information Element Examples ..............69 A.5.1. Example of Variable-Length Information Element with Length Less Than 255 Octets ...........................69 A.5.2. Example of Variable-Length Information Element with 3-Octet Length Encoding ...............................70 Normative References ..............................................71 Informative References ............................................71 Acknowledgments ...................................................74 Contributors ......................................................75
1. Introduction
Traffic on a data network can be seen as consisting of flows passing through network elements. For administrative or other purposes, it is often interesting, useful, or even necessary to have access to information about these flows that pass through the network elements. A Collecting Process should be able to receive the Flow information passing through multiple network elements within the data network. This requires uniformity in the method of representing the flow information and the means of communicating the flows from the network elements to the collection point. This document specifies a protocol to achieve these requirements. This document specifies in detail the representation of different flows, as well as the additional data required for flow interpretation, packet format, transport mechanisms used, security concerns, etc.1.1. Changes since RFC 5101
This document obsoletes the Proposed Standard revision of the IPFIX Protocol Specification [RFC5101]. The protocol specified by this document is interoperable with the protocol as specified in [RFC5101]. The following changes have been made to this document with respect to the previous document: - All outstanding technical and editorial errata on [RFC5101] have been addressed. - As the [IANA-IPFIX] registry is now the normative reference for all Information Element definitions (see [RFC7012]), all definitions of Information Elements in Section 4 have been replaced with references to that registry. - The encoding of the dateTimeSeconds, dateTimeMilliseconds, dateTimeMicroseconds, and dateTimeNanoseconds data types, and the related encoding of the IPFIX Message Header Export Time field, have been clarified, especially with respect to the epoch upon which the timestamp data types are based. - A new Section 5.2 has been added to address wraparound of these timestamp data types after they overflow in the years 2032-2038. - Clarifications on encoding, especially in Section 6, have been made: all IPFIX values are encoded in network byte order.
- Template management, as described in Section 8, has been simplified and clarified: the specification has been relaxed with respect to [RFC5101], especially concerning potential failures in Template ID reuse. Additional corner cases in template management have been addressed. The new template management language is interoperable with that in [RFC5101] to the extent that the behavior was defined in the prior specification. - Section 11.3 (Mutual Authentication) has been improved to refer to current practices in Transport Layer Security (TLS) mutual authentication; references to Punycode were removed, as these are covered in [RFC6125]. - Editorial improvements, including structural changes to Sections 8, 9, and 10 to improve readability, have been applied. Behavior common to all transport protocols has been separated out, with exceptions per transport specifically noted. All template management language (on both Exporting and Collecting Processes) has been unified in Section 8. - A new Section 12 on management considerations has been added.1.2. IPFIX Documents Overview
The IPFIX protocol provides network administrators with access to IP Flow information. The architecture for the export of measured IP Flow information out of an IPFIX Exporting Process to a Collecting Process is defined in [RFC5470], per the requirements defined in [RFC3917]. This document specifies how IPFIX Data Records and Templates are carried via a number of transport protocols from IPFIX Exporting Processes to IPFIX Collecting Processes. Four IPFIX optimizations/extensions are currently specified: a bandwidth-saving method for the IPFIX protocol [RFC5473], an efficient method for exporting bidirectional flows [RFC5103], a method for the definition and export of complex data structures [RFC6313], and the specification of the Protocol on IPFIX Mediators [IPFIX-MED-PROTO] based on the IPFIX Mediation Framework [RFC6183]. A "file-based transport" for IPFIX, which defines how IPFIX Messages can be stored in files for document-based workflows and for archival purposes, is discussed in [RFC5655]. IPFIX has a formal description of IPFIX Information Elements -- their names, data types, and additional semantic information -- as specified in [RFC7012]. The registry is maintained by IANA [IANA-IPFIX]. The inline export of the Information Element type information is specified in [RFC5610].
The framework for packet selection and reporting [RFC5474] enables network elements to select subsets of packets by statistical and other methods, and to export a stream of reports on the selected packets to a Collector. The set of packet selection techniques (Sampling, Filtering, and hashing) standardized by the Packet Sampling (PSAMP) protocol is described in [RFC5475]. The PSAMP protocol [RFC5476], which uses IPFIX as its export protocol, specifies the export of packet information from a PSAMP Exporting Process to a PSAMP Collector. Instead of exporting PSAMP Packet Reports, the stream of selected packets may also serve as input to the generation of IPFIX Flow Records. Like IPFIX, PSAMP has a formal description of its Information Elements: their names, types, and additional semantic information. The PSAMP information model is defined in [RFC5477]. [RFC6615] specifies a MIB module for monitoring, and [RFC6728] specifies a data model for configuring and monitoring IPFIX and PSAMP-compliant devices using the Network Configuration Protocol (NETCONF). [RFC6727] specifies the PSAMP MIB module as an extension of the IPFIX SELECTOR MIB module defined in [RFC6615]. In terms of development, [RFC5153] provides guidelines for the implementation and use of the IPFIX protocol, while [RFC5471] provides guidelines for testing. Finally, [RFC5472] describes what types of applications can use the IPFIX protocol and how they can use the information provided. It furthermore shows how the IPFIX framework relates to other architectures and frameworks.2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. The definitions of the basic terms like Traffic Flow, Exporting Process, Collecting Process, Observation Points, etc. are semantically identical to those found in the IPFIX requirements document [RFC3917]. Some of the terms have been expanded for more clarity when defining the protocol. Additional terms required for the protocol have also been defined. Definitions in this document and in [RFC5470] are equivalent; definitions that are only relevant to the IPFIX protocol only appear here.
The terminology summary table in Section 2.1 gives a quick overview of the relationships among some of the different terms defined. Observation Point An Observation Point is a location in the network where packets can be observed. Examples include a line to which a probe is attached; a shared medium, such as an Ethernet-based LAN; a single port of a router; or a set of interfaces (physical or logical) of a router. Note that every Observation Point is associated with an Observation Domain (defined below) and that one Observation Point may be a superset of several other Observation Points. For example, one Observation Point can be an entire line card. That would be the superset of the individual Observation Points at the line card's interfaces. Observation Domain An Observation Domain is the largest set of Observation Points for which Flow information can be aggregated by a Metering Process. For example, a router line card may be an Observation Domain if it is composed of several interfaces, each of which is an Observation Point. In the IPFIX Message it generates, the Observation Domain includes its Observation Domain ID, which is unique per Exporting Process. That way, the Collecting Process can identify the specific Observation Domain from the Exporter that sends the IPFIX Messages. Every Observation Point is associated with an Observation Domain. It is RECOMMENDED that Observation Domain IDs also be unique per IPFIX Device. Packet Treatment "Packet Treatment" refers to action(s) performed on a packet by a forwarding device or other middlebox, including forwarding, dropping, delaying for traffic-shaping purposes, etc.
Traffic Flow or Flow
There are several definitions of the term 'flow' being used by the
Internet community. Within the context of IPFIX, we use the
following definition:
A Flow is defined as a set of packets or frames passing an
Observation Point in the network during a certain time interval.
All packets belonging to a particular Flow have a set of common
properties. Each property is defined as the result of applying a
function to the values of:
1. one or more packet header fields (e.g., destination IP
address), transport header fields (e.g., destination port
number), or application header fields (e.g., RTP header fields
[RFC3550]).
2. one or more characteristics of the packet itself (e.g., number
of MPLS labels, etc.).
3. one or more of the fields derived from Packet Treatment (e.g.,
next-hop IP address, the output interface, etc.).
A packet is defined as belonging to a Flow if it completely
satisfies all the defined properties of the Flow.
Note that the set of packets represented by a Flow may be empty;
that is, a Flow may represent zero or more packets. As sampling
is a Packet Treatment, this definition includes packets selected
by a sampling mechanism.
Flow Key
Each of the fields that:
1. belong to the packet header (e.g., destination IP address), or
2. are a property of the packet itself (e.g., packet length), or
3. are derived from Packet Treatment (e.g., Autonomous System (AS)
number),
and that are used to define a Flow (i.e., are the properties
common to all packets in the Flow) are termed Flow Keys. As an
example, the traditional '5-tuple' Flow Key of source and
destination IP address, source and destination transport port, and
transport protocol, groups together all packets belonging to a
single direction of communication on a single socket.
Flow Record
A Flow Record contains information about a specific Flow that was
observed at an Observation Point. A Flow Record contains measured
properties of the Flow (e.g., the total number of bytes for all
the Flow's packets) and usually contains characteristic properties
of the Flow (e.g., source IP address).
Metering Process
The Metering Process generates Flow Records. Inputs to the
process are packet headers, characteristics, and Packet Treatment
observed at one or more Observation Points.
The Metering Process consists of a set of functions that includes
packet header capturing, timestamping, sampling, classifying, and
maintaining Flow Records.
The maintenance of Flow Records may include creating new records,
updating existing ones, computing Flow statistics, deriving
further Flow properties, detecting Flow expiration, passing Flow
Records to the Exporting Process, and deleting Flow Records.
Exporting Process
The Exporting Process sends IPFIX Messages to one or more
Collecting Processes. The Flow Records in the Messages are
generated by one or more Metering Processes.
Exporter
A device that hosts one or more Exporting Processes is termed an
Exporter.
IPFIX Device
An IPFIX Device hosts at least one Exporting Process. It may host
further Exporting Processes as well as arbitrary numbers of
Observation Points and Metering Processes.
Collecting Process
A Collecting Process receives IPFIX Messages from one or more
Exporting Processes. The Collecting Process might process or
store Flow Records received within these Messages, but such
actions are out of scope for this document.
Collector
A device that hosts one or more Collecting Processes is termed a
Collector.
Template
A Template is an ordered sequence of <type, length> pairs used to
completely specify the structure and semantics of a particular set
of information that needs to be communicated from an IPFIX Device
to a Collector. Each Template is uniquely identifiable by means
of a Template ID.
IPFIX Message
An IPFIX Message is a message that originates at the Exporting
Process and carries the IPFIX records of this Exporting Process,
and whose destination is a Collecting Process. An IPFIX Message
is encapsulated at the transport layer.
Message Header
The Message Header is the first part of an IPFIX Message; the
Message Header provides basic information about the message, such
as the IPFIX version, length of the message, message sequence
number, etc.
Template Record
A Template Record defines the structure and interpretation of
fields in a Data Record.
Data Record
A Data Record is a record that contains values of the parameters
corresponding to a Template Record.
Options Template Record
An Options Template Record is a Template Record that defines the
structure and interpretation of fields in a Data Record, including
defining how to scope the applicability of the Data Record.
Set
A Set is a collection of records that have a similar structure,
prefixed by a header. In an IPFIX Message, zero or more Sets
follow the Message Header. There are three different types of
Sets: Template Sets, Options Template Sets, and Data Sets.
Template Set
A Template Set is a collection of one or more Template Records
that have been grouped together in an IPFIX Message.
Options Template Set
An Options Template Set is a collection of one or more Options
Template Records that have been grouped together in an IPFIX
Message.
Data Set
A Data Set is one or more Data Records, of the same type, that are
grouped together in an IPFIX Message. Each Data Record is
previously defined by a Template Record or an Options Template
Record.
Information Element
An Information Element is a protocol- and encoding-independent
description of an attribute that may appear in an IPFIX Record.
Information Elements are defined in the IANA "IPFIX Information
Elements" registry [IANA-IPFIX]. The type associated with an
Information Element indicates constraints on what it may contain
and also determines the valid encoding mechanisms for use in
IPFIX.
Transport Session
In the Stream Control Transmission Protocol (SCTP), the Transport
Session is known as the SCTP association, which is uniquely
identified by the SCTP endpoints [RFC4960]; in TCP, the Transport
Session is known as the TCP connection, which is uniquely
identified by the combination of IP addresses and TCP ports used.
In UDP, the Transport Session is known as the UDP session, which
is uniquely identified by the combination of IP addresses and UDP
ports used.
2.1. Terminology Summary Table
Figure A shows a summary of IPFIX terminology. +------------------+---------------------------------------------+ | | Contents | | +--------------------+------------------------+ | Set | Template | Record | +------------------+--------------------+------------------------+ | Data Set | / | Data Record(s) | +------------------+--------------------+------------------------+ | Template Set | Template Record(s) | / | +------------------+--------------------+------------------------+ | Options Template | Options Template | / | | Set | Record(s) | | +------------------+--------------------+------------------------+ Figure A: Terminology Summary Table A Data Set is composed of Data Record(s). No Template Record is included. A Template Record or an Options Template Record defines the Data Record. A Template Set contains only Template Record(s). An Options Template Set contains only Options Template Record(s).
(page 13 continued on part 2)
