RFC 8038
Exporting MIB Variables Using the IP Flow Information Export (IPFIX) Protocol
Pages: 85
Proposed Standard
Proposed Standard
Part 1 of 4 – Pages 1 to 17
Internet Engineering Task Force (IETF) P. Aitken, Ed. Request for Comments: 8038 Brocade Category: Standards Track B. Claise ISSN: 2070-1721 Cisco Systems, Inc. S. B S Mojo Networks, Inc. C. McDowall Brocade J. Schoenwaelder Jacobs University Bremen May 2017 Exporting MIB Variables Using the IP Flow Information Export (IPFIX) ProtocolAbstract
This document specifies a way to complement IP Flow Information Export (IPFIX) Data Records with Management Information Base (MIB) objects, avoiding the need to define new IPFIX Information Elements for existing MIB objects that are already fully specified. Two IPFIX Options Templates, as well as a method for creating IPFIX Options Templates that are used to export the extra data required to fully describe Simple Network Management Protocol (SNMP) MIB objects in IPFIX, are specified herein. Status of This Memo This is an Internet Standards Track document. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc8038.
Copyright Notice Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.Table of Contents
1. Introduction ....................................................4 2. Motivation ......................................................5 3. Terminology .....................................................7 4. High-Level Solution Overview ....................................8 5. MIB Object Value Information Elements and the MIB Field Options Template ...............................................10 5.1. MIB Field Options Architecture ............................11 5.2. IPFIX and MIB Data Model ..................................13 5.3. MIB Field Options - Specifications and Required Fields ....15 5.3.1. MIB Field Options Template .........................16 5.3.2. MIB Type Options Template ..........................16 5.4. MIB Field Options Template Formats ........................17 5.4.1. Data Template Containing a mibObjectValue Field ....17 5.4.2. MIB Field Options Template .........................19 5.4.3. MIB Field Options Data Records .....................20 5.4.4. Options Template Containing a mibObjectValue Field ...............................21 5.4.5. MIB Field Options Template with Semantics Fields ...23 5.4.6. MIB Field Options Template with Extra MIB Object Details .....................................24 5.5. Use of Field Order in the MIB Field Options Template ......27 5.6. Identifying the SNMP Context ..............................27 5.7. Template Management .......................................28 5.7.1. Large Messages .....................................28 5.7.2. Template Withdrawal and Reuse ......................29
5.8. Exporting Conceptual Rows and Tables ......................29
5.8.1. Exporting Conceptual Rows - Indexing ...............30
5.8.2. Exporting Conceptual Rows - mibObjectValueRow ......30
5.8.3. Exporting Conceptual Rows - AUGMENTS ...............36
5.8.4. Exporting Conceptual Tables - mibObjectValueTable ..37
5.8.5. Exporting Columnar Objects: Using
mibIndexIndicator ..................................38
6. Example Use Cases ..............................................39
6.1. Non-columnar MIB Object: Established TCP Connections ......39
6.2. Enterprise-Specific MIB Object: Detailing CPU Load
History ...................................................42
6.3. Exporting a Conceptual Row: The OSPF Neighbor Row .........45
6.4. Exporting Augmented Conceptual Row: Mapping IF-MIB
ID to Name ................................................49
6.5. Exporting a Columnar Object: ipIfStatsInForwDatagrams .....55
6.6. Exporting a Columnar Object Indexed by Information
Elements: ifOutQLen .......................................58
6.7. Exporting with Multiple Contexts: The OSPF
Neighbor Row Revisited ....................................62
7. Configuration Considerations ...................................65
8. The Collecting Process's Side ..................................66
9. Applicability ..................................................66
10. Security Considerations .......................................67
11. IANA Considerations ...........................................68
11.1. New IPFIX Semantics ......................................68
11.1.1. snmpCounter .......................................68
11.1.2. snmpGauge .........................................68
11.2. New IPFIX Information Elements ...........................69
11.2.1. New MIB Object Value Information Elements .........69
11.2.2. New MIB Field Options Information Elements ........75
11.2.3. New MIB Type Information Elements .................79
12. References ....................................................81
12.1. Normative References .....................................81
12.2. Informative References ...................................82
Acknowledgments ...................................................84
Authors' Addresses ................................................84
1. Introduction
There is growing interest in using IP Flow Information Export (IPFIX) as a push mechanism for exporting management information. Using a push protocol such as IPFIX instead of a polling protocol like SNMP is especially interesting in situations where large chunks of repetitive data need to be exported periodically. While initially targeted at different problems, there is a large parallel between the information transported via IPFIX and SNMP. Furthermore, certain Management Information Base (MIB) objects are highly relevant to Flows as they are understood today. For example, in the IPFIX Information Model [IANA-IPFIX], Information Elements coming from the SNMP world have already been specified, e.g., ingressInterface and egressInterface both refer to the ifIndex object as defined in [RFC2863]. In particular, the Management Information Base was designed as a separate system of definitions; this opens up the possibility of exporting objects defined via the MIB over other protocols. Rather than mapping existing MIB objects to IPFIX Information Elements on a case-by-case basis, it would be advantageous to enable the export of any existing or future MIB objects as part of an IPFIX Data Record. This way, the duplication of Data Models [RFC3444], both as SMIv2 MIB objects and IPFIX Information Elements, out of the same Information Model [RFC3444] would be avoided. Therefore, the primary goals of this document are: o to specify a way to complement IPFIX Data Records with MIB objects; o to avoid the need to define new IPFIX Information Elements for existing MIB objects that are already fully specified; o to allow the correlation of SNMP and IPFIX sourced data by exporting them together; and o to allow SNMP push data from SNMP-only devices to be more easily integrated into IPFIX-based collection infrastructures.
2. Motivation
The intended scope of this work is the addition of MIB variable(s) to IPFIX Information Elements in Data Records, in order to complement the Data Records with useful and already-standardized information. Special consideration is given to the case of an existing Template Record that needs to be augmented with some MIB variables whose index is already present in the Template Record as an IPFIX Information Element -- for example, a 7-tuple Data Record containing the ingressInterface Information Element, which needs to be augmented by interface counters [RFC2863] that are indexed by the respective ingressInterface values already contained in the Data Records. See Section 3 for terminology definitions. Many Data Records contain the ingressInterface and/or the egressInterface Information Elements. These Information Elements carry an ifIndex value, a MIB object defined in [RFC2863]. In order to retrieve additional information about the identified interface, a Collector could simply poll relevant objects from the device running the Exporter via SNMP. However, that approach has several problems: o It requires implementing a mediation function between two Data Models, i.e., MIB objects and IPFIX Information Elements. o Confirming the validity of simple mappings (e.g., ifIndex to ifName) requires either checking on a regular basis that the Exporter's network management system did not reload or imposing ifIndex persistence across an Exporter's reload. o Synchronization problems occur because counters carried in Data Records and counters carried in SNMP messages are retrieved from the Exporter at different points in time and thus cannot be correlated. In the best case, assuming very tight integration of an IPFIX Collector with an SNMP polling engine, SNMP data is retrieved shortly after Data Records have been received, which implies a delay of the sum of the active or idle timeouts (if not null) plus the time to export the Data Record to the Collector. If, however, the SNMP data is retrieved by a generic Network Management Station (NMS) polling interface statistics, then the time lag between IPFIX counters and SNMP counters can be significantly higher. See [RFC5102] for details regarding active and idle timeouts. This document does not specify how to carry SNMP notifications in IPFIX, even if the specifications in this document could potentially allow this.
Since IPFIX is a push mechanism, initiated from the Exporter with no acknowledgment method, this specification does not provide the ability to execute configuration changes. The Distributed Management Expression MIB [RFC2982], which is a mechanism to create new MIB variables based on the content of existing ones, could also be advantageous in the context of this specification. Indeed, newly created MIB objects (for example, the link utilization MIB variable), created with the Distributed Management Expression MIB [RFC2982], could nicely complement Data Records. Another advantage of exporting MIB objects via IPFIX is that IPFIX would benefit from an extended series of types to be exported. The simple and application-wide data types specified in SMIv2 [RFC2578], along with new textual conventions, can be exported within IPFIX and then decoded in the Collector. However, since a textual convention can contain almost any name, this document does not extend the existing "IPFIX Information Elements" subregistry [IANA-IPFIX] that contains informationElementDataType. The overall architectural model is depicted in Figure 1. The IPFIX Exporter accesses the device's instrumentation, which follows the specifications contained in MIB modules. Other management interfaces, such as the Network Configuration Protocol (NETCONF) or the device's Command Line Interface (CLI), may provide access to the same instrumentation. +------+ +-------+ +.........+ +.....+ | SNMP | | IPFIX | : NETCONF : : CLI : +------+ +-------+ +.........+ +.....+ | | | | +--------------------------------------------+ | Instrumentation (specified in MIB modules) | +--------------------------------------------+ Figure 1: Architectural Model
3. Terminology
IPFIX-specific terminology (Information Element, Template, Template Record, Options Template Record, Template Set, Collector, Exporter, Data Record, Transport Session, Exporting Process, Collecting Process, etc.) used in this document is defined in Section 2 of [RFC7011]. As in [RFC7011], these IPFIX-specific terms have the first letter of a word capitalized. This document prefers the more generic term "Data Record" (as opposed to "Flow Record") in relation to the export of MIB objects. Object Identifier (MIB OID) An Object Identifier value is an ordered list of non-negative numbers. For SMIv2, each number in the list is referred to as a sub-identifier. There are at most 128 sub-identifiers in a value, and each sub-identifier has a maximum value of 2^32 - 1 (4294967295 decimal). See [RFC2578], Section 3.5. MIB Object Identifier Information Element An IPFIX Information Element ("mibObjectIdentifier") that denotes that a MIB Object Identifier (MIB OID) is exported in the (Options) Data Record. See Section 11.2.2.1. SMIv2 Terminology The key words "MIB module", "MIB object", "INDEX", "AUGMENTS", "textual convention", "columnar object", "conceptual row", and "conceptual table" in this document are to be interpreted as described in SMIv2 [RFC2578]. SMIv2 SYNTAX The SYNTAX key words "INTEGER", "Integer32", "OCTET STRING", "OBJECT IDENTIFIER", "BITS", "IpAddress", "Counter32", "Gauge32", "TimeTicks", "Opaque", "Counter64", "Unsigned32", "SEQUENCE", and "SEQUENCE OF" in this document are to be interpreted as described in SMIv2 [RFC2578]. SNMP Context Terminology The key words "snmpEngineID", "contextEngineID", and "contextName" in this document are to be interpreted as described in [RFC3411].
mibObjectValue Information Elements
"mibObjectValue Information Elements" refers to any and all of the
mibObjectValue Information Elements generically. Any restriction
or requirement in this document that refers to "mibObjectValue"
applies to the following Information Elements as defined in
Section 11.2.1: mibObjectValueInteger, mibObjectValueOctetString,
mibObjectValueOID, mibObjectValueBits, mibObjectValueIPAddress,
mibObjectValueCounter, mibObjectValueGauge,
mibObjectValueTimeTicks, mibObjectValueUnsigned,
mibObjectValueTable, and mibObjectValueRow.
Abstract Data Type
Abstract Data Types for IPFIX are defined in Section 3.1 of
[RFC7012]. This specification uses the Abstract Data Types
"unsigned8", "unsigned32", "unsigned64", "signed32", "octetArray",
"string", "ipv4Address", and "subTemplateList".
IE
Used as shorthand for "Information Element" [RFC7011] in the
figures.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in
[RFC2119].
4. High-Level Solution Overview
This document specifies a method for creating IPFIX Options Templates
that are used to export the extra data required to describe MIB
variables (see Section 5.1).
This allows IPFIX Templates to contain any combination of fields
defined by traditional IPFIX Information Element(s) and/or MIB Object
Identifier(s). The MIB Object Identifiers can reference either
non-columnar or columnar MIB object(s). Enterprise-specific MIB
Object Identifiers are also supported.
This document also defines two standard IPFIX Options Templates (see
Section 5.3) that are used as part of the mechanism to export MIB
object metadata:
o MIB Field Options Template (Section 5.3.1)
o MIB Type Options Template (Section 5.3.2)
This document defines three classes of new IPFIX Information Elements. These are used to export values from the MIB, export required Object Identifier information, and optionally export type data from a MIB module: o MIB Object Value Information Elements (Section 11.2.1) o MIB Field Options Information Elements (Section 11.2.2) o MIB Type Information Elements (Section 11.2.3) Additionally, this document defines two new IPFIX semantics that are required for the new Information Elements: o snmpCounter (Section 11.1.1) o snmpGauge (Section 11.1.2) Two common types defined in SMIv2 are conceptual rows and conceptual tables. It is desirable that exporting a complete or partial conceptual row be simple and efficient. This is accomplished by using IPFIX Structured Data [RFC6313] to reduce repetition of Object Identifier and indexing data. To allow the use of individual columnar objects that make up a conceptual row, a method is also specified to explain that a MIB object is indexed by other fields in the same Data Flow. For an individually indexed mibObjectValue, the index fields are sent in the same way as any of the other fields in the same Data Record and may be mibObjectValue Information Element(s) or other existing Information Element(s). Also, in some cases Exporters may not want (or be able) to export the full information on how the MIB objects being exported are indexed. This may be because the MIB object is being used purely as type information or the Exporting Process may not have knowledge of the indexing required. Therefore, providing index information for columnar objects is optional.
5. MIB Object Value Information Elements and the MIB Field Options Template
This document defines new mibObjectValue Information Elements (in Section 11.2.1). These are used to export MIB objects as part of standard IPFIX Templates. The mibObjectValue Information Elements contain the actual data values. The Metering Process or Exporting Process may extract the data values for mibObjectValue Information Elements from a Process that resides on the same device or may capture or create the data required to match the definition of the MIB object. In particular, exporting a value of a MIB object defined in a certain MIB module does not imply that the SNMP process on the device supports that MIB module. The main issue that arises from exporting values of MIB objects in IPFIX is that MIB Object Identifiers do not fit into the standard IPFIX Template format [RFC7011], as this only provides a 16-bit Information Element identifier. The values of a MIB object could be exported using a MIB-specific Information Element, without providing any Object Identifiers. However, without exporting the actual MIB OID, the full type of the data would be unknown, and every field containing MIB object data would appear identical. Without knowing which OID the contents of a field map to, the data would be incomprehensible to a Collector. For the values in the mibObjectValue Information Elements to be understandable, more meta-information about the mibObjectValue Information Elements must be sent as part of the IPFIX export. The required minimum information to understand each field that is being exported is provided in Section 5.3.1. One approach to this problem would be to extend the IPFIX standard to allow extended Field Specifiers so that metadata about fields can be included in Data Templates. This would, however, require a new version of the IPFIX standard that may not be backward compatible. However, future versions of IPFIX may export the required MIB metadata as part of newly defined IPFIX Set versions. This document defines a MIB Field Options Template to export the extra meta-information required for mibObjectValue Information Elements. This is a standard IPFIX Options Template Set that includes a minimum set of required fields (see Section 5.3.1) and may include extra fields to provide more meta-information about one of the mibObjectValue Information Elements.
The MIB Field Options export tells the Collecting Process the OID for the MIB object type definition for the following (Template, field).5.1. MIB Field Options Architecture
Four IPFIX Sets are used together to export a Flow that contains mibObjectValue Information Elements. These are: 1. A Template Set that includes the mibObjectValue Information Element. The Template Set informs the Collector that a MIB object value of length N will be exported. This Set may also be an Options Template Set. 2. A MIB Field Options Template Set. The MIB Field Options Template describes which metadata will be sent for each mibObjectValue Information Element being exported. 3. A MIB Field Options Data Set. The MIB Field Options Data Set includes the metadata for each MIB object (i.e., the mibObjectIdentifier or mibSubIdentifier). The metadata about the mibObjectValue Information Elements only needs to be resent as per normal Template refreshes or resends. 4. A Data Set. The Data Set contains only the actual data extracted from the MIB or described by the MIB module.
Figure 2 shows the IPFIX Message structure for a MIB field in a Template Set. +-------------------------------------------------------+ | IPFIX Message Header | +-------------------------------------------------------+ | Template Set (A) | +-------------------------------------------------------+ | Options Template Set (B) (MIB Field Options Template) | +-------------------------------------------------------+ | Data Set (B) (MIB Field Options Data) | +-------------------------------------------------------+ | Data Set (A) | +-------------------------------------------------------+ Figure 2: IPFIX Message Structure for a MIB Field in a Template Set The MIB Field Options Template defines MIB Field Options Data Records. The MIB Field Options Data Records annotate the Data Template with mibObjectValue metadata. Together, the Data Template and MIB Field Options Data Records define the Data Records that will be exported. The Data Records (A) have a dependency on the two Templates and the MIB Field Options Data Records. More Data Sets that use the same mibObjectValue Information Element can then be sent in subsequent packets.
Figure 3 shows the relationships between the Sets discussed above. +------------------------------+ |MIB Field Options Template (B)| +------------------------------+ |(templateId, elementIndex) | +------------------------------+ | mibOID | +------------------------------+ | | Defines V +------------------------+ +--------------------------+ | Data Template (A) | |MIB Field Options Data (B)| +------------------------+ +--------------------------+ |Field 0 - regular IE | | | +------------------------+ +--------------------------+ |Field 1-mibObjectValue | <----------- | (X,1) = OID | +------------------------+ Annotates +--------------------------+ |Field 2-mibObjectValue | <----------- | (X,2) = OID | +------------------------+ +--------------------------+ | | |------------------------------------/ | | Defines | V +------------------------+ | Data Records (A) | |------------------------| | Field 0 data | +------------------------+ | Field 1 data | +------------------------+ | Field 2 data | +------------------------+ Figure 3: Relationships between Sets5.2. IPFIX and MIB Data Model
[RFC2578], Section 7.1 specifies that the SYNTAX clause for a MIB object defines the abstract data structure of an object and what it must contain: "The data structure must be one of the following: a base type, BITS, or a textual convention. (SEQUENCE OF and SEQUENCE are also possible for conceptual tables, see section 7.1.12)."
For each of the SYNTAX clause options, this document specifies exactly which mibObjectValue Information Element to use. If a MIB object to be exported is a textual convention, the definition of the textual convention must be consulted and the SYNTAX clause used to determine the correct base type. This may recurse if the textual convention is defined in terms of another textual convention, but this should end at a base type. If the SYNTAX clause contains a textual convention or sub-typing (e.g., integerSubType, octetStringSubType) [RFC2578], the mibObjectSyntax Information Element SHOULD be used to export this detail to the Collecting Process. The options for the SYNTAX clause are then mapped as follows: +-------------+-------------------+---------------------------------+ | Section in | SYNTAX | mibObjectValue Information | | RFC 2578 | | Element | +-------------+-------------------+---------------------------------+ | 7.1.1 | INTEGER/Integer32 | mibObjectValueInteger | | 7.1.2 | OCTET STRING | mibObjectValueOctetString | | 7.1.3 | OBJECT IDENTIFIER | mibObjectValueOID | | 7.1.4 | BITS | mibObjectValueBits | | 7.1.5 | IpAddress | mibObjectValueIPAddress | | 7.1.6 | Counter32 | mibObjectValueCounter | | 7.1.7 | Gauge32 | mibObjectValueGauge | | 7.1.8 | TimeTicks | mibObjectValueTimeTicks | | 7.1.9 | Opaque | mibObjectValueOctetString | | 7.1.10 | Counter64 | mibObjectValueCounter | | 7.1.11 | Unsigned32 | mibObjectValueUnsigned | | 7.1.12 | SEQUENCE | mibObjectValueRow | | 7.1.12 | SEQUENCE OF | mibObjectValueTable | +-------------+-------------------+---------------------------------+ Table 1: SMIv2 SYNTAX to mibObjectValue Types Values are encoded as per the standard IPFIX encoding of Abstract Data Types. The only new encoding reference in this document is that Object Identifiers (OIDs) will be encoded as per ASN.1/BER [X.690] in an octetArray. The mibObjectValue and mibObjectIdentifier Information Elements are standard IPFIX fields. Therefore, the E bit of the mibObjectValue or mibObjectIdentifier Information Elements is set to 0.
The MIB object being exported may be defined in an enterprise- specific MIB module, but the Information Elements defined in this standard are still exported with the E bit set to 0. The OID being exported indicates that the MIB object was defined in an enterprise-specific MIB module.5.3. MIB Field Options - Specifications and Required Fields
For each mibObjectValue Information Element that is defined in an IPFIX Template, a MIB Field Options Data Record will be exported that provides the required minimum information to define the MIB object that is being exported (see Section 5.3.1). The MIB Field Options Data Records are defined in a Template referred to in this document as a MIB Field Options Template with the format specified in Section 5.4. The MIB Field Options Template and MIB Field Options Data Records MUST be exported in the same IPFIX Message as any Template that is using a mibObjectValue Information Element. Note that this places an implicit size constraint on the export. This whole set of Templates and MIB Field Options Data Records MUST all be exported prior to the corresponding Data Records that depend upon them. That is, the export order MUST be: 1. Data Template for mibObjectValue Information Elements (Set ID 2) 2. MIB Field Options Template (Set ID 3) 3. MIB Field Options Data Records (Set ID >= 256) 4. MIB Object Value Data Records (Set ID >= 256) Note that the ID of an identical MIB Field Options Template that has already been exported MAY be reused without exporting the Template again. IPFIX Set IDs are defined in Section 3.3.2 of [RFC7011]. A value of 2 indicates a Template Set, a value of 3 indicates an Options Template Set, and values 256 and above indicate Data Sets.
5.3.1. MIB Field Options Template
Three fields are REQUIRED to unambiguously export a standalone mibObjectValue Information Element with a MIB Field Options Template: o (scope) templateId [IANA-IPFIX] o (scope) informationElementIndex [IANA-IPFIX] o mibObjectIdentifier (Section 11.2.2.1) or mibSubIdentifier (Section 11.2.2.2) These are the minimum fields required in a MIB Field Options Template (see Section 5.4.2). The mibObjectIdentifier is used to provide the OID for all mibObjectValue Information Elements exported, except when IPFIX Structured Data [RFC6313] is being used to export a conceptual row (see Section 5.8.2). While the following are optional, they are nevertheless RECOMMENDED in certain circumstances, as described in the referenced sections: o mibCaptureTimeSemantics (discussed in Section 5.4.5; Information Element defined in Section 11.2.2.4) o mibIndexIndicator (discussed in Section 5.8.5; Information Element defined in Section 11.2.2.3) o mibContextEngineID (discussed in Section 5.6; Information Element defined in Section 11.2.2.5) o mibContextName (discussed in Section 5.6; Information Element defined in Section 11.2.2.6)5.3.2. MIB Type Options Template
There are also fields that provide type information from a MIB object definition that MAY be exported to a Collecting Process. Type information is statically defined in a MIB module; it is not expected to change. However, the additional information about the MIB object may help a Collecting Process that does not have access to the MIB module.
To export a MIB Type Options Template, the mibObjectIdentifier is RECOMMENDED as a Scope Field so that it matches the MIB Field Options Template. Any combination of the other MIB Type fields may be included. o (scope) mibObjectIdentifier (see Section 11.2.2.1) o mibObjectName (see Section 11.2.3.1) o mibObjectDescription (see Section 11.2.3.2) o mibObjectSyntax (see Section 11.2.3.3) o mibModuleName (see Section 11.2.3.4)
(page 17 continued on part 2)