RFC 5812
Forwarding and Control Element Separation (ForCES) Forwarding Element Model
3. ForCES Model Concepts
Some of the important ForCES concepts used throughout this document are introduced in this section. These include the capability and state abstraction, the FE and LFB model construction, and the unique addressing of the different model structures. Details of these aspects are described in Section 4 and Section 5. The intent of this section is to discuss these concepts at the high level and lay the foundation for the detailed description in the following sections. The ForCES FE model includes both a capability and a state abstraction. o The FE/LFB capability model describes the capabilities and capacities of an FE/LFB by specifying the variation in functions supported and any limitations. Capacity describes the limits of specific components (an example would be a table size limit).
o The state model describes the current state of the FE/LFB, that
is, the instantaneous values or operational behavior of the FE/
LFB.
Section 3.1 explains the difference between a capability model and a
state model, and describes how the two can be combined in the FE
model.
The ForCES model construction laid out in this document allows an FE
to provide information about its structure for operation. This can
be thought of as FE-level information and information about the
individual instances of LFBs provided by the FE.
o The ForCES model includes the constructions for defining the class
of Logical Functional Blocks (LFBs) that an FE may support. These
classes are defined in this and other documents. The definition
of such a class provides the information content for monitoring
and controlling instances of the LFB class for ForCES purposes.
Each LFB model class formally defines the operational LFB
components, LFB capabilities, and LFB events. Essentially,
Section 3.2 introduces the concept of LFBs as the basic functional
building blocks in the ForCES model.
o The FE model also provides the construction necessary to monitor
and control the FE as a whole for ForCES purposes. For
consistency of operation and simplicity, this information is
represented as an LFB, the FE Object LFB class and a singular LFB
instance of that class, defined using the LFB model. The FE
Object class defines the components to provide information at the
FE level, particularly the capabilities of the FE at a coarse
level, i.e., not all possible capabilities or all details about
the capabilities of the FE. Part of the FE-level information is
the LFB topology, which expresses the logical inter-connection
between the LFB instances along the data path(s) within the FE.
Section 3.3 discusses the LFB topology. The FE Object also
includes information about what LFB classes the FE can support.
The ForCES model allows for unique identification of the different
constructs it defines. This includes identification of the LFB
classes, and of LFB instances within those classes, as well as
identification of components within those instances.
The ForCES protocol [RFC5810] encapsulates target address(es) to
eventually get to a fine-grained entity being referenced by the CE.
The addressing hierarchy is broken into the following:
o An FE is uniquely identified by a 32-bit FEID.
o Each class of LFB is uniquely identified by a 32-bit LFB ClassID.
The LFB ClassIDs are global within the network element and may be
issued by IANA.
o Within an FE, there can be multiple instances of each LFB class.
Each LFB class instance is identified by a 32-bit identifier that
is unique within a particular LFB class on that FE.
o All the components within an LFB instance are further defined
using 32-bit identifiers.
Refer to Section 3.3 for more details on addressing.
3.1. ForCES Capability Model and State Model
Capability and state modeling applies to both the FE and LFB
abstraction.
Figure 1 shows the concepts of FE state, capabilities, and
configuration in the context of CE-FE communication via the ForCES
protocol.
+-------+ +-------+
| | FE capabilities: what it can/cannot do. | |
| |<-----------------------------------------| |
| | | |
| CE | FE state: what it is now. | FE |
| |<-----------------------------------------| |
| | | |
| | FE configuration: what it should be. | |
| |----------------------------------------->| |
+-------+ +-------+
Figure 1: Illustration of FE capabilities, state, and configuration
exchange in the context of CE-FE communication via ForCES.
3.1.1. FE Capability Model and State Model
Conceptually, the FE capability model tells the CE which states are
allowed on an FE, with capacity information indicating certain
quantitative limits or constraints. Thus, the CE has general
knowledge about configurations that are applicable to a particular
FE.
3.1.1.1. FE Capability Model
The FE capability model may be used to describe an FE at a coarse level. For example, an FE might be defined as follows: o the FE can handle IPv4 and IPv6 forwarding; o the FE can perform classification based on the following fields: source IP address, destination IP address, source port number, destination port number, etc.; o the FE can perform metering; o the FE can handle up to N queues (capacity); and o the FE can add and remove encapsulating headers of types including IPsec, GRE, L2TP. While one could try to build an object model to fully represent the FE capabilities, other efforts found this approach to be a significant undertaking. The main difficulty arises in describing detailed limits, such as the maximum number of classifiers, queues, buffer pools, and meters that the FE can provide. We believe that a good balance between simplicity and flexibility can be achieved for the FE model by combining coarse-level-capability reporting with an error reporting mechanism. That is, if the CE attempts to instruct the FE to set up some specific behavior it cannot support, the FE will return an error indicating the problem. Examples of similar approaches include Diffserv PIB RFC 3317 [RFC3317] and framework PIB RFC 3318 [RFC3318].3.1.1.2. FE State Model
The FE state model presents the snapshot view of the FE to the CE. For example, using an FE state model, an FE might be described to its corresponding CE as the following: o on a given port, the packets are classified using a given classification filter; o the given classifier results in packets being metered in a certain way and then marked in a certain way; o the packets coming from specific markers are delivered into a shared queue for handling, while other packets are delivered to a different queue; and
o a specific scheduler with specific behavior and parameters will
service these collected queues.
3.1.1.3. LFB Capability and State Model
Both LFB capability and state information are defined formally using
the LFB modeling XML schema.
Capability information at the LFB level is an integral part of the
LFB model and provides for powerful semantics. For example, when
certain features of an LFB class are optional, the CE needs to be
able to determine whether those optional features are supported by a
given LFB instance. The schema for the definition of LFB classes
provides a means for identifying such components.
State information is defined formally using LFB component constructs.
3.1.2. Relating LFB and FE Capability and State Model
Capability information at the FE level describes the LFB classes that
the FE can instantiate, the number of instances of each that can be
created, the topological (linkage) limitations between these LFB
instances, etc. Section 5 defines the FE-level components including
capability information. Since all information is represented as
LFBs, this is provided by a single instance of the FE Object LFB
class. By using a single instance with a known LFB class and a known
instance identification, the ForCES protocol can allow a CE to access
this information whenever it needs to, including while the CE is
establishing the control of the FE.
Once the FE capability is described to the CE, the FE state
information can be represented at two levels. The first level is the
logically separable and distinct packet processing functions, called
LFBs. The second level of information describes how these individual
LFBs are ordered and placed along the data path to deliver a complete
forwarding plane service. The interconnection and ordering of the
LFBs is called LFB topology. Section 3.2 discusses high-level
concepts around LFBs, whereas Section 3.3 discusses LFB topology
issues. This topology information is represented as components of
the FE Object LFB instance, to allow the CE to fetch and manipulate
this.
3.2. Logical Functional Block (LFB) Modeling
Each LFB performs a well-defined action or computation on the packets
passing through it. Upon completion of its prescribed function,
either the packets are modified in certain ways (e.g., decapsulator,
marker), or some results are generated and stored, often in the form
of metadata (e.g., classifier). Each LFB typically performs a single action. Classifiers, shapers, and meters are all examples of such LFBs. Modeling LFBs at such a fine granularity allows us to use a small number of LFBs to express the higher-order FE functions (such as an IPv4 forwarder) precisely, which in turn can describe more complex networking functions and vendor implementations of software and hardware. These fine-grained LFBs will be defined in detail in one or more documents to be published separately, using the material in this model. It is also the case that LFBs may exist in order to provide a set of components for control of FE operation by the CE (i.e., a locus of control), without tying that control to specific packets or specific parts of the data path. An example of such an LFB is the FE Object, which provides the CE with information about the FE as a whole, and allows the FE to control some aspects of the FE, such as the data path itself. Such LFBs will not have the packet-oriented properties described in this section. In general, multiple LFBs are contained in one FE, as shown in Figure 2, and all the LFBs share the same ForCES protocol (Fp) termination point that implements the ForCES protocol logic and maintains the communication channel to and from the CE.
+-----------+ | CE | +-----------+ ^ | Fp reference point | +--------------------------|-----------------------------------+ | FE | | | v | | +----------------------------------------------------------+ | | | ForCES protocol | | | | termination point | | | +----------------------------------------------------------+ | | ^ ^ | | : : Internal control | | : : | | +---:----------+ +---:----------| | | | :LFB1 | | : LFB2 | | | =====>| v |============>| v |======>...| | Inputs| +----------+ |Outputs | +----------+ | | | (P,M) | |Components| |(P',M') | |Components| |(P",M") | | | +----------+ | | +----------+ | | | +--------------+ +--------------+ | | | +--------------------------------------------------------------+ Figure 2: Generic LFB diagram. An LFB, as shown in Figure 2, may have inputs, outputs, and components that can be queried and manipulated by the CE via an Fp reference point (defined in RFC 3746 [RFC3746]) and the ForCES protocol termination point. The horizontal axis is in the forwarding plane for connecting the inputs and outputs of LFBs within the same FE. P (with marks to indicate modification) indicates a data packet, while M (with marks to indicate modification) indicates the metadata associated with a packet. The vertical axis between the CE and the FE denotes the Fp reference point where bidirectional communication between the CE and FE occurs: the CE-to-FE communication is for configuration, control, and packet injection, while the FE-to-CE communication is used for packet redirection to the control plane, reporting of monitoring and accounting information, reporting of errors, etc. Note that the interaction between the CE and the LFB is only abstract and indirect. The result of such an interaction is for the CE to manipulate the components of the LFB instances. An LFB can have one or more inputs. Each input takes a pair of a packet and its associated metadata. Depending upon the LFB input port definition, the packet or the metadata may be allowed to be
empty (or equivalently to not be provided). When input arrives at an LFB, either the packet or its associated metadata must be non-empty or there is effectively no input. (LFB operation generally may be triggered by input arrival, by timers, or by other system state. It is only in the case where the goal is to have input drive operation that the input must be non-empty.) The LFB processes the input, and produces one or more outputs, each of which is a pair of a packet and its associated metadata. Again, depending upon the LFB output port definition, either the packet or the metadata may be allowed to be empty (or equivalently to be absent). Metadata attached to packets on output may be metadata that was received, or may be information about the packet processing that may be used by later LFBs in the FEs packet processing. A namespace is used to associate a unique name and ID with each LFB class. The namespace MUST be extensible so that a new LFB class can be added later to accommodate future innovation in the forwarding plane. LFB operation is specified in the model to allow the CE to understand the behavior of the forwarding data path. For instance, the CE needs to understand at what point in the data path the IPv4 header TTL is decremented by the FE. That is, the CE needs to know if a control packet could be delivered to it either before or after this point in the data path. In addition, the CE needs to understand where and what type of header modifications (e.g., tunnel header append or strip) are performed by the FEs. Further, the CE works to verify that the various LFBs along a data path within an FE are compatible to link together. Connecting incompatible LFB instances will produce a non-working data path. So the model is designed to provide sufficient information for the CE to make this determination. Selecting the right granularity for describing the functions of the LFBs is an important aspect of this model. There is value to vendors if the operation of LFB classes can be expressed in sufficient detail so that physical devices implementing different LFB functions can be integrated easily into an FE design. However, the model, and the associated library of LFBs, must not be so detailed and so specific as to significantly constrain implementations. Therefore, a semi- formal specification is needed; that is, a text description of the LFB operation (human readable), but sufficiently specific and unambiguous to allow conformance testing and efficient design, so that interoperability between different CEs and FEs can be achieved.
The LFB class model specifies the following, among other information:
o number of inputs and outputs (and whether they are configurable)
o metadata read/consumed from inputs
o metadata produced at the outputs
o packet types accepted at the inputs and emitted at the outputs
o packet content modifications (including encapsulation or
decapsulation)
o packet routing criteria (when multiple outputs on an LFB are
present)
o packet timing modifications
o packet flow ordering modifications
o LFB capability information components
o events that can be detected by the LFB, with notification to the
CE
o LFB operational components
Section 4 of this document provides a detailed discussion of the LFB
model with a formal specification of LFB class schema. The rest of
Section 3.2 only intends to provide a conceptual overview of some
important issues in LFB modeling, without covering all the specific
details.
3.2.1. LFB Outputs
An LFB output is a conceptual port on an LFB that can send
information to another LFB. The information sent on that port is a
pair of a packet and associated metadata, one of which may be empty.
(If both were empty, there would be no output.)
A single LFB output can be connected to only one LFB input. This is
required to make the packet flow through the LFB topology
unambiguous.
Some LFBs will have a single output, as depicted in Figure 3.a.
+---------------+ +-----------------+ | | | | | | | OUT +--> ... OUT +--> ... | | | | EXCEPTIONOUT +--> | | | | +---------------+ +-----------------+ a. One output b. Two distinct outputs +---------------+ +-----------------+ | | | EXCEPTIONOUT +--> | OUT:1 +--> | | ... OUT:2 +--> ... OUT:1 +--> | ... +... | OUT:2 +--> | OUT:n +--> | ... +... +---------------+ | OUT:n +--> +-----------------+ c. One output group d. One output and one output group Figure 3: Examples of LFBs with various output combinations. To accommodate a non-trivial LFB topology, multiple LFB outputs are needed so that an LFB class can fork the data path. Two mechanisms are provided for forking: multiple singleton outputs and output groups, which can be combined in the same LFB class. Multiple separate singleton outputs are defined in an LFB class to model a predetermined number of semantically different outputs. That is, the LFB class definition MUST include the number of outputs, implying the number of outputs is known when the LFB class is defined. Additional singleton outputs cannot be created at LFB instantiation time, nor can they be created on the fly after the LFB is instantiated. For example, an IPv4 LPM (Longest-Prefix-Matching) LFB may have one output (OUT) to send those packets for which the LPM look-up was successful, passing a META_ROUTEID as metadata; and have another output (EXCEPTIONOUT) for sending exception packets when the LPM look-up failed. This example is depicted in Figure 3.b. Packets emitted by these two outputs not only require different downstream treatment, but they are a result of two different conditions in the LFB and each output carries different metadata. This concept assumes that the number of distinct outputs is known when the LFB class is defined. For each singleton output, the LFB class definition defines the types of frames (packets) and metadata the output emits.
An output group, on the other hand, is used to model the case where a flow of similar packets with an identical set of permitted metadata needs to be split into multiple paths. In this case, the number of such paths is not known when the LFB class is defined because it is not an inherent property of the LFB class. An output group consists of a number of outputs, called the output instances of the group, where all output instances share the same frame (packet) and metadata emission definitions (see Figure 3.c). Each output instance can connect to a different downstream LFB, just as if they were separate singleton outputs, but the number of output instances can differ between LFB instances of the same LFB class. The class definition may include a lower and/or an upper limit on the number of outputs. In addition, for configurable FEs, the FE capability information may define further limits on the number of instances in specific output groups for certain LFBs. The actual number of output instances in a group is a component of the LFB instance, which is read-only for static topologies, and read-write for dynamic topologies. The output instances in a group are numbered sequentially, from 0 to N-1, and are addressable from within the LFB. To use Output Port groups, the LFB has to have a built-in mechanism to select one specific output instance for each packet. This mechanism is described in the textual definition of the class and is typically configurable via some attributes of the LFB. For example, consider a redirector LFB, whose sole purpose is to direct packets to one of N downstream paths based on one of the metadata associated with each arriving packet. Such an LFB is fairly versatile and can be used in many different places in a topology. For example, given LFBs that record the type of packet in a FRAMETYPE metadatum, or a packet rate class in a COLOR metadatum, one may uses these metadata for branching. A redirector can be used to divide the data path into an IPv4 and an IPv6 path based on a FRAMETYPE metadatum (N=2), or to fork into rate-specific paths after metering using the COLOR metadatum (red, yellow, green; N=3), etc. Using an output group in the above LFB class provides the desired flexibility to adapt each instance of this class to the required operation. The metadata to be used as a selector for the output instance is a property of the LFB. For each packet, the value of the specified metadata may be used as a direct index to the output instance. Alternatively, the LFB may have a configurable selector table that maps a metadatum value to output instance. Note that other LFBs may also use the output group concept to build in similar adaptive forking capability. For example, a classifier LFB with one input and N outputs can be defined easily by using the output group concept. Alternatively, a classifier LFB with one singleton output in combination with an explicit N-output re-director
LFB models the same processing behavior. The decision of whether to use the output group model for a certain LFB class is left to the LFB class designers. The model allows the output group to be combined with other singleton output(s) in the same class, as demonstrated in Figure 3.d. The LFB here has two types of outputs, OUT, for normal packet output, and EXCEPTIONOUT, for packets that triggered some exception. The normal OUT has multiple instances; thus, it is an output group. In summary, the LFB class may define one output, multiple singleton outputs, one or more output groups, or a combination thereof. Multiple singleton outputs should be used when the LFB must provide for forking the data path and at least one of the following conditions hold: o the number of downstream directions is inherent from the definition of the class and hence fixed o the frame type and set of permitted metadata emitted on any of the outputs are different from what is emitted on the other outputs (i.e., they cannot share their frametype and permitted metadata definitions) An output group is appropriate when the LFB must provide for forking the data path and at least one of the following conditions hold: o the number of downstream directions is not known when the LFB class is defined o the frame type and set of metadata emitted on these outputs are sufficiently similar or, ideally, identical, such they can share the same output definition3.2.2. LFB Inputs
An LFB input is a conceptual port on an LFB on which the LFB can receive information from other LFBs. The information is typically a pair of a packet and its associated metadata. Either the packet or the metadata may for some LFBs and some situations be empty. They cannot both be empty, as then there is no input. For LFB instances that receive packets from more than one other LFB instance (fan-in), there are three ways to model fan-in, all supported by the LFB model and can all be combined in the same LFB: o Implicit multiplexing via a single input
o Explicit multiplexing via multiple singleton inputs o Explicit multiplexing via a group of inputs (input group) The simplest form of multiplexing uses a singleton input (Figure 4.a). Most LFBs will have only one singleton input. Multiplexing into a single input is possible because the model allows more than one LFB output to connect to the same LFB input. This property applies to any LFB input without any special provisions in the LFB class. Multiplexing into a single input is applicable when the packets from the upstream LFBs are similar in frametype and accompanying metadata, and require similar processing. Note that this model does not address how potential contention is handled when multiple packets arrive simultaneously. If contention handling needs to be explicitly modeled, one of the other two modeling solutions must be used. The second method to model fan-in uses individually defined singleton inputs (Figure 4.b). This model is meant for situations where the LFB needs to handle distinct types of packet streams, requiring input-specific handling inside the LFB, and where the number of such distinct cases is known when the LFB class is defined. For example, an LFB that can perform both Layer 2 decapsulation (to Layer 3) and Layer 3 encapsulation (to Layer 2) may have two inputs, one for receiving Layer 2 frames for decapsulation, and one for receiving Layer 3 frames for encapsulation. This LFB type expects different frames (L2 versus L3) at its inputs, each with different sets of metadata, and would thus apply different processing on frames arriving at these inputs. This model is capable of explicitly addressing packet contention by defining how the LFB class handles the contending packets. +--------------+ +------------------------+ | LFB X +---+ | | +--------------+ | | | | | | +--------------+ v | | | LFB Y +---+-->|input Meter LFB | +--------------+ ^ | | | | | +--------------+ | | | | LFB Z |---+ | | +--------------+ +------------------------+ (a) An LFB connects with multiple upstream LFBs via a single input.
+--------------+ +------------------------+ | LFB X +---+ | | +--------------+ +-->|layer2 | +--------------+ | | | LFB Y +------>|layer3 LFB | +--------------+ +------------------------+ (b) An LFB connects with multiple upstream LFBs via two separate singleton inputs. +--------------+ +------------------------+ | Queue LFB #1 +---+ | | +--------------+ | | | | | | +--------------+ +-->|in:0 \ | | Queue LFB #2 +------>|in:1 | input group | +--------------+ |... | | +-->|in:N-1 / | ... | | | +--------------+ | | | | Queue LFB #N |---+ | Scheduler LFB | +--------------+ +------------------------+ (c) A Scheduler LFB uses an input group to differentiate which queue LFB packets are coming from. Figure 4: Examples of LFBs with various input combinations. The third method to model fan-in uses the concept of an input group. The concept is similar to the output group introduced in the previous section and is depicted in Figure 4.c. An input group consists of a number of input instances, all sharing the properties (same frame and metadata expectations). The input instances are numbered from 0 to N-1. From the outside, these inputs appear as normal inputs, i.e., any compatible upstream LFB can connect its output to one of these inputs. When a packet is presented to the LFB at a particular input instance, the index of the input where the packet arrived is known to the LFB and this information may be used in the internal processing. For example, the input index can be used as a table selector, or as an explicit precedence selector to resolve contention. As with output groups, the number of input instances in an input group is not defined in the LFB class. However, the class definition may include restrictions on the range of possible values. In addition, if an FE supports configurable topologies, it may impose further limitations on the number of instances for particular port group(s) of a particular LFB class. Within these limitations, different instances of the same class may have a different number of input instances.
The number of actual input instances in the group is a component defined in the LFB class, which is read-only for static topologies, and is read-write for configurable topologies. As an example for the input group, consider the Scheduler LFB depicted in Figure 4.c. Such an LFB receives packets from a number of Queue LFBs via a number of input instances, and uses the input index information to control contention resolution and scheduling. In summary, the LFB class may define one input, multiple singleton inputs, one or more input groups, or a combination thereof. Any input allows for implicit multiplexing of similar packet streams via connecting multiple outputs to the same input. Explicit multiple singleton inputs are useful when either the contention handling must be handled explicitly or when the LFB class must receive and process a known number of distinct types of packet streams. An input group is suitable when contention handling must be modeled explicitly, but the number of inputs is not inherent from the class (and hence is not known when the class is defined), or when it is critical for LFB operation to know exactly on which input the packet was received.3.2.3. Packet Type
When LFB classes are defined, the input and output packet formats (e.g., IPv4, IPv6, Ethernet) MUST be specified. These are the types of packets that a given LFB input is capable of receiving and processing, or that a given LFB output is capable of producing. This model requires that distinct packet types be uniquely labeled with a symbolic name and/or ID. Note that each LFB has a set of packet types that it operates on, but does not care whether the underlying implementation is passing a greater portion of the packets. For example, an IPv4 LFB might only operate on IPv4 packets, but the underlying implementation may or may not be stripping the L2 header before handing it over. Whether or not such processing is happening is opaque to the CE.3.2.4. Metadata
Metadata is state that is passed from one LFB to another alongside a packet. The metadata passed with the packet assists subsequent LFBs to process that packet. The ForCES model defines metadata as precise atomic definitions in the form of label, value pairs.
The ForCES model provides to the authors of LFB classes a way to formally define how to achieve metadata creation, modification, reading, as well as consumption (deletion). Inter-FE metadata, i.e., metadata crossing FEs, while it is likely to be semantically similar to this metadata, is out of scope for this document. Section 4 has informal details on metadata.3.2.4.1. Metadata Lifecycle within the ForCES Model
Each metadatum is modeled as a <label, value> pair, where the label identifies the type of information (e.g., "color"), and its value holds the actual information (e.g., "red"). The label here is shown as a textual label, but for protocol processing it is associated with a unique numeric value (identifier). To ensure inter-operability between LFBs, the LFB class specification must define what metadata the LFB class "reads" or "consumes" on its input(s) and what metadata it "produces" on its output(s). For maximum extensibility, this definition should specify neither which LFBs the metadata is expected to come from for a consumer LFB nor which LFBs are expected to consume metadata for a given producer LFB.3.2.4.2. Metadata Production and Consumption
For a given metadatum on a given packet path, there MUST be at least one producer LFB that creates that metadatum and SHOULD be at least one consumer LFB that needs that metadatum. In the ForCES model, the producer and consumer LFBs of a metadatum are not required to be adjacent. In addition, there may be multiple producers and consumers for the same metadatum. When a packet path involves multiple producers of the same metadatum, then subsequent producers overwrite that metadatum value. The metadata that is produced by an LFB is specified by the LFB class definition on a per-output-port-group basis. A producer may always generate the metadata on the port group, or may generate it only under certain conditions. We call the former "unconditional" metadata, whereas the latter is "conditional" metadata. For example, deep packet inspection LFB might produce several pieces of metadata about the packet. The first metadatum might be the IP protocol (TCP, UDP, SCTP, ...) being carried, and two additional metadata items might be the source and destination port number. These additional metadata items are conditional on the value of the first metadatum (IP carried protocol) as they are only produced for protocols that
use port numbers. In the case of conditional metadata, it should be possible to determine from the definition of the LFB when "conditional" metadata is produced. The consumer behavior of an LFB, that is, the metadata that the LFB needs for its operation, is defined in the LFB class definition on a per-input-port-group basis. An input port group may "require" a given metadatum, or may treat it as "optional" information. In the latter case, the LFB class definition MUST explicitly define what happens if any optional metadata is not provided. One approach is to specify a default value for each optional metadatum, and assume that the default value is used for any metadata that is not provided with the packet. When specifying the metadata tags, some harmonization effort must be made so that the producer LFB class uses the same tag as its intended consumer(s).3.2.4.3. LFB Operations on Metadata
When the packet is processed by an LFB (i.e., between the time it is received and forwarded by the LFB), the LFB may perform read, write, and/or consume operations on any active metadata associated with the packet. If the LFB is considered to be a black box, one of the following operations is performed on each active metadatum. * IGNORE: ignores and forwards the metadatum * READ: reads and forwards the metadatum * READ/RE-WRITE: reads, over-writes, and forwards the metadatum * WRITE: writes and forwards the metadatum (can also be used to create new metadata) * READ-AND-CONSUME: reads and consumes the metadatum * CONSUME: consumes metadatum without reading The last two operations terminate the life-cycle of the metadatum, meaning that the metadatum is not forwarded with the packet when the packet is sent to the next LFB. In the ForCES model, a new metadatum is generated by an LFB when the LFB applies a WRITE operation to a metadatum type that was not present when the packet was received by the LFB. Such implicit creation may be unintentional by the LFB; that is, the LFB may apply the WRITE operation without knowing or caring whether or not the given metadatum existed. If it existed, the metadatum gets over- written; if it did not exist, the metadatum is created.
For LFBs that insert packets into the model, WRITE is the only meaningful metadata operation. For LFBs that remove the packet from the model, they may either READ- AND-CONSUME (read) or CONSUME (ignore) each active metadatum associated with the packet.3.2.5. LFB Events
During operation, various conditions may occur that can be detected by LFBs. Examples range from link failure or restart to timer expiration in special purpose LFBs. The CE may wish to be notified of the occurrence of such events. The description of how such messages are sent, and their format, is part of the Forwarding and Control Element Separation (ForCES) protocol [RFC5810] document. Indicating how such conditions are understood is part of the job of this model. Events are declared in the LFB class definition. The LFB event declaration constitutes: o a unique 32-bit identifier. o An LFB component that is used to trigger the event. This entity is known as the event target. o A condition that will happen to the event target that will result in a generation of an event to the CE. Examples of a condition include something getting created or deleted, a config change, etc. o What should be reported to the CE by the FE if the declared condition is met. The declaration of an event within an LFB class essentially defines what part of the LFB component(s) need to be monitored for events, what condition on the LFB monitored LFB component an FE should detect to trigger such an event, and what to report to the CE when the event is triggered. While events may be declared by the LFB class definition, runtime activity is controlled using built-in event properties using LFB component properties (discussed in Section 3.2.6). A CE subscribes to the events on an LFB class instance by setting an event property for subscription. Each event has a subscription property that is by default off. A CE wishing to receive a specific event needs to turn on the subscription property at runtime.
Event properties also provide semantics for runtime event filtering.
A CE may set an event property to further suppress events to which it
has already subscribed. The LFB model defines such filters to
include threshold values, hysteresis, time intervals, number of
events, etc.
The contents of reports with events are designed to allow for the
common, closely related information that the CE can be strongly
expected to need to react to the event. It is not intended to carry
information that the CE already has, large volumes of information, or
information related in complex fashions.
From a conceptual point of view, at runtime, event processing is
split into:
1. Detection of something happening to the (declared during LFB
class definition) event target. Processing the next step happens
if the CE subscribed (at runtime) to the event.
2. Checking of the (declared during LFB class definition) condition
on the LFB event target. If the condition is met, proceed with
the next step.
3. Checking (runtime set) event filters if they exist to see if the
event should be reported or suppressed. If the event is to be
reported, proceed to the next step.
4. Submitting of the declared report to the CE.
Section 4.7.6 discusses events in more details.
3.2.6. Component Properties
LFBs and structures are made up of components, containing the
information that the CE needs to see and/or change about the
functioning of the LFB. These components, as described in detail in
Section 4.7, may be basic values, complex structures (containing
multiple components themselves, each of which can be values,
structures, or tables), or tables (which contain values, structures,
or tables). Components may be defined such that their appearance in
LFB instances is optional. Components may be readable or writable at
the discretion of the FE implementation. The CE needs to know these
properties. Additionally, certain kinds of components (arrays /
tables, aliases, and events) have additional property information
that the CE may need to read or write. This model defines the
structure of the property information for all defined data types.
Section 4.8 describes properties in more details.
3.2.7. LFB Versioning
LFB class versioning is a method to enable incremental evolution of LFB classes. In general, an FE is not allowed to contain an LFB instance for more than one version of a particular class. Inheritance (discussed next in Section 3.2.8) has special rules. If an FE data path model containing an LFB instance of a particular class C also simultaneously contains an LFB instance of a class C' inherited from class C; C could have a different version than C'. LFB class versioning is supported by requiring a version string in the class definition. CEs may support multiple versions of a particular LFB class to provide backward compatibility, but FEs MUST NOT support more than one version of a particular class. Versioning is not restricted to making backward-compatible changes. It is specifically expected to be used to make changes that cannot be represented by inheritance. Often this will be to correct errors, and hence may not be backward compatible. It may also be used to remove components that are not considered useful (particularly if they were previously mandatory, and hence were an implementation impediment).3.2.8. LFB Inheritance
LFB class inheritance is supported in the FE model as a method to define new LFB classes. This also allows FE vendors to add vendor- specific extensions to standardized LFBs. An LFB class specification MUST specify the base class and version number it inherits from (the default is the base LFB class). Multiple inheritance is not allowed, however, to avoid unnecessary complexity. Inheritance should be used only when there is significant reuse of the base LFB class definition. A separate LFB class should be defined if little or no reuse is possible between the derived and the base LFB class. An interesting issue related to class inheritance is backward compatibility between a descendant and an ancestor class. Consider the following hypothetical scenario where a standardized LFB class "L1" exists. Vendor A builds an FE that implements LFB "L1", and vendor B builds a CE that can recognize and operate on LFB "L1". Suppose that a new LFB class, "L2", is defined based on the existing "L1" class by extending its capabilities incrementally. Let us examine the FE backward-compatibility issue by considering what would happen if vendor B upgrades its FE from "L1" to "L2" and vendor C's
CE is not changed. The old L1-based CE can interoperate with the new
L2-based FE if the derived LFB class "L2" is indeed backward
compatible with the base class "L1".
The reverse scenario is a much less problematic case, i.e., when CE
vendor B upgrades to the new LFB class "L2", but the FE is not
upgraded. Note that as long as the CE is capable of working with
older LFB classes, this problem does not affect the model; hence we
will use the term "backward compatibility" to refer to the first
scenario concerning FE backward compatibility.
Backward compatibility can be designed into the inheritance model by
constraining LFB inheritance to require that the derived class be a
functional superset of the base class (i.e., the derived class can
only add functions to the base class, but not remove functions).
Additionally, the following mechanisms are required to support FE
backward compatibility:
1. When detecting an LFB instance of an LFB type that is unknown to
the CE, the CE MUST be able to query the base class of such an
LFB from the FE.
2. The LFB instance on the FE SHOULD support a backward-
compatibility mode (meaning the LFB instance reverts itself back
to the base class instance), and the CE SHOULD be able to
configure the LFB to run in such a mode.
3.3. ForCES Model Addressing
Figure 5 demonstrates the abstraction of the different ForCES model
entities. The ForCES protocol provides the mechanism to uniquely
identify any of the LFB class instance components.
FE Address = FE01
+--------------------------------------------------------------+
| |
| +--------------+ +--------------+ |
| | LFB ClassID 1| |LFB ClassID 91| |
| | InstanceID 3 |============>|InstanceID 3 |======>... |
| | +----------+ | | +----------+ | |
| | |Components| | | |Components| | |
| | +----------+ | | +----------+ | |
| +--------------+ +--------------+ |
| |
+--------------------------------------------------------------+
Figure 5: FE entity hierarchy.
At the top of the addressing hierarchy is the FE identifier. In the example above, the 32-bit FE identifier is illustrated with the mnemonic FE01. The next 32-bit entity selector is the LFB ClassID. In the illustration above, two LFB classes with identifiers 1 and 91 are demonstrated. The example above further illustrates one instance of each of the two classes. The scope of the 32-bit LFB class instance identifier is valid only within the LFB class. To emphasize that point, each of class 1 and 91 has an instance of 3. Using the described addressing scheme, a message could be sent to address FE01, LFB ClassID 1, LFB InstanceID 3, utilizing the ForCES protocol. However, to be effective, such a message would have to target entities within an LFB. These entities could be carrying state, capability, etc. These are further illustrated in Figure 6 below. LFB Class ID 1,InstanceID 3 Components +-------------------------------------+ | | | LFB ComponentID 1 | | +----------------------+ | | | | | | +----------------------+ | | | | LFB ComponentID 31 | | +----------------------+ | | | | | | +----------------------+ | | | | LFB ComponentID 51 | | +----------------------+ | | | LFB ComponentID 89 | | | | +-----------------+ | | | | | | | | | | +-----------------+ | | | +----------------------+ | | | | | +-------------------------------------+ Figure 6: LFB hierarchy. Figure 6 zooms into the components carried by LFB Class ID 1, LFB InstanceID 3 from Figure 5.
The example shows three components with 32-bit component identifiers 1, 31, and 51. LFB ComponentID 51 is a complex structure encapsulating within it an entity with LFB ComponentID 89. LFB ComponentID 89 could be a complex structure itself, but is restricted in the example for the sake of clarity.3.3.1. Addressing LFB Components: Paths and Keys
As mentioned above, LFB components could be complex structures, such as a table, or even more complex structures such as a table whose cells are further tables, etc. The ForCES model XML schema (Section 4) allows for uniquely identifying anything with such complexity, utilizing the concept of dot-annotated static paths and content addressing of paths as derived from keys. As an example, if LFB ComponentID 51 were a structure, then the path to LFB ComponentID 89 above will be 51.89. LFB ComponentID 51 might represent a table (an array). In that case, to select the LFB component with ID 89 from within the 7th entry of the table, one would use the path 51.7.89. In addition to supporting explicit table element selection by including an index in the dotted path, the model supports identifying table elements by their contents. This is referred to as using keys, or key indexing. So, as a further example, if ComponentID 51 was a table that was key index-able, then a key describing content could also be passed by the CE, along with path 51 to select the table, and followed by the path 89 to select the table structure element, which upon computation by the FE would resolve to the LFB ComponentID 89 within the specified table entry.3.4. FE Data Path Modeling
Packets coming into the FE from ingress ports generally flow through one or more LFBs before leaving out of the egress ports. How an FE treats a packet depends on many factors, such as type of the packet (e.g., IPv4, IPv6, or MPLS), header values, time of arrival, etc. The result of LFB processing may have an impact on how the packet is to be treated in downstream LFBs. This differentiation of packet treatment downstream can be conceptualized as having alternative data paths in the FE. For example, the result of a 6-tuple classification performed by a classifier LFB could control which rate meter is applied to the packet by a rate meter LFB in a later stage in the data path. LFB topology is a directed graph representation of the logical data paths within an FE, with the nodes representing the LFB instances and the directed link depicting the packet flow direction from one LFB to
the next. Section 3.4.1 discusses how the FE data paths can be modeled as LFB topology, while Section 3.4.2 focuses on issues related to LFB topology reconfiguration.3.4.1. Alternative Approaches for Modeling FE Data Paths
There are two basic ways to express the differentiation in packet treatment within an FE; one represents the data path directly and graphically (topological approach) and the other utilizes metadata (the encoded state approach). o Topological Approach Using this approach, differential packet treatment is expressed by splitting the LFB topology into alternative paths. In other words, if the result of an LFB operation controls how the packet is further processed, then such an LFB will have separate output ports, one for each alternative treatment, connected to separate sub-graphs, each expressing the respective treatment downstream. o Encoded State Approach An alternate way of expressing differential treatment is by using metadata. The result of the operation of an LFB can be encoded in a metadatum, which is passed along with the packet to downstream LFBs. A downstream LFB, in turn, can use the metadata and its value (e.g., as an index into some table) to determine how to treat the packet. Theoretically, either approach could substitute for the other, so one could consider using a single pure approach to describe all data paths in an FE. However, neither model by itself results in the best representation for all practically relevant cases. For a given FE with certain logical data paths, applying the two different modeling approaches will result in very different looking LFB topology graphs. A model using only the topological approach may require a very large graph with many links or paths, and nodes (i.e., LFB instances) to express all alternative data paths. On the other hand, a model using only the encoded state model would be restricted to a string of LFBs, which is not an intuitive way to describe different data paths (such as MPLS and IPv4). Therefore, a mix of these two approaches will likely be used for a practical model. In fact, as we illustrate below, the two approaches can be mixed even within the same LFB. Using a simple example of a classifier with N classification outputs followed by other LFBs, Figure 7.a shows what the LFB topology looks like when using the pure topological approach. Each output from the classifier goes to one of the N LFBs where no metadata is needed. The topological approach is simple, straightforward, and graphically
intuitive. However, if N is large and the N nodes following the classifier (LFB#1, LFB#2, ..., LFB#N) all belong to the same LFB type (e.g., meter), but each has its own independent components, the encoded state approach gives a much simpler topology representation, as shown in Figure 7.b. The encoded state approach requires that a table of N rows of meter components be provided in the Meter node itself, with each row representing the attributes for one meter instance. A metadatum M is also needed to pass along with the packet P from the classifier to the meter, so that the meter can use M as a look-up key (index) to find the corresponding row of the attributes that should be used for any particular packet P. What if those N nodes (LFB#1, LFB#2, ..., LFB#N) are not of the same type? For example, if LFB#1 is a queue while the rest are all meters, what is the best way to represent such data paths? While it is still possible to use either the pure topological approach or the pure encoded state approach, the natural combination of the two appears to be the best option. Figure 7.c depicts two different functional data paths using the topological approach while leaving the N-1 meter instances distinguished by metadata only, as shown in Figure 7.c. +----------+ P | LFB#1 | +--------->|(Compon-1)| +-------------+ | +----------+ | 1|------+ P +----------+ | 2|---------------->| LFB#2 | | classifier 3| |(Compon-2)| | ...|... +----------+ | N|------+ ... +-------------+ | P +----------+ +--------->| LFB#N | |(Compon-N)| +----------+ (a) Using pure topological approach
+-------------+ +-------------+ | 1| | Meter | | 2| (P, M) | (Compon-1) | | 3|---------------->| (Compon-2) | | ...| | ... | | N| | (Compon-N) | +-------------+ +-------------+ (b) Using pure encoded state approach to represent the LFB topology in 5(a), if LFB#1, LFB#2, ..., and LFB#N are of the same type (e.g., meter). +-------------+ +-------------+ (P, M) | queue | | 1|------------->| (Compon-1) | | 2| +-------------+ | 3| (P, M) +-------------+ | ...|------------->| Meter | | N| | (Compon-2) | +-------------+ | ... | | (Compon-N) | +-------------+ (c) Using a combination of the two, if LFB#1, LFB#2, ..., and LFB#N are of different types (e.g., queue and meter). Figure 7: An example of how to model FE data paths. From this example, we demonstrate that each approach has a distinct advantage depending on the situation. Using the encoded state approach, fewer connections are typically needed between a fan-out node and its next LFB instances of the same type because each packet carries metadata the following nodes can interpret and hence invoke a different packet treatment. For those cases, a pure topological approach forces one to build elaborate graphs with many more connections and often results in an unwieldy graph. On the other hand, a topological approach is the most intuitive for representing functionally different data paths. For complex topologies, a combination of the two is the most flexible. A general design guideline is provided to indicate which approach is best used for a particular situation. The topological approach should primarily be used when the packet data path forks to distinct LFB classes (not just distinct parameterizations of the same LFB class), and when the fan-outs do not require changes, such as adding/removing LFB outputs, or require only very infrequent changes.
Configuration information that needs to change frequently should be expressed by using the internal attributes of one or more LFBs (and hence using the encoded state approach). +---------------------------------------------+ | | +----------+ V +----------+ +------+ | | | | | |if IP-in-IP| | | ---->| ingress |->+----->|classifier|---------->|Decap.|---->---+ | ports | | |---+ | | +----------+ +----------+ |others +------+ | V (a) The LFB topology with a logical loop +-------+ +-----------+ +------+ +-----------+ | | | |if IP-in-IP | | | | --->|ingress|-->|classifier1|----------->|Decap.|-->+classifier2|-> | ports | | |----+ | | | | +-------+ +-----------+ |others +------+ +-----------+ | V (b) The LFB topology without the loop utilizing two independent classifier instances. Figure 8: An LFB topology example. It is important to point out that the LFB topology described here is the logical topology, not the physical topology of how the FE hardware is actually laid out. Nevertheless, the actual implementation may still influence how the functionality is mapped to the LFB topology. Figure 8 shows one simple FE example. In this example, an IP-in-IP packet from an IPsec application like VPN may go to the classifier first and have the classification done based on the outer IP header. Upon being classified as an IP-in-IP packet, the packet is then sent to a decapsulator to strip off the outer IP header, followed by a classifier again to perform classification on the inner IP header. If the same classifier hardware or software is used for both outer and inner IP header classification with the same set of filtering rules, a logical loop is naturally present in the LFB topology, as shown in Figure 8.a. However, if the classification is implemented by two different pieces of hardware or software with different filters (i.e., one set of filters for the outer IP header and another set for the inner IP header), then it is more natural to model them as two different instances of classifier LFB, as shown in Figure 8.b.
3.4.2. Configuring the LFB Topology
While there is little doubt that an individual LFB must be configurable, the configurability question is more complicated for LFB topology. Since the LFB topology is really the graphic representation of the data paths within an FE, configuring the LFB topology means dynamically changing the data paths, including changing the LFBs along the data paths on an FE (e.g., creating/ instantiating, updating, or deleting LFBs) and setting up or deleting interconnections between outputs of upstream LFBs to inputs of downstream LFBs. Why would the data paths on an FE ever change dynamically? The data paths on an FE are set up by the CE to provide certain data plane services (e.g., Diffserv, VPN) to the network element's (NE) customers. The purpose of reconfiguring the data paths is to enable the CE to customize the services the NE is delivering at run time. The CE needs to change the data paths when the service requirements change, such as adding a new customer or when an existing customer changes their service. However, note that not all data path changes result in changes in the LFB topology graph. Changes in the graph are dependent on the approach used to map the data paths into LFB topology. As discussed in Section 3.4.1, the topological approach and encoded state approach can result in very different looking LFB topologies for the same data paths. In general, an LFB topology based on a pure topological approach is likely to experience more frequent topology reconfiguration than one based on an encoded state approach. However, even an LFB topology based entirely on an encoded state approach may have to change the topology at times, for example, to bypass some LFBs or insert new LFBs. Since a mix of these two approaches is used to model the data paths, LFB topology reconfiguration is considered an important aspect of the FE model. We want to point out that allowing a configurable LFB topology in the FE model does not mandate that all FEs are required to have this capability. Even if an FE supports configurable LFB topology, the FE may impose limitations on what can actually be configured. Performance-optimized hardware implementations may have zero or very limited configurability, while FE implementations running on network processors may provide more flexibility and configurability. It is entirely up to the FE designers to decide whether or not the FE actually implements reconfiguration and if so, how much. Whether a simple runtime switch is used to enable or disable (i.e., bypass) certain LFBs, or more flexible software reconfiguration is used, is an implementation detail internal to the FE and outside the scope of the FE model. In either case, the CE(s) MUST be able to learn the FE's configuration capabilities. Therefore, the FE model MUST
provide a mechanism for describing the LFB topology configuration capabilities of an FE. These capabilities may include (see Section 5 for full details): o Which LFB classes the FE can instantiate o The maximum number of instances of the same LFB class that can be created o Any topological limitations, for example: * The maximum number of instances of the same class or any class that can be created on any given branch of the graph * Ordering restrictions on LFBs (e.g., any instance of LFB class A must be always downstream of any instance of LFB class B) The CE needs some programming help in order to cope with the range of complexity. In other words, even when the CE is allowed to configure LFB topology for the FE, the CE is not expected to be able to interpret an arbitrary LFB topology and determine which specific service or application (e.g., VPN, Diffserv) is supported by the FE. However, once the CE understands the coarse capability of an FE, the CE MUST configure the LFB topology to implement the network service the NE is supposed to provide. Thus, the mapping the CE has to understand is from the high-level NE service to a specific LFB topology, not the other way around. The CE is not expected to have the ultimate intelligence to translate any high-level service policy into the configuration data for the FEs. However, it is conceivable that within a given network service domain, a certain amount of intelligence can be programmed into the CE to give the CE a general understanding of the LFBs involved to allow the translation from a high-level service policy to the low-level FE configuration to be done automatically. Note that this is considered an implementation issue internal to the control plane and outside the scope of the FE model. Therefore, it is not discussed any further in this document.
+----------+ +-----------+ ---->| Ingress |---->|classifier |--------------+ | | |chip | | +----------+ +-----------+ | v +-------------------------------------------+ +--------+ | Network Processor | <----| Egress | | +------+ +------+ +-------+ | +--------+ | |Meter | |Marker| |Dropper| | ^ | +------+ +------+ +-------+ | | | | +----------+-------+ | | | | | +---------+ +---------+ +------+ +---------+ | | |Forwarder|<------|Scheduler|<--|Queue | |Counter | | | +---------+ +---------+ +------+ +---------+ | +--------------------------------------------------------------+ Figure 9: The capability of an FE as reported to the CE. Figure 9 shows an example where a QoS-enabled (quality-of-service) router has several line cards that have a few ingress ports and egress ports, a specialized classification chip, and a network processor containing codes for FE blocks like meter, marker, dropper, counter, queue, scheduler, and IPv4 forwarder. Some of the LFB topology is already fixed and has to remain static due to the physical layout of the line cards. For example, all of the ingress ports might be hardwired into the classification chip so all packets flow from the ingress port into the classification engine. On the other hand, the LFBs on the network processor and their execution order are programmable. However, certain capacity limits and linkage constraints could exist between these LFBs. Examples of the capacity limits might be: o 8 meters o 16 queues in one FE o the scheduler can handle at most up to 16 queues o The linkage constraints might dictate that: * the classification engine may be followed by: + a meter + marker
+ dropper
+ counter
+ queue or IPv4 forwarder, but not a scheduler
* queues can only be followed by a scheduler
* a scheduler must be followed by the IPv4 forwarder
* the last LFB in the data path before going into the egress
ports must be the IPv4 forwarder
+-----+ +-------+ +---+
| A|--->|Queue1 |--------------------->| |
------>| | +-------+ | | +---+
| | | | | |
| | +-------+ +-------+ | | | |
| B|--->|Meter1 |----->|Queue2 |------>| |->| |
| | | | +-------+ | | | |
| | | |--+ | | | |
+-----+ +-------+ | +-------+ | | +---+
classifier +-->|Dropper| | | IPv4
+-------+ +---+ Fwd.
Scheduler
Figure 10: An LFB topology as configured
by the CE and accepted by the FE.
Once the FE reports these capabilities and capacity limits to the CE,
it is now up to the CE to translate the QoS policy into a desirable
configuration for the FE. Figure 9 depicts the FE capability, while
Figure 10 and Figure 11 depict two different topologies that the CE
may request the FE to configure. Note that Figure 11 is not fully
drawn, as inter-LFB links are included to suggest potential
complexity, without drawing in the endpoints of all such links.
Queue1 +---+ +--+ | A|------------------->| |--+ +->| | | | | | | B|--+ +--+ +--+ +--+ | | +---+ | | | | | | | Meter1 +->| |-->| | | | | | | | | | +--+ +--+ | IPv4 | Counter1 Dropper1 Queue2| +--+ Fwd. +---+ | +--+ +--->|A | +-+ | A|---+ | |------>|B | | | ------>| B|------------------------------>| | +-->|C |->| |-> | C|---+ +--+ | +>|D | | | | D|-+ | | | +--+ +-+ +---+ | | +---+ Queue3 | |Scheduler Classifier1 | | | A|------------> +--+ | | | +->| | | |-+ | | | B|--+ +--+ +-------->| | | | +---+ | | | | +--+ | | Meter2 +->| |-+ | | | | | | +--+ Queue4 | | Marker1 +--+ | +---------------------------->| |---+ | | +--+ Figure 11: Another LFB topology as configured by the CE and accepted by the FE. Note that both the ingress and egress are omitted in Figure 10 and Figure 11 to simplify the representation. The topology in Figure 11 is considerably more complex than Figure 10, but both are feasible within the FE capabilities, and so the FE should accept either configuration request from the CE.
(page 41 continued on part 3)
