RFC 3821
Fibre Channel Over TCP/IP (FCIP)
Network Working Group M. Rajagopal Request for Comments: 3821 E. Rodriguez Category: Standards Track R. Weber July 2004 Fibre Channel Over TCP/IP (FCIP) Status of this Memo This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. Please refer to the current edition of the "Internet Official Protocol Standards" (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited. Copyright Notice Copyright (C) The Internet Society (2004).Abstract
Fibre Channel Over TCP/IP (FCIP) describes mechanisms that allow the interconnection of islands of Fibre Channel storage area networks over IP-based networks to form a unified storage area network in a single Fibre Channel fabric. FCIP relies on IP-based network services to provide the connectivity between the storage area network islands over local area networks, metropolitan area networks, or wide area networks.Table Of Contents
1. Purpose, Motivation, and Objectives. . . . . . . . . . . . . . 3 2. Relationship to Fibre Channel Standards. . . . . . . . . . . . 4 2.1. Relevant Fibre Channel Standards . . . . . . . . . . . . 4 2.2. This Specification and Fibre Channel Standards . . . . . 5 3. Terminology. . . . . . . . . . . . . . . . . . . . . . . . . . 5 4. Protocol Summary . . . . . . . . . . . . . . . . . . . . . . . 7 5. The FCIP Model . . . . . . . . . . . . . . . . . . . . . . . . 9 5.1. FCIP Protocol Model. . . . . . . . . . . . . . . . . . . 9 5.2. FCIP Link. . . . . . . . . . . . . . . . . . . . . . . . 10 5.3. FC Entity. . . . . . . . . . . . . . . . . . . . . . . . 11 5.4. FCIP Entity. . . . . . . . . . . . . . . . . . . . . . . 12 5.5. FCIP Link Endpoint (FCIP_LEP). . . . . . . . . . . . . . 13 5.6. FCIP Data Engine (FCIP_DE) . . . . . . . . . . . . . . . 14 5.6.1. FCIP Encapsulation of FC Frames. . . . . . . . . 16 5.6.2. FCIP Data Engine Error Detection and Recovery. . 19 6. Checking FC Frame Transit Times in the IP Network. . . . . . . 22
7. The FCIP Special Frame (FSF) . . . . . . . . . . . . . . . . . 23 7.1. FCIP Special Frame Format. . . . . . . . . . . . . . . . 23 7.2. Overview of FSF Usage in Connection Establishment. . . . 26 8. TCP Connection Management. . . . . . . . . . . . . . . . . . . 28 8.1. TCP Connection Establishment . . . . . . . . . . . . . . 28 8.1.1. Connection Establishment Model . . . . . . . . . 28 8.1.2. Creating New TCP Connections . . . . . . . . . . 29 8.1.3. Processing Incoming TCP Connect Requests . . . . 32 8.1.4. Simultaneous Connection Establishment. . . . . . 36 8.2. Closing TCP Connections. . . . . . . . . . . . . . . . . 36 8.3. TCP Connection Parameters. . . . . . . . . . . . . . . . 36 8.3.1. TCP Selective Acknowledgement Option . . . . . . 36 8.3.2. TCP Window Scale Option. . . . . . . . . . . . . 36 8.3.3. Protection Against Sequence Number Wrap. . . . . 37 8.3.4. TCP_NODELAY Option . . . . . . . . . . . . . . . 37 8.4. TCP Connection Considerations. . . . . . . . . . . . . . 37 8.5. Flow Control Mapping between TCP and FC. . . . . . . . . 37 9. Security . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 9.1. Threat Models. . . . . . . . . . . . . . . . . . . . . . 38 9.2. FC Fabric and IP Network Deployment Models . . . . . . . 40 9.3. FCIP Security Components . . . . . . . . . . . . . . . . 40 9.3.1. IPsec ESP Authentication and Confidentiality . . 40 9.3.2. Key Management . . . . . . . . . . . . . . . . . 41 9.3.3. ESP Replay Protection and Rekeying Issues. . . . 43 9.4. Secure FCIP Link Operation . . . . . . . . . . . . . . . 44 9.4.1. FCIP Link Initialization Steps . . . . . . . . . 44 9.4.2. TCP Connection Security Associations (SAs) . . . 44 9.4.3. Handling Data Integrity and Confidentiality Violations . . . . . . . . . . . . . . . . . . . 45 10. Performance. . . . . . . . . . . . . . . . . . . . . . . . . . 45 10.1. Performance Considerations . . . . . . . . . . . . . . . 45 10.2. IP Quality of Service (QoS) Support. . . . . . . . . . . 46 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 47 11.1. Normative References . . . . . . . . . . . . . . . . . . 47 11.2. Informative References . . . . . . . . . . . . . . . . . 49 12. Acknowledgments. . . . . . . . . . . . . . . . . . . . . . . . 50 Appendix A. Fibre Channel Bit and Byte Numbering Guidance . . . . 51 Appendix B. IANA Considerations . . . . . . . . . . . . . . . . . 51 Appendix C. FCIP Usage of Addresses and Identifiers . . . . . . . 52 Appendix D. Example of synchronization Recovery Algorithm . . . . 53 Appendix E. Relationship between FCIP and IP over FC (IPFC) . . . 58 Appendix F. FC Frame Format . . . . . . . . . . . . . . . . . . . 59 Appendix G. FC Encapsulation Format . . . . . . . . . . . . . . . 61 Appendix H. FCIP Requirements on an FC Entity . . . . . . . . . . 63 Editors and Contributors Acknowledgements. . . . . . . . . . . . . 69 Editors and Contributors Addresses . . . . . . . . . . . . . . . . 70 Full Copyright Statement . . . . . . . . . . . . . . . . . . . . . 74
1. Purpose, Motivation, and Objectives
Warning to Readers Familiar With Fibre Channel: Both Fibre Channel and IETF standards use the same byte transmission order. However, the bit and byte numbering is different. See appendix A for guidance. Fibre Channel (FC) is a gigabit or multi-gigabit speed networking technology primarily used to implement Storage Area Networks (SANs). See section 2 for information about how Fibre Channel is standardized and the relationship of this specification to Fibre Channel standards. An overview of Fibre Channel can be found in [34]. This specification describes mechanisms that allow the interconnection of islands of Fibre Channel SANs over IP Networks to form a unified SAN in a single Fibre Channel fabric. The motivation behind defining these interconnection mechanisms is a desire to connect physically remote FC sites allowing remote disk access, tape backup, and live mirroring. Fibre Channel standards have chosen nominal distances between switch elements that are less than the distances available in an IP Network. Since Fibre Channel and IP Networking technologies are compatible, it is logical to turn to IP Networking for extending the allowable distances between Fibre Channel switch elements. The fundamental assumption made in this specification is that the Fibre Channel traffic is carried over the IP Network in such a manner that the Fibre Channel Fabric and all Fibre Channel devices on the Fabric are unaware of the presence of the IP Network. This means that the FC datagrams must be delivered in such time as to comply with existing Fibre Channel specifications. The FC traffic may span LANs, MANs, and WANs, so long as this fundamental assumption is adhered to. The objectives of this document are to: 1) specify the encapsulation and mapping of Fibre Channel (FC) frames employing FC Frame Encapsulation [19]. 2) apply the mechanism described in 1) to an FC Fabric using an IP network as an interconnect between two or more islands in an FC Fabric. 3) address any FC concerns arising from tunneling FC traffic over an IP-based network, including security, data integrity (loss), congestion, and performance. This will be accomplished by utilizing the existing IETF-specified suite of protocols.
4) be compatible with the referenced FC standards. While new work
may be undertaken in T11 to optimize and enhance FC Fabrics, this
specification REQUIRES conformance only to the referenced FC
standards.
5) be compatible with all applicable IETF standards so that the IP
Network used to extend an FC Fabric can be used concurrently for
other reasonable purposes.
The objectives of this document do not include using an IP Network as
a replacement for the Fibre Channel Arbitrated Loop interconnect. No
definition is provided for encapsulating loop primitive signals for
transmission over an IP Network.
Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14, RFC 2119 [1].
2. Relationship to Fibre Channel Standards
2.1. Relevant Fibre Channel Standards
FC is standardized as a family of American National Standards
developed by the T11 technical committee of INCITS (InterNational
Committee for Information Technology Standards). T11 has specified a
number of documents describing FC protocols, operations, and
services. T11 documents of interest to readers of this specification
include (but are not limited to):
- FC-BB - Fibre Channel Backbone [2]
- FC-BB-2 - Fibre Channel Backbone -2 [3]
- FC-SW-2 - Fibre Channel Switch Fabric -2 [4]
- FC-FS - Fibre Channel Framing and Signaling [5]
FC-BB and FC-BB-2 describe the relationship between an FC Fabric and
interconnect technologies not defined by Fibre Channel standards
(e.g., ATM and SONET). FC-BB-2 is the Fibre Channel document
describing the relationships between FC and TCP/IP, including the FC
use of FCIP.
FC-SW-2 describes the switch components of an FC Fabric and FC-FS
describes the FC Frame format and basic control features of Fibre
Channel.
Additional information regarding T11 activities is available on the
committee's web site www.t11.org.
2.2. This Specification and Fibre Channel Standards
When considering the challenge of transporting FC Frames over an IP Network, it is logical to divide the standardization effort between TCP/IP requirements and Fibre Channel requirements. This specification covers the TCP/IP requirements for transporting FC Frames; the Fibre Channel documents described in section 2.1 cover the Fibre Channel requirements. This specification addresses only the requirements necessary to properly utilize an IP Network as a conduit for FC Frames. The result is a specification for an FCIP Entity (see section 5.4). A product that tunnels an FC Fabric through an IP Network MUST combine the FCIP Entity with an FC Entity (see section 5.3) using an implementation specific interface. The requirements placed on an FC Entity by this specification to achieve proper delivery of FC Frames are summarized in appendix H. More information about FC Entities can be found in the Fibre Channel standards and an example of an FC Entity can be found in FC-BB-2 [3]. No attempt is being made to define a specific API between an FCIP Entity and an FC Entity. The approach is to specify required functional interactions between an FCIP Entity and an FC Entity (both of which are required to forward FC frames across an IP Network), but allow implementers to choose how these interactions will be realized.3. Terminology
Terms used to describe FCIP concepts are defined in this section. FC End Node - An FC device that uses the connection services provided by the FC Fabric. FC Entity - The Fibre Channel specific functional component that combines with an FCIP Entity to form an interface between an FC Fabric and an IP Network (see section 5.3). FC Fabric - An entity that interconnects various Nx_Ports (see [5]) attached to it, and is capable of routing FC Frames using only the destination ID information in an FC Frame header (see appendix F). FC Fabric Entity - A Fibre Channel specific element containing one or more Interconnect_Ports (see FC-SW-2 [4]) and one or more FC/FCIP Entity pairs. See FC-BB-2 [3] for details about FC Fabric Entities.
FC Frame - The basic unit of Fibre Channel data transfer (see
appendix F).
FC Frame Receiver Portal - The access point through which an FC
Frame and time stamp enter an FCIP Data Engine from the FC Entity.
FC Frame Transmitter Portal - The access point through which a
reconstituted FC Frame and time stamp leave an FCIP Data Engine to
the FC Entity.
FC/FCIP Entity pair - The combination of one FC Entity and one FCIP
entity.
FCIP Data Engine (FCIP_DE) - The component of an FCIP Entity that
handles FC Frame encapsulation, de-encapsulation, and transmission
FCIP Frames through a single TCP Connection (see section 5.6).
FCIP Entity - The entity responsible for the FCIP protocol exchanges
on the IP Network and encompasses FCIP_LEP(s) and FCIP Control and
Services module (see section 5.4).
FCIP Frame - An FC Frame plus the FC Frame Encapsulation [19]
header, encoded SOF and encoded EOF that contains the FC Frame
(see section 5.6.1).
FCIP Link - One or more TCP Connections that connect one FCIP_LEP to
another (see section 5.2).
FCIP Link Endpoint (FCIP_LEP) - The component of an FCIP Entity
that handles a single FCIP Link and contains one or more FCIP_DEs
(see section 5.5).
Encapsulated Frame Receiver Portal - The TCP access point through
which an FCIP Frame is received from the IP Network by an FCIP
Data Engine.
Encapsulated Frame Transmitter Portal - The TCP access point through
which an FCIP Frame is transmitted to the IP Network by an FCIP
Data Engine.
FCIP Special Frame (FSF) - A specially formatted FC Frame containing
information used by the FCIP protocol (see section 7).
4. Protocol Summary
The FCIP protocol is summarized as follows: 1) The primary function of an FCIP Entity is forwarding FC Frames, employing FC Frame Encapsulation described in [19]. 2) Viewed from the IP Network perspective, FCIP Entities are peers and communicate using TCP/IP. Each FCIP Entity contains one or more TCP endpoints in the IP-based network. 3) Viewed from the FC Fabric perspective, pairs of FCIP Entities, in combination with their associated FC Entities, forward FC Frames between FC Fabric elements. The FC End Nodes are unaware of the existence of the FCIP Link. 4) FC Primitive Signals, Primitive Sequences, and Class 1 FC Frames are not transmitted across an FCIP Link because they cannot be encoded using FC Frame Encapsulation [19]. 5) The path (route) taken by an encapsulated FC Frame follows the normal routing procedures of the IP Network. 6) An FCIP Entity MAY contain multiple FCIP Link Endpoints, but each FCIP Link Endpoint (FCIP_LEP) communicates with exactly one other FCIP_LEP. 7) When multiple FCIP_LEPs with multiple FCIP_DEs are in use, selection of which FCIP_DE to use for encapsulating and transmitting a given FC Frame is covered in FC-BB-2 [3]. FCIP Entities do not actively participate in FC Frame routing. 8) The FCIP Control and Services module MAY use TCP/IP quality of service features (see section 10.2). 9) It is necessary to statically or dynamically configure each FCIP entity with the IP addresses and TCP port numbers corresponding to FCIP Entities with which it is expected to initiate communication. If dynamic discovery of participating FCIP Entities is supported, the function SHALL be performed using the Service Location Protocol (SLPv2) [17]. It is outside the scope of this specification to describe any static configuration method for participating FCIP Entity discovery. Refer to section 8.1.2.2 for a detailed description of dynamic discovery of participating FCIP Entities using SLPv2.
10) Before creating a TCP Connection to a peer FCIP Entity, the FCIP
Entity attempting to create the TCP connection SHALL statically or
dynamically determine the IP address, TCP port, expected FC Fabric
Entity World Wide Name, TCP Connection Parameters, and Quality of
Service Information.
11) FCIP Entities do not actively participate in the discovery of FC
source and destination identifiers. Discovery of FC addresses
(accessible via the FCIP Entity) is provided by techniques and
protocols within the FC architecture as described in FC-FS [5] and
FC-SW-2 [4].
12) To support IP Network security (see section 9), FCIP Entities
MUST:
1) implement cryptographically protected authentication and
cryptographic data integrity keyed to the authentication
process, and
2) implement data confidentiality security features.
13) On an individual TCP Connection, this specification relies on
TCP/IP to deliver a byte stream in the same order that it was
sent.
14) This specification assumes the presence of and requires the use of
TCP and FC data loss and corruption mechanisms. The error
detection and recovery features described in this specification
complement and support these existing mechanisms.
5. The FCIP Model
5.1. FCIP Protocol Model
The relationship between FCIP and other protocols is illustrated in figure 1. +------------------------+ FCIP Link +------------------------+ | FCIP |===========| FCIP | +--------+------+--------+ +--------+------+--------+ | FC-2 | | TCP | | TCP | | FC-2 | +--------+ +--------+ +--------+ +--------+ | FC-1 | | IP | | IP | | FC-1 | +--------+ +--------+ +--------+ +--------+ | FC-0 | | LINK | | LINK | | FC-0 | +--------+ +--------+ +--------+ +--------+ | | PHY | | PHY | | | +--------+ +--------+ | | | | | | | IP Network | | V +--------------------+ V to Fibre to Fibre Channel Channel Fabric Fabric Key: FC-0 - Fibre Channel Physical Media Layer FC-1 - Fibre Channel Encode and Decode Layer FC-2 - Fibre Channel Framing and Flow Control Layer TCP - Transmission Control Protocol IP - Internet Protocol LINK - IP Link Layer PHY - IP Physical Layer Figure 1: FCIP Protocol Stack Model Note that the objective of the FCIP Protocol is to create and maintain one or more FCIP Links to transport data.
5.2. FCIP Link
The FCIP Link is the basic unit of service provided by the FCIP Protocol to an FC Fabric. As shown in figure 2, an FCIP Link connects two portions of an FC Fabric using an IP Network as a transport to form a single FC Fabric. /\/\/\/\/\/\ /\/\/\/\/\/\ /\/\/\/\/\/\ \ FC / \ IP / \ FC / / Fabric \=========/ Network \=========/ Fabric \ \/\/\/\/\/\/ \/\/\/\/\/\/ \/\/\/\/\/\/ | | |<--------- FCIP Link -------->| Figure: 2 FCIP Link Model At the points where the ends of the FCIP Link meet portions of the FC Fabric, an FCIP Entity (see section 5.4) combines with an FC Entity as described in section 5.3 to serve as the interface between FC and IP. An FCIP Link SHALL contain at least one TCP Connection and MAY contain more than one TCP Connection. The endpoints of a single TCP Connection are FCIP Data Engines (see section 5.6). The endpoints of a single FCIP Link are FCIP Link Endpoints (see section 5.5).
5.3. FC Entity
An implementation that tunnels an FC Fabric through an IP Network MUST combine an FC Entity with an FCIP Entity (see section 5.4) to form a complete interface between the FC Fabric and IP Network as shown in figure 3. An FC Fabric Entity may contain multiple instances of the FC/FCIP Entity pair shown on either the right-hand or left-hand side of figure 3. |<--------- FCIP Link -------->| | | +----------+ /\/\/\/\/\/\ +----------+ | FCIP | \ IP / | FCIP | | Entity |=========/ Network \=========| Entity | +----------+ \/\/\/\/\/\/ +----------+ | FC | | FC | | Entity | | Entity | +----------+ +----------+ | | /\/\/\/\/\/\ /\/\/\/\/\/\ \ FC / \ FC / / Fabric \ / Fabric \ \/\/\/\/\/\/ \/\/\/\/\/\/ Figure 3: Model for Two Connected FC/FCIP Entity Pairs In general, the combination of an FCIP Link and two FC/FCIP Entity pairs is intended to provide a non-Fibre Channel backbone transport between Fibre Channel components. For example, this combination can be used to function as the hard-wire connection between two Fibre Channel switches. The interface between the FC and FCIP Entities is implementation specific. The functional requirements placed on an FC Entity by this specification are listed in appendix H. More information about FC Entities can be found in the Fibre Channel standards and an example of an FC Entity can be found in FC-BB-2 [3].
5.4. FCIP Entity
The model for an FCIP Entity is shown in figure 4. ....................................................... : FCIP Entity : : : : +-----------+ : : | FCIP | : : |Control and|------------------------------------+ : : | Services | | : : | Module | | : : +-----------+ | : : | +--------------------+ | : : | +-------+--------------------+|----+ | : : | |+-----+--------------------+|----+| | : : | ||+----| FCIP Link Endpoint |----+|| | : : | ||| +--------------------+ ||| | : :.............................................|||.....: | ||| ||| | | ||| ||| o<--+ | ||| unique TCP ||| | | | ||| connections-->||| | | | ||| ||| | | +----------+ /\/\/\/\/\/\ | | FC | \ IP / | | Entity | / Network \ | +----------+ \/\/\/\/\/\/ | | | /\/\/\/\/\/\ +------------------+ \ FC / +->TCP port for / Fabric \ incoming \/\/\/\/\/\/ connections Figure 4: FCIP Entity Model The FCIP Entity receives TCP connect requests on behalf of the FCIP_LEPs that it manages. In support of this, the FCIP Entity is the sole owner of at least one TCP port/IP Address combination used to form TCP Connections. The TCP port may be the FCIP well known port at a given IP Address. An FC Fabric to IP Network interface product SHALL provide each FC/FCIP Entity pair contained in the product with a unique combination of FC Fabric Entity World Wide Identifier and FC/FCIP Entity Identifier values (see section 7). An FCIP Entity contains an FCIP Control and Services Module to control FCIP link initialization, FCIP link dissolution, and to provide the FC Entity with an interface to key IP Network features.
The interfaces to the IP Network features are implementation specific, however, REQUIRED TCP/IP functional support is specified in this document, including: - TCP Connections - see section 8 - Security - see section 9 - Performance - see section 10 - Dynamic Discovery - see section 8.1.2.2 The FCIP Link Endpoints in an FCIP Entity provide the FC Frame encapsulation and transmission features of FCIP.5.5. FCIP Link Endpoint (FCIP_LEP)
As shown in figure 5, the FCIP Link Endpoint contains one FCIP Data Engine for each TCP Connection in the FCIP Link. ................................................ : FCIP Link Endpoint : : +------------------+ : : +-------+------------------+|----+ : : |+-----+------------------+|----+| : : ||+----| FCIP Data Engine |----+|| : : ||| +------------------+ ||| : :..............................................: ||| ||| +----------+ /\/\/\/\/\/\ | FC | \ IP / | Entity | / Network \ +----------+ \/\/\/\/\/\/ | /\/\/\/\/\/\ \ FC / / Fabric \ \/\/\/\/\/\/ Figure 5: FCIP Link Endpoint Model Each time a TCP Connection is formed with a new FC/FCIP Entity pair (including all the actions described in section 8.1), the FCIP Entity SHALL create a new FCIP Link Endpoint containing one FCIP Data Engine. An FCIP_LEP is a transparent data translation point between an FC Entity and an IP Network. A pair of FCIP_LEPs communicating over one or more TCP Connections create an FCIP Link to join two islands of an FC Fabric, producing a single FC Fabric.
The IP Network over which the two FCIP_LEPs communicate is not aware of the FC payloads that it is carrying. Likewise, the FC End Nodes connected to the FC Fabric are unaware of the TCP/IP based transport employed in the structure of the FC Fabric. An FCIP_LEP uses normal TCP based flow control mechanisms for managing its internal resources and matching them with the advertised TCP Receiver Window Size (see sections 8.3.2, 8.5). An FCIP_LEP MAY communicate with its local FC Entity counterpart to coordinate flow control.5.6. FCIP Data Engine (FCIP_DE)
The model for one of the multiple FCIP_DEs that MAY be present in an FCIP_LEP is shown in figure 6. +--------------------------------+ | | F |-+ +------------------+ +-| C |p| | Encapsulation | |p| N -->|1|--->| Engine |--->|2|--> e E |-+ +------------------+ +-| t n | | I w t |-+ +------------------+ +-| P o i |p| | De-Encapsulation | |p| r t <--|4|<---| Engine |<---|3|<-- k y |-+ +------------------+ +-| | | +--------------------------------+ Figure 6: FCIP Data Engine Model Data enters and leaves the FCIP_DE through four portals (p1 - p4). The portals do not process or examine the data that passes through them. They are only the named access points where the FCIP_DE interfaces with the external world. The names of the portals are as follows: p1) FC Frame Receiver Portal - The interface through which an FC Frame and time stamp enters an FCIP_DE from the FC Entity. p2) Encapsulated Frame Transmitter Portal - The TCP interface through which an FCIP Frame is transmitted to the IP Network by an FCIP_DE. p3) Encapsulated Frame Receiver Portal - The TCP interface through which an FCIP Frame is received from the IP Network by an FCIP_DE.
p4) FC Frame Transmitter Portal - The interface through which a
reconstituted FC Frame and time stamp exits an FCIP_DE to the FC
Entity.
The work of the FCIP_DE is done by the Encapsulation and De-
Encapsulation Engines. The Engines have two functions:
1) Encapsulating and de-encapsulating FC Frames using the
encapsulation format described in FC Frame Encapsulation [19] and
in section 5.6.1 of this document, and
2) Detecting some data transmission errors and performing minimal
error recovery as described in section 5.6.2.
Data flows through a pair of IP Network connected FCIP_DEs in the
following seven steps:
1) An FC Frame and time stamp arrives at the FC Frame Receiver Portal
and is passed to the Encapsulation Engine. The FC Frame is
assumed to have been processed by the FC Entity according to the
applicable FC rules and is not validated by the FCIP_DE. If the
FC Entity is in the Unsynchronized state with respect to a time
base as described in the FC Frame Encapsulation [19]
specification, the time stamp delivered with the FC Frame SHALL be
zero.
2) In the Encapsulation Engine, the encapsulation format described in
FC Frame Encapsulation [19] and in section 5.6.1 of this document
SHALL be applied to prepare the FC Frame and associated time stamp
for transmission over the IP Network.
3) The entire encapsulated FC Frame (a.k.a. the FCIP Frame) SHALL be
passed to the Encapsulated Frame Transmitter Portal where it SHALL
be inserted in the TCP byte stream.
4) Transmission of the FCIP Frame over the IP Network follows all the
TCP rules of operation. This includes, but is not limited to, the
in-order delivery of bytes in the stream, as specified by TCP [6].
5) The FCIP Frame arrives at the partner FCIP Entity where it enters
the FCIP_DE through the Encapsulated Frame Receiver Portal and is
passed to the De-Encapsulation Engine for processing.
6) The De-Encapsulation Engine SHALL validate the incoming TCP byte
stream as described in section 5.6.2.2 and SHALL de-encapsulate
the FC Frame and associated time stamp according to the
encapsulation format described in FC Frame Encapsulation [19] and
in section 5.6.1 of this document.
7) In the absence of errors, the de-encapsulated FC Frame and time
stamp SHALL be passed to the FC Frame Transmitter Portal for
delivery to the FC Entity. Error handling is discussed in section
5.6.2.2.
Every FC Frame that arrives at the FC Frame Receiver Portal SHALL be
transmitted on the IP Network as described in steps 1 through 4
above. In the absence of errors, data bytes arriving at the
Encapsulated Frame Receiver Portal SHALL be de-encapsulated and
forwarded to the FC Frame Transmitter Portal as described in steps 5
through 7.
5.6.1. FCIP Encapsulation of FC Frames
The FCIP encapsulation of FC Frames employs FC Frame Encapsulation
[19].
The features from FC Frame Encapsulation that are unique to
individual protocols SHALL be applied as follows for the FCIP
encapsulation of FC Frames.
The Protocol# field SHALL contain 1 in accordance with the IANA
Considerations annex of FC Frame Encapsulation [19].
The Protocol Specific field SHALL have the format shown in figure 7.
Note: the word numbers in figure 7 are relative to the complete FC
Frame Encapsulation header, not to the Protocol Specific field.
W|------------------------------Bit------------------------------|
o| |
r| 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3|
d|0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1|
+---------------------------------------------------------------+
1| replication of encapsulation word 0 |
+---------------+---------------+---------------+---------------+
2| pFlags | Reserved | -pFlags | -Reserved |
+---------------+---------------+---------------+---------------+
Figure 7: FCIP Usage of FC Frame Encapsulation Protocol Specific
field
Word 1 of the Protocol Specific field SHALL contain an exact copy of
word 0 in FC Frame Encapsulation [19].
The pFlags (protocol specific flags) field provides information about
the protocol specific usage of the FC Encapsulation Header. Figure 8
shows the defined pFlags bits.
|----------------Bit--------------------| | | | 0 1 2 3 4 5 6 7 | +----+-----------------------------+----+ | Ch | Reserved | SF | +----+-----------------------------+----+ Figure 8: pFlags Field Bits The SF (Special Frame) bit indicates whether the FCIP Frame is an encapsulated FC Frame or an FSF (FCIP Special Frame, see section 7). When the FCIP Frame contains an encapsulated FC Frame, the SF bit SHALL be 0. When the FCIP Frame is an FSF, the SF bit SHALL be 1. The FSF SHALL only be sent as the first bytes transmitted in each direction on a newly formed TCP Connection and only one FSF SHALL be transmitted in each direction at that time (see section 8.1). After that all FCIP Frames SHALL have the SF bit set to 0. The Ch (Changed) bit indicates whether an echoed FSF has been intentionally altered (see section 8.1.3). The Ch bit SHALL be 0 unless the FSF bit is 1. When the initial TCP Connection FSF is sent, the Ch bit SHALL be 0. If the recipient of a TCP connect request echoes the FSF without any changes, then the Ch bit SHALL continue to be 0. If the recipient of a TCP connect request alters the FSF before echoing it, then the Ch bit SHALL be changed to 1. The -pFlags field SHALL contain the ones complement of the contents of the pFlags field.
Table 1 summarizes the usage of the pFlags SF and Ch bits. +----+----+------------+--------------------------------------+ | | | Originated | | | SF | Ch | or Echoed | Validity/Description | +----+----+------------+--------------------------------------+ | 0 | 0 | n/a | Encapsulated FC Frame | +----+----+------------+--------------------------------------+ | 0 | 1 | n/a | Always Illegal | +----+----+------------+--------------------------------------+ | 1 | 0 | Originated | Originated FSF | +----+----+------------+--------------------------------------+ | 1 | 1 | Originated | Always Illegal | +----+----+------------+--------------------------------------+ | 1 | 0 | Echoed | Echoed FSF without changes | +----+----+------------+--------------------------------------+ | 1 | 1 | Echoed | Echoed FSF with changes | +----+----+------------+--------------------------------------+ | Note 1: Echoed FSFs may contain changes resulting from | | transmission errors, necessitating the comparison between | | sent and received FSF bytes by the FSF originator described | | in section 8.1.2.3. | | | | Note 2: Column positions in this table do not reflect the | | bit positions of the SF and Ch bits in the pFlags field. | +-------------------------------------------------------------+ Table 1: pFlags SF and Ch bit usage summary The Reserved pFlags bits SHALL be 0. The Reserved field (bits 23-16 in word 2): SHALL contain 0. The -Reserved field (bits 7-0 in word 2): SHALL contain 255 (or 0xFF). The CRCV (CRC Valid) Flag SHALL be set to 0. The CRC field SHALL be set to 0. In FCIP, the SOF and EOF codes listed as Class 2, Class 3, and Class 4 in the FC Frame Encapsulation [19] are legal.
5.6.2. FCIP Data Engine Error Detection and Recovery
5.6.2.1. TCP Assistance With Error Detection and Recovery
TCP [6] requires in order delivery, generation of TCP checksums, and checking of TCP checksums. Thus, the byte stream passed from TCP to the FCIP_LEP will be in order and free of errors detectable by the TCP checksum. The FCIP_LEP relies on TCP to perform these functions.5.6.2.2. Errors in FCIP Headers and Discarding FCIP Frames
Bytes delivered through the Encapsulated Frame Receiver Portal that are not correctly delimited as defined by the FC Frame Encapsulation [19] are considered to be in error. The failure of the Protocol# and Version fields in the FCIP Frame header to contain the values defined for an FCIP Frame SHALL be considered an error. Further, some errors in the encapsulation will result in the FCIP_DE losing synchronization with the FC Frames in the byte stream entering through the Encapsulated Frame Receiver Portal. The Frame Length field in the FC Frame Encapsulation header is used to determine where in the data stream the next FC Encapsulated Header is located. The following tests SHALL be performed to verify synchronization with the byte stream entering the Encapsulated Frame Receiver Portal, and synchronization SHALL be considered lost if any of the tests fail: 1) Frame Length field validation -- 15 < Frame Length < 545; 2) Comparison of Frame Length field to its ones complement; and 3) A valid EOF is found in the word preceding the start of the next FCIP header as indicated by the Frame Length field, to be tested as follows: 1) Bits 24-31 and 16-23 contain identical legal EOF values (the list of legal EOF values is in the FC Frame Encapsulation [19]); and 2) Bits 8-15 and 0-7 contain the ones complement of the EOF value found in bits 24-31. Note: The range of valid Frame Length values is derived as follows. The FCIP Frame header is seven words, one word each is required for the encoded SOF and EOF values, the FC Frame header is six words, and
the FC CRC requires one word, yielding a base Frame Length of 16 (7+1+1+6+1) words, if no FC Payload is present. Since the FC Payload is optional, any Frame Length value greater than 15 is valid. The maximum FC Payload size is 528 words, meaning that any Frame Length value up to and including 544 (528+16) is valid. If synchronization is lost, the FC Frame SHALL NOT be forwarded on to the FC Entity and further recovery SHALL be handled as defined by section 5.6.2.3. In addition to the tests above, the validity and positioning of the following FCIP Frame information SHOULD be used to detect encapsulation errors that may or may not affect synchronization: a) Protocol# ones complement field (1 test); b) Version ones complement field (1 test); c) Replication of encapsulation word 0 in word 1 (1 test); d) Reserved field and its ones complement (2 tests); e) Flags field and its ones complement (2 tests); f) CRC field is equal to zero (1 test); g) SOF fields and ones complement fields (4 tests); h) Format and values of FC header (1 test); i) CRC of FC Frame (2 tests); j) FC Frame Encapsulation header information in the next FCIP Frame (1 test). At least 3 of the 16 tests listed above SHALL be performed. Failure of any of the above tests actually performed SHALL indicate an encapsulation error and the FC Frame SHALL NOT be forwarded on to the FC Entity. Further, such errors SHOULD be considered carefully, since some may be synchronization errors. Whenever an FCIP_DE discards bytes delivered through the Encapsulated Frame Receiver Portal, it SHALL cause the FCIP Entity to notify the FC Entity of the condition and provide a suitable description of the reason bytes were discarded. The burden for recovering from discarded data falls on the FC Entity and other components of the FC Fabric, and is outside the scope of this specification.
5.6.2.3. Synchronization Failures
If an FCIP_DE determines that it cannot find the next FCIP Frame header in the byte stream entering through the Encapsulated Frame Receiver Portal, the FCIP_DE SHALL do one of the following: a) close the TCP Connection [6] [7] and notify the FC Entity with the reason for the closure; b) recover synchronization by searching the bytes delivered by the Encapsulated Frame Receiver Portal for a valid FCIP Frame header having the correct properties (see section 5.6.2.2), and discarding bytes delivered by the Encapsulated Frame Receiver Portal until a valid FCIP Frame header is found; or c) attempt to recover synchronization as described in b) and if synchronization cannot be recovered, close the TCP Connection as described in a), including notification of the FC Entity with the reason for the closure. If the FCIP_DE attempts to recover synchronization, the resynchronization algorithm used SHALL meet the following requirements: a) discard or identify with an EOFa (see appendix section F.1) those FC Frames and fragments of FC Frames identified before synchronization has again been completely verified. The number of FC Frames not forwarded may vary based on the algorithm used; b) return to forwarding FC Frames through the FC Frame Transmitter Portal only after synchronization on the transmitted FCIP Frame stream has been verified; and c) close the TCP/IP connection if the algorithm ends without verifying successful synchronization. The probability of failing to synchronize successfully and the time necessary to determine whether or not synchronization was successful may vary with the algorithm used. An example algorithm meeting these requirements can be found in appendix D. The burden for recovering from the discarding of FCIP Frames during the optional resynchronization process described in this section falls on the FC Entity and other components of the FC Fabric, and is outside the scope of this specification.