RFC 5042
Direct Data Placement Protocol (DDP) / Remote Direct Memory Access Protocol (RDMAP) Security
Network Working Group J. Pinkerton Request for Comments: 5042 Microsoft Corporation Category: Standards Track E. Deleganes Self October 2007 Direct Data Placement Protocol (DDP) / Remote Direct Memory Access Protocol (RDMAP) Security Status of This Memo This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. Please refer to the current edition of the "Internet Official Protocol Standards" (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited.Abstract
This document analyzes security issues around implementation and use of the Direct Data Placement Protocol (DDP) and Remote Direct Memory Access Protocol (RDMAP). It first defines an architectural model for an RDMA Network Interface Card (RNIC), which can implement DDP or RDMAP and DDP. The document reviews various attacks against the resources defined in the architectural model and the countermeasures that can be used to protect the system. Attacks are grouped into those that can be mitigated by using secure communication channels across the network, attacks from Remote Peers, and attacks from Local Peers. Attack categories include spoofing, tampering, information disclosure, denial of service, and elevation of privilege.
Table of Contents
1. Introduction ....................................................4 2. Architectural Model .............................................6 2.1. Components .................................................7 2.2. Resources ..................................................9 2.2.1. Stream Context Memory ...............................9 2.2.2. Data Buffers .......................................10 2.2.3. Page Translation Tables ............................10 2.2.4. Protection Domain (PD) .............................11 2.2.5. STag Namespace and Scope ...........................11 2.2.6. Completion Queues ..................................12 2.2.7. Asynchronous Event Queue ...........................12 2.2.8. RDMA Read Request Queue ............................13 2.3. RNIC Interactions .........................................13 2.3.1. Privileged Control Interface Semantics .............13 2.3.2. Non-Privileged Data Interface Semantics ............13 2.3.3. Privileged Data Interface Semantics ................14 2.3.4. Initialization of RNIC Data Structures for Data Transfer ......................................14 2.3.5. RNIC Data Transfer Interactions ....................16 3. Trust and Resource Sharing .....................................17 4. Attacker Capabilities ..........................................18 5. Attacks That Can Be Mitigated with End-to-End Security .........18 5.1. Spoofing ..................................................19 5.1.1. Impersonation ......................................19 5.1.2. Stream Hijacking ...................................20 5.1.3. Man-in-the-Middle Attack ...........................20 5.2. Tampering - Network-Based Modification of Buffer Content ..21 5.3. Information Disclosure - Network-Based Eavesdropping ......21 5.4. Specific Requirements for Security Services ...............21 5.4.1. Introduction to Security Options ...................21 5.4.2. TLS Is Inappropriate for DDP/RDMAP Security ........22 5.4.3. DTLS and RDDP ......................................23 5.4.4. ULPs That Provide Security .........................23 5.4.5. Requirements for IPsec Encapsulation of DDP ........23 6. Attacks from Remote Peers ......................................24 6.1. Spoofing ..................................................25 6.1.1. Using an STag on a Different Stream ................25 6.2. Tampering .................................................26 6.2.1. Buffer Overrun - RDMA Write or Read Response .......26 6.2.2. Modifying a Buffer after Indication ................27 6.2.3. Multiple STags to Access the Same Buffer ...........27 6.3. Information Disclosure ....................................28 6.3.1. Probing Memory Outside of the Buffer Bounds ........28 6.3.2. Using RDMA Read to Access Stale Data ...............28 6.3.3. Accessing a Buffer after the Transfer ..............28 6.3.4. Accessing Unintended Data with a Valid STag ........29
6.3.5. RDMA Read into an RDMA Write Buffer ................29
6.3.6. Using Multiple STags That Alias to the Same
Buffer .............................................29
6.4. Denial of Service (DOS) ...................................30
6.4.1. RNIC Resource Consumption ..........................30
6.4.2. Resource Consumption by Idle ULPs ..................31
6.4.3. Resource Consumption by Active ULPs ................32
6.4.3.1. Multiple Streams Sharing Receive Buffers ..32
6.4.3.2. Remote or Local Peer Attacking a
Shared CQ .................................34
6.4.3.3. Attacking the RDMA Read Request Queue .....36
6.4.4. Exercise of Non-Optimal Code Paths .................37
6.4.5. Remote Invalidate an STag Shared on
Multiple Streams ...................................37
6.4.6. Remote Peer Attacking an Unshared CQ ...............38
6.5. Elevation of Privilege ....................................38
7. Attacks from Local Peers .......................................38
7.1. Local ULP Attacking a Shared CQ ...........................39
7.2. Local Peer Attacking the RDMA Read Request Queue ..........39
7.3. Local ULP Attacking the PTT and STag Mapping ..............39
8. Security considerations ........................................40
9. IANA Considerations ............................................40
10. References ....................................................40
10.1. Normative References .....................................40
10.2. Informative References ...................................41
Appendix A. ULP Issues for RDDP Client/Server Protocols ...........43
Appendix B. Summary of RNIC and ULP Implementation Requirements ...46
Appendix C. Partial Trust Taxonomy ................................47
Acknowledgments ...................................................49
1. Introduction
RDMA enables new levels of flexibility when communicating between two parties compared to current conventional networking practice (e.g., a stream-based model or datagram model). This flexibility brings new security issues that must be carefully understood when designing Upper Layer Protocols (ULPs) utilizing RDMA and when implementing RDMA-aware NICs (RNICs). Note that for the purposes of this security analysis, an RNIC may implement RDMAP [RDMAP] and DDP [DDP], or just DDP. Also, a ULP may be an application or it may be a middleware library. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. Additionally, the security terminology defined in [RFC4949] is used in this specification. The document first develops an architectural model that is relevant for the security analysis. Section 2 details components, resources, and system properties that may be attacked. The document uses Local Peer to represent the RDMA/DDP protocol implementation on the local end of a Stream (implemented with a transport protocol, such as [RFC793] or [RFC4960]). The local Upper-Layer-Protocol (ULP) is used to represent the application or middle-ware layer above the Local Peer. The document does not attempt to differentiate between a Remote Peer and a Remote ULP (an RDMA/DDP protocol implementation on the remote end of a Stream versus the application on the remote end) for several reasons: often, the source of the attack is difficult to know for sure and, regardless of the source, the mitigations required of the Local Peer or local ULP are the same. Thus, the document generically refers to a Remote Peer rather than trying to further delineate the attacker. The document then defines what resources a local ULP may share across Streams and what resources the local ULP may share with the Remote Peer across Streams in Section 3. Intentional sharing of resources between multiple Streams may imply some level of trust between the Streams. However, some types of resource sharing have unmitigated security attacks, which would mandate not sharing a specific type of resource unless there is some level of trust between the Streams sharing resources.
This document defines a new term, "Partial Mutual Trust", to address
this concept:
Partial Mutual Trust - a collection of RDMAP/DDP Streams, which
represent the local and remote end points of the Stream that are
willing to assume that the Streams from the collection will not
perform malicious attacks against any of the other Streams in the
collection.
ULPs have explicit control of which collection of endpoints is in a
Partial Mutual Trust collection through tools discussed in Appendix
C, Partial Trust Taxonomy.
An untrusted peer relationship is appropriate when a ULP wishes to
ensure that it will be robust and uncompromised even in the face of a
deliberate attack by its peer. For example, a single ULP that
concurrently supports multiple unrelated Streams (e.g., a server)
would presumably treat each of its peers as an untrusted peer. For a
collection of Streams that share Partial Mutual Trust, the assumption
is that any Stream not in the collection is untrusted. For the
untrusted peer, a brief list of capabilities is enumerated in Section
4.
The rest of the document is focused on analyzing attacks and
recommending specific mitigations to the attacks. Attacks are
categorized into attacks mitigated by end-to-end security, attacks
initiated by Remote Peers, and attacks initiated by Local Peers. For
each attack, possible countermeasures are reviewed.
ULPs within a host are divided into two categories - Privileged and
Non-Privileged. Both ULP types can send and receive data and request
resources. The key differences between the two are:
The Privileged ULP is trusted by the local system not to
maliciously attack the operating environment, but it is not
trusted to optimize resource allocation globally. For example,
the Privileged ULP could be a kernel ULP; thus, the kernel
presumably has in some way vetted the ULP before allowing it to
execute.
A Non-Privileged ULP's capabilities are a logical sub-set of the
Privileged ULP's. It is assumed by the local system that a Non-
Privileged ULP is untrusted. All Non-Privileged ULP interactions
with the RNIC Engine that could affect other ULPs need to be done
through a trusted intermediary that can verify the Non-Privileged
ULP requests.
The appendices provide focused summaries of this specification. Appendix A, ULP Issues for RDDP Client/Server Protocols, focuses on implementers of traditional client/server protocols. Appendix B, Summary of RNIC and ULP Implementation Requirements, summarizes all normative requirements in this specification. Appendix C, Partial Trust Taxonomy, provides an abstract model for categorizing trust boundaries. If an RDMAP/DDP protocol implementation uses the mitigations recommended in this document, that implementation should not exhibit additional security vulnerabilities above and beyond those of an implementation of the transport protocol (i.e., TCP or SCTP) and protocols beneath it (e.g., IP) without RDMAP/DDP.2. Architectural Model
This section describes an RDMA architectural reference model that is used as security issues are examined. It introduces the components of the model, the resources that can be attacked, the types of interactions possible between components and resources, and the system properties that must be preserved. Figure 1 shows the components comprising the architecture and the interfaces where potential security attacks could be launched. External attacks can be injected into the system from a ULP that sits above the RNIC Interface or from the network. The intent here is to describe high level components and capabilities that affect threat analysis, and not focus on specific implementation options. Also note that the architectural model is an abstraction, and an actual implementation may choose to subdivide its components along different boundary lines from those defined here. For example, the Privileged Resource Manager may be partially or completely encapsulated in the Privileged ULP. Regardless, it is expected that the security analysis of the potential threats and countermeasures still apply. Note that the model below is derived from several specific RDMA implementations. A few of note are [VERBS-RDMAC], [VERBS-RDMAC- Overview], and [INFINIBAND].
+-------------+ | Privileged | | Resource | Admin<-+>| Manager | ULP Control Interface | | |<------+-------------------+ | +-------------+ | | | ^ v v | | +-------------+ +-----------------+ +---------------->| Privileged | | Non-Privileged | | | ULP | | ULP | | +-------------+ +-----------------+ | ^ ^ |Privileged |Privileged |Non-Privileged |Control |Data |Data |Interface |Interface |Interface RNIC | | | Interface v v v ================================================================= +--------------------------------------+ | | | RNIC Engine | | | +--------------------------------------+ ^ | v Internet Figure 1 - RDMA Security Model2.1. Components
The components shown in Figure 1 - RDMA Security Model are: * RDMA Network Interface Controller Engine (RNIC) - The component that implements the RDMA protocol and/or DDP protocol. * Privileged Resource Manager - The component responsible for managing and allocating resources associated with the RNIC Engine. The Resource Manager does not send or receive data. Note that whether the Resource Manager is an independent component, part of the RNIC, or part of the ULP is implementation dependent.
* Privileged ULP - See Section 1, Introduction, for a definition of Privileged ULP. The local host infrastructure can enable the Privileged ULP to map a Data Buffer directly from the RNIC Engine to the host through the RNIC Interface, but it does not allow the Privileged ULP to directly consume RNIC Engine resources. * Non-Privileged ULP - See Section 1, Introduction, for a definition of Non-Privileged ULP. A design goal of the DDP and RDMAP protocols is to allow, under constrained conditions, Non-Privileged ULP to send and receive data directly to/from the RDMA Engine without Privileged Resource Manager intervention, while ensuring that the host remains secure. Thus, one of the primary goals of this document is to analyze this usage model for the enforcement that is required in the RNIC Engine to ensure that the system remains secure. DDP provides two mechanisms for transferring data: * Untagged Data Transfer - The incoming payload simply consumes the first buffer in a queue of buffers that are in the order specified by the receiving Peer (commonly referred to as the Receive Queue), and * Tagged Data Transfer - The Peer transmitting the payload explicitly states which destination buffer is targeted, through use of an STag. STag-based transfers allow the receiving ULP to be indifferent to what order (or in what messages) the opposite Peer sent the data, or in what order packets are received. Both data transfer mechanisms are also enabled through RDMAP, with additional control semantics. Typically, Tagged Data Transfer can be used for payload transfer, while Untagged Data Transfer is best used for control messages. However, each Upper Layer Protocol can determine the optimal use of Tagged and Untagged messages for itself. See [APPLICABILITY] for more information on application applicability for the two transfer mechanisms. For DDP, the two forms correspond to Untagged and Tagged DDP Messages, respectively. For RDMAP, the two forms correspond to Send Type Messages and RDMA Messages (either RDMA Read or RDMA Write Messages), respectively.
The host interfaces that could be exercised include:
* Privileged Control Interface - A Privileged Resource Manager uses
the RNIC Interface to allocate and manage RNIC Engine resources,
control the state within the RNIC Engine, and monitor various
events from the RNIC Engine. It also uses this interface to act
as a proxy for some operations that a Non-Privileged ULP may
require (after performing appropriate countermeasures).
* ULP Control Interface - A ULP uses this interface to the
Privileged Resource Manager to allocate RNIC Engine resources.
The Privileged Resource Manager implements countermeasures to
ensure that, if the Non-Privileged ULP launches an attack, it can
prevent the attack from affecting other ULPs.
* Non-Privileged Data Transfer Interface - A Non-Privileged ULP
uses this interface to initiate and check the status of data
transfer operations.
* Privileged Data Transfer Interface - A superset of the
functionality provided by the Non-Privileged Data Transfer
Interface. The ULP is allowed to directly manipulate RNIC Engine
mapping resources to map an STag to a ULP Data Buffer.
If Internet control messages, such as ICMP, ARP, RIPv4, etc. are
processed by the RNIC Engine, the threat analyses for those protocols
is also applicable, but outside the scope of this document.
2.2. Resources
This section describes the primary resources in the RNIC Engine that
could be affected if under attack. For RDMAP, all the defined
resources apply. For DDP, all the resources except the RDMA Read
Queue apply.
2.2.1. Stream Context Memory
The state information for each Stream is maintained in memory, which
could be located in a number of places - on the NIC, inside RAM
attached to the NIC, in host memory, or in any combination of the
three, depending on the implementation.
Stream Context Memory includes state associated with Data Buffers.
For Tagged Buffers, this includes how STag names, Data Buffers, and
Page Translation Tables (see Section 2.2.3) interrelate. It also
includes the list of Untagged Data Buffers posted for reception of
Untagged Messages (commonly called the Receive Queue), and a list of
operations to perform to send data (commonly called the Send Queue).
2.2.2. Data Buffers
As mentioned previously, there are two different ways to expose a local ULP's Data Buffers for data transfer: Untagged Data Transfer, where a buffer can be exposed for receiving RDMAP Send Type Messages (a.k.a. DDP Untagged Messages) on DDP Queue zero, or Tagged Data Transfer, where the buffer can be exposed for remote access through STags (a.k.a. DDP Tagged Messages). This distinction is important because the attacks and the countermeasures used to protect against the attack are different depending on the method for exposing the buffer to the network. For the purposes of the security discussion, for Tagged Data Transfer, a single logical Data Buffer is exposed with a single STag on a given Stream. Actual implementations may support scatter/gather capabilities to enable multiple physical data buffers to be accessed with a single STag, but from a threat analysis perspective, it is assumed that a single STag enables access to a single logical Data Buffer. In any event, it is the responsibility of the Privileged Resource Manager to ensure that no STag can be created that exposes memory that the consumer had no authority to expose. A Data Buffer has specific access rights. The local ULP can control whether a Data Buffer is exposed for local only, or local and remote access, and assign specific access privileges (read, write, read and write) on a per Stream basis. For DDP, when an STag is Advertised, the Remote Peer is presumably given write access rights to the data (otherwise, there would not be much point to the Advertisement). For RDMAP, when a ULP Advertises an STag, it can enable write-only, read-only, or both write and read access rights. Similarly, some ULPs may wish to provide a single buffer with different access rights on a per Stream basis. For example, some Streams may have read-only access, some may have remote read and write access, while on other Streams, only the local ULP/Local Peer is allowed access.2.2.3. Page Translation Tables
Page Translation Tables are the structures used by the RNIC to be able to access ULP memory for data transfer operations. Even though these structures are called "Page" Translation Tables, they may not reference a page at all - conceptually, they are used to map a ULP address space representation (e.g., a virtual address) of a buffer to
the physical addresses that are used by the RNIC Engine to move data. If, on a specific system, a mapping is not used, then a subset of the attacks examined may be appropriate. Note that the Page Translation Table may or may not be a shared resource.2.2.4. Protection Domain (PD)
A Protection Domain (PD) is a local construct to the RDMA implementation, and never visible over the wire. Protection Domains are assigned to three of the resources of concern - Stream Context Memory, STags associated with Page Translation Table entries, and Data Buffers. A correct implementation of a Protection Domain requires that resources that belong to a given Protection Domain cannot be used on a resource belonging to another Protection Domain, because Protection Domain membership is checked by the RNIC prior to taking any action involving such a resource. Protection Domains are therefore used to ensure that an STag can only be used to access an associated Data Buffer on one or more Streams that are associated with the same Protection Domain as the specific STag. If an implementation chooses not to share resources between Streams, it is recommended that each Stream be associated with its own, unique Protection Domain. If an implementation chooses to allow resource sharing, it is recommended that Protection Domain be limited to the collection of Streams that have Partial Mutual Trust with each other. Note that a ULP (either Privileged or Non-Privileged) can potentially have multiple Protection Domains. This could be used, for example, to ensure that multiple clients of a server do not have the ability to corrupt each other. The server would allocate a Protection Domain per client to ensure that resources covered by the Protection Domain could not be used by another (untrusted) client.2.2.5. STag Namespace and Scope
The DDP specification defines a 32-bit namespace for the STag. Implementations may vary in terms of the actual number of STags that are supported. In any case, this is a bounded resource that can come under attack. Depending upon STag namespace allocation algorithms, the actual name space to attack may be significantly less than 2^32. The scope of an STag is the set of DDP/RDMAP Streams on which the STag is valid. If an STag is valid on a particular DDP/RDMAP Stream, then that stream can modify the buffer, subject to the access rights that the stream has for the STag (see Section 2.2.2, Data Buffers, for additional information).
The analysis presented in this document assumes two mechanisms for
limiting the scope of Streams for which the STag is valid:
* Protection Domain scope. The STag is valid if used on any Stream
within a specific Protection Domain, and is invalid if used on
any Stream that is not a member of the Protection Domain.
* Single Stream scope. The STag is valid on a single Stream,
regardless of what the Stream association is to a Protection
Domain. If used on any other Stream, it is invalid.
2.2.6. Completion Queues
Completion Queues (CQ) are used in this document to conceptually
represent how the RNIC Engine notifies the ULP about the completion
of the transmission of data, or the completion of the reception of
data through the Data Transfer Interface (specifically for Untagged
Data Transfer; Tagged Data Transfer cannot cause a completion to
occur). Because there could be many transmissions or receptions in
flight at any one time, completions are modeled as a queue rather
than as a single event. An implementation may also use the
Completion Queue to notify the ULP of other activities; for example,
the completion of a mapping of an STag to a specific ULP buffer.
Completion Queues may be shared by a group of Streams, or may be
designated to handle a specific Stream's traffic. Limiting
Completion Queue association to one, or a small number, of RDMAP/DDP
Streams can prevent several forms of attacks by sharply limiting the
scope of the attack's effect.
Some implementations may allow this queue to be manipulated directly
by both Non-Privileged and Privileged ULPs.
2.2.7. Asynchronous Event Queue
The Asynchronous Event Queue is a queue from the RNIC to the
Privileged Resource Manager of bounded size. It is used by the RNIC
to notify the host of various events that might require management
action, including protocol violations, Stream state changes, local
operation errors, low water marks on receive queues, and possibly
other events.
The Asynchronous Event Queue is a resource that can be attacked
because Remote or Local Peers and/or ULPs can cause events to occur
that have the potential of overflowing the queue.
Note that an implementation is at liberty to implement the functions
of the Asynchronous Event Queue in a variety of ways, including
multiple queues or even simple callbacks. All vulnerabilities
identified are intended to apply, regardless of the implementation of the Asynchronous Event Queue. For example, a callback function may be viewed simply as a very short queue.2.2.8. RDMA Read Request Queue
The RDMA Read Request Queue is the memory that holds state information for one or more RDMA Read Request Messages that have arrived, but for which the RDMA Read Response Messages have not yet been completely sent. Because potentially more than one RDMA Read Request can be outstanding at one time, the memory is modeled as a queue of bounded size. Some implementations may enable sharing of a single RDMA Read Request Queue across multiple Streams.2.3. RNIC Interactions
With RNIC resources and interfaces defined, it is now possible to examine the interactions supported by the generic RNIC functional interfaces through each of the 3 interfaces: Privileged Control Interface, Privileged Data Interface, and Non-Privileged Data Interface. As mentioned previously in Section 2.1, Components, there are two data transfer mechanisms to be examined, Untagged Data Transfer and Tagged Data Transfer.2.3.1. Privileged Control Interface Semantics
Generically, the Privileged Control Interface controls the RNIC's allocation, de-allocation, and initialization of RNIC global resources. This includes allocation and de-allocation of Stream Context Memory, Page Translation Tables, STag names, Completion Queues, RDMA Read Request Queues, and Asynchronous Event Queues. The Privileged Control Interface is also typically used for managing Non-Privileged ULP resources for the Non-Privileged ULP (and possibly for the Privileged ULP as well). This includes initialization and removal of Page Translation Table resources, and managing RNIC events (possibly managing all events for the Asynchronous Event Queue).2.3.2. Non-Privileged Data Interface Semantics
The Non-Privileged Data Interface enables data transfer (transmit and receive) but does not allow initialization of the Page Translation Table resources. However, once the Page Translation Table resources have been initialized, the interface may enable a specific STag mapping to be enabled and disabled by directly communicating with the RNIC, or create an STag mapping for a buffer that has been previously initialized in the RNIC.
For RDMAP, ULP data can be sent by one of the previously described data transfer mechanisms: Untagged Data Transfer or Tagged Data Transfer. Two RDMAP data transfer mechanisms are defined, one using Untagged Data Transfer (Send Type Messages), and one using Tagged Data Transfer (RDMA Read Responses and RDMA Writes). ULP data reception through RDMAP can be done by receiving Send Type Messages into buffers that have been posted on the Receive Queue or Shared Receive Queue. Thus, a Receive Queue or Shared Receive Queue can only be affected by Untagged Data Transfer. Data reception can also be done by receiving RDMA Write and RDMA Read Response Messages into buffers that have previously been exposed for external write access through Advertisement of an STag (i.e., Tagged Data Transfer). Additionally, to cause ULP data to be pulled (read) across the network, RDMAP uses an RDMA Read Request Message (which only contains RDMAP control information necessary to access the ULP buffer to be read), to cause an RDMA Read Response Message to be generated that contains the ULP data. For DDP, transmitting data means sending DDP Tagged or Untagged Messages. For data reception, DDP can receive Untagged Messages into buffers that have been posted on the Receive Queue or Shared Receive Queue. It can also receive Tagged DDP Messages into buffers that have previously been exposed for external write access through Advertisement of an STag. Completion of data transmission or reception generally entails informing the ULP of the completed work by placing completion information on the Completion Queue. For data reception, only an Untagged Data Transfer can cause completion information to be put in the Completion Queue.2.3.3. Privileged Data Interface Semantics
The Privileged Data Interface semantics are a superset of the Non- Privileged Data Transfer semantics. The interface can do everything defined in the prior section, as well as create/destroy buffer to STag mappings directly. This generally entails initialization or clearing of Page Translation Table state in the RNIC.2.3.4. Initialization of RNIC Data Structures for Data Transfer
Initialization of the mapping between an STag and a Data Buffer can be viewed in the abstract as two separate operations: a. Initialization of the allocated Page Translation Table entries with the location of the Data Buffer, and
b. Initialization of a mapping from an allocated STag name to a set
of Page Translation Table entry(s) or partial entries.
Note that an implementation may not have a Page Translation Table
(i.e., it may support a direct mapping between an STag and a Data
Buffer). If there is no Page Translation Table, then attacks based
on changing its contents or exhausting its resources are not
possible.
Initialization of the contents of the Page Translation Table can be
done by either the Privileged ULP or by the Privileged Resource
Manager as a proxy for the Non-Privileged ULP. By definition, the
Non-Privileged ULP is not trusted to directly manipulate the Page
Translation Table. In general, the concern is that the Non-
Privileged ULP may try to maliciously initialize the Page Translation
Table to access a buffer for which it does not have permission.
The exact resource allocation algorithm for the Page Translation
Table is outside the scope of this document. It may be allocated for
a specific Data Buffer, or as a pooled resource to be consumed by
potentially multiple Data Buffers, or be managed in some other way.
This document attempts to abstract implementation dependent issues,
and group them into higher level security issues, such as resource
starvation and sharing of resources between Streams.
The next issue is how an STag name is associated with a Data Buffer.
For the case of an Untagged Data Buffer (i.e., Untagged Data
Transfer), there is no wire visible mapping between an STag and the
Data Buffer. Note that there may, in fact, be an STag that
represents the buffer, if an implementation chooses to internally
represent Untagged Data Buffer using STags. However, because the
STag, by definition, is not visible on the wire, this is a local
host, implementation-specific issue that should be analyzed in the
context of a local host implementation-specific security analysis,
and thus, is outside the scope of this document.
For a Tagged Data Buffer (i.e., Tagged Data Transfer), either the
Privileged ULP or the Privileged Resource Manager acting on behalf of
the Non-Privileged ULP may initialize a mapping from an STag to a
Page Translation Table, or may have the ability to simply
enable/disable an existing STag to Page Translation Table mapping.
There may also be multiple STag names that map to a specific group of
Page Translation Table entries (or sub-entries). Specific security
issues with this level of flexibility are examined in Section 6.2.3,
Multiple STags to Access the Same Buffer.
There are a variety of implementation options for initialization of Page Translation Table entries and mapping an STag to a group of Page Translation Table entries that have security repercussions. This includes support for separation of mapping an STag versus mapping a set of Page Translation Table entries, and support for ULPs directly manipulating STag to Page Translation Table entry mappings (versus requiring access through the Privileged Resource Manager).2.3.5. RNIC Data Transfer Interactions
RNIC Data Transfer operations can be subdivided into send and receive operations. For send operations, there is typically a queue that enables the ULP to post multiple operation requests to send data (referred to as the Send Queue). Depending upon the implementation, Data Buffers used in the operations may or may not have Page Translation Table entries associated with them, and may or may not have STags associated with them. Because this is a local host specific implementation issue rather than a protocol issue, the security analysis of threats and mitigations is left to the host implementation. Receive operations are different for Tagged Data Buffers versus Untagged Data Buffers (i.e., Tagged Data Transfer vs. Untagged Data Transfer). For Untagged Data Transfer, if more than one Untagged Data Buffer can be posted by the ULP, the DDP specification requires that they be consumed in sequential order (the RDMAP specification also requires this). Thus, the most general implementation is that there is a sequential queue of receive Untagged Data Buffers (Receive Queue). Some implementations may also support sharing of the sequential queue between multiple Streams. In this case, defining "sequential" becomes non-trivial - in general, the buffers for a single Stream are consumed from the queue in the order that they were placed on the queue, but there is no consumption order guarantee between Streams. For receive Tagged Data Transfer (i.e., Tagged Data Buffers, RDMA Write Buffers, or RDMA Read Buffers), at some time prior to data transfer, the mapping of the STag to specific Page Translation Table entries (if present) and the mapping from the Page Translation Table entries to the Data Buffer must have been initialized (see Section 2.3.4 for interaction details).
3. Trust and Resource Sharing
It is assumed that, in general, the Local and Remote Peer are untrusted, and thus attacks by either should have mitigations in place. A separate, but related issue is resource sharing between multiple Streams. If local resources are not shared, the resources are dedicated on a per Stream basis. Resources are defined in Section 2.2, Resources. The advantage of not sharing resources between Streams is that it reduces the types of attacks that are possible. The disadvantage of not sharing resources is that ULPs might run out of resources. Thus, there can be a strong incentive for sharing resources, if the security issues associated with the sharing of resources can be mitigated. It is assumed in this document that the component that implements the mechanism to control sharing of the RNIC Engine resources is the Privileged Resource Manager. The RNIC Engine exposes its resources through the RNIC Interface to the Privileged Resource Manager. All Privileged and Non-Privileged ULPs request resources from the Resource Manager (note that by definition both the Non-Privileged and the Privileged application might try to greedily consume resources, thus creating a potential Denial of Service (DOS) attack). The Resource Manager implements resource management policies to ensure fair access to resources. The Resource Manager should be designed to take into account security attacks detailed in this document. Note that for some systems the Privileged Resource Manager may be implemented within the Privileged ULP. All Non-Privileged ULP interactions with the RNIC Engine that could affect other ULPs MUST be done using the Privileged Resource Manager as a proxy. All ULP resource allocation requests for scarce resources MUST also be done using a Privileged Resource Manager. The sharing of resources across Streams should be under the control of the ULP, both in terms of the trust model the ULP wishes to operate under, as well as the level of resource sharing the ULP wishes to give local processes. For more discussion on types of trust models that combine partial trust and sharing of resources, see Appendix C, Partial Trust Taxonomy. The Privileged Resource Manager MUST NOT assume that different Streams share Partial Mutual Trust unless there is a mechanism to ensure that the Streams do indeed share Partial Mutual Trust. This can be done in several ways, including explicit notification from the ULP that owns the Streams.
4. Attacker Capabilities
An attacker's capabilities delimit the types of attacks that the attacker is able to launch. RDMAP and DDP require that the initial LLP Stream (and connection) be set up prior to transferring RDMAP/DDP Messages. This requires at least one round-trip handshake to occur. If the attacker is not the Remote Peer that created the initial connection, then the attacker's capabilities can be segmented into send only capabilities or send and receive capabilities. Attacking with send only capabilities requires the attacker to first guess the current LLP Stream parameters before they can attack RNIC resources (e.g., TCP sequence number). If this class of attacker also has receive capabilities and the ability to pose as the receiver to the sender and the sender to the receiver, they are typically referred to as a "man-in-the-middle" attacker [RFC3552]. A man-in-the-middle attacker has a much wider ability to attack RNIC resources. The breadth of attack is essentially the same as that of an attacking Remote Peer (i.e., the Remote Peer that set up the initial LLP Stream).
(page 18 continued on part 2)
