Internet Engineering Task Force (IETF)                      S. Kitterman
Request for Comments: 7208                  Kitterman Technical Services
Obsoletes: 4408                                               April 2014
Category: Standards Track
ISSN: 2070-1721


                     Sender Policy Framework (SPF)
           for Authorizing Use of Domains in Email, Version 1

Abstract

   Email on the Internet can be forged in a number of ways.  In
   particular, existing protocols place no restriction on what a sending
   host can use as the "MAIL FROM" of a message or the domain given on
   the SMTP HELO/EHLO commands.  This document describes version 1 of
   the Sender Policy Framework (SPF) protocol, whereby ADministrative
   Management Domains (ADMDs) can explicitly authorize the hosts that
   are allowed to use their domain names, and a receiving host can check
   such authorization.

   This document obsoletes RFC 4408.

Status of This Memo

   This is an Internet Standards Track document.

   This document is a product of the Internet Engineering Task Force
   (IETF).  It represents the consensus of the IETF community.  It has
   received public review and has been approved for publication by the
   Internet Engineering Steering Group (IESG).  Further information on
   Internet Standards is available in Section 2 of RFC 5741.

   Information about the current status of this document, any errata,
   and how to provide feedback on it may be obtained at
   http://www.rfc-editor.org/info/rfc7208.

Copyright Notice

   Copyright (c) 2014 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

   This document may contain material from IETF Documents or IETF
   Contributions published or made publicly available before November
   10, 2008.  The person(s) controlling the copyright in some of this
   material may not have granted the IETF Trust the right to allow
   modifications of such material outside the IETF Standards Process.
   Without obtaining an adequate license from the person(s) controlling
   the copyright in such materials, this document may not be modified
   outside the IETF Standards Process, and derivative works of it may
   not be created outside the IETF Standards Process, except to format
   it for publication as an RFC or to translate it into languages other
   than English.

Table of Contents

   1. Introduction ....................................................5
      1.1. Terminology ................................................5
           1.1.1. Key Words ...........................................5
           1.1.2. Imported Definitions ................................5
           1.1.3. MAIL FROM Definition ................................6
           1.1.4. HELO Definition .....................................6
      1.2. check_host() ...............................................6
   2. Operational Overview ............................................6
      2.1. Publishing Authorization ...................................6
      2.2. Checking Authorization .....................................7
      2.3. The "HELO" Identity ........................................8
      2.4. The "MAIL FROM" Identity ...................................9
      2.5. Location of Checks .........................................9
      2.6. Results of Evaluation ......................................9
           2.6.1. None ...............................................10
           2.6.2. Neutral ............................................10
           2.6.3. Pass ...............................................10
           2.6.4. Fail ...............................................10

           2.6.5. Softfail ...........................................10
           2.6.6. Temperror ..........................................10
           2.6.7. Permerror ..........................................10
   3. SPF Records ....................................................11
      3.1. DNS Resource Records ......................................11
      3.2. Multiple DNS Records ......................................12
      3.3. Multiple Strings in a Single DNS Record ...................12
      3.4. Record Size ...............................................13
      3.5. Wildcard Records ..........................................13
   4. The check_host() Function ......................................14
      4.1. Arguments .................................................14
      4.2. Results ...................................................15
      4.3. Initial Processing ........................................15
      4.4. Record Lookup .............................................15
      4.5. Selecting Records .........................................15
      4.6. Record Evaluation .........................................16
           4.6.1. Term Evaluation ....................................16
           4.6.2. Mechanisms .........................................16
           4.6.3. Modifiers ..........................................17
           4.6.4. DNS Lookup Limits ..................................17
      4.7. Default Result ............................................18
      4.8. Domain Specification ......................................19
   5. Mechanism Definitions ..........................................20
      5.1. "all" .....................................................21
      5.2. "include" .................................................21
      5.3. "a" .......................................................23
      5.4. "mx" ......................................................23
      5.5. "ptr" (do not use) ........................................23
      5.6. "ip4" and "ip6" ...........................................25
      5.7. "exists" ..................................................25
   6. Modifier Definitions ...........................................26
      6.1. redirect: Redirected Query ................................26
      6.2. exp: Explanation ..........................................27
   7. Macros .........................................................28
      7.1. Formal Specification ......................................29
      7.2. Macro Definitions .........................................29
      7.3. Macro Processing Details ..................................30
      7.4. Expansion Examples ........................................32
   8. Result Handling ................................................33
      8.1. None ......................................................34
      8.2. Neutral ...................................................34
      8.3. Pass ......................................................34
      8.4. Fail ......................................................35
      8.5. Softfail ..................................................35
      8.6. Temperror .................................................36
      8.7. Permerror .................................................36

   9. Recording the Result ...........................................36
      9.1. The Received-SPF Header Field .............................37
      9.2. SPF Results in the Authentication-Results Header Field ....39
   10. Effects on Infrastructure .....................................39
      10.1. Sending Domains ..........................................40
           10.1.1. DNS Resource Considerations .......................40
           10.1.2. Administrator's Considerations ....................41
           10.1.3. Bounces ...........................................41
      10.2. Receivers ................................................42
      10.3. Mediators ................................................42
   11. Security Considerations .......................................43
      11.1. Processing Limits ........................................43
      11.2. SPF-Authorized Email May Contain Other False Identities ..44
      11.3. Spoofed DNS and IP Data ..................................44
      11.4. Cross-User Forgery .......................................44
      11.5. Untrusted Information Sources ............................45
           11.5.1. Recorded Results ..................................45
           11.5.2. External Explanations .............................45
           11.5.3. Macro Expansion ...................................46
      11.6. Privacy Exposure .........................................46
      11.7. Delivering Mail Producing a "Fail" Result ................46
   12. Collected ABNF ................................................46
   13. Contributors and Acknowledgements .............................48
   14. IANA Considerations ...........................................49
      14.1. The SPF DNS Record Type ..................................49
      14.2. The Received-SPF Mail Header Field .......................50
      14.3. SPF Modifier Registry ....................................50
   15. References ....................................................50
      15.1. Normative References .....................................50
      15.2. Informative References ...................................51
   Appendix A. Extended Examples .....................................54
     A.1. Simple Examples ............................................55
     A.2. Multiple Domain Example ....................................56
     A.3. DNS Blacklist (DNSBL) Style Example ........................56
     A.4. Multiple Requirements Example ..............................57
   Appendix B. Changes in Implementation Requirements from RFC 4408 ..57
   Appendix C. Further Testing Advice ................................58
   Appendix D. SPF/Mediator Interactions .............................59
     D.1. Originating ADMDs ..........................................59
     D.2. Mediators ..................................................60
     D.3. Receiving ADMDs ............................................60
   Appendix E. Mail Services .........................................61
   Appendix F. MTA Relays ............................................61
   Appendix G. Local Policy Considerations ...........................62
     G.1. Policy for SPF Pass ........................................62
     G.2. Policy for SPF Fail ........................................62
     G.3. Policy for SPF Permerror ...................................63
     G.4. Policy for SPF Temperror ...................................63

1.  Introduction

   The current email infrastructure has the property that any host
   injecting mail into the system can use any DNS domain name it wants
   in each of the various identifiers specified by [RFC5321] and
   [RFC5322].  Although this feature is desirable in some circumstances,
   it is a major obstacle to reducing Unsolicited Bulk Email (UBE, aka
   spam).  Furthermore, ADMDs (as described in [RFC5598]) are
   understandably concerned about the ease with which other entities can
   make use of their domain names, often with malicious intent.

   This document defines a protocol by which ADMDs can authorize hosts
   to use their domain names in the "MAIL FROM" or "HELO" identities.
   Compliant ADMDs publish Sender Policy Framework (SPF) records in the
   DNS specifying which hosts are permitted to use their names, and
   compliant mail receivers use the published SPF records to test the
   authorization of sending Mail Transfer Agents (MTAs) using a given
   "HELO" or "MAIL FROM" identity during a mail transaction.

   An additional benefit to mail receivers is that after the use of an
   identity is verified, local policy decisions about the mail can be
   made based on the sender's domain, rather than the host's IP address.
   This is advantageous because reputation of domain names is likely to
   be more accurate than reputation of host IP addresses since domains
   are likely to be more stable over a longer period.  Furthermore, if a
   claimed identity fails verification, local policy can take stronger
   action against such email, such as rejecting it.

1.1.  Terminology

1.1.1.  Key Words

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in
   [RFC2119].

1.1.2.  Imported Definitions

   ABNF (Augmented Backus-Naur Form) ABNF is defined in [RFC5234], as
   are the tokens "ALPHA", "DIGIT", and "SP" (space).

   The tokens "Local-part", "Domain", and "Mailbox" are defined in
   [RFC5321].

   "dot-atom", "quoted-string", "comment", "CFWS" (comment folded white
   space), "FWS" (folded white space), and "CRLF" (carriage-return/
   line-feed) are defined in [RFC5322].

1.1.3.  MAIL FROM Definition

   This document is concerned with the identity of the sender of a mail
   message, as referred to in [RFC5321]:

      The transaction starts with a MAIL command that gives the sender
      identification.

   Since there are many other names for this identity, it is important
   to choose a name that is:

   1.  commonly used

   2.  well defined

   As such, throughout this document the term "MAIL FROM" will be used,
   which is defined as the RFC5321.MailFrom (reverse-path) identity
   described in [RFC5598].

1.1.4.  HELO Definition

   This document also makes use of the HELO/EHLO identity.  The "HELO"
   identity derives from either the SMTP HELO or EHLO command (see
   [RFC5321]).  Since HELO and EHLO can, in many cases, be used
   interchangeably, they are identified commonly as "HELO" in this
   document.  This means RFC5321.HELO/.EHLO as defined in [RFC5598].
   These commands supply the identity of the SMTP client (sending host)
   for the SMTP session.

1.2.  check_host()

   Section 4 introduces an algorithm to evaluate an SPF policy against
   an arriving email transaction.  In an early implementation, this
   algorithm was encoded in a function called check_host().  That name
   is used in this document as symbolic of the SPF evaluation algorithm,
   but of course implementers are not required to use this name.

2.  Operational Overview

2.1.  Publishing Authorization

   An SPF-compliant domain publishes valid SPF records as described in
   Section 3.  These records authorize the use of the relevant domain
   names in the "HELO" and "MAIL FROM" identities by the MTAs specified
   therein.

   SPF results can be used to make both positive (source is authorized)
   and negative (source is not authorized) determinations.  If ADMDs
   choose to publish SPF records and want to support receivers making
   negative authorization determinations, it is necessary for them to
   publish records that end in "-all", or redirect to other records that
   do; otherwise, no definitive determination of authorization can be
   made.  Potential issues and mitigations associated with negative
   determinations are discussed in Section 10.

   ADMDs that wish to declare that no hosts are authorized to use their
   DNS domain names in the HELO or MAIL FROM commands during SMTP
   sessions can publish SPF records that say so for domain names that
   are neither used in the domain part of email addresses nor expected
   to originate mail.

   When changing SPF records, care has to be taken to ensure that there
   is a transition period so that the old policy remains valid until all
   legitimate email can reasonably expect to have been checked.
   [RFC5321], Section 4.5.4.1 discusses how long a message might be in
   transit.  While offline checks are possible, the closer to the
   original transmission time checks are performed, the more likely they
   are to get an SPF result that matches the sending ADMD intent at the
   time the message was sent.

2.2.  Checking Authorization

   A mail receiver can perform a set of SPF checks for each mail message
   it receives.  An SPF check tests the authorization of a client host
   to emit mail with a given identity.  Typically, such checks are done
   by a receiving MTA, but can be performed elsewhere in the mail
   processing chain so long as the required information is available and
   reliable.  The "MAIL FROM" and "HELO" identities are checked as
   described in Sections 2.4 and 2.3, respectively.

   Without explicit approval of the publishing ADMD, checking other
   identities against SPF version 1 records is NOT RECOMMENDED because
   there are cases that are known to give incorrect results.  For
   example, almost all mailing lists rewrite the "MAIL FROM" identity
   (see Section 10.3), but some do not change any other identities in
   the message.  Documents that define other identities will have to
   define the method for explicit approval.

   It is possible that mail receivers will use the SPF check as part of
   a larger set of tests on incoming mail.  The results of other tests
   might influence whether or not a particular SPF check is performed.
   For example, finding the sending host's IP address on a local
   whitelist might cause all other tests to be skipped and all mail from
   that host to be accepted.

   When a mail receiver decides to perform an SPF check, it has to use a
   correctly implemented check_host() function (Section 4) evaluated
   with the correct parameters.  Although the test as a whole is
   optional, once it has been decided to perform a test it has to be
   performed as specified so that the correct semantics are preserved
   between publisher and receiver.

   To make the test, the mail receiver MUST evaluate the check_host()
   function with the arguments described in Section 4.1.

   Although invalid, malformed, or non-existent domains cause SPF checks
   to return "none" because no SPF record can be found, it has long been
   the policy of many MTAs to reject email from such domains, especially
   in the case of invalid "MAIL FROM".  Rejecting email will prevent one
   method of circumventing of SPF records.

   Implementations have to take care to correctly extract the <domain>
   from the data given with the SMTP MAIL FROM command as many MTAs will
   still accept such things as source routes (see Appendix C of
   [RFC5321]), the %-hack (see [RFC1123]), and bang paths (see
   [RFC1983]).  These archaic features have been maliciously used to
   bypass security systems.

2.3.  The "HELO" Identity

   It is RECOMMENDED that SPF verifiers not only check the "MAIL FROM"
   identity but also separately check the "HELO" identity by applying
   the check_host() function (Section 4) to the "HELO" identity as the
   <sender>.  Checking "HELO" promotes consistency of results and can
   reduce DNS resource usage.  If a conclusive determination about the
   message can be made based on a check of "HELO", then the use of DNS
   resources to process the typically more complex "MAIL FROM" can be
   avoided.  Additionally, since SPF records published for "HELO"
   identities refer to a single host, when available, they are a very
   reliable source of host authorization status.  Checking "HELO" before
   "MAIL FROM" is the RECOMMENDED sequence if both are checked.

   Note that requirements for the domain presented in the EHLO or HELO
   command are not always clear to the sending party, and SPF verifiers
   have to be prepared for the identity to be an IP address literal (see
   [RFC5321], Section 4.1.3) or simply be malformed.  This SPF check can
   only be performed when the "HELO" string is a valid, multi-label
   domain name.

2.4.  The "MAIL FROM" Identity

   SPF verifiers MUST check the "MAIL FROM" identity if a "HELO" check
   either has not been performed or has not reached a definitive policy
   result by applying the check_host() function to the "MAIL FROM"
   identity as the <sender>.

   [RFC5321] allows the reverse-path to be null (see Section 4.5.5 in
   [RFC5321]).  In this case, there is no explicit sender mailbox, and
   such a message can be assumed to be a notification message from the
   mail system itself.  When the reverse-path is null, this document
   defines the "MAIL FROM" identity to be the mailbox composed of the
   local-part "postmaster" and the "HELO" identity (which might or might
   not have been checked separately before).

2.5.  Location of Checks

   The authorization check SHOULD be performed during the processing of
   the SMTP transaction that receives the mail.  This reduces the
   complexity of determining the correct IP address to use as an input
   to check_host() and allows errors to be returned directly to the
   sending MTA by way of SMTP replies.  Appendix D of [RFC7001] provides
   a more thorough discussion of this topic.

   The authorization check is performed during the SMTP transaction at
   the time of the MAIL command, and uses the MAIL FROM value and the
   client IP address.  Performing the check at later times or with other
   input can cause problems such as the following:

   o  It might be difficult to accurately extract the required
      information from potentially deceptive headers.

   o  Legitimate email might fail the authorization check because the
      sender's policy has since changed.

   Generating non-delivery notifications to forged identities that have
   failed the authorization check often constitutes backscatter, i.e.,
   nuisance rejection notices that are not actionable.  Operators are
   strongly advised to avoid such practices.  Section 2 of [RFC3834]
   describes backscatter and the problems it causes.

2.6.  Results of Evaluation

   Section 4 defines check_host(), a model function definition that uses
   the inputs defined above and the sender's policy published in the DNS
   to reach a conclusion about client authorization.  An SPF verifier
   implements something semantically equivalent to the function defined
   there.

   This section enumerates and briefly defines the possible outputs of
   that function.  Note, however, that the protocol establishes no
   normative requirements for handling any particular result.
   Discussion of handling options for each result can be found in
   Section 8.

2.6.1.  None

   A result of "none" means either (a) no syntactically valid DNS domain
   name was extracted from the SMTP session that could be used as the
   one to be authorized, or (b) no SPF records were retrieved from
   the DNS.

2.6.2.  Neutral

   A "neutral" result means the ADMD has explicitly stated that it is
   not asserting whether the IP address is authorized.

2.6.3.  Pass

   A "pass" result is an explicit statement that the client is
   authorized to inject mail with the given identity.

2.6.4.  Fail

   A "fail" result is an explicit statement that the client is not
   authorized to use the domain in the given identity.

2.6.5.  Softfail

   A "softfail" result is a weak statement by the publishing ADMD that
   the host is probably not authorized.  It has not published a
   stronger, more definitive policy that results in a "fail".

2.6.6.  Temperror

   A "temperror" result means the SPF verifier encountered a transient
   (generally DNS) error while performing the check.  A later retry may
   succeed without further DNS operator action.

2.6.7.  Permerror

   A "permerror" result means the domain's published records could not
   be correctly interpreted.  This signals an error condition that
   definitely requires DNS operator intervention to be resolved.

3.  SPF Records

   An SPF record is a DNS record that declares which hosts are, and are
   not, authorized to use a domain name for the "HELO" and "MAIL FROM"
   identities.  Loosely, the record partitions hosts into permitted and
   not-permitted sets (though some hosts might fall into neither
   category).

   The SPF record is expressed as a single string of text found in the
   RDATA of a single DNS TXT resource record; multiple SPF records are
   not permitted for the same owner name.  The record format and the
   process for selecting records are described below in Section 4.  An
   example record is the following:

      v=spf1 +mx a:colo.example.com/28 -all

   This record has a version of "spf1" and three directives: "+mx",
   "a:colo.example.com/28" (the "+" is implied), and "-all".

   Each SPF record is placed in the DNS tree at the owner name it
   pertains to, not in a subdomain under the owner name.  This is
   similar to how SRV records [RFC2782] are done.

   The example in this section might be published via these lines in a
   domain zone file:

      example.com.          TXT "v=spf1 +mx a:colo.example.com/28 -all"

   Since TXT records have multiple uses, beware of other TXT records
   published there for other purposes.  They might cause problems with
   size limits (see Section 3.4), and care has to be taken to ensure
   that only SPF records are used for SPF processing.

   ADMDs publishing SPF records ought to keep the amount of DNS
   information needed to evaluate a record to a minimum.  Sections 4.6.4
   and 10.1.1 provide some suggestions about "include" mechanisms and
   chained "redirect" modifiers.

3.1.  DNS Resource Records

   SPF records MUST be published as a DNS TXT (type 16) Resource Record
   (RR) [RFC1035] only.  The character content of the record is encoded
   as [US-ASCII].  Use of alternative DNS RR types was supported in
   SPF's experimental phase but has been discontinued.

   In 2003, when SPF was first being developed, the requirements for
   assignment of a new DNS RR type were considerably more stringent than
   they are now.  Additionally, support for easy deployment of new DNS

   RR types was not widely deployed in DNS servers and provisioning
   systems.  As a result, developers of SPF found it easier and more
   practical to use the TXT RR type for SPF records.

   In its review of [RFC4408], the SPFbis working group concluded that
   its dual RR type transition model was fundamentally flawed since it
   contained no common RR type that implementers were required to serve
   and required to check.  Many alternatives were considered to resolve
   this issue, but ultimately the working group concluded that
   significant migration to the SPF RR type in the foreseeable future
   was very unlikely and that the best solution for resolving this
   interoperability issue was to drop support for the SPF RR type from
   SPF version 1.  See Appendix A of [RFC6686] for further information.

   The circumstances surrounding SPF's initial deployment a decade ago
   are unique.  If a future update to SPF were developed that did not
   reuse existing SPF records, it could use the SPF RR type.  SPF's use
   of the TXT RR type for structured data should in no way be taken as
   precedent for future protocol designers.  Further discussion of
   design considerations when using new DNS RR types can be found in
   [RFC5507].

3.2.  Multiple DNS Records

   A domain name MUST NOT have multiple records that would cause an
   authorization check to select more than one record.  See Section 4.5
   for the selection rules.

3.3.  Multiple Strings in a Single DNS Record

   As defined in [RFC1035], Sections 3.3 and 3.3.14, a single text DNS
   record can be composed of more than one string.  If a published
   record contains multiple character-strings, then the record MUST be
   treated as if those strings are concatenated together without adding
   spaces.  For example:

      IN TXT "v=spf1 .... first" "second string..."

   is equivalent to:

      IN TXT "v=spf1 .... firstsecond string..."

   TXT records containing multiple strings are useful in constructing
   records that would exceed the 255-octet maximum length of a
   character-string within a single TXT record.

3.4.  Record Size

   The published SPF record for a given domain name SHOULD remain small
   enough that the results of a query for it will fit within 512 octets.
   Otherwise, there is a possibility of exceeding a DNS protocol limit.
   This UDP limit is defined in [RFC1035], Section 2.3.4, although it
   was raised by [RFC2671].  Staying below 512 octets ought to prevent
   older DNS implementations from failing over to TCP and will work with
   UDP in the absence of EDNS0 [RFC6891] support.  Since the answer size
   is dependent on many things outside the scope of this document, it is
   only possible to give this guideline: If the size of the DNS message,
   the combined length of the DNS name and the text of all the records
   of a given type is under 450 octets, then DNS answers ought to fit in
   UDP packets.  Records that are too long to fit in a single UDP packet
   could be silently ignored by SPF verifiers due to firewall and other
   issues that interfere with the operation of DNS over TCP or using
   ENDS0.

   Note that when computing the sizes for replies to queries of the TXT
   format, one has to take into account any other TXT records published
   at the domain name.  Similarly, the sizes for replies to all queries
   related to SPF have to be evaluated to fit in a single 512-octet UDP
   packet (i.e., DNS message size limited to 450 octets).

3.5.  Wildcard Records

   Use of wildcard records for publishing is discouraged, and care has
   to be taken if they are used.  If a zone includes wildcard MX
   records, it might want to publish wildcard declarations, subject to
   the same requirements and problems.  In particular, the declaration
   MUST be repeated for any host that has any RR records at all, and for
   subdomains thereof.  Consider the example in [RFC1034],
   Section 4.3.3.  Based on that, we can do the following:

      EXAMPLE.COM.          MX      10      A.EXAMPLE.COM
      EXAMPLE.COM.          TXT     "v=spf1 a:A.EXAMPLE.COM -all"

      *.EXAMPLE.COM.        MX      10      A.EXAMPLE.COM
      *.EXAMPLE.COM.        TXT     "v=spf1 a:A.EXAMPLE.COM -all"

      A.EXAMPLE.COM.        A       203.0.113.1
      A.EXAMPLE.COM.        MX      10      A.EXAMPLE.COM
      A.EXAMPLE.COM.        TXT     "v=spf1 a:A.EXAMPLE.COM -all"

      *.A.EXAMPLE.COM.      MX      10      A.EXAMPLE.COM
      *.A.EXAMPLE.COM.      TXT     "v=spf1 a:A.EXAMPLE.COM -all"

   SPF records have to be listed twice for every name within the zone:
   once for the name, and once with a wildcard to cover the tree under
   the name, in order to cover all domains in use in outgoing mail.