Network Working Group                                       G. Vaudreuil
Request for Comments: 1428                                          CNRI
                                                           February 1993


                   Transition of Internet Mail from
                              Just-Send-8
                           to 8bit-SMTP/MIME

Status of this Memo

   This RFC provides information for the Internet community.  It does
   not specify an Internet standard.  Distribution of this memo is
   unlimited.

Abstract

   Protocols for extending SMTP to pass 8bit characters have been
   defined [3] [4]. These protocols require that messages transported by
   the extended SMTP are to be encoded in MIME [1] [2].  Before work
   began on these protocols, several SMTP implementations adopted ad-hoc
   mechanisms for sending 8bit data. It is desirable for the extended
   SMTP environment and these ad hoc mechanisms interoperate.  This
   document outlines the problems in this environment and an approach to
   minimizing the cost of transition from current usage of non-MIME 8bit
   messages to MIME.

1. Terminology

   RFC 821 defines a 7bit transport.  A transport agent which does not
   clear the high order bit upon receipt of octets with this bit set in
   SMTP messages is called 8 bit transparent in this document. An
   implementation of the general SMTP Extensions document [3] and the
   8bit extensions protocol [4] which passes MIME messages using all 8
   bits of an octet is called 8bit ESMTP.  An implementation of extended
   SMTP which does not accept 8bit characters is called 7bit ESMTP.  A
   gateway is defined to be a transport agent with User Agent authority
   to alter or convert the content of a message.

2. The Problem

   SMTP as defined in RFC 821 limits the sending of Internet Mail to
   US-ASCII [5] characters.  As the Internet has grown to include non-
   English correspondents, the need to communicate with character sets
   other than US-ASCII has prompted many vendors and users to extend
   SMTP or RFC 822 to use non-ASCII character sets.  Common approaches
   are to send 7 bit national variant ISO 646 character sets over
   current RFC822/SMTP, to extend SMTP and RFC822 to use 8bit ISO 8859

   character sets, and to use proprietary PC character sets.

   A third approach is used for Japanese mail.  Japanese characters are
   represented by pairs of octets with the high order bit cleared.
   Switching between 14 bit character sets and 7 bit character sets is
   indicated within the message by ISO 2022 escape sequences.

   So long as these implementations can communicate without intermediate
   transformations and have a loose private agreement on the use of a
   specific character set without tagging, basic mail service can be
   provided.

   In the transition to the negotiated 8bit ESMTP/MIME environment, it
   is important that mail sent by a currently non-conforming user can be
   read by another non-conforming user.  This existing functionality is
   reduced by conversion from 8bit text to text encoded in unreadable
   Base-64 or "garbled" text encoded in quoted printable.

   There are several interesting non-interoperable cases that currently
   exist in non US-ASCII mail and several new ones likely to emerge in a
   transition to 8bit/MIME.  Below is a listing of the transition-to-
   mime cases.  Only solutions to (4) in the context of a translating
   gateway are discussed in this memo.

                \ Receiver
                  \    7bit     8bit          MIME/
             Sender \| only   | transparent | ESMTP
           ----------------------------------------
           7bit only |  (1)   |    (1)      | (1)
           ----------------------------------------
    8bit transparent |  (2)   |    (3)      | (4)
           ----------------------------------------
          MIME/ESMTP |  (5)   |    (5)      | (6)


   (1) 7Bit non-MIME sender to 7bit, MIME, or 8bit transparent receiver

      This will continue to work unchanged with nationally varient ISO
      646 or ISO 2022 character set shifting if an external "out of
      band" agreement exists between the sender and the receiver.  A
      7bit to 8bit/ESMTP gateway need not alter the content of this
      message.

   (2) 8bit sender to 7bit non-MIME receiver

      The receiver will receive bit-stripped mail which results in the
      mis-interpretation of the data and the wrong character being
      displayed or printed.  Mail sent using languages where most

      characters are in the US-ASCII subset of ISO 8859 may be somewhat
      readable.

   (3) 8bit transparent sender to 8bit transparent receiver

      Will work if an external agreement "out of band" to use a
      particular character set without tagging exists between the sender
      and the receiver.

   (4) 8bit transparent sender to MIME/ESMTP conformant receiver

      Will work if a reasonable upgrade path is provided via gateways,
      the indicated character set tag inserted by the gateway is correct
      and the receiver supports the character set chosen by the sender.
      This case is the focus of this memo.

   (5) MIME/ESMTP sender to non-MIME 7bit receiver

      Because the ESMTP/MIME sender cannot know if the receiver will
      understand 8bits, the sender will encode the text into base-64 or
      quoted-printable which may be considered "garbled" by the
      receiver.  To provide a useful downgrade path the gateway must
      have some knowledge about the capabilities of the receiver. When
      the character set can be clearly identified, techniques like the
      menmonic MNEM encoding described in RFC 1345 may be helpful in
      this case.

   (6) MIME/ESMTP sender to MIME/ESMTP receiver

      Interoperability will be attained provided the receiver supports
      the character set chosen by the sender.

3. Upgrade Path from 8bit Transparent to ESMTP/MIME

   A gateway which has been upgraded to support Extended SMTP may
   upgrade an 8bit message received to MIME.  This is consistent with
   the requirement that all 8bit mail sent by ESMTP be encoded in MIME.
   The upgrade should be done using the best available information.

   A site may "upgrade" to MIME en-masse by implementing MIME conversion
   for all messages leaving the site.  For text messages, the body can
   be converted by adding a MIME-version header and a Content-Type:
   Text/Plain with the character set in use in the site, provided the
   site uses a single character set.

   An appropriate Content-Transfer-Encoding header line must be added to
   indicate any encoding that may be necessary.

    Example:

       MIME-Version: 1.0
       Content-Type: Text/Plain; Charset = ISO-8859-1
       Content-Transfer-Encoding: 8bit

   If no information about the character set in use is available, the
   gateway should upgrade the content by using the character set
   "unknown-8bit". The unknown-8bit value of the charset parameter
   indicates only that no reliable information about the character
   set(s) used in the message was available.

   If a message body has been upgraded to MIME, the RFC 822 headers
   containing non US-ASCII characters must be upgraded to conform with
   the header encoding rules of RFC1342. A gateway should recode all
   unstructered header fields as well as RFC 822 "comment"s and
   "phrase"s according to the rules of RFC 1342. There is no equivalent
   in RFC 1342 to the "8bit" Content-Transfer-Encoding value for message
   bodies so all 8bit header text must be transformed according to
   either the "B" or the "Q" encoding method.  For ISO 8859 character
   sets, the "Q" encoding will generally result in somewhat readable
   headers.

   Trace information should be added to the document with a convert
   clause: "rfc822-to-8bit", "rfc822-to-base-64" or "rfc822-to-quoted-
   printable" e.g.,

   Received: from dbc.mtview.ca.us by dbc.mtview.ca.us
             convert rfc822-to-8bit; Tue, 01 Sep 1992 01:18:00 -0700

Appendix - The "unknown-8bit" Character Set

   This section defines a "charset" parameter, for use in a MIME
   Content-Type field.

   A special purpose character set called "unknown-8bit" is defined to
   be an unknown 8bit character set, encoded into a sequence of octets.
   It can be used as a label for any character set from any language,
   using any encoding.  It must not be further defined.

   The use of this token in a "charset=" field of a message indicates
   that nothing is known about the character set used. This marker is
   intended for use by non-MIME to MIME gateways; specifically in those
   which translate from SMTP to 8bit ESMTP/MIME.

   This character set is not intended to be used by mail composers.  It
   is assumed that the mail composer knows the character set in use and
   will mark it with a character set value as specified in [1], as

   amended by current Assigned Numbers documents [6].

   The use of the "unknown-8bit" label is intended only by mail gateway
   agents which cannot determine via out-of-band information the
   intended character set.

   The interpretation of the "unknown-8bit" is up to the mail reader.
   It is assumed that in many cases the human user will be able to
   interpret the information and choose an appropriate character set or
   pre-processor.

Acknowledgements

   This document originated as a hallway conversation between Ned Freed,
   Neil Katin, and the author.  Substantive input was received from
   Jonathan Laventhol, Craig Everhart, Olle Jarnefors, and Olafur
   Gudmundsson.  The document was refined with the input of many
   participants in the IETF SMTP Extensions Working Group.

References

   [1] Borenstein, N., and N. Freed, "Multipurpose Internet Mail
       Extensions", RFC 1341, Bellcore, Innosoft, June 1992.

   [2] Moore, K., "Representation of Non-ASCII Text in Internet Message
       Headers", RFC 1342, University of Tennessee, June 1992.

   [3] Klensin, J., WG Chair, Freed, N., Editor, Rose, M., Stefferud,
       E., and D. Crocker, "SMTP Service Extensions" RFC 1425, United
       Nations University, Innosoft International, Inc., Dover Beach
       Consulting, Inc., Network Management Associates, Inc., The Branch
       Office, February 1993.

   [4] Klensin, J., WG Chair, Freed, N., Editor, Rose, M., Stefferud,
       E., and D. Crocker, "SMTP Service Extensions for 8bit
       MIMEtransport", RFC 1426, United Nations University, Innosoft
       International, Inc., Dover Beach Consulting, Inc., Network
       Management Associates, Inc., The Branch Office, February 1993.

   [5] Coded Character Set--7-Bit American Standard Code for Information
       Interchange, ANSI X3.4-1986.

   [6] Reynolds, J., and J. Postel, "Assigned Numbers", STD 2, RFC 1340,
       USC/Information Sciences Institute, July 1992.

Security Considerations

   Security issues are not discussed in this memo.

Author's Address

   Greg Vaudreuil
   Corporation for National Research Initiatives
   1895 Preston White Drive, Suite 100
   Reston, VA 22091 USA

   Phone: (703) 620-8990
   EMail: GVaudre@CNRI.Reston.VA.US