RFC 5537
Netnews Architecture and Protocols
Pages: 48
Proposed Standard
→ Errata
Obsoletes: 1036 1849
Updated by: 8315
Proposed Standard
→ Errata
Obsoletes: 1036 1849
Updated by: 8315
Part 2 of 2 – Pages 24 to 48
3.10. Duties of a Gateway
A gateway transforms an article into the native message format of another medium, or translates the messages of another medium into news articles, or transforms articles into proto-articles for injection into a separate Netnews network. Encapsulation of a news article into a message of MIME type application/news-transmission, or the subsequent undoing of that encapsulation, is not gatewaying since it involves no transformation of the article. There are two basic types of gateway, the outgoing gateway that transforms a news article into a different type of message, and the incoming gateway that transforms a message from another network into a news proto-article and injects it into a Netnews network. These are handled separately below. Transformation of an article into another medium stands a very high chance of discarding or interfering with the protection inherent in the news system against duplicate articles. The most common problem caused by gateways is loops that repeatedly reinject previously posted articles. To prevent this, a gateway MUST take precautions against loops, as detailed below. The transformations applied to the message SHOULD be as minimal as possible while still accomplishing the gatewaying. Every change made by a gateway potentially breaks a property of one of the media or loses information, and therefore only those transformations made necessary by the differences between the media should be applied. If bidirectional gatewaying (both an incoming and an outgoing gateway) is being set up between Netnews and some other medium, the incoming and outgoing gateways SHOULD be coordinated to avoid unintended reinjection of gated articles. Circular gatewaying (gatewaying a message into another medium and then back into Netnews) SHOULD NOT be done; encapsulation of the article SHOULD be used instead where this is necessary. Safe bidirectional gatewaying between a mailing list and a newsgroup is far easier if the newsgroup is moderated. Posts to the moderated group and submissions to the mailing list can then go through a single point that does the necessary gatewaying and then sends the message out to both the newsgroup and the mailing list at the same time, eliminating most of the possibility of loops. Bidirectional gatewaying between a mailing list and an unmoderated newsgroup, in contrast, is difficult to do correctly and is far more fragile. Newsgroups intended to be bidirectionally gated to a mailing list SHOULD therefore be moderated where possible, even if the moderator
is a simple gateway and injecting agent that correctly handles crossposting to other moderated groups and otherwise passes all traffic.3.10.1. Duties of an Outgoing Gateway
From the perspective of Netnews, an outgoing gateway is just a special type of reading agent. The exact nature of what the outgoing gateway will need to do to articles depends on the medium to which the articles are being gated. Because it raises the danger of loops due to the possibility of one or more corresponding incoming gateways back from that medium to Netnews, the operation of the outgoing gateway is subject to additional constraints. The following practices are encouraged for all outgoing gateways, regardless of whether there is known to be a related incoming gateway, both as precautionary measures and as guidelines to quality of implementation: 1. The message identifier of the news article should be preserved if at all possible, preferably as or within the corresponding unique identifier of the other medium. However, if it is not preserved in this way, then at least it should be preserved as a comment in the message. This helps greatly with preventing loops. 2. The Date and Injection-Date of the news article should also be preserved if possible, for similar reasons. 3. The message should be tagged in some way so as to prevent its reinjection into Netnews. This may be impossible to do without knowledge of potential incoming gateways, but it is better to try to provide some indication even if not successful; at the least, a human-readable indication that the article should not be gated back to Netnews can help locate a human problem. 4. Netnews control messages should not be gated to another medium unless they would somehow be meaningful in that medium.3.10.2. Duties of an Incoming Gateway
The incoming gateway has the responsibility of ensuring that all of the requirements of this protocol are met by the articles that it forms. In addition to its special duties as a gateway, it bears all of the duties and responsibilities of a posting agent, and it has the same responsibility of a relaying agent to reject articles that it has already gatewayed.
An incoming gateway MUST NOT gate the same message twice. It may not be possible to ensure this in the face of mangling or modification of the message, but at the very least a gateway, when given a copy of a message that it has already gated and that is identical except for trace header fields (like Received in Email or Path in Netnews), MUST NOT gate the message again. An incoming gateway SHOULD take precautions against having this rule bypassed by modifications of the message that can be anticipated. News articles prepared by gateways MUST be valid news proto-articles (see Section 3.4.1). This often requires the gateway to synthesize a conforming article from non-conforming input. The gateway MUST then pass the article to an injecting agent, not directly to a relaying agent. Incoming gateways MUST NOT pass control messages (articles containing a Control or Supersedes header field) without removing or renaming that header field. Gateways MAY, however, generate cancel control messages for messages they have gatewayed. If a gateway receives a message that it can determine is a valid equivalent of a cancel control message in the medium it is gatewaying, it SHOULD discard that message without gatewaying it, generate a corresponding cancel control message of its own, and inject that cancel control message. NOTE: It is not unheard of for mail-to-news gateways to be used to post control messages, but encapsulation should be used for these cases instead. Gateways by their very nature are particularly prone to loops. Spews of normal articles are bad enough; spews of control messages with special significance to the news system, possibly resulting in high processing load or even in emails being sent for every message received, are catastrophic. It is far preferable to construct a system specifically for posting control messages that can do appropriate consistency checks and authentication of the originator of the message. If there is a message identifier that fills a role similar to that of the Message-ID header field in news, it SHOULD be used in the formation of the message identifier of the news article, perhaps with transformations required to meet the uniqueness requirement of Netnews and with the removal of any comments so as to comply with the syntax in Section 3.1.3 of [RFC5536]. Such transformations SHOULD be designed so that two messages with the same identifier generate the same Message-ID header field. NOTE: Message identifiers play a central role in the prevention of duplicates, and their correct use by gateways will do much to prevent loops. Netnews does, however, require that message identifiers be unique, and therefore message identifiers from
other media may not be suitable for use without modification. A
balance must be struck by the gateway between preserving
information used to prevent loops and generating unique message
identifiers.
Exceptionally, if there are multiple incoming gateways for a
particular set of messages, each to a different newsgroup(s), each
one SHOULD generate a message identifier unique to that gateway.
Each incoming gateway nonetheless MUST ensure that it does not gate
the same message twice.
NOTE: Consider the example of two gateways of a given mailing list
into two separate Usenet newsgroups, both of which preserve the
email message identifier. Each newsgroup may then receive a
portion of the messages (different sites seeing different
portions). In these cases, where there is no one "official"
gateway, some other method of generating message identifiers has
to be used to avoid collisions. It would obviously be preferable
for there to be only one gateway that crossposts, but this may not
be possible to coordinate.
If no date information is available, the gateway MAY supply a Date
header field with the gateway's current date. If only partial
information is available (such as date but not time), this SHOULD be
fleshed out to a full Date by adding default values rather than by
discarding this information. Only in very exceptional circumstances
should Date information be discarded, as it plays an important role
in preventing reinjection of old messages.
An incoming gateway MUST add a Sender header field to the news
article it forms by containing the <mailbox> of the administrator of
the gateway. Problems with the gateway may be reported to this
<mailbox>. The <display-name> portion of this <mailbox> SHOULD
indicate that the entity responsible for injection of the message is
a gateway. If the original message already had a Sender header
field, it SHOULD be renamed to Original-Sender so that its contents
can be preserved. See Section 3.10.3 for the specification of that
header field.
3.10.3. Original-Sender Header Field
The Original-Sender header field holds the content of a Sender header
field in an original message received by an incoming gateway,
preserving it while the incoming gateway adds its own Sender header
field. The content syntax makes use of syntax defined in [RFC5536]
and [RFC5322].
header =/ Original-Sender-header
Original-Sender-header
= "Original-Sender" ":" SP
Original-Sender-content
Original-Sender-content
= mailbox
The Permanent Message Header Field Repository entry for this header
field is:
Header field name: Original-Sender
Applicable protocol: Netnews
Status: standard
Author/Change controller: IETF
Specification document(s): RFC 5537
3.10.4. Gateway Example
To illustrate the type of precautions that should be taken against
loops, here is an example of the measures taken by one particular
combination of mail-to-news and news-to-mail gateways designed to
handle bidirectional gatewaying between mailing lists and unmoderated
groups:
1. The news-to-mail gateway preserves the message identifier of the
news article in the generated email message. The mail-to-news
gateway likewise preserves the email message identifier, provided
that it is syntactically valid for Netnews. This allows the news
system's built-in suppression of duplicates to serve as the first
line of defense against loops.
2. The news-to-mail gateway adds an X-* header field to all messages
it generates. The mail-to-news gateway discards any incoming
messages containing this header field. This is robust against
mailing list managers that replace the message identifier and
against any number of email hops, provided that the other message
header fields are preserved.
3. The mail-to-news gateway prepends the host name from which it
received the email message to the content of the Path header
field. The news-to-mail gateway refuses to gateway any message
that contains the list server name in its Path header field
(including in the tail section). This is robust against any
amount of munging of the message header fields by the mailing
list, provided that the email only goes through one hop.
4. The mail-to-news gateway is designed never to generate bounces to
the envelope sender. Instead, articles that are rejected by the
news server (for reasons not warranting silent discarding of the
message) result in a bounce message sent to an errors address
that is known not to forward to any mailing lists. In this way,
they can be handled by the news administrators.
These precautions have proven effective in practice at preventing
loops for this particular application (bidirectional gatewaying
between mailing lists and locally distributed newsgroups where both
gateways can be designed together). General gatewaying to world-wide
newsgroups poses additional difficulties; one must be very wary of
strange configurations, such as a newsgroup gated to a mailing list
that is in turn gated to a different newsgroup.
4. Media Types
This document defines several media types, which have been registered
with IANA as provided for in [RFC4288].
The media type message/news, as previously registered with IANA, is
hereby declared obsolete. The intent of this media type was to
define a standard way of transmitting news articles via mail for
human reading. However, it was never widely implemented, and its
default treatment as application/octet-stream by agents that did not
recognize it was counter-productive. The media type message/rfc822
(defined in Section 5.2.1 of [RFC2046]) SHOULD be used in its place.
The updated MIME media type definition of message/news is:
MIME type name: message
MIME subtype name: news
Required parameters: none
Optional parameters: none
Encoding considerations: same as message/rfc822
Security considerations: News articles may constitute "control
messages", which can have effects on a
host's news system beyond just addition
of information. Since control messages
may occur in normal news flow, most hosts
are suitably defended against undesired
effects already, but transmission of news
articles via mail may bypass
firewall-type defenses. Reading a news
article transmitted by mail involves no
hazards beyond those of mail, but
submitting it to news software for
processing should be done with care.
Interoperability considerations:
Rarely used, and therefore often
handled as application/octet-stream.
Disposition should by default be inline.
Published specification: RFC 5537
Applications that use this media type:
Some old mail and news user agents.
Intended usage: OBSOLETE
Author: Henry Spencer
Change controller: IETF
4.1. application/news-transmission
The media type application/news-transmission is intended for the
encapsulation of complete news articles where the intention is that
the recipient should then inject them into Netnews. This application
type provides one of the methods for mailing articles to moderators
(see Section 3.5.1) and may be used to convey messages to an
injecting agent. This encapsulation removes the need to transform an
email message into a Netnews proto-article and provides a way to send
a Netnews article using MIME through a transport medium that does not
support 8bit data.
The MIME media type definition of application/news-transmission is:
MIME type name: application
MIME subtype name: news-transmission
Required parameters: none
Optional parameters: One and only one of "usage=moderate",
"usage=inject", or "usage=relay".
Encoding considerations: A transfer-encoding different from that
of the article transmitted MAY be
supplied to ensure correct transmission
over some 7bit transport medium.
Security considerations: News articles may constitute "control
messages", which can have effects on a
host's news system beyond just addition
of information. Since control messages
may occur in normal news flow, most hosts
are suitably defended against undesired
effects already, but transmission of news
articles via mail may bypass
firewall-type defenses.
Published specification: RFC 5537
Body part: A complete proto-article ready for
injection into Netnews or an article
being relayed to another agent.
Applications that use this media type:
Injecting agents, Netnews moderators.
Intended usage: COMMON
Change controller: IETF
usage=moderate indicates the article is intended for a moderator,
usage=inject for an injecting agent, and usage=relay for a relaying
agent. The entity receiving the article may only implement one type
of agent, in which case the parameter MAY be omitted.
Contrary to the prior registration of this media type, article
batches are not permitted as a body part. Multiple messages or a
message with multiple application/news-transmission parts may be used
instead.
4.2. application/news-groupinfo
The application/news-groupinfo media type is used in conjunction with
the newgroup control message (see Section 5.2.1). Its body part
contains brief information about a newsgroup: the newsgroup name, its
description, and its moderation status.
The MIME media type definition of application/news-groupinfo is:
MIME type name: application
MIME subtype name: news-groupinfo
Required parameters: none
Optional parameters: charset, which MUST be a charset
registered for use with MIME text types.
It has the same syntax as the parameter
defined for text/plain [RFC2046].
Specifies the charset of the body part.
If not given, the charset defaults to
US-ASCII [ASCII].
Encoding considerations: 7bit or 8bit encoding MUST be used to
maintain compatibility.
Security considerations: None.
Interoperability considerations:
Disposition should by default be inline.
Applications that use this media type:
Control message issuers, relaying
agents, serving agents.
Published specification: RFC 5537
Intended usage: COMMON
Change controller: IETF
The content of the application/news-groupinfo body part is defined
as:
groupinfo-body = [ newsgroups-tag CRLF ]
newsgroups-line CRLF
newsgroups-tag = %x46.6F.72 SP %x79.6F.75.72 SP
%x6E.65.77.73.67.72.6F.75.70.73 SP
%x66.69.6C.65.3A
; case sensitive
; "For your newsgroups file:"
newsgroups-line = newsgroup-name
[ 1*HTAB newsgroup-description ]
[ *WSP moderation-flag ]
newsgroup-description
= eightbit-utext *( *WSP eightbit-utext )
moderation-flag = SP "(" %x4D.6F.64.65.72.61.74.65.64 ")"
; SPACE + case sensitive "(Moderated)"
eightbit-utext = VCHAR / %d127-255
This unusual format is backward-compatible with the scanning of the
body of newgroup control messages for descriptions done by Netnews
implementations that predate this specification. Although optional,
the <newsgroups-tag> SHOULD be included for backward compatibility.
The <newsgroup-description> MUST NOT contain any occurrence of the
string "(Moderated)" within it. Moderated newsgroups MUST be marked
by appending the case-sensitive text " (Moderated)" at the end.
While a charset parameter is defined for this media type, most
existing software does not understand MIME header fields or correctly
handle descriptions in a variety of charsets. Using a charset of US-
ASCII where possible is therefore RECOMMENDED; if not possible, UTF-8
[RFC3629] SHOULD be used. Regardless of the charset used, the
constraints of the above grammar MUST be met and the <newsgroup-name>
MUST be represented in that charset using the same octets as would be
used with a charset of US-ASCII.
4.3. application/news-checkgroups
The application/news-checkgroups media type contains a list of
newsgroups within a hierarchy or hierarchies, including their
descriptions and moderation status. It is primarily for use with the
checkgroups control message (see Section 5.2.3).
The MIME media type definition of application/news-checkgroups is:
MIME type name: application
MIME subtype name: news-checkgroups
Required parameters: none
Optional parameters: charset, which MUST be a charset
registered for use with MIME text types.
It has the same syntax as the parameter
defined for text/plain [RFC2046].
Specifies the charset of the body part.
If not given, the charset defaults to
US-ASCII [ASCII].
Encoding considerations: 7bit or 8bit encoding MUST be used to
maintain compatibility.
Security considerations: This media type provides only a means
for conveying a list of newsgroups and
does not provide any information
indicating whether the sender is
authorized to state which newsgroups
should exist within a hierarchy. Such
authorization must be accomplished by
other means.
Interoperability considerations:
Disposition should by default be inline.
Applications that use this media type:
Control message issuers, relaying
agents, serving agents.
Published specification: RFC 5537
Intended usage: COMMON
Change controller: IETF
The content of the application/news-checkgroups body part is defined
as:
checkgroups-body = *( valid-group CRLF )
valid-group = newsgroups-line ; see Section 4.2
The same restrictions on charset, <newsgroup-name>, and <newsgroup-
description> apply for this media type as for application/
news-groupinfo.
One application/news-checkgroups message may contain information for
one or more hierarchies and is considered complete for any hierarchy
for which it contains a <valid-group> unless accompanied by external
information limiting its scope (such as a <chkscope> parameter to a
checkgroups control message, as described in Section 5.2.3). In
other words, an application/news-checkgroups body part consisting of
example.moderated A moderated newsgroup (Moderated)
example.test An unmoderated test group
is a statement that the example.* hierarchy contains two newsgroups,
example.moderated and example.test, and no others. This media type
therefore MUST NOT be used for conveying partial information about a
hierarchy; if a group from a given hierarchy is present, all groups
that exist in that hierarchy MUST be listed unless its scope is limited by external information, in which case all groups SHOULD be listed. Spaces are used in this example for formatting reasons. In an actual message, the newsgroup name and description MUST be separated by one or more tabs (HTAB, ASCII %d09), not spaces.5. Control Messages
A control message is an article that contains a Control header field and thereby indicates that some action should be taken by an agent other than distribution and display. Any article containing a Control header field (defined in Section 3.2.3 of [RFC5536]) is a control message. Additionally, the action of an article containing a Supersedes header field is described here; while such an article is not a control message, it specifies an action similar to the cancel control message. The <control-command> of a Control header field comprises a <verb>, which indicates the action to be taken, and one or more <argument> values, which supply the details. For some control messages, the body of the article is also significant. Each recognized <verb> (the control message type) is described in a separate section below. Agents MAY accept other control message types than those specified below, and MUST either ignore or reject control messages with unrecognized types. Syntactic definitions of valid <argument> values and restrictions on control message bodies are given in the section for each control message type. Contrary to [RFC1036], the presence of a Subject header field starting with the string "cmsg " MUST NOT cause an article to be interpreted as a control message. Agents MAY reject an article that has such a Subject header field and no Control header field as ambiguous. Likewise, the presence of a <newsgroup-name> ending in ".ctl" in the Newsgroups header field or the presence of an Also- Control header field MUST NOT cause the article to be interpreted as a control message.5.1. Authentication and Authorization
Control messages specify actions above and beyond the normal processing of an article and are therefore potential vectors of abuse and unauthorized action. There is, at present, no standardized means of authenticating the sender of a control message or verifying that the contents of a control message were sent by the claimed sender. There are, however, some unstandardized authentication mechanisms in common use, such as [PGPVERIFY].
Agents acting on control messages SHOULD take steps to authenticate control messages before acting on them, as determined by local authorization policy. Whether this is done via the use of an unstandardized authentication protocol, by comparison with information obtained through another protocol, by human review, or by some other means is left unspecified by this document. Further extensions or revisions of this protocol are expected to standardize a digital signature mechanism. Agents are expected to have their own local authorization policies for which control messages will be honored. No Netnews agent is ever required to act on any control message. The following descriptions specify the actions that a control message requests, but an agent MAY always decline to act on any given control message.5.2. Group Control Messages
A group control message is any control message type that requests some update to the list of newsgroups known to a news server. The standard group control message types are "newgroup", "rmgroup", and "checkgroups". Before honoring any group control message, an agent MUST check the newsgroup or newsgroups affected by that control message and decline to create any newsgroups not in conformance with the restrictions in Section 3.1.4 of [RFC5536]. All of the group control messages MUST have an Approved header field (Section 3.2.1 of [RFC5536]). Group control messages without an Approved header field SHOULD NOT be honored. Group control messages affecting specific groups (newgroup and rmgroup control messages, for example) SHOULD include the <newsgroup- name> for the group or groups affected in their Newsgroups header field. Other newsgroups MAY be included in the Newsgroups header field so that the control message will reach more news servers, but due to the special relaying rules for group control messages (see Section 3.6) this is normally unnecessary and may be excessive.5.2.1. The newgroup Control Message
The newgroup control message requests that the specified group be created or, if already existing, that its moderation status or description be changed. The syntax of its Control header field is: control-command =/ Newgroup-command Newgroup-command = "newgroup" Newgroup-arguments Newgroup-arguments = 1*WSP newsgroup-name
[ 1*WSP newgroup-flag ]
newgroup-flag = "moderated"
If the request is honored, the moderation status of the group SHOULD
be set in accordance with the presence or absence of the <newgroup-
flag> "moderated". "moderated" is the only flag defined by this
protocol. Other flags MAY be defined by extensions to this protocol
and accepted by agents. If an agent does not recognize the
<newgroup-flag> of a newgroup control message, it SHOULD ignore that
control message.
The body of a newgroup message SHOULD contain an entity of type
application/news-groupinfo specifying the description of the
newsgroup, either as the entire body or as an entity within a
multipart/mixed object [RFC2046]. If such an entity is present, the
moderation status specified therein MUST match the moderation status
specified by the <newgroup-flag>. The body of a newgroup message MAY
contain other entities (encapsulated in multipart/mixed) that provide
additional information about the newsgroup or the circumstances of
the control message.
In the absence of an application/news-groupinfo entity, a news server
MAY search the body of the message for the line "For your newsgroups
file:" and take the following line as a <newsgroups-line>. Prior to
the standardization of application/news-groupinfo, this was the
convention for providing a newsgroup description.
If the request is honored and contains a newsgroup description, and
if the news server honoring it stores newsgroup descriptions, the
stored newsgroup description SHOULD be updated to the description
specified in the control message, even if no other property of the
group has changed.
5.2.1.1. newgroup Control Message Example
A newgroup control message requesting creation of the moderated
newsgroup example.admin.info.
From: "example.* Administrator" <admin@noc.example>
Newsgroups: example.admin.info
Date: 27 Feb 2002 12:50:22 +0200
Subject: cmsg newgroup example.admin.info moderated
Approved: admin@noc.example
Control: newgroup example.admin.info moderated
Message-ID: <ng-example.admin.info-20020227@noc.example>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="nxtprt"
Content-Transfer-Encoding: 8bit
This is a MIME control message.
--nxtprt
Content-Type: application/news-groupinfo; charset=us-ascii
For your newsgroups file:
example.admin.info About the example.* groups (Moderated)
--nxtprt
Content-Type: text/plain; charset=us-ascii
A moderated newsgroup for announcements about new newsgroups in
the example.* hierarchy.
--nxtprt--
Spaces are used in this example for formatting reasons. In an actual
message, the newsgroup name and description MUST be separated by one
or more tabs (HTAB, ASCII %d09), not spaces.
5.2.2. The rmgroup Control Message
The rmgroup control message requests that the specified group be
removed from a news server's list of valid groups. The syntax of its
Control header field is:
control-command =/ Rmgroup-command
Rmgroup-command = "rmgroup" Rmgroup-arguments
Rmgroup-arguments = 1*WSP newsgroup-name
The body of the control message MAY contain anything, usually an
explanatory text.
5.2.3. The checkgroups Control Message
The checkgroups control message contains a list of all the valid
groups in a hierarchy with descriptions and moderation status. It
requests that a news server update its valid newsgroup list for that
hierarchy to include the groups specified, remove any groups not
specified, and update group descriptions and moderation status to
match those given in the checkgroups control message. The syntax of
its Control header field is:
control-command =/ Checkgroup-command
Checkgroup-command = "checkgroups" Checkgroup-arguments
Checkgroup-arguments= [ chkscope ] [ chksernr ]
chkscope = 1*( 1*WSP ["!"] newsgroup-name )
chksernr = 1*WSP "#" 1*DIGIT
A checkgroups message is interpreted as an exhaustive list of the
valid groups in all hierarchies or sub-hierarchies with a prefix
listed in the <chkscope> argument, excluding any sub-hierarchy where
the <chkscope> argument is prefixed by "!". For complex cases with
multiple <chkscope> arguments, start from an empty list of groups,
include all groups in the checkgroups control message matching
<chkscope> arguments without a "!" prefix, and then exclude all
groups matching <chkscope> arguments with a "!" prefix. Follow this
method regardless of the order of the <chkscope> arguments in the
Control header field.
If no <chkscope> argument is given, it applies to all hierarchies for
which group statements appear in the body of the message.
Since much existing software does not honor the <chkscope> argument,
the body of the checkgroups control message MUST NOT contain group
statements for newsgroups outside the intended scope and SHOULD
contain a correct newsgroup list even for sub-hierarchies excluded
with "!" <chkscope> terms. News servers, however, MUST honor
<chkscope> as specified here.
The <chksernr> argument may be any positive integer. If present, it
MUST increase with every change to the newsgroup list, MUST NOT ever
decrease, and MUST be included in all subsequent checkgroups control
messages with the same scope. If provided, news servers SHOULD
remember the <chksernr> value of the previous checkgroups control
message honored for a particular hierarchy or sub-hierarchy and
decline to honor any subsequent checkgroups control message for the
same hierarchy or sub-hierarchy with a smaller <chksernr> value or
with no <chksernr> value.
There is no upper limit on the length of <chksernr>, other than the
limitation on the length of header fields. Implementations may
therefore want to do comparisons by zero-padding the shorter of two
<chksernr> values on the left and then doing a string comparison,
rather than assuming <chksernr> can be manipulated as a number.
For example, the following Control header field
Control: checkgroups de !de.alt #2009021301
indicates that the body of the message will list every newsgroup in
the de.* hierarchy, excepting the de.alt.* sub-hierarchy, and should
not be honored if a checkgroups control message with a serial number
greater than 2009021301 was previously honored. The serial number in
this example was formed from the date in YYYYMMDD format, followed by
a two-digit sequence number within that date.
The body of the message is an entity of type application/ news-checkgroups. It SHOULD be declared as such with appropriate MIME headers, but news servers SHOULD interpret checkgroups messages that lack the appropriate MIME headers as if the body were of type application/news-checkgroups for backward compatibility.5.3. The cancel Control Message
The cancel control message requests that a target article be withdrawn from circulation and access. The syntax of its Control header field is: control-command =/ Cancel-command Cancel-command = "cancel" Cancel-arguments Cancel-arguments = 1*WSP msg-id The argument identifies the article to be cancelled by its message identifier. The body of the control message MAY contain anything, usually an explanatory text. A serving agent that elects to honor a cancel message SHOULD make the article unavailable to reading agents (perhaps by deleting it completely). If the cancel control message arrives before the article it targets, news servers choosing to honor it SHOULD remember the message identifier that was cancelled and reject the cancelled article when it arrives. To best ensure that it will be relayed to the same news servers as the original message, a cancel control message SHOULD have the same Newsgroups header field as the message it is cancelling. Cancel control messages listing moderated newsgroups in their Newsgroups header field MUST contain an Approved header field like any other article in a moderated newsgroup. This means that cancels posted to a moderated newsgroup will normally be sent to the moderator first for approval. Outside of moderated newsgroups, cancel messages are not required to contain an Approved header field. Contrary to [RFC1036], cancel control messages are not required to contain From and Sender header fields matching the target message. This requirement only encouraged cancel issuers to conceal their identity and provided no security.5.4. The Supersedes Header Field
The presence of a Supersedes header field in an article requests that the message identifier given in that header field be withdrawn in exactly the same manner as if it were the target of a cancel control
message. Accordingly, news servers SHOULD apply to a Supersedes header field the same authentication and authorization checks as they would apply to cancel control messages. If the Supersedes header field is honored, the news server SHOULD take the same actions as it would take when honoring a cancel control message for the given target article. The article containing the Supersedes header field, whether or not the Supersedes header field is honored, SHOULD be handled as a normal article and SHOULD NOT receive the special treatment of control messages described in Section 3.7.5.5. The ihave and sendme Control Messages
The ihave and sendme control messages implement a predecessor of the NNTP [RFC3977] protocol. They are largely obsolete on the Internet but still see use in conjunction with some transport protocols such as UUCP [UUCP]. News servers are not required to support them. ihave and sendme control messages share similar syntax for their Control header fields and bodies: control-command =/ Ihave-command Ihave-command = "ihave" Ihave-arguments Ihave-arguments = 1*WSP *( msg-id 1*WSP ) relayer-name control-command =/ Sendme-command Sendme-command = "sendme" Sendme-arguments Sendme-arguments = Ihave-arguments relayer-name = path-identity ; see 3.1.5 of [RFC5536] ihave-body = *( msg-id CRLF ) sendme-body = ihave-body The body of the article consists of a list of <msg-id>s, one per line. The message identifiers SHOULD be put in the body of the article, not in the Control header field, but news servers MAY recognize and process message identifiers in the Control header field for backward compatibility. Message identifiers MUST NOT be put in the Control header field if they are present in the body of the control message. The ihave message states that the named relaying agent has received articles with the specified message identifiers, which may be of interest to the relaying agents receiving the ihave message. The sendme message requests that the agent receiving it send the articles having the specified message identifiers to the named relaying agent. Contrary to [RFC1036], the relayer-name MUST be given as the last argument in the Control header field.
Upon receipt of the sendme message (and a decision to honor it), the receiving agent sends the article or articles requested. The mechanism for this transmission is unspecified by this document and is arranged between the sites involved. These control messages are normally sent as point-to-point articles between two sites and not then sent on to other sites. Newsgroups beginning with "to." are reserved for such point-to-point communications and are formed by prepending "to." to a <relayer-name> to form a <newsgroup-name>. Articles with such a group in their Newsgroups header fields SHOULD NOT be sent to any news server other than the one identified by <relayer-name>.5.6. Obsolete Control Messages
The following control message types are declared obsolete by this document and SHOULD NOT be sent or honored: sendsys version whogets senduuname6. Security Considerations
Netnews is designed for broad dissemination of public messages and offers little in the way of security. What protection Netnews has against abuse and impersonation is provided primarily by the underlying transport layer. In large Netnews networks where news servers cannot be relied upon to enforce authentication and authorization requirements at the transport layer, articles may be trivially forged and widely read, and the identities of article senders and the privacy of articles cannot be assured. See Section 5 of [RFC5536] for further security considerations related to the format of articles.6.1. Compromise of System Integrity
Control messages pose a particular security concern since acting on unauthorized control messages may cause newsgroups to be removed, articles to be deleted, and unwanted newsgroups to be created. Administrators of news servers SHOULD therefore take steps to verify the authenticity of control messages as discussed in Section 5.1. Articles containing Supersedes header fields are effectively cancel control messages and SHOULD be subject to the same checks as
discussed in Section 5.4. Currently, many sites are ignoring all cancel control messages and Supersedes header fields due to the difficulty of authenticating them and their widespread abuse. Cancel control messages are not required to have the same Newsgroups header field as the messages they are cancelling. Since they are sometimes processed before the original message is received, it may not be possible to check that the Newsgroup header fields match. This allows a malicious poster to inject a cancel control message for an article in a moderated newsgroup without adding an Approved header field to the control message, and to hide malicious cancel control messages from some reading agents by using a different Newsgroups header field so that the cancel control message is not accepted by all news servers that accepted the original message. All agents should be aware that all article content, most notably including the content of the Control header field, is potentially untrustworthy and malicious. Articles may be constructed in syntactically invalid ways to attempt to overflow internal buffers, violate hidden assumptions, or exploit implementation weaknesses. For example, some news server implementations have been successfully attacked via inclusion of Unix shell code in the Control header field. All article contents, and particularly control message contents, SHOULD be handled with care and rigorously verified before any action is taken on the basis of the contents of the article. A malicious poster may add an Approved header field to bypass the moderation process of a moderated newsgroup. Injecting agents SHOULD verify that messages approved for a moderated newsgroup are being injected by the moderator using authentication information from the underlying transport or some other authentication mechanism arranged with the moderator. There is, at present, no standardized method of authenticating approval of messages to moderated groups, although some unstandardized authentication methods such as [PGPMOOSE] are in common use. A malicious news server participating in a Netnews network may bypass checks performed by injecting agents, forge Path header fields and other trace information (such as Injection-Info header fields), and otherwise compromise the authorization requirements of a Netnews network. News servers SHOULD use the facilities of the underlying transport to authenticate their peers and reject articles from injecting and relaying agents that do not follow the requirements of this protocol or the Netnews network.
6.2. Denial of Service
The proper functioning of individual newsgroups can be disrupted by the excessive posting of unwanted articles; by the repeated posting of identical or near identical articles; by posting followups that either are unrelated to their precursors or that quote their precursors in full with the addition of minimal extra material (especially if this process is iterated); by crossposting to, or requesting followups to, totally unrelated newsgroups; and by abusing control messages and the Supersedes header field to delete articles or newsgroups. Such articles intended to deny service, or other articles of an inflammatory nature, may also have their From or Reply-To addresses set to valid but incorrect email addresses, thus causing large volumes of email to descend on the true owners of those addresses. Users and agents should always be aware that identifying information in articles may be forged. A malicious poster may prevent an article from being seen at a particular site by including in the Path header field of the proto- article the <path-identity> of that site. Use of the <diag-keyword> "POSTED" by injecting agents to mark the point of injection can prevent this attack. Primary responsibility for preventing such attacks lies with injecting agents, which can apply authentication and authorization checks via the underlying transport and prevent those attempting denial-of-service attacks from posting messages. If specific injecting agents fail to live up to their responsibilities, they may be excluded from the Netnews network by configuring relaying agents to reject articles originating from them. A malicious complainer may submit a modified copy of an article (with an altered Injection-Info header field, for instance) to the administrator of an injecting agent in an attempt to discredit the author of that article and even to have his posting privileges removed. Administrators SHOULD therefore obtain a genuine copy of the article from their own serving agent before taking action in response to such a complaint.6.3. Leakage
Articles that are intended to have restricted distribution are dependent on the goodwill of every site receiving them. Restrictions on dissemination and retention of articles may be requested via the Archive and Distribution header fields, but such requests cannot be enforced by the protocol.
The flooding algorithm used by Netnews transports such as NNTP [RFC3977] is extremely good at finding any path by which articles can leave a subnet with supposedly restrictive boundaries, and substantial administrative effort is required to avoid this. Organizations wishing to control such leakage are advised to designate a small number of gateways to handle all news exchanges with the outside world. The sendme control message (Section 5.5), insofar as it is still used, can be used to request articles that the requester should not have access to.7. IANA Considerations
IANA has registered the following media types, described elsewhere in this document, for use with the Content-Type header field, in the IETF tree in accordance with the procedures set out in [RFC4288]. application/news-transmission (4.1) application/news-groupinfo (4.2) application/news-checkgroups (4.3) application/news-transmission is a change to a previous registration. IANA has registered the new header field, Original-Sender, in the Permanent Message Header Field Repository, using the template in Section 3.10.3. IANA has changed the status of the message/news media type to "OBSOLETE". message/rfc822 should be used instead. An updated template is included in Section 4.8. References
8.1. Normative References
[ASCII] American National Standard for Information Systems, "Coded Character Sets - 7-Bit American National Standard Code for Information Interchange (7-Bit ASCII)", ANSI X3.4, 1986. [RFC2046] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types", RFC 2046, November 1996. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, November 2003. [RFC4288] Freed, N. and J. Klensin, "Media Type Specifications and Registration Procedures", BCP 13, RFC 4288, December 2005. [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008. [RFC5322] Resnick, P., Ed., "Internet Message Format", RFC 5322, October 2008. [RFC5536] Murchison, K., Ed., Lindsey, C., and D. Kohn, "Netnews Article Format", RFC 5536, November 2009.8.2. Informative References
[PGPMOOSE] Rose, G., "PGP Moose", November 1998. [PGPVERIFY] Lawrence, D., "Signing Control Messages", August 2001, <ftp://ftp.isc.org/pub/pgpcontrol/FORMAT>. [RFC1036] Horton, M. and R. Adams, "Standard for interchange of USENET messages", RFC 1036, December 1987. [RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, November 1996. [RFC2606] Eastlake, D. and A. Panitz, "Reserved Top Level DNS Names", BCP 32, RFC 2606, June 1999. [RFC3798] Hansen, T. and G. Vaudreuil, "Message Disposition Notification", RFC 3798, May 2004. [RFC3977] Feather, C., "Network News Transfer Protocol (NNTP)", RFC 3977, October 2006. [SON-OF-1036] Spencer, H., "News Article Format and Transmission", Work in Progress, May 2009. [USEAGE] Lindsey, C., "Usenet Best Practice", Work in Progress, March 2005. [UUCP] O'Reilly, T. and G. Todino, "Managing UUCP and Usenet", O'Reilly & Associates Ltd., January 1992.
Appendix A. Changes to the Existing Protocols
This document prescribes many changes, clarifications, and new features since the protocol described in [RFC1036]. Most notably: o A new, backward-compatible Path header field format that permits standardized embedding of additional trace and authentication information is now RECOMMENDED. See Section 3.2. Folding of the Path header is permitted. o Trimming of the References header field is REQUIRED, and a mechanism for doing so is defined. o Addition of the new Injection-Date header field is required in some circumstances for posting agents (Section 3.4.2) and injecting agents (Section 3.5), and MUST be used by news servers for date checks (Section 3.6). Injecting agents are also strongly encouraged to put all local trace information in the new Injection-Info header field. o A new media type is defined for transmitting Netnews articles through other media (Section 4.1), and moderators SHOULD prepare to receive submissions in that format (Section 3.5.1). o Certain control messages (Section 5.6) are declared obsolete, and the special significance of "cmsg" at the start of a Subject header field is removed. o Additional media types are defined for improved structuring, specification, and automated processing of control messages (Sections 4.2 and 4.3). o Two new optional parameters are added to the checkgroups control message. o The message/news media type is declared obsolete. o Cancel control messages are no longer required to have From and Sender header fields matching those of the target message, as this requirement added no real security. o The relayer-name parameter in the Control header field of ihave and sendme control messages is now required. In addition, many protocol steps and article verification requirements that are unmentioned or left ambiguous by [RFC1036] but are widely implemented by Netnews servers have been standardized and specified in detail.
Appendix B. Acknowledgements
This document is the result of a twelve-year effort and the number of people that have contributed to its content are too numerous to mention individually. Many thanks go out to all past and present members of the USEFOR Working Group of the Internet Engineering Task Force (IETF) and the accompanying mailing list. Special thanks are due to Henry Spencer, whose [SON-OF-1036] draft served as the initial basis for this document.Authors' Addresses
Russ Allbery (editor) Stanford University P.O. Box 20066 Stanford, CA 94309 US EMail: rra@stanford.edu URI: http://www.eyrie.org/~eagle/ Charles H. Lindsey 5 Clerewood Avenue Heald Green Cheadle Cheshire SK8 3JU United Kingdom Phone: +44 161 436 6131 EMail: chl@clerew.man.ac.uk