7 Remote variant selection algorithms A remote variant selection algorithm is a standardized algorithm by which a server can choose a best variant on behalf of a negotiating user agent. The use of a remote algorithm can speed up the negotiation process by eliminating a request-response round trip. A remote algorithm typically computes whether the Accept- headers in the request contain sufficient information to allow a choice, and if so, which variant is the best variant. This specification does not define any remote algorithms, but does define a mechanism to negotiate on the use of such algorithms. 7.1 Version numbers A version numbering scheme is used to distinguish between different remote variant selection algorithms. rvsa-version = major "." minor major = 1*4DIGIT minor = 1*4DIGIT An algorithm with the version number X.Y, with Y>0, MUST be downwards compatible with all algorithms from X.0 up to X.Y. Downwards compatibility means that, if supplied with the same information, the newer algorithm MUST make the same choice, or a better choice, as the old algorithm. There are no compatibility requirements between algorithms with different major version numbers. 8 Content negotiation status codes and headers This specification adds one new HTTP status code, and introduces six new HTTP headers. It also extends the semantics of an existing HTTP/1.1 header. 8.1 506 Variant Also Negotiates The 506 status code indicates that the server has an internal configuration error: the chosen variant resource is configured to engage in transparent content negotiation itself, and is therefore not a proper end point in the negotiation process.
8.2 Accept-Features The Accept-Features request header can be used by a user agent to give information about the presence or absence of certain features in the feature set of the current request. Servers can use this information when running a remote variant selection algorithm. Note: the name `Accept-Features' for this header was chosen because of symmetry considerations with other Accept- headers, even though the Accept-Features header will generally not contain an exhaustive list of features which are somehow `accepted'. A more accurate name of this header would have been `Feature-Set- Info'. Accept-Features = "Accept-Features" ":" #( feature-expr *( ";" feature-extension ) ) feature-expr = [ "!" ] ftag | ftag ( "=" | "!=" ) tag-value | ftag "=" "{" tag-value "}" | "*" feature-extension = token [ "=" ( token | quoted-string ) ] No feature extensions are defined in this specification. An example is: Accept-Features: blex, !blebber, colordepth={5}, !screenwidth, paper = A4, paper!="A2", x-version=104, * The different feature expressions have the following meaning: ftag ftag is present !ftag ftag is absent ftag=V ftag is present with the value V ftag!=V ftag is present, but not with the value V ftag={V} ftag is present with the value V, and not with any other values * the expressions in this header do not fully describe the feature set: feature tags not mentioned in this header may also be present, and, except for the case ftag={V}, tags may be present with more values than mentioned.
Absence of the Accept-Features header in a request is equivalent to the inclusion of Accept-Features: * By using the Accept-Features header, a remote variant selection algorithm can sometimes determine the truth value of a feature predicate on behalf of the user agent. For example, with the header Accept-Features: blex, !blebber, colordepth={5}, !screenwidth, paper = A4, paper!="A2", x-version=104, * the algorithm can determine that the following predicates are true: blex, colordepth=[4-], colordepth!=6, colordepth, !screenwidth, paper=A4, colordepth=[4-6] and that the following predicates are false: !blex, blebber, colordepth=6, colordepth=foo, !colordepth, screenwidth, screenwidth=640, screenwidth!=640, but the truth value of the following predicates cannot be determined: UA-media=stationary, UA-media!=screen, paper!=a0, x-version=[100-300], x-version=[200-300], x-version=99, UA-media=screen, paper=A0, paper=a4, x-version=[100-199], wuxta 8.3 Alternates The Alternates response header is used to convey the list of variants bound to a negotiable resource. This list can also include directives for any content negotiation process. If a response from a transparently negotiable resource includes an Alternates header, this header MUST contain the complete variant list bound to the negotiable resource. Responses from resources which do not support transparent content negotiation MAY also use Alternates headers. Alternates = "Alternates" ":" variant-list variant-list = 1#( variant-description | fallback-variant | list-directive ) fallback-variant = "{" <"> URI <"> "}" list-directive = ( "proxy-rvsa" "=" <"> 0#rvsa-version <"> )
| extension-list-directive extension-list-directive = token [ "=" ( token | quoted-string ) ] An example is Alternates: {"paper.1" 0.9 {type text/html} {language en}}, {"paper.2" 0.7 {type text/html} {language fr}}, {"paper.3" 1.0 {type application/postscript} {language en}}, proxy-rvsa="1.0, 2.5" Any relative URI specified in a variant-description or fallback- variant field is relative to the request-URI. Only one fallback- variant field may be present. If the variant selection algorithm of the user agent finds that all described variants are unacceptable, then it SHOULD choose the fallback variant, if present, as the best variant. If the user agent computes the overall quality values of the described variants, and finds that several variants share the highest value, then the first variant with this value in the list SHOULD be chosen as the best variant. The proxy-rvsa directive restricts the use of remote variant selection algorithms by proxies. If present, a proxy MUST ONLY use algorithms which have one of the version numbers listed, or have the same major version number and a higher minor version number as one of the versions listed. Any restrictions set by proxy-rvsa come on top of the restrictions set by the user agent in the Negotiate request header. The directive proxy-rvsa="" will disable variant selection by proxies entirely. Clients SHOULD ignore all extension-list- directives they do not understand. A variant list may contain multiple differing descriptions of the same variant. This can be convenient if the variant uses conditional rendering constructs, or if the variant resource returns multiple representations using a multipart media type. 8.4 Negotiate The Negotiate request header can contain directives for any content negotiation process initiated by the request. Negotiate = "Negotiate" ":" 1#negotiate-directive negotiate-directive = "trans" | "vlist" | "guess-small"
| rvsa-version | "*" | negotiate-extension negotiate-extension = token [ "=" token ] Examples are Negotiate: 1.0, 2.5 Negotiate: * The negotiate directives have the following meaning "trans" The user agent supports transparent content negotiation for the current request. "vlist" The user agent requests that any transparently negotiated response for the current request includes an Alternates header with the variant list bound to the negotiable resource. Implies "trans". "guess-small" The user agent allows origin servers to run a custom algorithm which guesses the best variant for the request, and to return this variant in a choice response, if the resulting choice response is smaller than or not much larger than a list response. The definition of `not much larger' is left to origin server heuristics. Implies "vlist" and "trans". rvsa-version The user agent allows origin servers and proxies to run the remote variant selection algorithm with the indicated version number, or with the same major version number and a higher minor version number. If the algorithm has sufficient information to choose a best, neighboring variant, the origin server or proxy MAY return a choice response with this variant. Implies "trans". "*" The user agent allows origin servers and proxies to run any remote variant selection algorithm. The origin server may even run algorithms which have not been standardized. If the algorithm has sufficient information to choose a best, neighboring variant, the origin server or proxy MAY return a choice response with this variant. Implies "trans".
Servers SHOULD ignore all negotiate-directives they do not understand. If the Negotiate header allows a choice between multiple remote variant selection algorithms which are all supported by the server, the server SHOULD use some internal precedence heuristics to select the best algorithm. 8.5 TCN The TCN response header is used by a server to signal that the resource is transparently negotiated. TCN = "TCN" ":" #( response-type | server-side-override-directive | tcn-extension ) response-type = "list" | "choice" | "adhoc" server-side-override-directive = "re-choose" | "keep" tcn-extension = token [ "=" ( token | quoted-string ) ] If the resource is not transparently negotiated, a TCN header MUST NOT be included in any response. If the resource is transparently negotiated, a TCN header, which includes the response-type value of the response, MUST be included in every response with a 2xx status code or any 3xx status code, except 304, in which it MAY be included. A TCN header MAY also be included, without a response-type value, in other responses from transparently negotiated resources. A server-side override directive MUST be included if the origin server performed a server-side override when choosing the response. If the directive is "re-choose", the server MUST include an Alternates header with the variant bound to the negotiable resource in the response, and user agent SHOULD use its internal variant selection algorithm to choose, retrieve, and display the best variant from this list. If the directive is "keep" the user agent SHOULD NOT renegotiate on the response, but display it directly, or act on it directly if it is a redirection response. Clients SHOULD ignore all tcn-extensions they do not understand. 8.6 Variant-Vary The Variant-Vary response header can be used in a choice response to record any vary information which applies to the variant data (the entity body combined with some of the entity headers) contained in the response, rather than to the response as a whole.
Variant-Vary = "Variant-Vary" ":" ( "*" | 1#field-name ) Use of the Variant-Vary header is discussed in section 10.2. 9 Cache validators To allow for correct and efficient caching and revalidation of negotiated responses, this specification extends the caching model of HTTP/1.1 [1] in various ways. This specification does not introduce a `variant-list-max-age' directive which explicitly bounds the freshness lifetime of a cached variant list, like the `max-age' Cache-Control directive bounds the freshness lifetime of a cached response. However, this specification does ensure that a variant list which is sent at a time T by the origin server will never be re-used without revalidation by semantically transparent caches after the time T+M. This M is the maximum of all freshness lifetimes assigned (using max-age directives or Expires headers) by the origin server to a. the responses from the negotiable resource itself, and b. the responses from its neighboring variant resources If no freshness lifetimes are assigned by the origin server, M is the maximum of the freshness lifetimes which were heuristically assigned by all caches which can re-use the variant list. 9.1 Variant list validators A variant list validator is an opaque value which acts as the cache validator of a variant list bound to a negotiable resource. variant-list-validator = <quoted-string not containing any ";"> If two responses contain the same variant list validator, a cache can treat the Alternates headers in these responses as equivalent (though the headers themselves need not be identical). 9.2 Structured entity tags A structured entity tag consists of a normal entity tag of which the opaque string is extended with a semicolon followed by the text (without the surrounding quotes) of a variant list validator:
normal | variant list | structured entity tag | validator | entity tag -------------+----------------+----------------- "etag" | "vlv" | "etag;vlv" W/"etag" | "vlv" | W/"etag;vlv" Note that a structured entity tag is itself also an entity tag. The structured nature of the tag allows caching proxies capable of transparent content negotiation to perform some optimizations defined in section 10. When not performing such optimizations, a structured tag SHOULD be treated as a single opaque value, according to the general rules in HTTP/1.1. Examples of structured entity tags are: "xyzzy;1234" W/"xyzzy;1234" "gonkxxxx;1234" "a;b;c;;1234" In the last example, the normal entity tag is "a;b;c;" and the variant list validator is "1234". If a transparently negotiated response includes an entity tag, it MUST be a structured entity tag. The variant list validator in the structured tag MUST act as a validator for the variant list contained in the Alternates header. The normal entity tag in the structured tag MUST act as a validator of the entity body in the response and of all entity headers except Alternates. 9.3 Assigning entity tags to variants To allow for correct revalidation of transparently negotiated responses by clients, origin servers SHOULD generate all normal entity tags for the neighboring variant resources of the negotiable resource in such a way that 1. the same tag is never used by two different variants, unless this tag labels exactly the same entity on all occasions, 2. if one normal tag "X" is a prefix of another normal tag "XY", then "Y" must never be a semicolon followed by a variant list validator. 10 Content negotiation responses If a request on a transparently negotiated resource yields a response with a 2xx status code or any 3xx status code except 304, this response MUST always be either a list response, a choice response, or an adhoc response. These responses MUST always include a TCN header which specifies their type. Transparently negotiated responses with other status codes MAY also include a TCN header.
The conditions under which the different content negotiation responses may be sent are defined in section 12.1 for origin servers and in section 13 for proxies. After having constructed a list, choice, or adhoc response, a server MAY process any If-No-Match or If-Range headers in the request message and shorten the response to a 304 (Not Modified) or 206 (Partial Content) response, following the rules in the HTTP/1.1 specification [1]. In this case, the entity tag of the shortened response will identify it indirectly as a list, choice, or adhoc response. 10.1 List response A list response returns the variant list of the negotiable resource, but no variant data. It can be generated when the server does not want to, or is not allowed to, return a particular best variant for the request. If the user agent supports transparent content negotiation, the list response will cause it to select a best variant and retrieve it. A list response MUST contain (besides the normal headers required by HTTP) a TCN header which specifies the "list" response-type, the Alternates header bound to the negotiable resource, a Vary header and (unless it was a HEAD request) an entity body which allows the user to manually select the best variant. An example of a list response is HTTP/1.1 300 Multiple Choices Date: Tue, 11 Jun 1996 20:02:21 GMT TCN: list Alternates: {"paper.1" 0.9 {type text/html} {language en}}, {"paper.2" 0.7 {type text/html} {language fr}}, {"paper.3" 1.0 {type application/postscript} {language en}} Vary: negotiate, accept, accept-language ETag: "blah;1234" Cache-control: max-age=86400 Content-Type: text/html Content-Length: 227 <h2>Multiple Choices:</h2> <ul> <li><a href=paper.1>HTML, English version</a> <li><a href=paper.2>HTML, French version</a> <li><a href=paper.3>Postscript, English version</a> </ul>
Note: A list response can have any status code, but the 300 (Multiple Choices) code is the most appropriate one for HTTP/1.1 clients. Some existing versions of HTTP/1.0 clients are known to silently ignore 300 responses, instead of handling them according to the HTTP/1.0 specification [2]. Servers should therefore be careful in sending 300 responses to non-negotiating HTTP/1.0 user agents, and in making these responses cacheable. The 200 (OK) status code can be used instead. The Vary header in the response SHOULD ensure correct handling by plain HTTP/1.1 caching proxies. This header can either be Vary: * or a more elaborate header; see section 10.6.1. Only the origin server may construct list responses. Depending on the status code, a list response is cacheable unless indicated otherwise. According to the HTTP/1.1 specification [1], a user agent which does not support transparent content negotiation will, when receiving a list response with the 300 status code, display the entity body included in the response. If the response contains a Location header, however, the user agent MAY automatically redirect to this location. The handling of list responses by clients supporting transparent content negotiation is described in sections 11.1 and 13. 10.2 Choice response A choice response returns a representation of the best variant for the request, and may also return the variant list of the negotiable resource. It can be generated when the server has sufficient information to be able to choose the best variant on behalf the user agent, but may only be generated if this best variant is a neighboring variant. For request from user agents which do not support transparent content negotiation, a server may always generate a choice response, provided that the variant returned is a neighboring variant. The variant returned in a choice response need not necessarily be listed in the variant list bound to the negotiable resource.
A choice response merges a normal HTTP response from the chosen variant, a TCN header which specifies the "choice" response-type, and a Content-Location header giving the location of the variant. Depending on the status code, a choice response is cacheable unless indicated otherwise. Origin servers and proxy caches MUST construct choice responses with the following algorithm (or any other algorithm which gives equal end results for the client). In this algorithm, `the current Alternates header' refers to the Alternates header containing the variant list which was used to choose the best variant, and `the current variant list validator' refers to the validator of this list. Section 10.4 specifies how these two items can be obtained by a proxy cache. The algorithm consists of four steps. 1. Construct a HTTP request message on the best variant resource by rewriting the request-URI and Host header (if appropriate) of the received request message on the negotiable resource. 2. Generate a valid HTTP response message, but not one with the 304 (Not Modified) code, for the request message constructed in step 1. In a proxy cache, the response can be obtained from cache memory, or by passing the constructed HTTP request towards the origin server. If the request is passed on, the proxy MAY add, modify, or delete If-None-Match and If-Range headers to optimize the transaction with the upstream server. Note: the proxy should be careful not to add entity tags of non-neighboring variants to If-* (conditional) headers of the request, as there are no global uniqueness requirements for these tags. 3. Only in origin servers: check for an origin server configuration error. If the HTTP response message generated in step 2 contains a TCN header, then the best variant resource is not a proper end point in the transparent negotiation process, and a 506 (Variant Also Negotiates) error response message SHOULD be generated instead of going to step 4. 4. Add a number of headers to the HTTP response message generated in step 2.
a. Add a TCN header which specifies the "choice" response-type. b. Add a Content-Location header giving the location of the chosen variant. Delete any Content-Location header which was already present. Note: According to the HTTP/1.1 specification [1], if the Content-Location header contains a relative URI, this URI is relative to the URI in the Content-Base header, if present, and relative to the request-URI if no Content- Base header is present. c. If any Vary headers are present in the response message from step 2, add, for every Vary header, a Variant-Vary header with a copy of the contents of this Vary header. d. Delete any Alternates headers which are present in in the response. Now, the current Alternates header MUST be added if this is required by the Negotiate request header, or if the server returns "re-choose" in the TCN response header. Otherwise, the current Alternates header MAY be added. Note: It is usually a good strategy to always add the current Alternates header, unless it is very large compared to the rest of the response. e. Add a Vary header to ensure correct handling by plain HTTP/1.1 caching proxies. This header can either be Vary: * or a more elaborate header, see section 10.6. f. To ensure compatibility with HTTP/1.0 caching proxies which do not recognize the Vary header, an Expires header with a date in the past MAY be added. See section 10.7 for more information. g. If an ETag header is present in the response message from step 2, then extend the entity tag in that header with the current variant list validator, as specified in section 9.2. Note: Step g. is required even if the variant list itself is not added in step d. h. Only in proxy caches: set the Age header of the response to max( variant_age , alternates_age )
where variant_age is the age of the variant response obtained in step 2, calculated according to the rules in the HTTP/1.1 specification [1], and alternates_age is the age of the Alternates header added in step d, calculated according to the rules in section 10.4. Note that a server can shorten the response produced by the above algorithm to a 304 (Not Modified) response if an If-None-Match header in the original request allows it. If this is the case, an implementation of the above algorithm can avoid the unnecessary internal construction of full response message in step 2, it need only construct the parts which end up in the final 304 response. A proxy cache which implements this optimization can sometimes generate a legal 304 response even if it has not cached the variant data itself. An example of a choice response is: HTTP/1.1 200 OK Date: Tue, 11 Jun 1996 20:05:31 GMT TCN: choice Content-Type: text/html Last-Modified: Mon, 10 Jun 1996 10:01:14 GMT Content-Length: 5327 Cache-control: max-age=604800 Content-Location: paper.1 Alternates: {"paper.1" 0.9 {type text/html} {language en}}, {"paper.2" 0.7 {type text/html} {language fr}}, {"paper.3" 1.0 {type application/postscript} {language en}} Etag: "gonkyyyy;1234" Vary: negotiate, accept, accept-language Expires: Thu, 01 Jan 1980 00:00:00 GMT <title>A paper about .... 10.3 Adhoc response An adhoc response can be sent by an origin server as an extreme measure, to achieve compatibility with a non-negotiating or buggy client if this compatibility cannot be achieved by sending a list or choice response. There are very little requirements on the contents of an adhoc response. An adhoc response MUST have a TCN header which specifies the "adhoc" response-type, and a Vary header if the response is cacheable. It MAY contain the Alternates header bound to the negotiable resource.
Any Vary header in the response SHOULD ensure correct handling by plain HTTP/1.1 caching proxies. This header can either be Vary: * or a more elaborate header, see section 10.6.1. Depending on the status code, an adhoc response is cacheable unless indicated otherwise. As an example of the use of an adhoc response, suppose that the variant resource "redirect-to-blah" yields redirection (302) responses. A choice response with this variant could look as follows: HTTP/1.1 302 Moved Temporarily Date: Tue, 11 Jun 1996 20:02:28 GMT TCN: choice Content-location: redirect-to-blah Location: http://blah.org/ Content-Type: text/html Content-Length: 62 This document is available <a href=http://blah.org/>here</a>. Suppose that the server knows that the receiving user agent has a bug, which causes it to crash on responses which contain both a Content-Location and a Location header. The server could then work around this bug by performing a server-side override and sending the following adhoc response instead: HTTP/1.1 302 Moved Temporarily Date: Tue, 11 Jun 1996 20:02:28 GMT TCN: adhoc, keep Location: http://blah.org/ Content-Type: text/html Content-Length: 62 This document is available <a href=http://blah.org/>here</a>. 10.4 Reusing the Alternates header If a proxy cache has available a negotiated response which is cacheable, fresh, and has ETag and Alternates headers, then it MAY extract the Alternates header and associated variant list validator from the response, and reuse them (without unnecessary delay) to
negotiate on behalf of the user agent (section 13) or to construct a choice response (section 10.2). The age of the extracted Alternates header is the age of the response from which it is extracted, calculated according to the rules in the HTTP/1.1 specification [1]. 10.5 Extracting a normal response from a choice response If a proxy receives a choice response, it MAY extract and cache the normal HTTP response contained therein. The normal response can be extracted by taking a copy of the choice response and then deleting any Content-Location, Alternates, and Vary headers, renaming any Variant-Vary headers to Vary headers, and shortening the structured entity tag in any ETag header to a normal entity tag. This normal response MAY be cached (as a HTTP response to the variant request as constructed in step 1. of section 10.2) and reused to answer future direct requests on the variant resource, according to the rules in the HTTP/1.1 specification [1]. Note: The caching of extracted responses can decrease the upstream bandwidth usage with up to a factor 2, because two independent HTTP/1.1 cache entries, one associated with the negotiable resource URI and one with the variant URI, are created in the same transaction. Without this optimization, both HTTP/1.1 cache entries can only be created by transmitting the variant data twice. For security reasons (see section 14.2), an extracted normal response MUST NEVER be cached if belongs to a non-neighboring variant resource. If the choice response claims to contain data for a non- neighboring variant resource, the proxy SHOULD reject the choice response as a probable spoofing attempt. 10.6 Elaborate Vary headers If a HTTP/1.1 [1] server can generate varying responses for a request on some resource, then the server MUST include a Vary header in these responses if they are cacheable. This Vary header is a signal to HTTP/1.1 caches that something special is going on. It prevents the caches from returning the currently chosen response for every future request on the resource. Servers engaging in transparent content negotiation will generate varying responses. Therefore, cacheable list, choice, and adhoc responses MUST always include a Vary header.
The most simple Vary header which can be included is Vary: * This header leaves the way in which the response is selected by the server completely unspecified. A more elaborate Vary header MAY be used to allow for certain optimizations in HTTP/1.1 caches which do not have specific optimizations for transparent content negotiation, but which do cache multiple variant responses for one resource. Such a more elaborate Vary header lists all request headers which can be used by the server when selecting a response for a request on the resource. 10.6.1 Construction of an elaborate Vary header Origin servers can construct a more elaborate Vary header in the following way. First, start with the header Vary: negotiate `negotiate' is always included because servers use the information in the Negotiate header when choosing between a list, choice, or adhoc response. Then, if any of the following attributes is present in any variant description in the Alternates header, add the corresponding header name to the Vary header attribute | header name to add -----------+--------------------- type | accept charset | accept-charset language | accept-language features | accept-features The Vary header constructed in this way specifies the response variation which can be caused by the use of a variant selection algorithm in proxies. If the origin server will in some cases, for example if contacted by a non-negotiating user agent, use a custom negotiation algorithm which takes additional headers into account, these names of these headers SHOULD also be added to the Vary header.
10.6.2 Caching of an elaborate Vary header A proxy cache cannot construct an elaborate vary header using the method above, because this method requires exact knowledge of any custom algorithms present in the origin server. However, when extracting an Alternates header from a response (section 10.4) caches MAY also extract the Vary header in the response, and reuse it along with the Alternates header. A clean Vary header can however only be extracted if the variant does not vary itself, i.e. if a Variant-Vary header is absent. 10.7 Adding an Expires header for HTTP/1.0 compatibility To ensure compatibility with HTTP/1.0 caching proxies which do not recognize the Vary header, an Expires header with a date in the past can be added to the response, for example Expires: Thu, 01 Jan 1980 00:00:00 GMT If this is done by an origin server, the server SHOULD usually also include a Cache-Control header for the benefit of HTTP/1.1 caches, for example Cache-Control: max-age=604800 which overrides the freshness lifetime of zero seconds specified by the included Expires header. Note: This specification only claims downwards compatibility with the HTTP/1.0 proxy caches which implement the HTTP/1.0 specification [2]. Some legacy proxy caches which return the HTTP/1.0 protocol version number do not honor the HTTP/1.0 Expires header as specified in [2]. Methods for achieving compatibility with such proxy caches are beyond the scope of this specification. 10.8 Negotiation on content encoding Negotiation on the content encoding of a response is orthogonal to transparent content negotiation. The rules for when a content encoding may be applied are the same as in HTTP/1.1: servers MAY content-encode responses that are the result of transparent content negotiation whenever an Accept-Encoding header in the request allows it. When negotiating on the content encoding of a cacheable response, servers MUST add the accept-encoding header name to the Vary header of the response, or add `Vary: *'.
Servers SHOULD always be able to provide unencoded versions of every transparently negotiated response. This means in particular that every variant in the variant list SHOULD at least be available in an unencoded form. Like HTTP/1.1, this specification allows proxies to encode or decode relayed or cached responses on the fly, unless explicitly forbidden by a Cache-Control directive. The encoded or decoded response still contains the same variant as far as transparent content negotiation is concerned. Note that HTTP/1.1 requires proxies to add a Warning header if the encoding of a response is changed. 11 User agent support for transparent negotiation This section specifies the requirements a user agent needs to satisfy in order to support transparent negotiation. If the user agent contains an internal cache, this cache MUST conform to the rules for proxy caches in section 13. 11.1 Handling of responses If a list response is received when a resource is accessed, the user agent MUST be able to automatically choose, retrieve, and display the best variant, or display an error message if none of the variants are acceptable. If a choice response is received when a resource is accessed, the usual action is to automatically display the enclosed entity. However, if a remote variant selection algorithm which was enabled could have made a choice different from the choice the local algorithm would make, the user agent MAY apply its local algorithm to any variant list in the response, and automatically retrieve and display another variant if the local algorithm makes an other choice. When receiving a choice response, a user agent SHOULD check if variant resource is a neighboring variant resource of the negotiable resource. If this is not the case, the user agent SHOULD reject the choice response as a probable spoofing attempt and display an error message, for example by internally replacing the choice response with a 502 (bad gateway) response. 11.2 Presentation of a transparently negotiated resource If the user agent is displaying a variant which is not an embedded or inlined object and which is the result of transparent content negotiation, the following requirements apply.
1. The user agent SHOULD allow the user to review a list of all variants bound to the negotiable resource, and to manually retrieve another variant if desired. There are two general ways of providing such a list. First, the information in the Alternates header of the negotiable resource could be used to make an annotated menu of variants. Second, the entity included in a list response of the negotiable resource could be displayed. Note that a list response can be obtained by doing a GET request which only has the "trans" directive in the Negotiate header. 2. The user agent SHOULD make available though its user interface some indication that the resource being displayed is a negotiated resource instead of a plain resource. It SHOULD also allow the user to examine the variant list included in the Alternates header. Such a notification and review mechanism is needed because of privacy considerations, see section 14.1. 3. If the user agent shows the URI of the displayed information to the user, it SHOULD be the negotiable resource URI, not the variant URI that is shown. This encourages third parties, who want to refer to the displayed information in their own documents, to make a hyperlink to the negotiable resource as a whole, rather than to the variant resource which happens to be shown. Such correct linking is vital for the interoperability of content across sites. The user agent SHOULD however also provide a means for reviewing the URI of the particular variant which is currently being displayed. 4. Similarly, if the user agent stores a reference to the displayed information for future use, for example in a hotlist, it SHOULD store the negotiable resource URI, not the variant URI. It is encouraged, but not required, that some of the above functionality is also made available for inlined or embedded objects, and when a variant which was selected manually is being displayed. 12 Origin server support for transparent negotiation 12.1 Requirements To implement transparent negotiation on a resource, the origin server MUST be able to send a list response when getting a GET request on the resource. It SHOULD also be able to send appropriate list responses for HEAD requests. When getting a request on a transparently negotiable resource, the origin server MUST NEVER return a response with a 2xx status code or any 3xx status code, except 304, which is not a list, choice, or adhoc response.
If the request includes a Negotiate header with a "vlist" or "trans" directive, but without any directive which allows the server to select a best variant, a list response MUST ALWAYS be sent, except when the server is performing a server-side override for bug compatibility. If the request includes a Negotiate header with a "vlist" or "guess-small" directive, an Alternates header with the variant list bound to the negotiable resource MUST ALWAYS be sent in any list, choice, or adhoc response, except when the server is performing a server-side override for bug compatibility. If the Negotiate header allows it, the origin server MAY run a remote variant selection algorithm. If the algorithm has sufficient information to choose a best variant, and if the best variant is a neighboring variant, the origin server MAY return a choice response with this variant. When getting a request on a transparently negotiable resource from a user agent which does not support transparent content negotiation, the origin server MAY use a custom algorithm to select between sending a list, choice, or adhoc response. The following table summarizes the rules above. |Req on |Usr agnt|server- | Response may be: | |trans neg|capable |side +------+------+------+------+------+ |resource?|of TCN? |override?|list |choice|adhoc |normal|error | +---------+--------+---------+------+------+------+------+------+ | Yes | Yes | No |always|smt(*)|never |never |always| +---------+--------+---------+------+------+------+------+------+ | Yes | Yes | Yes |always|always|always|never |always| +---------+--------+---------+------+------+------+------+------+ | Yes | No | - |always|always|always|never |always| +---------+--------+---------+------+------+------+------+------+ | No | - | - |never |never |never |always|always| +---------+--------+---------+------+------+------+------+------+ (*) sometimes, when allowed by the Negotiate request header Negotiability is a binary property: a resource is either transparently negotiated, or it is not. Origin servers SHOULD NOT vary the negotiability of a resource, or the variant list bound to that resource, based on the request headers which are received. The variant list and the property of being negotiated MAY however change through time. The Cache-Control header can be used to control the propagation of such time-dependent changes through caches. It is the responsibility of the author of the negotiable resource to ensure that all resources in the variant list serve the intended content, and that the variant resources do not engage in transparent
content negotiation themselves. 12.2 Negotiation on transactions other than GET and HEAD If a resource is transparently negotiable, this only has an impact on the GET and HEAD transactions on the resource. It is not possible (under this specification) to do transparent content negotiation on the direct result of a POST request. However, a POST request can return an unnegotiated 303 (See Other) response which causes the user agent to do a GET request on a second resource. This second resource could then use transparent content negotiation to return an appropriate final response. The figure below illustrates this. Server ______ proxy ______ proxy ______ user x.org cache cache agent < ------------------------------------- | POST http://x.org/cgi/submit | <form contents in request body> | -------------------------------------- > 303 See Other | Location: http://x.org/result/OK | | < ------------------------------------- | GET http://x.org/result/OK | small Accept- headers | able to choose on behalf of user agent | ------------------------------------- > choice response with | ..result/OK.nl variant | displays OK.nl See the HTTP/1.1 specification [1] for details on the 303 (See Other) status code. Note that this status code is not understood by some HTTP/1.0 clients. 13 Proxy support for transparent negotiation Transparent content negotiation is an extension on top of HTTP/1.x. It is designed to work through any proxy which only implements the HTTP/1.1 specification [1]. If Expires headers are added as discussed in section 10.7, negotiation will also work though proxies
which implement HTTP/1.0 [2]. Thus, every HTTP/1.0 or HTTP/1.1 proxy provides support for transparent content negotiation. However, if it is to be claimed that a HTTP/1.x proxy offers transparent content negotiation services, at least one of the specific optimizations below MUST be implemented. An HTTP/1.x proxy MUST ONLY optimize (change) the HTTP traffic flowing through it in ways which are explicitly allowed by the specification(s) it conforms to. A proxy which supports transparent content negotiation on top of HTTP/1.x MAY perform the optimizations allowed for by HTTP/1.x. In addition, it MAY perform three additional optimizations, defined below, on the HTTP traffic for transparently negotiated resources and their neighboring variant resources. First, when getting a request on a transparently negotiable resource from a user agent which supports transparent content negotiation, the proxy MAY return any cached, fresh list response from that resource, even if the selecting request headers, as specified by the Vary header, do not match. Second, when allowed by the user agent and origin server, a proxy MAY reuse an Alternates header taken from a previous response (section 10.4) to run a remote variant selection algorithm. If the algorithm has sufficient information to choose a best variant, and if the best variant is a neighboring variant, the proxy MAY return a choice response with this variant. Third, if a proxy receives a choice response, it MAY extract and cache the normal response embedded therein, as described in section 10.5. 14 Security and privacy considerations 14.1 Accept- headers revealing personal information Accept- headers, in particular Accept-Language headers, may reveal information which the user would rather keep private unless it will directly improve the quality of service. For example, a user may not want to send language preferences to sites which do not offer multi- lingual content. The transparent content negotiation mechanism allows user agents to omit sending of the Accept-Language header by default, without adversely affecting the outcome of the negotiation process if transparently negotiated multi-lingual content is accessed.
However, even if Accept- headers are never sent, the automatic selection and retrieval of a variant by a user agent will reveal a preference for this variant to the server. A malicious service author could provide a page with `fake' negotiability on (ethnicity- correlated) languages, with all variants actually being the same English document, as a means of obtaining privacy-sensitive information. Such a plot would however be visible to an alert victim if the list of available variants and their properties is reviewed. Some additional privacy considerations connected to Accept- headers are discussed in [1]. 14.2 Spoofing of responses from variant resources The caching optimization in section 10.5 gives the implementer of a negotiable resource control over the responses cached for all neighboring variant resources. This is a security problem if a neighboring variant resource belongs to another author. To provide security in this case, the HTTP server will have to filter the Content-Location headers in the choice responses generated by the negotiable resource implementation. 14.3 Security holes revealed by negotiation Malicious servers could use transparent content negotiation as a means of obtaining information about security holes which may be present in user agents. This is a risk in particular for negotiation on the availability of scripting languages and libraries. 15 Internationalization considerations This protocol defines negotiation facilities which can be used for the internationalization of web content. For the internationalization of list response bodies (section 10.1), HTTP/1.0 style negotiation (section 4.2) can be used. 16 Acknowledgments Work on HTTP content negotiation has been done since at least 1993. The authors are unable to trace the origin of many of the ideas incorporated in this document. Many members of the HTTP working group have contributed to the negotiation model in this specification. The authors wish to thank the individuals who have commented on earlier versions of this document, including Brian Behlendorf, Daniel DuBois, Martin J. Duerst, Roy T. Fielding, Jim Gettys, Yaron Goland, Dirk van Gulik, Ted Hardie, Graham Klyne, Scott Lawrence, Larry Masinter, Jeffrey Mogul, Henrik Frystyk Nielsen, Frederick G.M. Roeber, Paul Sutton, and Klaus Weide and Mark Wood.
17 References [1] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2068, January 1997. [2] Berners-Lee, T., Fielding, R., and H. Frystyk, "Hypertext Transfer Protocol -- HTTP/1.0", RFC 1945, May 1996. [3] Holtman, K., and A. Mutz, "HTTP Remote Variant Selection Algorithm -- RVSA/1.0", RFC 2296, March 1998. [4] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [5] Yergeau, F., "UTF-8, a transformation format of Unicode and ISO 10646", RFC 2044, October 1996. 18 Authors' Addresses Koen Holtman Technische Universiteit Eindhoven Postbus 513 Kamer HG 6.57 5600 MB Eindhoven (The Netherlands) EMail: koen@win.tue.nl Andrew H. Mutz Hewlett-Packard Company 1501 Page Mill Road 3U-3 Palo Alto CA 94304, USA Fax +1 415 857 4691 EMail: mutz@hpl.hp.com
19 Appendix: Example of a local variant selection algorithm A negotiating user agent will choose the best variant from a variant list with a local variant selection algorithm. This appendix contains an example of such an algorithm. The inputs of the algorithm are a variant list from an Alternates header, and an agent-side configuration database, which contains - the feature set of the current request, - a collection of quality values assigned to media types, languages, and charsets for the current request, following the model of the corresponding HTTP/1.1 [1] Accept- headers, - a table which lists `forbidden' combinations of media types and charsets, i.e. combinations which cannot be displayed because of some internal user agent limitation. The output of the algorithm is either the best variant, or the conclusion that none of the variants are acceptable. 19.1 Computing overall quality values As a first step in the local variant selection algorithm, the overall qualities associated with all variant descriptions in the list are computed. The overall quality Q of a variant description is the value Q = round5( qs * qt * qc * ql * qf * qa ) where rounds5 is a function which rounds a floating point value to 5 decimal places after the point. It is assumed that the user agent can run on multiple platforms: the rounding function makes the algorithm independent of the exact characteristics of the underlying floating point hardware. The factors qs, qt, qc, ql, qf, and qa are determined as follows. qs Is the source quality factor in the variant description. qt The media type quality factor is 1 if there is no type attribute in the variant description. Otherwise, it is the quality value assigned to this type by the configuration database. If the database does not assign a value, then the factor is 0.
qc The charset quality factor is 1 if there is no charset attribute in the variant description. Otherwise, it is the quality value assigned to this charset by the configuration database. If the database does not assign a value, then the factor is 0. ql The language quality factor is 1 if there is no language attribute in the variant description. Otherwise, it is the highest quality value the configuration database assigns to any of the languages listed in the language attribute. If the database does not assign a value to any of the languages listed, then the factor is 0. qf The features quality factor is 1 if there is no features attribute in the variant description. Otherwise, it is the quality degradation factor computed for the features attribute using the feature set of the current request. qa The quality adjustment factor is 0 if the variant description lists a media type - charset combination which is `forbidden' by the table, and 1 otherwise. As an example, if a variant list contains the variant description {"paper.2" 0.7 {type text/html} {language fr}} and if the configuration database contains the quality value assignments types: text/html;q=1.0, type application/postscript;q=0.8 languages: en;q=1.0, fr;q=0.5 then the local variant selection algorithm will compute the overall quality for the variant description as follows: {"paper.2" 0.7 {type text/html} {language fr}} | | | | | | V V V round5 ( 0.7 * 1.0 * 0.5 ) = 0.35000 With same configuration database, the variant list {"paper.1" 0.9 {type text/html} {language en}}, {"paper.2" 0.7 {type text/html} {language fr}}, {"paper.3" 1.0 {type application/postscript} {language en}} would yield the following computations:
round5 ( qs * qt * qc * ql * qf * qa ) = Q --- --- --- --- --- --- paper.1: 0.9 * 1.0 * 1.0 * 1.0 * 1.0 * 1.0 = 0.90000 paper.1: 0.7 * 1.0 * 1.0 * 0.5 * 1.0 * 1.0 = 0.35000 paper.3: 1.0 * 0.8 * 1.0 * 1.0 * 1.0 * 1.0 = 0.80000 19.2 Determining the result Using all computed overall quality values, the end result of the local variant selection algorithm is determined as follows. If all overall quality values are 0, then the best variant is the fallback variant, if there is one in the list, else the result is the conclusion that none of the variants are acceptable. If at least one overall quality value is greater than 0, then the best variant is the variant which has the description with the highest overall quality value, or, if there are multiple variant descriptions which share the highest overall quality value, the variant of the first variant description in the list which has this highest overall quality value. 19.3 Ranking dimensions Consider the following variant list: {"paper.greek" 1.0 {language el} {charset ISO-8859-7}}, {"paper.english" 1.0 {language en} {charset ISO-8859-1}} It could be the case that the user prefers the language "el" over "en", while the user agent can render "ISO-8859-1" better than "ISO- 8859-7". The result is that in the language dimension, the first variant is best, while the second variant is best in the charset dimension. In this situation, it would be preferable to choose the first variant as the best variant: the user settings in the language dimension should take precedence over the hard-coded values in the charset dimension. To express this ranking between dimensions, the user agent configuration database should have a higher spread in the quality values for the language dimension than for the charset dimension. For example, with languages: el;q=1.0, en-gb;q=0.7, en;q=0.6, da;q=0, ... charsets: ISO-8859-1;q=1.0, ISO-8859-7;q=0.95, ISO-8859-5;q=0.97, unicode-1-1;q=0, ...
the first variant will have an overall quality of 0.95000, while the second variant will have an overall quality 0.70000. This makes the first variant the best variant. 20 Appendix: feature negotiation examples This appendix contains examples of the use of feature tags in variant descriptions. The tag names used here are examples only, they do not in general reflect the tag naming scheme proposed in [4]. 20.1 Use of feature tags Feature tags can be used in variant lists to express the quality degradation associated with the presence or absence of certain features. One example is {"index.html.plain" 0.7 }, {"index.html" 1.0 {features tables frames}} Here, the "{features tables frames}" part expresses that index.html uses the features tagged as tables and frames. If these features are absent, the overall quality of index.html degrades to 0. Another example is {"home.graphics" 1.0 {features !textonly}}, {"home.textonly" 0.7 } where the "{features !textonly}" part expresses that home.graphics requires the absence of the textonly feature. If the feature is present, the overall quality of home.graphics degrades to 0. The absence of a feature need not always degrade the overall quality to 0. In the example {"x.html.1" 1.0 {features fonts;-0.7}} the absence of the fonts feature degrades the quality with a factor of 0.7. Finally, in the example {"y.html" 1.0 {features [blebber wolx] }} The "[blebber wolx]" expresses that y.html requires the presence of the blebber feature or the wolx feature. This construct can be used in a number of cases: 1. blebber and wolx actually tag the same feature, but they were registered by different people, and some user agents say they support blebber while others say they support wolx.
2. blebber and wolx are HTML tags of different vendors which implement the same functionality, and which are used together in y.html without interference. 3. blebber and wolx are HTML tags of different vendors which implement the same functionality, and y.html uses the tags in a conditional HTML construct. 4. blebber is a complicated HTML tag with only a sketchy definition, implemented by one user agent vendor, and wolx indicates implementation of a well-defined subset of the blebber tag by some other vendor(s). y.html uses only this well-defined subset. 20.2 Use of numeric feature tags As an example of negotiation in a numeric area, the following variant list describes four variants with title graphics designed for increasing screen widths: {"home.pda" 1.0 {features screenwidth=[-199] }}, {"home.narrow" 1.0 {features screenwidth=[200-599] }}, {"home.normal" 1.0 {features screenwidth=[600-999] }}, {"home.wide" 1.0 {features screenwidth=[1000-] }}, {"home.normal"} The last element of the list specifies a safe default for user agents which do not implement screen width negotiation. Such user agents will reject the first four variants as unusable, as they seem to rely on a feature which they do not understand. 20.3 Feature tag design When designing a new feature tag, it is important to take into account that existing user agents, which do not recognize the new tag will treat the feature as absent. In general, a new feature tag needs to be designed in such a way that absence of the tag is the default case which reflects current practice. If this design principle is ignored, the resulting feature tag will generally be unusable. As an example, one could try to support negotiation between monochrome and color content by introducing a `color' feature tag, the presence of which would indicate the capability to display color graphics. However, if this new tag is used in a variant list, for example {"rainbow.gif" 1.0 {features color} }
{"rainbow.mono.gif" 0.6 {features !color}} then existing user agents, which would not recognize the color tag, would all display the monochrome rainbow. The color tag is therefore unusable in situations where optimal results for existing user agents are desired. To provide for negotiation in this area, one must introduce a `monochrome' feature tag; its presence indicates that the user agent can only render (or the user prefers to view) monochrome graphics. 21 Appendix: origin server implementation considerations 21.1 Implementation with a CGI script Transparent content negotiation has been designed to allow a broad range of implementation options at the origin server side. A very minimal implementation can be done using the CGI interface. The CGI script below is an example. #!/bin/sh cat - <<'blex' TCN: list Alternates: {"stats.tables.html" 1.0 {type text/html} {features tables}}, {"stats.html" 0.8 {type text/html}}, {"stats.ps" 0.95 {type application/postscript}} Vary: * Content-Type: text/html <title>Multiple Choices for Web Statistics</title> <h2>Multiple Choices for Web Statistics:</h2> <ul> <li><a href=stats.tables.html>Version with HTML tables</a> <p> <li><a href=stats.html>Version without HTML tables</a> <p> <li><a href=stats.ps>Postscript version</a> </ul> blex The Alternates header in the above script must be read as a single line. The script always generates a list response with the 200 (OK) code, which ensures compatibility with non-negotiating HTTP/1.0 agents.
21.2 Direct support by HTTP servers Sophisticated HTTP servers could make a transparent negotiation module available to content authors. Such a module could incorporate a remote variant selection algorithm and an implementation of the algorithm for generating choice responses (section 10.2). The definition of interfaces to such modules is beyond the scope of this specification. 21.3 Web publishing tools Web publishing tools could automatically generate several variants of a document (for example the original TeX version, a HTML version with tables, a HTML version without tables, and a Postscript version), together with an appropriate variant list in the interface format of a HTTP server transparent negotiation module. This would allow documents to be published as transparently negotiable resources. 22 Appendix: Example of choice response construction The following is an example of the construction of a choice response by a proxy cache which supports HTTP/1.1 and transparent content negotiation. The use of the HTTP/1.1 conditional request mechanisms is also shown. Assume that a user agent has cached a variant list with the validator "1234" for the negotiable resource http://x.org/paper. Also assume that it has cached responses from two neighboring variants, with the entity tags "gonkyyyy" and W/"a;b". Assume that all three user agent cache entries are stale: they would need to be revalidated before the user agent can use them. If http://x.org/paper accessed in this situation, the user agent could send the following request to its proxy cache: GET /paper HTTP/1.1 Host: x.org User-Agent: WuxtaWeb/2.4 Negotiate: 1.0 Accept: text/html, application/postscript;q=0.4, */* Accept-Language: en If-None-Match: "gonkyyyy;1234", W/"a;b;1234" Assume that the proxy cache has cached the same three items as the user agent, but that it has revalidated the variant list 8000 seconds ago, so that the list is still fresh for the proxy. This means that the proxy can run a remote variant selection algorithm on the list and the incoming request.
Assume that the remote algorithm is able to choose paper.html.en as the best variant. The proxy can now construct a choice response, using the algorithm in section 10.2. In steps 1 and 2 of the algorithm, the proxy can construct the following conditional request on the best variant, and send it to the origin server: GET /paper.html.en HTTP/1.1 Host: x.org User-Agent: WuxtaWeb/2.4 Negotiate: 1.0 Accept: text/html, application/postscript;q=0.4, */* Accept-Language: en If-None-Match: "gonkyyyy", W/"a;b" Via: 1.1 fred On receipt of the response HTTP/1.1 304 Not Modified Date: Tue, 11 Jun 1996 20:05:31 GMT Etag: "gonkyyyy" from the origin server, the proxy can use its freshly revalidated paper.html.en cache entry to expand the response to a non-304 response: HTTP/1.1 200 OK Date: Tue, 11 Jun 1996 20:05:31 GMT Content-Type: text/html Last-Modified: Mon, 10 Jun 1996 10:01:14 GMT Content-Length: 5327 Cache-control: max-age=604800 Etag: "gonkyyyy" Via: 1.1 fred Age: 0 <title>A paper about .... Using this 200 response, the proxy can construct a choice response in step 4 of the algorithm: HTTP/1.1 200 OK Date: Tue, 11 Jun 1996 20:05:31 GMT TCN: choice Content-Type: text/html Last-Modified: Mon, 10 Jun 1996 10:01:14 GMT Content-Length: 5327 Cache-control: max-age=604800 Content-Location: paper.html.en
Alternates: {"paper.html.en" 0.9 {type text/html} {language en}}, {"paper.html.fr" 0.7 {type text/html} {language fr}}, {"paper.ps.en" 1.0 {type application/postscript} {language en}} Etag: "gonkyyyy;1234" Vary: negotiate, accept, accept-language Expires: Thu, 01 Jan 1980 00:00:00 GMT Via: 1.1 fred Age: 8000 <title>A paper about .... The choice response can subsequently be shortened to a 304 response, because of the If-None-Match header in the original request from the user agent. Thus, the proxy can finally return HTTP/1.1 304 Not Modified Date: Tue, 11 Jun 1996 20:05:31 GMT Etag: "gonkyyyy;1234" Content-Location: paper.html.en Vary: negotiate, accept, accept-language Expires: Thu, 01 Jan 1980 00:00:00 GMT Via: 1.1 fred Age: 8000 to the user agent.
23 Full Copyright Statement Copyright (C) The Internet Society (1998). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.