RFC 8620
The JSON Meta Application Protocol (JMAP)
6. Binary Data
Binary data is referenced by a *blobId* in JMAP and uploaded/ downloaded separately to the core API. The blobId solely represents the raw bytes of data, not any associated metadata such as a file name or content type. Such metadata is stored alongside the blobId in the object referencing it. The data represented by a blobId is immutable. Any blobId that exists within an account may be used when creating/ updating another object in that account. For example, an Email type may have a blobId that represents the object in Internet Message Format [RFC5322]. A client could create a new Email object with an attachment and use this blobId, in effect attaching the old message to the new one. Similarly, it could attach any existing attachment of an old message without having to download and upload it again.
When the client uses a blobId in a create/update, the server MAY
assign a new blobId to refer to the same binary data within the new/
updated object. If it does so, it MUST return any properties that
contain a changed blobId in the created/updated response, so the
client gets the new ids.
A blob that is not referenced by a JMAP object (e.g., as a message
attachment) MAY be deleted by the server to free up resources.
Uploads (see below) are initially unreferenced blobs. To ensure
interoperability:
o The server SHOULD use a separate quota for unreferenced blobs to
the account's usual quota. In the case of shared accounts, this
quota SHOULD be separate per user.
o This quota SHOULD be at least the maximum total size that a single
object can reference on this server. For example, if supporting
JMAP Mail, this should be at least the maximum total attachments
size for a message.
o When an upload would take the user over quota, the server MUST
delete unreferenced blobs in date order, oldest first, until there
is room for the new blob.
o Except where quota restrictions force early deletion, an
unreferenced blob MUST NOT be deleted for at least 1 hour from the
time of upload; if reuploaded, the same blobId MAY be returned,
but this SHOULD reset the expiry time.
o A blob MUST NOT be deleted during the method call that removed the
last reference, so that a client can issue a create and a destroy
that both reference the blob within the same method call.
6.1. Uploading Binary Data
There is a single endpoint that handles all file uploads for an
account, regardless of what they are to be used for. The Session
object (see Section 2) has an "uploadUrl" property in URI Template
(level 1) format [RFC6570], which MUST contain a variable called
"accountId". The client may use this template in combination with an
"accountId" to get the URL of the file upload resource.
To upload a file, the client submits an authenticated POST request to
the file upload resource.
A successful request MUST return a single JSON object with the
following properties as the response:
o accountId: "Id"
The id of the account used for the call.
o blobId: "Id"
The id representing the binary data uploaded. The data for this
id is immutable. The id *only* refers to the binary data, not any
metadata.
o type: "String"
The media type of the file (as specified in [RFC6838],
Section 4.2) as set in the Content-Type header of the upload HTTP
request.
o size: "UnsignedInt"
The size of the file in octets.
If identical binary content to an existing blob in the account is
uploaded, the existing blobId MAY be returned.
Clients should use the blobId returned in a timely manner. Under
rare circumstances, the server may have deleted the blob before the
client uses it; the client should keep a reference to the local file
so it can upload it again in such a situation.
When an HTTP error response is returned to the client, the server
SHOULD return a JSON "problem details" object as the response body,
as per [RFC7807].
As access controls are often determined by the object holding the
reference to a blob, unreferenced blobs MUST only be accessible to
the uploader, even in shared accounts.
6.2. Downloading Binary Data
The Session object (see Section 2) has a "downloadUrl" property,
which is in URI Template (level 1) format [RFC6570]. The URL MUST
contain variables called "accountId", "blobId", "type", and "name".
To download a file, the client makes an authenticated GET request to
the download URL with the appropriate variables substituted in:
o "accountId": The id of the account to which the record with the
blobId belongs.
o "blobId": The blobId representing the data of the file to
download.
o "type": The type for the server to set in the "Content-Type"
header of the response; the blobId only represents the binary data
and does not have a content-type innately associated with it.
o "name": The name for the file; the server MUST return this as the
filename if it sets a "Content-Disposition" header.
As the data for a particular blobId is immutable, and thus the
response in the generated download URL is too, implementors are
recommended to set long cache times and use the "immutable" Cache-
Control extension [RFC8246] for successful responses, for example,
"Cache-Control: private, immutable, max-age=31536000".
When an HTTP error response is returned to the client, the server
SHOULD return a JSON "problem details" object as the response body,
as per [RFC7807].
6.3. Blob/copy
Binary data may be copied *between* two different accounts using the
"Blob/copy" method rather than having to download and then reupload
on the client.
The "Blob/copy" method takes the following arguments:
o fromAccountId: "Id"
The id of the account to copy blobs from.
o accountId: "Id"
The id of the account to copy blobs to.
o blobIds: "Id[]"
A list of ids of blobs to copy to the other account.
The response has the following arguments:
o fromAccountId: "Id"
The id of the account blobs were copied from.
o accountId: "Id"
The id of the account blobs were copied to.
o copied: "Id[Id]|null"
A map of the blobId in the fromAccount to the id for the blob in
the account it was copied to, or null if none were successfully
copied.
o notCopied: "Id[SetError]|null"
A map of blobId to a SetError object for each blob that failed to
be copied, or null if none.
The SetError may be any of the standard set errors that may be
returned for a create, as defined in Section 5.3. In addition, the
"notFound" SetError error may be returned if the blobId to be copied
cannot be found.
The following additional method-level error may be returned instead
of the "Blob/copy" response:
"fromAccountNotFound": The "fromAccountId" included with the request
does not correspond to a valid account.
7. Push
Push notifications allow clients to efficiently update (almost)
instantly to stay in sync with data changes on the server. The
general model for push is simple and sends minimal data over the push
channel: just enough for the client to know whether it needs to
resync. The format allows multiple changes to be coalesced into a
single push update and the frequency of pushes to be rate limited by
the server. It doesn't matter if some push events are dropped before
they reach the client; the next time it gets/sets any records of a
changed type, it will discover the data has changed and still sync
all changes.
There are two different mechanisms by which a client can receive push notifications, to allow for the different environments in which a client may exist. An event source resource (see Section 7.3) allows clients that can hold transport connections open to receive push notifications directly from the JMAP server. This is simple and avoids third parties, but it is often not feasible on constrained platforms such as mobile devices. Alternatively, clients can make use of any push service supported by their environment. A URL for the push service is registered with the JMAP server (see Section 7.2); the server then POSTs each notification to that URL. The push service is then responsible for routing these to the client.7.1. The StateChange Object
When something changes on the server, the server pushes a StateChange object to the client. A *StateChange* object has the following properties: o @type: "String" This MUST be the string "StateChange". o changed: "Id[TypeState]" A map of an "account id" to an object encoding the state of data types that have changed for that account since the last StateChange object was pushed, for each of the accounts to which the user has access and for which something has changed. A *TypeState* object is a map. The keys are the type name "Foo" (e.g., "Mailbox" or "Email"), and the value is the "state" property that would currently be returned by a call to "Foo/get". The client can compare the new state strings with its current values to see whether it has the current data for these types. If not, the changes can then be efficiently fetched in a single standard API request (using the /changes type methods).
7.1.1. Example
In this example, the server has amalgamated a few changes together across two different accounts the user has access to, before pushing the following StateChange object to the client: { "@type": "StateChange", "changed": { "a3123": { "Email": "d35ecb040aab", "EmailDelivery": "428d565f2440", "CalendarEvent": "87accfac587a" }, "a43461d": { "Mailbox": "0af7a512ce70", "CalendarEvent": "7a4297cecd76" } } } The client can compare the state strings with its current state for the Email, CalendarEvent, etc., object types in the appropriate accounts to see if it needs to fetch changes. If the client is itself making changes, it may receive a StateChange object while the /set API call is in flight. It can wait until the call completes and then compare if the new state string after the /set is the same as was pushed in the StateChange object; if so, and the old state of the /set response matches the client's previous state, it does not need to waste a request asking for changes it already knows.7.2. PushSubscription
Clients may create a PushSubscription to register a URL with the JMAP server. The JMAP server will then make an HTTP POST request to this URL for each push notification it wishes to send to the client. As a push subscription causes the JMAP server to make a number of requests to a previously unknown endpoint, it can be used as a vector for launching a denial-of-service attack. To prevent this, when a subscription is created, the JMAP server immediately sends a PushVerification object to that URL (see Section 7.2.2). The JMAP server MUST NOT make any further requests to the URL until the client receives the push and updates the subscription with the correct verification code.
A *PushSubscription* object has the following properties:
o id: "Id" (immutable; server-set)
The id of the push subscription.
o deviceClientId: "String" (immutable)
An id that uniquely identifies the client + device it is running
on. The purpose of this is to allow clients to identify which
PushSubscription objects they created even if they lose their
local state, so they can revoke or update them. This string MUST
be different on different devices and be different from apps from
other vendors. It SHOULD be easy to regenerate and not depend on
persisted state. It is RECOMMENDED to use a secure hash of a
string that contains:
1. A unique identifier associated with the device where the JMAP
client is running, normally supplied by the device's operating
system.
2. A custom vendor/app id, including a domain controlled by the
vendor of the JMAP client.
To protect the privacy of the user, the deviceClientId id MUST NOT
contain an unobfuscated device id.
o url: "String" (immutable)
An absolute URL where the JMAP server will POST the data for the
push message. This MUST begin with "https://".
o keys: "Object|null" (immutable)
Client-generated encryption keys. If supplied, the server MUST
use them as specified in [RFC8291] to encrypt all data sent to the
push subscription. The object MUST have the following properties:
* p256dh: "String"
The P-256 Elliptic Curve Diffie-Hellman (ECDH) public key as
described in [RFC8291], encoded in URL-safe base64
representation as defined in [RFC4648].
* auth: "String"
The authentication secret as described in [RFC8291], encoded in
URL-safe base64 representation as defined in [RFC4648].
o verificationCode: "String|null"
This MUST be null (or omitted) when the subscription is created.
The JMAP server then generates a verification code and sends it in
a push message, and the client updates the PushSubscription object
with the code; see Section 7.2.2 for details.
o expires: "UTCDate|null"
The time this push subscription expires. If specified, the JMAP
server MUST NOT make further requests to this resource after this
time. It MAY automatically destroy the push subscription at or
after this time.
The server MAY choose to set an expiry if none is given by the
client or modify the expiry time given by the client to a shorter
duration.
o types: "String[]|null"
A list of types the client is interested in (using the same names
as the keys in the TypeState object defined in the previous
section). A StateChange notification will only be sent if the
data for one of these types changes. Other types are omitted from
the TypeState object. If null, changes will be pushed for all
types.
The POST request MUST have a content type of "application/json" and
contain the UTF-8 JSON-encoded object as the body. The request MUST
have a "TTL" header and MAY have "Urgency" and/or "Topic" headers, as
specified in Section 5 of [RFC8030]. The JMAP server is expected to
understand and handle HTTP status responses in a reasonable manner.
A "429" (Too Many Requests) response MUST cause the JMAP server to
reduce the frequency of pushes; the JMAP push structure allows
multiple changes to be coalesced into a single minimal StateChange
object. See the security considerations in Section 8.6 for a
discussion of the risks in connecting to unknown servers.
The JMAP server acts as an application server as defined in
[RFC8030]. A client MAY use the rest of [RFC8030] in combination
with its own push service to form a complete end-to-end solution, or
it MAY rely on alternative mechanisms to ensure the delivery of the
pushed data after it leaves the JMAP server.
The push subscription is tied to the credentials used to authenticate
the API request that created it. Should these credentials expire or
be revoked, the push subscription MUST be destroyed by the JMAP
server. Only subscriptions created by these credentials are returned when the client fetches existing subscriptions. When these credentials have their own expiry (i.e., it is a session with a timeout), the server SHOULD NOT set or bound the expiry time for the push subscription given by the client but MUST expire it when the session expires. When these credentials are not time bounded (e.g., Basic authentication [RFC7617]), the server SHOULD set an expiry time for the push subscription if none is given and limit the expiry time if set too far in the future. This maximum expiry time MUST be at least 48 hours in the future and SHOULD be at least 7 days in the future. An app running on a mobile device may only be able to refresh the push subscription lifetime when it is in the foreground, so this gives a reasonable time frame to allow this to happen. In the case of separate access and refresh credentials, as in Oauth 2.0 [RFC6749], the server SHOULD tie the push subscription to the validity of the refresh token rather than the access token and behave according to whether this is time-limited or not. When a push subscription is destroyed, the server MUST securely erase the URL and encryption keys from memory and storage as soon as possible.7.2.1. PushSubscription/get
Standard /get method as described in Section 5.1, except it does *not* take or return an "accountId" argument, as push subscriptions are not tied to specific accounts. It also does *not* return a "state" argument. The "ids" argument may be null to fetch all at once. The server MUST only return push subscriptions that were created using the same authentication credentials as for this "PushSubscription/get" request. As the "url" and "keys" properties may contain data that is private to a particular device, the values for these properties MUST NOT be returned. If the "properties" argument is null or omitted, the server MUST default to all properties excluding these two. If one of them is explicitly requested, the method call MUST be rejected with a "forbidden" error.
7.2.2. PushSubscription/set
Standard /set method as described in Section 5.3, except it does *not* take or return an "accountId" argument, as push subscriptions are not tied to specific accounts. It also does *not* take an "ifInState" argument or return "oldState" or "newState" arguments. The "url" and "keys" properties are immutable; if the client wishes to change these, it must destroy the current push subscription and create a new one. When a PushSubscription is created, the server MUST immediately push a *PushVerification* object to the URL. It has the following properties: o @type: "String" This MUST be the string "PushVerification". o pushSubscriptionId: "String" The id of the push subscription that was created. o verificationCode: "String" The verification code to add to the push subscription. This MUST contain sufficient entropy to avoid the client being able to guess the code via brute force. The client MUST update the push subscription with the correct verification code before the server makes any further requests to the subscription's URL. Attempts to update the subscription with an invalid verification code MUST be rejected by the server with an "invalidProperties" SetError. The client may update the "expires" property to extend (or, less commonly, shorten) the lifetime of a push subscription. The server MAY modify the proposed new expiry time to enforce server-defined limits. Extending the lifetime does not require the subscription to be verified again. Clients SHOULD NOT update or destroy a push subscription that they did not create (i.e., has a "deviceClientId" that they do not recognise).
7.2.3. Example
At "2018-07-06T02:14:29Z", a client with deviceClientId "a889-ffea- 910" fetches the set of push subscriptions currently on the server, making an API request with: [[ "PushSubscription/get", { "ids": null }, "0" ]] Which returns: [[ "PushSubscription/get", { "list": [{ "id": "e50b2c1d-9553-41a3-b0a7-a7d26b599ee1", "deviceClientId": "b37ff8001ca0", "verificationCode": "b210ef734fe5f439c1ca386421359f7b", "expires": "2018-07-31T00:13:21Z", "types": [ "Todo" ] }, { "id": "f2d0aab5-e976-4e8b-ad4b-b380a5b987e4", "deviceClientId": "X8980fc", "verificationCode": "f3d4618a9ae15c8b7f5582533786d531", "expires": "2018-07-12T05:55:00Z", "types": [ "Mailbox", "Email", "EmailDelivery" ] }], "notFound": [] }, "0" ]] Since neither of the returned push subscription objects have the client's deviceClientId, it knows it does not have a current push subscription active on the server. So it creates one, sending this request: [[ "PushSubscription/set", { "create": { "4f29": { "deviceClientId": "a889-ffea-910", "url": "https://example.com/push/?device=X8980fc&client=12c6d086", "types": null } } }, "0" ]]
The server creates the push subscription but limits the expiry time
to 7 days in the future, returning this response:
[[ "PushSubscription/set", {
"created": {
"4f29": {
"id": "P43dcfa4-1dd4-41ef-9156-2c89b3b19c60",
"keys": null,
"expires": "2018-07-13T02:14:29Z"
}
}
}, "0" ]]
The server also immediately makes a POST request to
"https://example.com/push/?device=X8980fc&client=12c6d086" with the
data:
{
"@type": "PushVerification",
"pushSubscriptionId": "P43dcfa4-1dd4-41ef-9156-2c89b3b19c60",
"verificationCode": "da1f097b11ca17f06424e30bf02bfa67"
}
The client receives this and updates the subscription with the
verification code (note there is a potential race condition here; the
client MUST be able to handle receiving the push while the request
creating the subscription is still in progress):
[[ "PushSubscription/set", {
"update": {
"P43dcfa4-1dd4-41ef-9156-2c89b3b19c60": {
"verificationCode": "da1f097b11ca17f06424e30bf02bfa67"
}
}
}, "0" ]]
The server confirms the update was successful and will now make
requests to the registered URL when the state changes.
Two days later, the client updates the subscription to extend its
lifetime, sending this request:
[[ "PushSubscription/set", {
"update": {
"P43dcfa4-1dd4-41ef-9156-2c89b3b19c60": {
"expires": "2018-08-13T00:00:00Z"
}
}
}, "0" ]]
The server extends the expiry time, but only again to its maximum
limit of 7 days in the future, returning this response:
[[ "PushSubscription/set", {
"updated": {
"P43dcfa4-1dd4-41ef-9156-2c89b3b19c60": {
"expires": "2018-07-15T02:22:50Z"
}
}
}, "0" ]]
7.3. Event Source
Clients that can hold transport connections open can connect directly
to the JMAP server to receive push notifications via a "text/event-
stream" resource, as described in [EventSource]. This is a long
running HTTP request, where the server can push data to the client by
appending data without ending the response.
When a change occurs in the data on the server, it pushes an event
called "state" to any connected clients, with the StateChange object
as the data.
The server SHOULD also send a new event id that encodes the entire
server state visible to the user immediately after sending a "state"
event. When a new connection is made to the event-source endpoint, a
client following the server-sent events specification will send a
Last-Event-ID HTTP header field with the last id it saw, which the
server can use to work out whether the client has missed some
changes. If so, it SHOULD send these changes immediately on
connection.
The Session object (see Section 2) has an "eventSourceUrl" property,
which is in URI Template (level 1) format [RFC6570]. The URL MUST
contain variables called "types", "closeafter", and "ping".
To connect to the resource, the client makes an authenticated GET
request to the event-source URL with the appropriate variables
substituted in:
o "types": This MUST be either:
* A comma-separated list of type names, e.g.,
"Email,CalendarEvent". The server MUST only push changes for
the types in this list.
* The single character: "*". Changes to all types are pushed.
o "closeafter": This MUST be one of the following values:
* "state": The server MUST end the HTTP response after pushing a
state event. This can be used by clients in environments where
buffering proxies prevent the pushed data from arriving
immediately, or indeed at all, when operating in the usual
mode.
* "no": The connection is persisted by the server as a standard
event-source resource.
o "ping": A positive integer value representing a length of time in
seconds, e.g., "300". If non-zero, the server MUST send an event
called "ping" whenever this time elapses since the previous event
was sent. This MUST NOT set a new event id. If the value is "0",
the server MUST NOT send ping events.
The server MAY modify a requested ping interval to be subject to a
minimum and/or maximum value. For interoperability, servers MUST
NOT have a minimum allowed value higher than 30 or a maximum
allowed value less than 300.
The data for the ping event MUST be a JSON object containing an
"interval" property, the value (type "UnsignedInt") being the
interval in seconds the server is using to send pings (this may be
different to the requested value if the server clamped it to be
within a min/max value).
Clients can monitor for the ping event to help determine when the
closeafter mode may be required.
A client MAY hold open multiple connections to the event-source
resource, although it SHOULD try to use a single connection for
efficiency.
(next page on part 5)
