Allow protocol discovery between client and server.

[timemachine3030]

If writing an application that is able to upload to many different servers, the application will need to check if the remote server supports this protocol. Therefore, I suggest the following protocol extension to start a conversion on the topic.

(Note: that servers that do not support the tus protocol will reply to HEAD requests with either 200 OK or 404 Not Found)

Protocol Discovery

All implementations SHOULD include the protocol discovery API to allow clients to negotiate, with a server, a common supported version of this protocol.

When making a HEAD request for a new file the client MAY include a TUS-Resumable header with the version of this protocol.

When receiving a HEAD request with the TUS-Resumable header the server MAY reply with the 202 Accepted status and the current Offset.

Example

Request:

HEAD /files/24e533e02ec3bc40c387f1a0e460e216 HTTP/1.1
Host: tus.example.org
TUS-Resumable: 1.0

Response:

HTTP/1.1 202 Accepted
Offset: 0

[qsorix]

I'm interested in this subject. My use case will be different though. The clients I'm about to support are mobile nodes that will support a single version of the protocol, but different clients may support different versions. Server must support many versions, depending on what clients are deployed.

I want server to detect which version it is and adjust, so ideally, I would want client to include a header in their File Creation request, as that's probably the first thing they're going to send.

Request:

POST /files HTTP/1.1
Host: tus.example.org
TUS-Resumable: 0.2.2
Content-Length: 0
Entity-Length: 100

Response:

HTTP/1.1 201 Created
Location: http://tus.example.org/files/24e533e02ec3bc40c387f1a0e460e216

Or, when server no longer supports some version (not the best situation for me, but can happen and should be specified), it could return an error, and at the same time indicate the preferred version to be used.

HTTP/1.1 410 Gone
TUS-Resumable: 1.0.0

[qsorix]

On the other hand, are request headers the right way to do it? Perhaps there should be separate URLs to initiate uploads in separate versions? E.g. POST /0.2.2/files.

[tim-kos]

@qsorix But wouldn't that be solved with what @timemachine3030 proposed? That you first do a HEAD request to the server to figure out if your client's protocol version is supported?

Another idea: We could introduce an endpoint that you can send a HEAD request to which returns you a list of protocol versions that are supported. I don't really like this, but I wanted to throw it into the discussion anyways.

I think what @timemachine3030 proposed makes sense ... you could just send a HEAD request with the protocol version you'd LIKE to use. And if that is not available you would send more HEAD requests to figure out the latest version you can support with your client.

[qsorix]

I'm fine with @timemachine3030's proposal. In my comment i was trying to say that my use-case is reversed. This:

allow clients to negotiate, with a server, a common supported version of this protocol.

In my case is not important. I have thousands of remote embedded devices, and each support only a single version of protocol. Since upgrades are spread in time, on the server I need to support (almost) all versions of the protocol.

So my use case is to "allow the server to recognize the protocol version used by the client".

My clients MUST advertise the version of the protocol they're using in every request they make. There's no negotiation, and if there was, it would not be optional. So I prefer clients to just include a header and follow their version of the protocol, instead of asking them to do additional request, just to dicover what's supported on the server. My clients have no choice but to assume the version they're running is supported on the server.

And on the server, we must support all versions that are in use by some clients. As clients are being upgraded, we gradually drop support for older versions.

What @timemachine3030 wrote is OK for a typical web-oriented use case, where a single rich client may connect to several servers. Each server may run a different version of the protocol, so the client want to figure out the protocol version to use.

I'm happy with the original proposal. I just need to extend it so that:

• protocol version is advertised on File Creation request
• server can assume that during the upload client will continue to use the same version it advertised in File Creation
• server can reject a request if it does not support client's version
• ideally, protocol version is included in all requests (and if it's missing, it's asumed to be 0.2.1)

[tim-kos]

I see, these extensions make sense to me.

What do you think @timemachine3030 ?

[Gargaj]

There are additional perks to protocol discovery: In addition to the protocol version number, the server may also serve supplemental data that is specific to that implementation or configuration, like maximum request size allowed or session time length. These are important enough for the client to be aware of.

[timemachine3030]

Great comments from everyone.

@qsorix, Yes I see the need to send a version identifier with every request. It is important that the protocol's advise understands that there are situations where the server is sophisticated and the client is dumb and vise versa.

As an additional note, I think that the client should still make an initial handshake request in the form of a HEAD (or OPTIONS as #36 recommends) for initial protocol acceptance and discovery. This removes some complexity from the discovery vs creation error handling.

An example scenario is where the POST request generates an error (Forbidden or Unauthorized as examples) AND the protocol version is deprecated. As you can only respond with one status code in HTTP/1.1 you have a state where you need to know the error priority. Conversely, if you are checking the protocol version in an initial handshake request, the protocol is free to leave the example condition as undefined behavior as by the letter of the protocol it is a situation that will not be encountered.

Some items for additional discussion are,

• Fear of creating extra requests? #36 has, almost, convinced me that an OPTIONS request is preferred over HEAD for the discovery phase. My goal with originally including it in the HEAD request was to augment as existing request instead of creating an additional request type definition. However, now that it has been explored more I think it belongs in OPTIONS and is optional in HEAD, POST, and others. This fits both use cases of "server required versions" vs. "server supported versions".
• I suggest 409 Conflict over 410 Gone because Conflict allows for retires where Gone is a final state (I realize that in @qsorix's example retries are not possible, but this is for the general case).

[Acconut]

@timemachine3030 Yes I see the need to send a version identifier with every request.

We should include the version in every request and response. The client must send its version in the request and the server its version in the response. This is in conflict with @qsorix's idea to use the URL (#29 (comment)) since tus does not specify URL endpoints (and it should not start to do so).

@qsorix And on the server, we must support all versions that are in use by some clients. As clients are being upgraded, we gradually drop support for older versions.

For this case, the server supports multiple versions, we could allow multiple versions to be returned by the server, e.g.

TUS-Resumable: 1.0.0,0.2.2,0.2.1

where the first ones are preferred (use 1.0.0 if supported else 0.2.2 or 0.2.1).

@Gargaj In addition to the protocol version number, the server may also serve supplemental data that is specific to that implementation or configuration, like maximum request size allowed or session time length.

I agree although we should try to avoid to put every piece information into one request/response. Instead I would like to introduce another method to retrieve this data. In addition to maximum size and session time length I would suggest to list the supported extensions.

[Acconut]

I could thing of following:

OPTIONS /files
Host: master.tus.io
# Client's tus version
TUS-Resumable: 1.0.0
...

204 No Content
# Sever's tus version(s)
TUS-Resumable: 1.0.1
# Implemented and allowed extensions
TUS-Extensions: retries,checksum,upload-expires
# Maximum upload size in bytes
TUS-Max-Size: 1073741824
...

An OPTIONS request to /files/34th80buzawrvf039btr90zjl4a35 should return the same result.

[qsorix]

@Acconut, looks good. I like it.

TUS-Resumable can appear multiple times or contain a comma-separated list, right?

And TUS-Resumable (with a single version that's actually used) MUST be included in all other (not OPTIONS) requests, right? And in responses as well?

Which versions should server report back? All supported? The preferred one from what client advertised? Up to the implementation to decide?

Minor comment. Perhaps it should be TUS-Extension: (without 's'), to stay consistent with other HTTP headers that take multiple values.

[Acconut]

@qsorix

TUS-Resumable can appear multiple times or contain a comma-separated list, right?

It should be a comma-separated list of all supported versions where this first elements are the preferred ones. So TUS-Resumable: 1.0.0,0.2.1 translates to: The server prefers 1.0.0 but also supports 0.2.1.

And TUS-Resumable (with a single version that's actually used) MUST be included in all other (not OPTIONS) requests, right? And in responses as well?

Yes, that's how you can determine whether the client/server supports tus (and which version).

Minor comment. Perhaps it should be TUS-Extension: (without 's'), to stay consistent with other HTTP headers that take multiple values.

Good point, I agree.

[qsorix]

Cool. +1.

Two questions about this:

It should be a comma-separated list (...) where this first elements are the preferred ones.

First. It would also be valid to do two headers, right? E.g.

TUS-Resumable: 1.0.0
TUS-Resumable: 0.2.1

Second. Do we go with a plain list or bother with "Quality factors"?

HTTP uses quality factors (q=0..1) to declare preference. E.g.

       Accept: text/plain; q=0.5, text/html,
               text/x-dvi; q=0.8, text/x-c

As a programmer, I like the sorted list much better because it is easier to parse.

On the other hand, if existing libraries that deal with HTTP headers support quality factors, then programming convenience is not an issue, and I believe staying consistent with the established convention makes sense.

When there is no quality factor, the default is 1. In this example:

TUS-Resumable: 1.0.0,0.2.1

both versions have the same priority. It can be implementation defined how to pick one, and we can suggest to pick them in order (server/client SHOULD).

But if we agree to honour q=, then this

TUS-Resumable: 0.2.1; q=0.5, 1.0.0

Must also favour 1.0.0 over 0.2.1.

The library I use, does not parse this stuff. Hypothetically, there can be a library that goes too far and not only parses it, but also reorders the values to make it easier to the programmers to pick the most preferred one.

When deciding on this, I would consider two things:

• Are there known issues with proxies or server/client libraries that do not preserve the order HTTP headers?
• Are quality factors supported by existing HTTP libraries?

I'm happy to agree that a plain list is all we need.

[Acconut]

@qsorix

First. It would also be valid to do two headers, right? E.g.

As far as I know it's supported mostly everywhere (this is in the HTTP spec, see Set-Cookie) so theoretically it should work. But, I prefer the version using one header because you don't have to worry about the order and it's similar to the Accept-* headers.

Second. Do we go with a plain list or bother with "Quality factors"?

Again, good point. I don't have a strong opinion on this. The advantage of quality factors are that you can have multiple versions with the same factor but considering the price of parsing. I would prefer using the plain list just because of simplicity.

[Acconut]

Here's what I would suggest:

Add to 5.2. Headers:

---

TUS-Resumable

The TUS-Resumable header MUST be sent in every response and resquest. Its value is a string set to the current version of the used tus resumable upload protocol by the client or server.

TUS-Extension

This header MUST be a comma-separated list of the extensions supported by the server. If no extensions are supported TUS-Extension MAY be omitted.

TUS-Max-Size

The TUS-Max-Size header MUST be an integer indicating the maximum allowed size of a single fully uploaded file in bytes. If no hard-limit is presented or the server is not able to calculate it this header MUST be omitted.

TUS-Version

This header MUST be a comma-separated list of the supported versions of the tus resumable upload protocol by the server. The lists elements are sorted by the server's preference whereas the first element is the most preferred one.

---

Add to 5.3. Requests:

---

OPTIONS

An OPTIONS request MAY be used to gather information about the current configuration of the server. The response MUST contain the TUS-Extension, TUS-Version and TUS-Max-Size if available.

---

@qsorix As you see I added the TUS-Version header which contains the list of the supported versions. Its value may be different from TUS-Resumable which indicates the currently used version.

[Acconut]

See #48 for the proposed changes.

[qsorix]

Sorry for a late response, I stayed offline duringe christmas.

@Acconut, please remind me why for TUS-Version only server is mentioned? Clients are expected to first send OPTIONS, then pick the version and send requests using the proper one? If so, I'm +1.

Will it say anywhere what happens when server doesn't like client's version? I.e. what's the proper way to reject invalid requests?

I think it would help if every header was accompanied by a sensible example, to limit possible confusion regarding formating etc.

[Acconut]

Sorry for a late response, I stayed offline duringe christmas.

No problem, we still have time until 1.0 needs to be done. :)

I think it would help if every header was accompanied by a sensible example, to limit possible confusion regarding formating etc.

I agree. An example for an OPTIONS request and response including all headers will follow.

@Acconut, please remind me why for TUS-Version only server is mentioned? Clients are expected to first send OPTIONS, then pick the version and send requests using the proper one? If so, I'm +1.

Yes, that's how it is intended by me. The server doesn't need to know which versions the client supports, it must only be aware of the version the client is using in its requests.

Will it say anywhere what happens when server doesn't like client's version? I.e. what's the proper way to reject invalid requests?

Good point I forgot about! I vote for 412 Precondition Failed. It is intended to be used when the server can't process the request since the client specified conditions the server can't meet. And that's exactly what we need when the requester uses a version which is not supported by the responder.

[qsorix]

I'm fine with 412.

[Acconut]

See a7ba01a.

[qsorix]

+1

[Acconut]

Merged in 7c4fd4b.