Shares and Hubs expose a HTTP server at port 80 on their Onion Service. This document outlines a few general requirements on the HTTP features that all implementations must support.
- Servers must support both HTTP/1.0 and HTTP/1.1.
- The server must not require TLS.
- Servers must support HTTP/1.1 Keep-alive.
- Servers should not send 3xx response codes. Clients are not required to follow redirects.
Additionally, Share implementations have the following requirements:
- Pipelining of requests must be supported.
- Single-range requests on published files must be supported.
All other HTTP features, such as HTTP2, compression, caching, conditional requests, authentication, cookies, etc, are optional. Servers and clients must be able to interact with other implementations that do not support these features.
I very much dislike the HTTP date format and this response header is of questionable utility, but alas, it's required by HTTP for servers that can reasonably be expected to know the current date and time, and me not liking it is not a good enough reason to overrule HTTP and make the header optional. At least clients aren't required to parse the timestamp.
Implied in supporting HTTP/1.0 is that servers can't rely on receiving a "Host" header. I don't think they'll need it, anyway. In fact, having a single server handle multiple Onion Services may be a security risk. And so is having the client tell the server what their hostname is, if that hostname is then used for something important.
Disallowing HTTP redirects simplifies clients and prevents unexpected behavior. If clients support redirects, they may be tricked into contacting a different Onion service than what the user expected. Insecure clients could even be tricked into contacting a clearnet site. Redirects are deemed not necessary because APIs don't move around, and when a published file has been moved, this will be reflected in the metadata and clients can find the new path by looking up the file hash. (Note that clients already have an incentive to regularly contact Hub(s) to find new sources for the file hashes they are interested in - more sources implies faster and more reliable downloads)
If the server knows in advance how large the response is going to be, setting a "Content-Length" will always work. If it doesn't, HTTP/1.1 specifies a
Transfer-Encoding: chunked to send streaming data. Unfortunately, servers are permitted to send chunked responses even if the client did not indicate support for it with an "Accept-Encoding". In fact, there does not seem to be a way for clients to indicate (absense of) support for this feature. For this reason, clients must support chunked transfer encodings.
Chunked encoding can also be used in request bodies, but this does not seem to be commonly used. Servers are permitted to not support it by replying with a
411 Length Required response.
Shares must send
Accept-Ranges: bytes on published files and handle requests for a single range. Here's a full example with all the required request and response headers:
GET /somefile HTTP/1.1 Host: xyz.onion Connection: keep-alive Range: bytes=0-1023 HTTP/1.1 206 Partial Content Accept-Ranges: bytes Content-Range: bytes 0-1023/20348023 Content-Length: 1024 Connection: keep-alive Date: Sat, 29 Dec 2018 10:19:21 GMT <data>
HTTP also allows multipart ranges, but these do not seem to be worth the complexity and don't have to be supported. If the requested range is outside of the bounds of the file size, the server must respond with status code 416, "Requested Range Not Satisfiable".
Range requests for Hubs or for Share metadata do not make much sense: Metadata is mutable and lacks checksums, there is no way to guarantee integrity when downloading metadata over multiple requests.