ChiFS

Logo pending™

HTTP Requirements

Shares and Hubs expose a HTTP server at port 80 on their Onion Service. This document outlines a few general requirements on the HTTP features that all implementations must support.

General requirements:

Additionally, Share implementations have the following requirements:

All other HTTP features, such as HTTP2, compression, caching, conditional requests, authentication, cookies, etc, are optional. Servers and clients must be able to interact with other implementations that do not support these features.

Considerations

Date header

I very much dislike the HTTP date format and this response header is of questionable utility, but alas, it's required by HTTP for servers that can reasonably be expected to know the current date and time, and me not liking it is not a good enough reason to overrule HTTP and make the header optional. At least clients aren't required to parse the timestamp.

Host header

Implied in supporting HTTP/1.0 is that servers can't rely on receiving a "Host" header. I don't think they'll need it, anyway. In fact, having a single server handle multiple Onion Services may be a security risk. And so is having the client tell the server what their hostname is, if that hostname is then used for something important.

Redirects

Disallowing HTTP redirects simplifies clients and prevents unexpected behavior. If clients support redirects, they may be tricked into contacting a different Onion service than what the user expected. Insecure clients could even be tricked into contacting a clearnet site. Redirects are deemed not necessary because APIs don't move around, and when a published file has been moved, this will be reflected in the metadata and clients can find the new path by looking up the file hash. (Note that clients already have an incentive to regularly contact Hub(s) to find new sources for the file hashes they are interested in - more sources implies faster and more reliable downloads)

Transfer-Encoding

If the server knows in advance how large the response is going to be, setting a "Content-Length" will always work. If it doesn't, there's two strategies:

This is already part of the HTTP/1.1 spec. I'd like to enforce the "Content-Length" header and avoid the requirement that clients support chunked encoding, but that may exclude the use of some web servers or frameworks, especially for the dynamic pages that Hubs will generally have.

Range requests

Shares must send Accept-Ranges: bytes on published files and handle requests for a single range. Here's a full example with all the required request and response headers:

GET /somefile HTTP/1.1
Host: xyz.onion
Connection: keep-alive
Range: bytes=0-1023

HTTP/1.1 206 Partial Content
Accept-Ranges: bytes
Content-Range: bytes 0-1023/20348023
Content-Length: 1024
Connection: keep-alive
Date: Sat, 29 Dec 2018 10:19:21 GMT

<data>

HTTP also allows multipart ranges, but these do not seem to be worth the complexity and don't have to be supported. If the requested range is outside of the bounds of the file size, the server must respond with status code 416, "Requested Range Not Satisfiable".

Range requests for Hubs or for Share metadata do not make much sense: Metadata is mutable and lacks checksums, there is no way to guarantee integrity when downloading metadata over multiple requests.