ChiFS Protocol Specifications
ChiFS is still a work in progress, and so is the protocol. The protocol specifications are being written alongside the implementations.
High-level overview and design philosophy
As noted on the homepage, ChiFS consists of three entities (or roles): Shares, Hubs and Clients. Here's a more detailed explanation of what each entity does and its constraints:
A Share does two things: (1) It provides files that can be downloaded by Clients and (2) it provides metadata so that Hubs can index the files in a database in order to provide search and discovery. The operator of a Share has full control over which files are to be shared, its directory structure, and which metadata is included. File metadata includes information such as file size, type, hash and media info.
On a technical level, a Share is an HTTP server accessible over Tor through an Onion service. One of the goals is to make it easy to publish existing file repositories (such as Linux package repositories) on ChiFS and to make it easy to add ChiFS Share capabilities to existing websites and frameworks. In order to achieve that, it should be possible use an existing static file HTTP server to implement the Share API. No special configuration - other than setting the right paths - should be required on the part of the HTTP server. It should also be possible to run a Share on resource-constrained devices - if it is possible to run a HTTP server and an Onion service on the device, it should be possible to run a Share.
A Hub is responsible for synchronizing the file metadata from multiple Shares, indexing the metadata in a database and providing file search and discovery for Clients. Like Shares, Hubs provide an HTTP API through an Onion service. Unlike Shares, it is not expected that Hub functionality is integrated into existing services - a Hub is implemented as a custom piece of software - nor that it is possible to run a Hub on a resource-constrained device.
Ideally, a Hub is not much more than a "stupid" cache. The only real information that a Hub holds is a list of Shares, everything else should be cached information obtained from Share metadata. Hubs take a central place in the functioning of the ChiFS network, but because they hold no real information on their own, they can be easily replicated and replaced.
- Clients are the interface between the Shares, Hubs and the final end-user. Clients interact with one or more Hubs in order to find and browse through published files, and then interact with Shares to actually download the files.
Of course, a single application may implement multiple such roles. For example, a Client could also implement a Share, so that users can re-share the files that they have just downloaded.
Among the metadata generated by Shares is a file hash. File hashes provide two functions:
- They identify identical files among different Shares. This enables Clients to download a single file from multiple Shares, either as fallback when the original Share is unavailable, or in parallel in order to provide better downloading performance.
- They allow Clients to verify that the downloaded data indeed corresponds to the requested file. A Merkle-tree based hash is used in order to allow for incremental verification when file chunks are downloaded from multiple Shares.
At the moment the protocol is described in the following three documents:
- FileHash: The choice of the hash function used to identify files in the ChiFS network.
- ShareApi: The HTTP API provided by Shares.
- HttpRequirements: General requirements when implementing HTTP
Note that none of these specifications are final. Major changes are expected as the details of the ChiFS network are being worked out.
In addition to the above specifications, there are also a few 'idea' documents. These outline some directions the protocol could take in the future, but are not actively being worked on right now.
Improvements can be proposed and discussed at the /chifs/chifs git repository.