Greetings!
In order to improve our communication with fellow developers and users, we have decided to start a blog. We don't have an exact plan and would welcome any input on topics you'd like to see covered; please reach out to hello@private.storage if you have ideas.
That said, this will definitely lean towards the "technical" side, being written mostly by our developers.
This first blog post will outline the software we currently use and maintain for the PrivateStorage service.
At PrivateStorage we strive to keep our technology free and open source -- usually abbreviated "FOSS", for "Free and Open-Source Software". Partially, this is "merely" the preference of the developers involved, but this approach has tangible benefits in our security and privacy space by allowing other experts to validate our claims.
We continue to add features and bug-fixes to several major FOSS libraries and projects. Here we will talk about the "top-level" projects that represent "PrivateStorage" and the service. At the end of this post, I will list all the libraries used in just one of these projects, to give an idea of the amount of other work we are able to leverage thanks to fellow FOSS developers. Giving back to this vibrant ecosystem is another motivation for releasing our work.
The Gridsync program is the GUI front-end, written in Python and using the Qt graphics framework. Using Qt lets us release on Windows, MacOS and Linux from the same codebase.
Gridsync acts as a sort of "controller", too, running the underlying Magic Folder and Tahoe-LAFS clients that are required. It interacts with these components via a localhost HTTP API. Security between components is achieved by an on-disk token: Gridsync must prove that it has read-access to the file containing the token. This limits things like cross-protocol attacks from Web sites, and roughly meaning the attack surface is similar to any other local application running as the same user.
Gridsync also co-ordinates a backup of all important state. This backup exists in the Tahoe-LAFS Grid and is represented by file saved as "the recovery key" (optionally encrypted).
A lot of the "file synchronization" functionality of PrivateStorage is accomplished by Magic Folder. This is a local daemon process that is run by Gridsync in our setup, but it could be run alone or by a different sort of UI. Magic Folder is also a Python application (its origins are from the Tahoe-LAFS codebase). It uses an associated Tahoe-LAFS client to do read and write operations to the Tahoe-LAFS Grid.
It also ships with two CLI tools: magic-folder and magic-folder-api.
The former is a friendly front-end intended for human use, while the latter is a more direct wrapper around the underlying HTTP API (useful for debugging or testing with other integrations).
The Tahoe-LAFS project has several logical behaviours bundled as one application.
One of those is acting as a "client" of the storage-servers (that together form "a Grid"). This is what Magic Folder (and hence Gridsync) use. We run several instances of Tahoe-LAFS acting as storage-servers.
Tahoe-LAFS come from the Object Capability ("OCap") ideas, representing access to a particular resource through cryptographic primitives and ultimately a short piece of text called "a Capability".
One feature of these is that anyone possessing a Capability has enough information to re-constitute the original (plaintext) data. So, every Capability string must be considered secret data.
Magic Folder or Gridsync take care to handle these secrets for the user, so you normally shouldn't see them or need to use them directly.
Because the Tahoe-LAFS servers can't know with whom users may have shared Capabilities, the way data is kept accessible in the system is through "leases". These allow someone with particular Capabilities to tell storage servers: "please keep this data for 31 more days".
These "leases" are where we tie our anonymous payment system into Tahoe-LAFS.
In order to create or extend "a lease" in a Tahoe-LAFS storage-server, our deployment asks for tokens. We call these tokens ZKAPs and talk about them as "storage time" in PrivateStorage.
To obtain some tokens, you exchange a voucher. This process is based on Cloudflare's "Privacy Pass" protocol and uses cryptographic blinding so that we cannot connect an issued token to the voucher that allowed its creation.
To obtain a voucher, you pay with a credit-card (or are given a not-yet-redeemed voucher some other way). So although we can know which credit-cards have paid for which vouchers, we cannot connect that to the tokens ultimately issued. We also cannot connect those credit-card details to any usage of the system (even if we're forced to try).
We can, however, tell if a given token is valid (i.e. issued by us) and has not yet been spent.
The "ZKAP Authorizer"-configured storage-servers that we run demand tokens for certain actions, including lease-renewal. The "ZKAP Authorizer"-configured Tahoe-LAFS client that Gridsync is running thus spends tokens as it must in order to accomplish its tasks.
This is why you have to run your client at least once a month: one thing it does is renew leases on files you care about. Without those leases being renewed, the underlying ciphertext can be deleted. Remember: we still don't know whose ciphertext is whose, just that some client somewhere redeemed enough tokens to renew a lease on it.
I have tried to broadly outline the projects involved in PrivateStorage.
Each of those projects in turn depends on other libraries in order to accomplish its tasks. As one example, we will look at all of the dependencies of Magic Folder. You can see the complete list of dependencies that enable Magic Folder to operate.
Here are some of the more-used ones:
For completeness, the rest of the dependencies for Magic Wormhole (including all "dependencies of dependencies" etc) are, in alphabetical order: appdirs; bcrypt; boltons; cbor2; certifi; cffi; charset-normalizer; collections-extended; constantly; distro; filelock; foolscap; future; hkdf; humanize; hyperlink; idna; incremental; markupsafe; netifaces; psutil; pyasn1; pyasn1-modules; pycddl; pycparser; pynacl; pyopenssl; pyrsistent; pyutil; pyyaml; requests; service-identity; setuptools; six; spake2; tahoe-lafs; tqdm; treq; tubes; txaio; txtorcon; typing-extensions; urllib3; werkzeug; zfec; zope-interface;
The idea is to highlight the depth of the FOSS community that we call ourselves a part of. We couldn't have written all of this from scratch, and appreciate all the public work (both professional and volunteer) that exists.
If you made it to here, thanks a lot for your attention!
To keep track of future posts, you can subscribe to our RSS feed.