System Implementation

Design Goals

It seems best to begin with a number of design goals which guided us:

1. The system must not be account-based, but constitute a true digital bearer certificate exchange system, where digital wallets exist but accounts do not.
2. To engender user trust in the system, it must be 100% open source, not only for our code but for any code packages or libraries which are utilized.
3. The system must be distributed so that it cannot be shut down by highly organized crime.
4. All data must be encrypted, and handled in such a way that the user does not need to trust any of the system operators. (The sole exception being the Issuer, which must be trusted to store bona fide backing assets.)
5. Payments must be irrevocable, and untraceable. It must be physically impossible for any component, even the VP, to provide a transaction history for any user. That which is impossible cannot be compelled. (See goals 1 and 3.)
6. The system architecture must provide inherent economic benefits to its operators, while minimizing costs to users.
7. To mitigate the threats of DDoS, spam, and easy traffic analysis, HTTP browser and email traffic should be avoided.
8. Given the increasing ubiquity of wireless hand-held devices such as web-enabled cell phones, it is desirable not to restrict users needlessly to the desktop environment.

Platform

For these reasons, we chose SELinux, XMPP, and Java as our implementation mediums. (Though the client, naturally, will run anywhere that a current Java Virtual Machine (JVM) is available.)

This led us to search for a robust, well-supported open source implementation of an XMPP server and client, written in Java using a plugin architecture. We settled on the Openfire server and the Spark client. The client has seen nearly 2 million downloads; the server nearly 4 million (probably double-counting upgrades). Both are completely open source and have excellent plugin developer documentation, including full Javadocs.

We have implemented the VP, OFS, PKS, SDS, and DHT as Openfire plugins. The VSC is of course implemented as a Spark plugin. Server components communicate using a pair of s2s sockets (one in each direction), secured with TLS. The VSC uses a standard SSL connection to the OFS, and opens an anonymous SSL connection to the PKS, SDS, and DHT after successful login at the OFS.

Current Implementation Status

Our current implementation does not yet use a Tahoe grid, nor a true distributed hash table. (Note: the Tahoe grid SDS was implemented and can be selected in the XML config, but the performance of Tahoe is much too poor to utilize it as a flat-file database.) Instead, we created a layer on top of a MySQL database for each. Tahoe works by creating folders located at randomized hashes, and generating a "read capability" (permission) and a "read-write capability" to control access to each folder. It proved simple enough to emulate this functionality by creating an access gating table which allows the use of randomly generated hash indices and required "caps" in order to determine the access index of an underlying storage table.

Thus each voucher safe (VS) has a number of "folders" (aggregated rows in the database) to which access can be gained only by presenting the required hash and cap. The records themselves, whether vouchers, pending payments, receipts, login or recovery secrets, are always encrypted. With the addition of a passive slave mirror for the database in another location, we deem this to be good enough for an initial deployment, until and unless Tahoe performance improves substantially. The code has been written in such a way that only a single class need be replaced to substitute Tahoe for MySQL in the SDS, and this can be done in the static XML config.

Similarly, the DHT is presently written as a layer on top of a simple single-table MySQL database. However, it could easily be reworked to sit on top a suitable distributed database application. We consider that Project Voldemort represents a potentially good choice.

The SDS and DHT are simple plugins which parse XML commands (IQs with child element extensions) and retrieve or update data as requested. The SDS additionally requires the presentation of an actual hash index and associated cap. Nonexistent hash or bad cap, sorry no service. In this sense they operate similar to Loom, in that the odds of accidentally lucking into a working combination are truly astronomical. Even if one did, the records are always encrypted.

A given SDS can serve n Voucher Publishers, but each VP must perforce employ only a single SDS. Any distribution of the data (such as via Tahoe) takes place behind the SDS gateway.

Similarly, a DHT can serve m VPs, and any number of users.

An OFS can serve x VPs, and a given VP can interface with y OFSen, as might be needed based on traffic loads.

Interceptors and Data Integrity Safeguards

The SDS, DHT, and PKS plugins also implement packet interceptors, which impose the requirement that all XML packets allowed processing adhere to their defined namespaces. This prevents users from, for example, chatting with each other via the SDS rather than the OFS. An interceptor also protects the VP from any bizarre packets which might be forwarded to it from the OFS, PKS, or SDS. In general, we have made the assumption that any system component could "go rogue" at any time, and protect from damage if one does.

Required connections between Openfire plugin components are actively maintained by managers which detect timeouts and disconnections and perform automatic reconnects. The VP maintains a "hard queue" for its connections to the SDS and DHT. This means that should a timeout occur or its connection be lost, the VP is able to buffer up the messages in a disk-backed memory map until connectivity is restored, then catch up by draining the queue (in a measured way, not in a flood!). This allows the VP to guarantee that updates in progress will eventually be delivered, even though subsequent requests will fail immediately without the connection. This feature is critical to prevent user data loss.

A Word About Crypto

The standard Java virtual machine provides RSA and DSA key pairs up to 4096 bits in strength, symmetric encryption via 3DES, and hash digests up to SHA-512. Although Openfire deploys Bouncy Castle to deal with tasks such as X.509 certificate generation, we felt it would be undesirable to require additional non-standard crypto libraries for the client. (Key pairs are always generated in the client.) Therefore, we selected RSA keys of 2048 bits, and use triple-DES with 24 byte (192 bit) keys to encrypt the login secrets blobs which contain the VS private keys. (More details on login protocols are available elsewhere.) We considered that a key length of 2048 bits represents the best trade-off between security, operation speed, and size of encrypted objects (including signatures).

However, different assumptions could also be made. For this reason, the selected key algorithm and bit length are static configuration variables which can be set at the VP. (Although once set, they cannot easily be changed for an existing user base.)

Signatures are built using SHA-1 hashes. Concerns about the possible reversibility of SHA-1 are moot when the sole purpose is to verify a signature on clear text which the user already possesses. However, SHA-512 hashes are used when constructing login and recovery hashes, and the VP uses SHA-512 to build its hash of transaction data, which it must store in its database in order to allow the pickup of voucher payments (or the reversion of expired payments not picked up) by voucher safes which present the correct hash value. This stronger hash makes it impossible for an attacker to lift transaction details off of the VP.

Component Details

There are further details available (or will be soon) on the implementation architecture of each individual component. These are as follows:

• Design and Implementation of the OFS?
• Design and Implementation of the VP?
• Design and Implementation of the SDS?
• Design and Implementation of the DHT?
• Design and Implementation of the PKS?
• Design and Implementation of the Client?

---