Thanks to their resilience, integrity, and transparency properties, blockchains have gained much traction recently, with applications ranging from banking and energy sector to legal contracts and healthcare. Blockchains initially received attention as Bitcoin’s underlying technology. But for all its success as a popular cryptocurrency, Bitcoin suffers from scalability issues: with a current block size of 1MB and 10 minute inter-block interval, its throughput is capped at about 7 transactions per second, and a client that creates a transaction has to wait for about 10 minutes to confirm that it has been added to the blockchain. This is several orders of magnitude slower that what mainstream payment processing companies like Visa currently offer: transactions are confirmed within a few seconds, and have ahigh throughput of 2,000 transactions per second on average, peaking up to 56,000 transactions per second. A reparametrization of Bitcoin can somewhat assuage these issues, increasing throughput to to 27 transactions per second and 12 second latency. Smart contract platforms, such as Ethereum inherit those scalability limitations. More significant improvements, however, call for a fundamental redesign of the blockchain paradigm.
This week we published a pre-print of our new Chainspace system—a distributed ledger platform for high-integrity and transparent processing of transactions within a decentralized system. Chainspace uses smart contracts to offer extensibility, rather than catering to specific applications such as Bitcoin for a currency, or certificate transparency for certificate verification. Unlike Ethereum, Chainspace’s sharded architecture allows for a ledger linearly scalable since only the nodes concerned with the transaction have to process it. Our modest testbed of 60 cores achieves 350 transactions per second. In comparison, Bitcoin achieves a peak rate of less than 7 transactions per second for over 6k full nodes, and Ethereum currently processes 4 transactions per second (of a theoretical maximum of 25). Moreover, Chainspace is agnostic to the smart contract language, or identity infrastructure, and supports privacy features through modern zero-knowledge techniques. We have released the Chainspace whitepaper, and the code is available as an open-source project on GitHub.
System Overview
The figure above illustrates the system design of Chainspace. Chainspace is comprised of a network of infrastructure nodes that manage valid objects and ensure that only valid transactions on those objects are committed. Let’s look at the data model of Chainspace first. An object represents a unit of data in the Chainspace system (e.g., a bank account), and is in one of the following three states: active (can be used by a transaction), locked (is being processed by an existing transaction), or inactive (was used by a previous transaction). Objects also have a type that determines the unique identifier of the smart contract that defines them. Smart contract procedures can operate on active objects only, while inactive objects are retained just for the purposes of audit. Chainspace allows composition of smart contracts from different authors to provide ecosystem features. Each smart contract is associated with a checker to enable private processing of transactions on infrastructure nodes since checkers do not take any secret local parameters. Checkers are pure functions (i.e., deterministic, and have no side-effects) that return a boolean value.
Now, a valid transaction accepts active input objects along with other ancillary information, and generates output objects (e.g., transfers money to another bank account). To achieve high transaction throughput and low latency, Chainspace organizes nodes into shards that manage the state of objects, keep track of their validity, and record transactions aborted or committed. We implemented this using Sharded Byzantine Atomic Commit (S-BAC)—a protocol that composes existing Byzantine Fault Tolerant (BFT) agreement and atomic commit primitives in a novel way. Here is how the protocol works:
- Intra-shard agreement. Within each shard, all honest nodes ensure that they consistently agree on accepting or rejecting a transaction.
- Inter-shard agreement. Across shards, nodes must ensure that transactions are committed if all shards are willing to commit the transaction, and rejected (or aborted) if any shards decide to abort the transaction.
Consensus on committing (or aborting) transactions takes place in parallel across different shards. A nice property of S-BAC’s atomic commit protocol is that the entire shard—rather than a third party—acts as a coordinator. This is in contrast to other sharding-based systems with cryptocurrency application like OmniLedger or RSCoin where an untrusted client acts as the coordinator, and is incentivized to act honestly. Such incentives do not hold for a generalized platform like Chainspace where objects may have shared ownership.
Chainspace also supports transparency and auditability. Its operation is divided into time periods called epochs, and each shard maintains a hash chain that represents transactions processed in each epoch, as described below:
- Compute checkpoint. In every epoch, nodes in each shard agree on a checkpoint: a block (Merkle tree) of evidence including transactions processed in the current epoch, and signed promises from other nodes.
- Extend hash chain. They then extend their hash chain by hashing the root of this Merkle tree and a block sequence number, with the head hash of the chain so far, to create the new head of the hash chain. Each peer signs the new head of their chain, and shares it with all other peers in the shard, and anyone who requests it.
Threat Model and Security Properties
Chainspace supports security properties against two polynomial time bounded adversaries:
- Honest Shards (HS) may create arbitrary contracts, and input arbitrary transactions into Chainspace, however they are bound to only control up to f faulty nodes in any shard. As a result, and to ensure the correctness and liveness properties of Byzantine consensus, each shard must have a size of at least 3f+1 nodes.
- Dishonest Shards (DS) has, additionally to HS, managed to gain control of one or more shards, meaning that they control over f nodes in those shards. So its correctness or liveness may not be guaranteed.
Given this threat model, Chainspace supports the security properties below:
- Transparency. Chainspace ensures that anyone who possesses the identity of a valid object may authenticate the full history of transactions and objects that led to the creation of the object.
- Integrity. Subject to the HS threat model, when one or more transactions are submitted only a set of valid non-conflicting transactions will be executed within the system. This includes resolving conflicts—in terms of multiple transactions using the same objects—ensuring the validity of the transactions, and also making sure that all new objects are registered as active. Ultimately, Chainspace transactions are accepted, and the set of active objects changes, as if executed sequentially.
- Encapsulation. The smart contract checking system of Chainspace enforces strict isolation between smart contracts and their state—thus prohibiting one smart contract from directly interfering with objects from other contracts. Where cross-contract calls are supported, this is mediated by well defined interfaces providing encapsulation.
- Non-repudiation. In the case where conflicting or otherwise invalid transactions were accepted in honest shards (in the case of the DS threat model), then evidence exists to identify the parties or shards in the system that allowed the inconsistency to occur. As a result, failures outside the HS threat model are detectable; the guilty parties may be banned, and appropriate off-line recovery mechanisms could be deployed.
Implementation and Evaluation
We implemented a prototype of Chainspace (released as an open-source project) in about 10k lines of Python and Java code. The implementation consists of two components: a Python contracts environment and a Java node. The Python contracts environment allows developers to write, deploy and test smart contracts. The Java node implements a shard replica that accepts incoming transactions from clients and initiates, and executes, the S-BAC protocol. We developed a number of key system smart contracts (providing flexible policies about managing shards, smart contract creation, auditing and accounting) and application smart contracts (smart-meter private billing, and smart voting system) and evaluated their performance to validate support for high-integrity and high-privacy applications (see the whitepaper for details).
Here we present our evaluation of the performance and scalability of Chainspace, through deployments on Amazon EC2 containers. We launched up to 96 nodes on t2.medium virtual machines, each containing 8 GB of RAM on 2 virtual CPUs and running GNU/Linux Debian 8.1. We sent transactions to the network from a Chainspace client running on a t2.xlarge virtual machine, containing 16 GB of RAM and 4 virtual CPUs, also running GNU/Linux Debian 8.1. In our tests, we map objects to shards randomly.
The figure above measures the effect of the number of shards on transaction throughput. We see that the transaction throughput of Chainspace scales linearly with the number of shards: with 4 nodes per shard, the number of transactions per second (t/s) increases on average by 22 for 1-input transactions for each shard added. This is because as inputs are randomly assigned to shards based on their hashes, the transaction processing load is spread out over a larger number of shards.
Another important metric of scalability is the client-perceived latency—the time from when a client submits a transaction until it receives a decision about whether the transaction has been committed—under varying system loads expressed in terms of transactions received per second. The figure above shows the effect of transactions received by the system per second (all 1-input transactions) on client-perceived latency for 2 shards, each having 4 nodes. In the previous figure, we found that the average throughput for a Chainspace system with similar configuration is 75 1-input transactions per second. Consequently, we observe here that the increase in latency with varying system loads is smaller for 20 t/s–60 t/s (average 69 ms), but the values start to get bigger after 60 t/s (average 210 ms). This is when the system reaches its maximum transaction throughput, causing a backlog of transactions to be processed.
Limitations and Improvements
While Chainspace shows promising results with respect to scalability, it comes with some limitations and open areas that we plan to tackle in future work:
Avoiding dishonest shards. The integrity properties of Chainspace rely on all shards managing objects being honest, namely containing at most f fault nodes each. As Chainspace allows any set of nodes to create a shard, the function mapping objects to shards must avoid dishonest shards. Our isolation properties ensure that in the worst case a dishonest shard can affect state from contracts that have objects mapped to it. For this reason, Chainspace allows the contract creator to designate which shards manage objects from their contract. This embodies specific trust assumptions where users have to trust the contract creator both for the code (which is auditable) and also for the choice of shards to involve in transactions—which is also public.
Recovery from malicious shards. In case one or more shards are malicious, we provide an auditing mechanism for honest nodes in honest shards to detect the inconsistency and to trace the malicious shard. However, it is not clear how to automatically recover from detecting such an inconsistency. Options include: forcing a fork into one or many consistent worlds; applying a rule to collectively agree the canonical version; patching past transactions to recover consistency; or agree on a minimal common consistent state.
Suitable fee structure. Checkers involved in validating transactions can be costly. For this reason, we allow peers in a shard to accept transactions subject to a Chainspace payment to the peers. However, this ‘flat’ fee is not dependent on the cost or complexity of running the checker which might be more or less expensive. Ethereum instead charges gas according to the cost of executing the contract procedure—at the cost of implementing their own virtual machine and language.
Graceful degradation under high contention. The S-BAC protocol ensures correctness in all cases. But under high contention for the same object the rate of aborted transactions rises. This is expected, since the S-BAC protocol in effect implements a variant of optimistic concurrency control, that is known to result in aborts under high contention. There are strategies for dealing with this in the distributed systems literature, such as locking objects in some conventional order—however, none is immediately applicable to the byzantine setting.
Messaging complexity of BFT. Chainspace uses BFT-SMART’s implementation of Practical Byzantine Fault Tolerant protocol as a black box, and inherits its O(n2) messaging complexity. However, BFT- SMART can be replaced with any improved PBFT variant without breaking any security assumptions.
Summary
We presented the design of Chainspace—an open, distributed ledger platform for high-integrity and transparent processing of transactions. Chainspace offers extensibility though privacy-friendly smart contracts. We presented an instantiation of Chainspace by parameterizing it with a number of system and application contracts, along with their evaluation. Unlike existing smart-contract based systems such as Ethereum, Chainspace offers high scalability through sharding across nodes using a novel distributed atomic commit protocol called S-BAC, while offering high auditability. We presented an implementation and evaluation of S-BAC on a real cloud-based testbed under varying transaction loads and showed that Chainspace’s transaction throughput scales linearly with the number of shards by up to 22 transactions per second for each shard added, handling up to 350 transactions per second with 15 shards. As such, it offers a competitive alternative to both centralized and permissioned systems, as well as fully peer-to-peer, but unscalable systems like Ethereum.
We have released the Chainspace whitepaper, and the code is available as an open-source project on GitHub. We would be happy to receive your feedback, thoughts, and suggestions about Chainspace via comments on this blog post, or email.
The Chainspace project is developed, and funded, in the context of the EU H2020 Decode project, the EPSRC Glass Houses project and the Alan Turing Institute.