Decentralized Applications in the Cloud

[tl;dr Are decentralized applications implemented using cloud provider managed services really decentralized, and are they ready for prime time usage? Proof-of-stake is not cloud-friendly, and proof-of-stake still immature. Open source blockchain solutions provide mechanisms to ensure secure operational independence of organizations on the same network, even if they are using the same cloud provider. Integrating decentralized applications with cloud-native applications may help build highly integrated platform eco-systems.]

A “decentralized application” (or ‘dapp’) is a technology solution to a business process that spans a number of independent, geographically distributed and possibly untrustworthy, organizations. A list of dapps can be found here, but most current dapps are somewhat frivolous in nature, somewhat similar to websites appearing on the WWW in the early nineties after Mosaic was released.

A decentralized application relies on cryptographic techniques to ensure trust (i.e., immutability of history, validity of transactions), and uses peer-to-peer communication mechanisms to avoid a centralized bottleneck.

Today, in the financial world in particular, regulatory-mandated organizations such as clearing houses, SEF‘s and the SWIFT financial network broker trust between independent organizations. These are centralized services where trust is assumed (by rule of law). Other trust brokers, apart from government, include industry consortia, community groups, non-governmental organizations, etc. While trust brokers will always exist (in non-anarchic societies!), the way in which they participate in enterprise value chains will likely change significantly in the coming years, as cloud- and blockchain-based technologies enable processes to extend beyond the traditional enterprise boundaries.

The most famous decentralized application is BitCoin, based on blockchain technology, which assumes zero trust between individual participants. Many decentralized applications rely on blockchain technology to provide trust, or at least consensus. The concept of ‘blocks’ in a blockchain is a practical optimization for truly decentralized applications; applications which merely require immutability and verifiability of transactions do not require blocks, but will then have centrally managed state. (AWS’s new QLDB service is an example of this.)

The concept of ‘blockchain-as-a-service’ (BaaS) has been around for a while – as evidenced by the number of companies operating in this space; these are principally tools to enable the building of decentralized applications and aim to isolate businesses from the complexities of blockchain development.

Recently, the major cloud service providers have entered the ‘BaaS’ space – AWS and Azure offer BaaS solutions, currently leveraging open-source offerings from HyperLedger and Ethereum – neither of which rely on Bitcoin technology, although both are blockchain-based. Other blockchain solutions, such as R3’s Corda, have yet to be added as foundational cloud provider services, but Corda nodes can be deployed on any IaaS cloud provider (e.g., AWS and Azure, and GCP, etc.)

Implications of Blockchain on the cloud

There are some significant implications with respect to the use of blockchain technology in the cloud. The first is related to how trust is achieved effectively on cloud-based infrastructure, the second to how operationally decentralized applications built on cloud infrastructure really are.

Achieving Trust

In BitCoin, trust is achieved through ‘proof-of-work’ – i.e., a untrusted node has to consume considerable resources (mostly power) to prove they have correctly validated a block of transactions. Multiple nodes compete for this validation, and there is a reward for the ‘winner’, which a majority of nodes must accept as the winner. The basic idea is that it would be extremely expensive in terms of resource utilization and alliance building to be able to successfully validate ‘fake’ transactions.

The problem for cloud providers is that this is a very inefficient use of cloud resources. ‘Blockchain-as-a-service’ for proof-of-work-based protocols (i.e., Bitcoin and its derivatives) is essentially provided by mining pools. These are entities (people, organizations) who have specialised hardware who come together to solve Bitcoin proof-of-work puzzles, and share the proceeds as a group. This is not the business the major cloud providers are in.

Therefore, proof-of-work is essentially not feasible to provide on public cloud. Instead, other variants, such as ‘proof-of-stake‘, or permissioned-ledgers are necessary. Technically, this precludes ‘Ethereum-as-a-service’ from being offered by cloud providers, as Ethereum still relies on proof-of-work, but Ethereum’s intention is to (eventually) be proof-of-stake-based.

Unfortunately, these non proof-of-work-based alternatives are still relatively immature and unproven. In particular, there is a potential issue around combinatorial complexity management in permissioned ledgers, if each organization has to define and manage its own ‘universe’ of participants and the data they have access to. (Note – it’s not seen as an issue for public ledgers, as all participants have equal access to content on the network, so combinatorial complexity does not increase with number of participants. Also, the content of public ledgers is transparent to everyone,)

Decentralized Operations

If all application instances in a decentralized blockchain service are running on the same cloud, how ‘decentralized’ is that really – from an operational perspective?

Clearly, the decentralized blockchain service will fail if the cloud provider itself suffers an outright failure (i.e., all regions simultaneously unavailable). This could bring entire industries or supply chains to their knees..but it is also (we hope) very unlikely, given the focus that cloud providers rightly have on resilience. We are putting a lot of trust into the cloud providers to do the right thing..although at some point, to allay systemic risk concerns, this may perhaps need to be enshrined in law (in much the same way utilities have legal obligations for service provision).

Managed solutions like AWS Managed Blockchain currently assume the blockchain service is entirely hosted in the providers cloud, and that every organization with nodes on the network have their own cloud accounts in which they operate infrastructure they control.

In this architecture, every organization wholly controls its own infrastructure (in its own AWS account), the blockchain software running on that infrastructure, and the ledger data stored within it.

The blockchain service software itself (in the AWS case, Hyperledger Fabric) provides mechanisms to ensure new (ledger-updating) code is only deployed to the blockchain network when a quorum of participants have cryptographically signed the new code. This provides guarantees that ‘rogue’ code running on another member’s node cannot force an invalid consensus.

In this way, individual organisations retain control over the operation of their part of the blockchain network, while decoupling them sufficiently from other organizations and how they operate their parts of the network. This achieves operational decentralization, the equivalent of each organization running their nodes in their own data centers.

Achieving Decentralized Goals

A decentralized application is intended to remove process bottlenecks, or eliminate the ‘middle-man’ by ensuring multiple independent parties have the same reliable, consistent view of a shared dataset that cannot be tampered with, without relying on a single central 3rd party.

We have established that while the cloud service provider centralizes infrastructure and shared blockchain services, control over data (ledger state) and applications is truly decentralized. As long as the cloud provider itself is available, individual participants in the network can fail or be unavailable without impacting overall service availability.

Managing State in Decentralised Ledgers

A key challenge in any transactional system is transactionally recording an event as having occurred, as well as the impact of the event – e.g., recording that Bob requested to transfer $X to Alice’s account from his account, as well as updating Bob’s account balance and Alice’s account balance with the adjusted balance. But state is application specific, and many applications only need to know the event happened, and prefer to manage their own view of state – for example, analytics, machine-learning, AI, etc have very different states to manage for the same event.

For speed and integrity, some state may need to be managed in the decentralized ledger, but this should ideally be kept to a minimum. Instead, the event log is key, and most applications will likely need to maintain their own dynamic view of state which can be queried according to various application needs.

While blockchain solutions like Hyperledger offer some means to manage stage (e.g., using CouchDB to capture and query complex JSON objects), treating a distributed ledger like a traditional operational datastore is likely to cause problems down the line.

The Hyperledger Fabric is addressing this need by introducing the concept of the EventHub which in principle would allow organizations to build ‘listeners’ to act on committed events. Perhaps in the future this may be integrated with AWS EventBridge allowing events committed to the decentralized ledger to be integrated into more traditional (non-decentralized) application workflows, enabling a very powerful ecosystem of decentralized and cloud-native applications across multiple organizations to work together in a trusted manner.

Summary

As non proof-of-work blockchain solutions mature, decentralized business applications enabled via public cloud will likely be a major area of growth, as industries and special interest groups collaborate to solve common problems while retaining operational independence.

The nascent services provided by AWS and Azure are not yet ready for mission-critical large-scale use, but look like to be an excellent way for organizations to experiment with decentralized applications and re-imaging many legacy processes that currently rely on centralized intermediaries.

Finally, integration of decentralized applications with enterprise applications is likely to be significant area of growth in the future.

Decentralized Applications in the Cloud

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s