Say Goodbye to Miners, and Hello to Builders, Proposers and Verifiers: Sharding and Danksharding

Say Goodbye to Miners, and Hello to Builders, Proposers and Verifiers: Sharding and Danksharding

I have just moved house and currently live near other houses being built. The one thing you notice is that there is considerable activity in building parts of houses, and where each house is at different stages of development. This is a distributed model of building a house, and where many houses can be built at the same time. If we build one house at a time, we would end up with bottle-necks, and where we would continually have to gain approval for every part of the house build. In a distributed approach to house building, each house can be built independently of the others.

And, imagine if every time a worker had to do some work on the house build, they had to put a cost on their work, and then bid for the work to be done. For this, we normally bundle up all the costs of all the work into a contract and then get each builder to bid for the whole house build.

Well, the world of Ethereum is now taking these approaches and moving towards the increased distribution of its work. This will hopefully move away from slow processing and high gas fees. One of the key implementations in this new model of Ethereum is the move away from miners and proof of work, and towards builders, proposers and verifiers, and which should allow Ethereum to scale-up.

Introduction

It is well known that the success of Ethereum has outgrown its original model, especially its scaling issues and its usage of proof of work as identification of trust. One dream of the Internet is of an infrastructure which distributed processing and services around the world, but we have ended up with a centralised approach and where data is concentrated within centralised servers. In an Ethereum viewpoint of the world, data is distributed around the world and stored in many places, and where there is no central control of the current state of the Internet. In a perfect Internet, we contain local versions of the blockchain and communicate with that, and where we know that this is the same state that someone else on the other side of the planet is using.

We are now seeing sharding being rolled out into the Ethereum infrastructure, and which will significantly increase the number of transactions that can be processed at any given time. With sharding, the Ethereum nodes will only need to store a part of the distributed registry and relay this information to others in the network.

Sharding

So let’s look in detail at how sharding works.

The major problem that we are fixing is that, in order to create a consensus, each of the nodes must do all of the calculations and certify every transaction. This slows down the network when there is an increase in the number of transactions, and Ethereum has thus hit a bottleneck.

Now let’s consider that we have a definitive state of the Ethereum network for all of its transactions — a Global State. Everyone should be able to check this state and see all of the transactions. But now we can split the states with a Merkle Tree, and where the root defines the global state and where we break up the states into shard nodes. In the following, we have three shards which make up the global root, and where each shard gathers the transactions that they are responsible for:

We thus split the overall state of the infrastructure into shards, and where each unique account is in one shard, and where these accounts can only transact with other accounts within this shard. Vitalik Buterin defines this by imagining that the blockchain was split into thousands of islands, and where each island can do its own thing. On each island the people on that island trade with each other, and perform transactions, and do not require those from other islands to be involved. As long as we have a trusted agent on the island recording the transactions, this can feed into a global transaction ledger. If those on the island contact other islands, we can put in-place a mechanism to support this, and link the transactions, such as sending a receipt of a transaction from one island person to another, and for us to get a receipt back from the other island.

The sharding system works by creating a transaction group which has a transaction group header and a transaction group body (and which stores all of the transactions related to that shard). In the example below we have a Shard ID of 22, and in the transaction group header we have:

• Shard ID: This is identified as the shard that the transaction relates to.
• Pre-state root: This defines the state Shard 22 root before the new transactions.
• Post state root: This defines the state Shard 22 root after the new transactions.
• Receipt root: This is the receipt of Shard 22 for transactions.

We also see signatures within the transaction group header. These are nodes which have been selected to validate the transaction in the shard and are chosen at random. Each shard thus identifies itself with an ID, and then maintains its state with its own transactions, where it maps back to a previous state. This is similar to having an independent blockchain running for a shard, and where the transactions within the shard do not interact with others outside the shard.

But what happens when the accounts in one shard want to transact with another shard? Within this, we record a transaction and send a proof of receipt to Shard B. This shard then sends back the consumption of the receipt.

There are major challenges with sharding, and the main one is whether we can make sure that sharing works on trusted nodes and where shards have trust between each other.

Danksharding

On the way we implement a fully sharded infrastructure, we need to migrate our existing methods to a fully sharded infrastructure, and one of the methods being implemented is Danksharding (named after Dankrad Feist, an Ethereum developer). Danksharding aims to deliver improved speed, reduced gas fees and great storage of data. It is basically part of the move towards a fully sharded infrastructure for Ethereum. In this, we say goodbye to miners and now have proposers, builders and verifiers.

Overall, we use trusted block builders to select the data and transactions that go into a slot and then make a request to a proposer. The builders propose a fee for the transactions to the proposer, and then the proposer selects the highest bidder. If the builder gets chosen for a given slot in a block, they will then be responsible for adding the data and transactions to this slot (Figure 3). A verifier then verifies that the slot has been built correctly. The core advantage is that the builder proposes a merged-free from the transactions, and, from this, it is hoped that the gas fees for transactions will be significantly reduced, as each transaction does not have an associated gas fee. As long as there are not too many bidders, the overall cost of the slot filling will likely be fairly low.

The miners in our existing Ethereum infrastructure are basically just block proposers, but once the new infrastructure is implemented, they will become validators, and where new software agents known as block builders will take a key role in gathering and proposing data and transactions for the slots. If a slot is verified as being invalid, an erasure code can be proposed, and where the data can eventually be marked for deleted from the slot.

EIP-4844 and Blobs

Any roll-out of the new infrastructure needs to be done with care, as a single flaw could lead to a large-scale compromise. And so EIP-4844 has been proposed as a prototype specification for danksharding [here]:

The full name of the proposal is related to Binary Large Objects — or blobs. While data blobs exist in many other application areas, the main difference in the new proposal is that they could be associated with smart contracts. For blobs, a proposer proposes that a data bundle should exist and that will be relevant to transactions. These can have a header and a main body of data and will be digitally signed by a proposer. The maximum size of this data blob is proposed at 1MB. This will thus allow transactions to gain access to shared data.

Conclusions

If Bitcoin is a Ford T, Ethereum is a Tesla. Along the way, the Ford T has been upgraded, and without pulling it in for a complete upgrade. It is like upgrading a speeding car. So let’s see how one of the largest computational engines ever created will cope with this new upgrade.