Last update: 25 Feb. 2022.
For the latest Shard Blob Transactions EIP, see https://github.com/ethereum/EIPs/pull/4844
Experimental implementation by Proto, Matt and George: https://github.com/protolambda/go-ethereum/pull/1
Ongoing work by Prysm team (Prysmatic Labs): https://github.com/prysmaticlabs/prysm/compare/kiln…blobs
This specification modifies and extends the Sharding-format data blob transactions proposal by Vitalik and Dankrad: https://notes.ethereum.org/@vbuterin/blob_transactions
Introduce a new transaction format for "blob-carrying transactions" which contain a large amount of data that cannot be
accessed by EVM execution, but whose commitment can be accessed.
The format is intended to be fully compatible with the format that will be used in full sharding.
Rollups are in the short and medium term, and possibly in the long term, the only trustless scaling solution for Ethereum.
Transaction fees on L1 have been very high for months and there is greater urgency in doing anything required to help facilitate an ecosystem-wide move to rollups.
Rollups are significantly reducing fees for many Ethereum users: Optimism and Arbitrum frequently provide fees that are ~3-8x lower than the Ethereum base layer itself,
and ZK rollups, which have better data compression and can avoid including signatures, have fees ~40-100x lower than the base layer.
However, even these fees are too expensive for many users. The long-term solution to the long-term inadequacy of rollups
by themselves has always been data sharding, which would add ~16 MB per block of dedicated data space to the chain that rollups could use.
However, data sharding will still take a considerable amount of time to finish implementing and deploying.
This EIP provides a stop-gap solution until that point by implementing the transaction format that would be used in sharding,
but not actually sharding those transactions. Instead, they would simply be part of the beacon chain and would need to be downloaded
by all consensus nodes (but can be deleted after only a relatively short delay).
There would be a reduced cap on the number of these transactions that can be included, corresponding to a target of ~1 MB per block and a limit of ~2 MB.
Constant | Value |
---|---|
SYSTEM_STATE_ADDRESS |
0x000.....0100 |
TOTAL_BLOB_TXS_STORAGE_SLOT |
0 |
BLOB_TX_TYPE |
Bytes1(0x05) |
FIELD_ELEMENTS_PER_BLOB |
4096 |
BLS_MODULUS |
52435875175126190479447740508185965837690552500527637822603658699938581184513 |
KZG_SETUP_G2 |
Vector[G2Point, FIELD_ELEMENTS_PER_BLOB] , contents TBD |
KZG_SETUP_LAGRANGE |
Vector[BLSCommitment, FIELD_ELEMENTS_PER_BLOB] , contents TBD |
BLOB_COMMITMENT_VERSION_KZG |
Bytes1(0x01) |
BLOB_VERIFICATION_PRECOMPILE_ADDRESS |
Bytes20(0x13) |
BLOB_VERIFICATION_PRECOMPILE_GAS |
1800000 |
POINT_EVALUATION_PRECOMPILE_ADDRESS |
Bytes20(0x14) |
POINT_EVALUATION_PRECOMPILE_GAS |
50000 |
MAX_BLOBS_PER_BLOCK |
16 |
TARGET_BLOBS_PER_BLOCK |
8 |
MAX_BLOBS_PER_TX |
2 |
GASPRICE_UPDATE_FRACTION_PER_BLOB |
64 |
MAX_VERSIONED_HASHES_LIST_SIZE |
2**24 |
MAX_CALLDATA_SIZE |
2**24 |
MAX_ACCESS_LIST_SIZE |
2**24 |
MAX_ACCESS_LIST_STORAGE_KEYS |
2**24 |
MAX_TX_WRAP_KZG_COMMITMENTS |
2**24 |
LIMIT_BLOBS_PER_TX |
2**24 |
GAS_PER_BLOB |
120000 |
HASH_OPCODE_BYTE |
Bytes1(0x49) |
HASH_OPCODE_GAS |
3 |
Type | Base type | Additional checks |
---|---|---|
BLSFieldElement |
uint256 |
x < BLS_MODULUS |
Blob |
Vector[BLSFieldElement, FIELD_ELEMENTS_PER_BLOB] |
|
VersionedHash |
Bytes32 |
|
KZGCommitment |
Bytes48 |
Same as BLS standard "is valid pubkey" check but also allows 0x00..00 for point-at-infinity |
Converts a blob to its corresponding KZG point:
Converts a KZG point into a versioned hash:
Verifies a KZG evaluation proof:
Approximates 2 ** (numerator / denominator)
, with the simplest possible approximation that is continuous and has a continuous derivative:
We introduce a new EIP-2718 transaction type,
with the format being the single byte BLOB_TX_TYPE
followed by an SSZ encoding of the
SignedBlobTransaction
container comprising the transaction contents:
EIP-2718 is extended with a "wrapper data", the typed transaction can be encoded in two forms, dependent on the context:
TransactionType || TransactionNetworkPayload
, or LegacyTransaction
TransactionType || TransactionPayload
, or LegacyTransaction
Execution-payloads / blocks use the minimal encoding of transactions.
In the transaction-pool and local transaction-journal the network encoding is used.
For previous types of transactions the network encoding is no different, i.e. TransactionNetworkPayload == TransactionPayload
.
The TransactionNetworkPayload
wraps a TransactionPayload
with additional data:
this wrapping data SHOULD be verified directly before or after signature verification.
When a blob transaction is passed through the network (see the Networking section below),
the TransactionNetworkPayload
version of the transaction also includes blobs
and kzgs
(commitments list).
The execution layer verifies the wrapper validity against the inner TransactionPayload
after signature verification as:
blob_versioned_hashes
must start with the byte BLOB_COMMITMENT_VERSION_KZG
MAX_BLOBS_PER_TX
blob commitments in any single transaction.MAX_BLOBS_PER_BLOCK
total blob commitments in a valid block.kzg_to_versioned_hash(kzg[i]) == versioned_hash[i]
The signature is verified and tx.origin
is calculated as follows:
On the consensus-layer the blobs are now referenced, but not fully encoded, in the beacon block body.
Instead of embedding the full contents in the body, the contents of the blobs are propagated separately, as a "sidecar".
This "sidecar" design provides forward compatibility for further data increases:
with data-availability-sampling (DAS) not all blobs will have to be downloaded by all beacon nodes on the network.
We add to the BeaconBlockBody
a new object, blob_kzgs: List[KZGCommitment, MAX_OBJECT_LIST_SIZE]
,
and update BeaconBlock
and SignedBeaconBlock
to reflect this updated BeaconBlockBody
.
At the end of the process_block
state-transition, the BeaconBlock
and its contained ExecutionPayload
need to be cross-validated with cross_validate(block.body)
.
We introduce a BlobsSidecar
container:
Nodes broadcast the SignedBeaconBlock
along with its corresponding
BlobsSidecar
over the network instead of plain beacon blocks.
When a node receives both objects, it first calls verify_blobs
and if this call passes it runs the usual processing on the beacon block.
A verified BlobsSidecar
is a strict requirement for the validity of the beacon block.
Without the sidecar the block may be processed but MUST NOT be considered valid until a valid BlobsSidecar
has been downloaded.
This download requirement can be substituted with an availability-check in the future with data-availability-sampling.
Full peer-to-peer specification is out of scope for this EIP, the consensus-specs expands on this instead.
In summary, the peer to peer layer changes involve:
SignedBeaconBlock
, the beacon_block
gossip topic is updatedbeacon_blocks_by_range/2/
and beacon_blocks_by_root/2/
type tables are updated.blobs_sidecar
/eth2/beacon_chain/req/blobs_sidecars_by_range/1/
and /eth2/beacon_chain/req/blobs_sidecars_by_root/1/
,Blocks are created through the engine API, passing payload-attributes to start building an execution payload
after a forkchoice update.
Note: The final EIP design may return the blobs along with the execution payload, creating engine_getPayloadV2
.
For encapsulation in the EIP draft phase, the blobs are retrieved separately, through engine_getBlobV1
.
engine_getBlobV1
:
payload ID (bytes8, DATA, hex string)
versioned hash (bytes32, DATA, hex string)
Note that the execution-layer may lose and/or prune the Blob, resulting in an error upon calling engine_getBlobV1
.
The consensus-layer should then build a new execution payload.
We add an opcode DATAHASH
(with byte value HASH_OPCODE_BYTE
) which takes as input one stack argument index
,
and returns tx.message.blob_versioned_hashes[index]
if index < len(tx.message.blob_versioned_hashes)
,
and otherwise zero.
The opcode has a gas cost of HASH_OPCODE_GAS
.
Add a precompile at BLOB_VERIFICATION_PRECOMPILE_ADDRESS
that checks a blob against a versioned hash.
The precompile costs BLOB_VERIFICATION_PRECOMPILE_GAS
and executes the following logic:
Its logic is designed so that it is future-proof, and new versions of commitments can be added if needed.
Add a precompile at POINT_EVALUATION_PRECOMPILE_ADDRESS
that evaluates a proof that a particular blob resolves
to a particular value at a point.
The precompile costs POINT_EVALUATION_PRECOMPILE_GAS
and executes the following logic:
For early draft implementations, we simply charge GAS_PER_BLOB
per blob.
We propose a simple independent EIP 1559-style targeting rule to compute the gas cost of the transaction.
We use the TOTAL_BLOB_TXS_STORAGE_SLOT
storage slot of the SYSTEM_STATE_ADDRESS
address
to store persistent data needed to compute the cost.
Note that unlike existing transaction types, the gas cost is dependent on the pre-state of the block.
We update at the end of a block, as follows:
Transactions are presented as TransactionType || TransactionNetworkPayload
on the execution layer network,
the payload is a SSZ encoded container:
We do network-level validation of BlobTransactionNetworkWrapper
objects as follows:
This EIP introduces blob transactions in the same format in which they are expected to exist in the final sharding specification.
This provides a temporary but significant scaling relief for rollups by allowing them to scale to 2 MB per slot,
with a separate fee market allowing fees to be very low while usage of this system is limited.
The core goal of rollup scaling stopgaps is to provide temporary scaling relief,
without imposing extra development burdens on rollups to take advantage of this relief.
Today, rollups use calldata. In the future, rollups will have no choice but to use sharded data (also called "blobs")
because sharded data will be much cheaper.
Hence, rollups cannot avoid making a large upgrade to how they process data at least once along the way.
But what we can do is ensure that rollups need to only upgrade once.
This immediately implies that there are exactly two possibilities for a stopgap: (i) reducing the gas costs of existing calldata,
and (ii) bringing forward the format that will be used for sharded data, but not yet actually sharding it.
EIP-4488 is a solution of category (i); this is a solution of category (ii).
The main tradeoff in designing this EIP is that of implementing more now versus having to implement more later:
do we implement 25% of the work on the way to full sharding, or 50%, or 75%?
The work that is already done in this EIP includes:
BeaconBlock
verification and data availability sampling blobsBeaconBlock
logic required for full shardingThe work that remains to be done to get to full sharding includes:
blob_kzgs
in the consensus layer to allow 2D samplingThis EIP also sets the stage for longer-term protocol cleanups:
TransactionNetworkPayload
to separate network and block encodings of a transaction typeInstead of putting rollup block data in transaction calldata, rollups would expect rollup block submitters
to put the data into blobs. This guarantees availability (which is what rollups need) but would be much cheaper than calldata.
Rollups need data to be available once, long enough to ensure honest actors can construct the rollup state, but not forever.
Optimistic rollups only need to actually proof the underlying data when fraud proofs are being submitted.
The fraud proof submission function would require the full contents of the fraudulent blob to be submitted as part of calldata.
It would use the blob verification function to verify the data against the versioned hash that was submitted before,
and then perform the fraud proof verification on that data as is done today.
ZK rollups would provide two commitments to their transaction or state delta data:
the kzg in the blob and some commitment using whatever proof system the ZK rollup uses internally.
They would use the commitment proof of equivalence protoco,
using the point evaluation precompile, to prove that the kzg (which the protocol ensures points to available data) and the ZK rollup's own commitment refer to the same data.
We use versioned hashes (rather than kzgs) as references to blobs in the execution layer to ensure forward compatibility with future changes.
For example, if we need to switch to Merkle trees + STARKs for quantum-safety reasons, then we would add a new version,
allowing the precompiles to work with the new format.
Rollups would not have to make any EVM-level changes to how they work;
sequencers would simply have to switch over to using a new transaction type at the appropriate time.
The blob gasprice update rule is intended to approximate the formula blob_gas = 2**(excess_blobs / GASPRICE_UPDATE_FRACTION_PER_BLOB)
,
where excess_blobs
is the total "extra" number of blobs that the chain has accepted relative to the "targeted" number (TARGET_BLOB_TXS_PER_BLOCK
per block).
Like EIP 1559, it's a self-correcting formula: as the excess goes higher, the blob_gas
increases exponentially, reducing usage and eventually forcing the excess back down.
The block-by-block behavior is roughly as follows.
If in block N
, blob_gas = G1
, and block N
has X
blobs, then in block N+1
, excess_blobs
increases by X - TARGET_BLOB_TXS_PER_BLOCK
,
and so the blob_gas
of block N+1
increases by a factor of 2**((X - TARGET_BLOB_TXS_PER_BLOCK) / GASPRICE_UPDATE_FRACTION_PER_BLOB)
.
Hence, it has a similar effect to the existing EIP 1559, but is more "stable" in the sense that it responds in the same way to the same total usage regardless of how it's distributed.
This EIP introduces a transaction type that has a distinct mempool version (BlobTransactionNetworkWrapper
) and execution-payload version (SignedBlobTransaction
),
with only one-way convertibility between the two. The blobs are in the BlobTransactionNetworkWrapper
and not in the SignedBlobTransaction
;
instead, they go into the BeaconBlockBody
. This means that there is now a part of a transaction that will not be accessible from the web3 API.
Blob transactions are unique in that they have a variable intrinsic gas cost. Hence, a transaction that could be included in one block may be invalid for the next.
To prevent mempool attacks, we recommend a simple technique: only propagate transactions whose gas
is at least twice the current minimum.
Additionally, blob transactions have a large data size at the mempool layer, which poses a mempool DoS risk,
though not an unprecedented one as this also applies to transactions with large amounts of calldata.
The risk is that an attacker makes and publishes a series of large blob transactions with fees f9 > f8 > ... > f1
,
where each fee is the 10% minimum increment higher than the previous, and finishes it off with a 21000-gas basic transaction with fee f10
.
Hence, an attacker could impose millions of gas worth of load on the network and only pay 21000 gas worth of fees.
We recommend a simple solution: both for blob transactions and for transactions carrying a large amount of calldata,
increase the minimum increment for mempool replacement from 1.1x to 2x, decreasing the number of resubmissions an attacker can do at any given fee level by ~7x.
Work in progress. Prototype implementations are actively being tested and used to understand the testing surface in more depth.
The reference implementation is a work in progress while the specification is being reviewed and improved.
Execution layer implementation: https://github.com/protolambda/go-ethereum/pull/1
Consensus layer implementation: https://github.com/prysmaticlabs/prysm/compare/kiln…blobs
This EIP increases the storage requirements per Beacon block by a maximum of ~2 MB.
This is equal to the theoretical maximum size of a block today (30M gas / 16 gas per calldata byte = 1.875M bytes), and so it will not greatly increase worst-case bandwidth.
Post-merge, block times are expected to be static rather than an unpredictable Poisson distribution, giving a guaranteed period of time for large blocks to propagate.
The sustained load of this EIP is much lower than alternatives (eg. EIP-4488 and EIP-4490),
because there is no existing software that stores the blobs and there is no expectation that they need to be stored for as long as execution payload.
This makes it easier to implement a policy that these blobs should be deleted after eg. 30-60 days
(a much shorter delay than the delay after which EIP-4444 intends to delete ExecutionPayload
history).
Copyright and related rights waived via CC0.