# WALRUS Storage Protocol — Research Overview

## Protocol Overview

### Epochs and Committees

Walrus operates in a quasi-permissioned environment, using a delegated proof-of-stake (DPoS) mechanism to determine which storage providers participate in core protocol operations. The amount of stake delegated to a provider directly influences its eligibility and responsibilities within the system.

The protocol advances in multi-stage epochs, with each epoch governed by an active storage committee. This committee consists of a subset of providers selected to hold and serve data during that epoch. Membership rotates at each epoch boundary to reflect stake changes and to tolerate churn in the provider set.

Each epoch is divided into two distinct phases:

**Storage phase**: The active committee handles all read and write operations.

**Transition phase**: A new committee for epoch $e+1$ is introduced. Members of the incoming committee are expected to recover their shard assignments by downloading the necessary data from the outgoing epoch-$e$ committee.

The Walrus paper notes that committee selection aims to minimize the total data transferred during transitions, though the exact algorithm is not specified.

To reduce bottlenecks during transitions, the protocol also proposes an optimization: splitting responsibilities between epochs. Writes are directed to the new committee ($e+1$), while reads continue to be served by the outgoing committee ($e$). This prevents race conditions between read and write operations and ensures system availability even as membership changes.

---

### Storage Model

Walrus’ storage architecture is based on a fixed set of logical **shards** (currently 1000), each uniquely identified by a shard ID and assigned to a node in the active storage committee. Shard assignment is proportional to stake and does not account for physical capacity, which may obligate some nodes to store data beyond their actual limits.

The central innovation of Walrus is **RedStuff**, a two-dimensional erasure coding scheme that achieves durability with only a 4.5× replication factor. Unlike traditional replication systems, RedStuff enables **self-healing recovery** with bandwidth proportional to the fraction of lost data:

$$
O\!\left(\frac{|blob|}{n}\right) \quad \text{vs.} \quad O(|blob|)
$$
for conventional Reed–Solomon recovery.
**Encoding process**:
- The blob is split into an $(f+1) \times (2f+1)$ matrix of symbols.
- **Primary encoding**: Each column of length $f+1$ is extended to $n$ symbols using RaptorQ, forming *primary slivers*.
- **Secondary encoding**: Each row of length $2f+1$ is extended to $n$ symbols, forming *secondary slivers*.
- Each shard stores one pair $(S^{p,i}, S^{s,i})$, ensuring redundancy across both dimensions.

Each sliver is associated with a vector commitment to its symbols. Rather than replicating all commitments to every node (which would incur quadratic overhead), Walrus applies a one-dimensional $(f+1)$-out-of-$n$ erasure code to the commitments themselves. Each node stores a single encoded share, allowing the full metadata set to be reconstructed from any $f+1$ nodes. This reduces overhead to $O(n)$ while maintaining verifiability.

This dual encoding underpins the design of the Walrus read, write, and recovery workflows, which integrate erasure coding with vector commitments, blockchain registration, and distributed storage protocols.

---

## Write

**Preprocessing**: The client encodes the blob with RedStuff, generating $n$ primary and $n$ secondary slivers. For each of the $2n$ slivers, the client computes vector commitments, enabling symbol-level verification without revealing the entire sliver.


A blob commitment is then derived by concatenating the commitments of each primary–secondary sliver pair and constructing a Merkle tree over these values. This structure was chosen to reduce hashing overhead compared to treating each sliver independently. The final blob ID is obtained by hashing the blob commitment together with metadata, consisting of:

- 32B Merkle root hash for each primary and secondary sliver
- 32B blob ID
- 1B erasure code type identifier
- 8B unencoded blob size


**Blockchain Registration**: The client registers the blob ID, size, expiry, and payment on the Sui blockchain. Smart contracts validate available capacity and atomically reserve space. Storage fees are paid in WAL tokens and pooled to compensate storage nodes over the contract duration.

**Data Dissemination**: Once registration is confirmed, the client initiates parallel communication with all $n$ storage nodes. For each node $i$, the client sends a storage request containing:

- Blob ID
- Full sliver commitment metadata ($2n$ commitments)
- Node $i$’s primary sliver $S^{p,i}$
- Node $i$’s secondary sliver $S^{s,i}$
- Inclusion proofs showing consistency with the blob commitment

Nodes verify the slivers and return signed acknowledgments.

**Availability Certification**: When the client collects $2f+1$ valid acknowledgments, it submits them on-chain as a **Point of Availability (PoA)**. From this point forward, the blob is considered publicly available, and providers are obligated to retain their assigned slivers. Nodes receiving the PoA event also verify completeness and, if missing slivers, trigger the recovery process.

---

## Read

**Committee and Metadata Discovery**
The client queries the blockchain to determine the active storage committee responsible for the blob ID. The blob’s metadata is then reconstructed from encoded fragments held by the committee and validated.

> **Note:** Walrus encodes and distributes metadata across all nodes rather than storing full copies at each provider. As a result, metadata recovery is a necessary step before blob reconstruction.

**Sliver Retrieval and Verification**: The client requests secondary slivers in parallel from committee members. Each returned sliver is validated against its vector commitment. Once $2f+1$ valid responses are obtained, the blob is reconstructed and re-encoded. The recomputed blob commitment and metadata are then used to regenerate the blob ID, which must match the on-chain record for the data to be considered valid.

---

## Recovery

Each shard stores a pair consisting of one primary and one secondary sliver. When a node loses its assigned slivers, recovery is performed in two stages:

- **Secondary sliver recovery**: The node queries $f+1$ peers for overlapping elements from their primary slivers. These responses are sufficient to reconstruct the missing secondary sliver.
- **Primary sliver recovery**: After restoring the secondary sliver, the node queries $2f+1$ peers for the required primary elements. This enables the reconstruction of the missing primary sliver.

This recovery process is bandwidth-efficient, requiring only:

$$
O\!\left(\frac{|Blob|}{n}\right)
$$

data transfer, proportional to the number of lost elements rather than the entire blob size.

---

## Storage Challenges

To ensure that storage providers continue to retain their assigned data, Walrus employs a **fully asynchronous challenge mechanism**. This process is triggered near the end of each epoch when the blockchain emits a challenge event at a specific block height. Upon detecting this event, all nodes acknowledge participation. Once $2f+1$ acknowledgments are observed, the challenge phase begins.

### Challenge Phase

During the challenge phase, nodes suspend responses to read and recovery requests. This restriction prevents adversarial providers from opportunistically reconstructing missing slivers at the last moment. Each challenged node must produce a set of *common symbols* for every blob it stores, accompanied by cryptographic proofs. These proofs are transmitted to peer nodes, which verify correctness and return signed confirmations. Once a node collects $2f+1$ valid confirmations, it aggregates them into a certificate and submits the result on-chain as evidence of availability.

While this protocol enforces strong guarantees, it is **bandwidth-intensive** and imposes significant operational cost. To reduce overhead, the paper also proposes a **lightweight sampling variant**, in which only a randomly chosen subset of blobs are verified. This approach improves efficiency and availability during challenges but trades off durability and the strength of security guarantees.

> **Note:** The effectiveness of this mechanism is constrained by epoch length. With epochs averaging ~2 weeks, running challenges only once per epoch introduces potential durability risks in practice.

---

## Storage Costs

Storage is bought and sold on Walrus as storage resources, represented on the Sui blockchain. These resources serve as reservations for storage space with a defined starting epoch, ending epoch, and size. Users register resources to hold specific blobs when ready to store data, then write the data and establish the Point of Availability. Before being attached to a blob, storage resources can be split across time or space, and can be traded. Storage resources can also be disassociated from deleted blobs and reallocated to new ones.

**Storage Quantity**: The total storage available for sale is determined by voting on the shard size. Since both shard size and replication factor are fixed, the network can calculate its total capacity and unused storage available for sale.

Active storage providers vote for their storage capacity and prices one epoch in advance. At the end of epoch e, all shard size submissions are ordered in decreasing order. The system selects the 66.67th percentile (by stake weight) as the shard size, ensuring that 2/3 of submissions are for larger sizes and 1/3 are for smaller sizes.

**Storage Price**: Nodes set not only shard size but also prices for storage and writes. One full epoch in advance, nodes submit prices for storage resources (per unit of storage, per epoch) and for writes (per unit of storage).

These prices are independently ordered in ascending order, with the 66.67th percentile (by stake weight) selected. This ensures 2/3 of submissions are for lower prices and 1/3 are for higher prices. The storage price is applied as is, while the write price is multiplied by a hardcoded factor greater than one, reflecting an additional refundable deposit.

**Payments**: Users pay storage fees in WAL upfront for the entire contract period when registering the blob. These fees are collected in a storage pool and distributed to storage providers at the end of each epoch.

> **Note:** The core design does not specify any mechanism for **read costs**. The paper proposes external solutions for pricing reads.

---

## Economic Security

Walrus uses a delegated proof of stake model with fixed token supply. This token drives the security of the protocol. There are 3 primary utility drivers for this token:

1. **Staking**: All storage providers or delegators must stake WAL tokens. The total staked value determines shard allocations and membership in the active storage committee.
2. **Storage Fees**: As mentioned above, users pay storage fees in WAL tokens.
3. **Governance**: Token holders receive proportionate voting power in network governance proposals.

**Rewards & Penalties**: At the end of every epoch, rewards for each storage provider are calculated based on their relative stake and behavior. Behavior includes successfully submitting challenge proofs and participating in hard recovery processes during epoch transitions. Note that rewards are calculated based on overall network revenue, not just individual contributions.

If a storage provider fails to submit proofs during the challenge phase, their stake is slashed and rewards are dismissed.

---

## Shortcomings

1. The protocol lacks privacy measures, as all information about storage providers and contracts is publicly available on the blockchain.
2. The core design's fixed number of shards and rigid association of storage providers with specific shards significantly limits protocol scalability.
3. The delegated proof-of-stake model creates substantial entry barriers for newcomers, potentially leading to network centralization over time.
4. Epoch-end shard migration represents considerable overhead and resource waste, despite the paper's proposed migration strategies, which seems counterproductive.
5. The challenge mechanism design fails to provide strong durability guarantees by sacrificing security for availability and network efficiency.