changed 2 months ago  
Published Linked with GitHub

On Hardware Requirements For Validators

Summary

  • We recommend a total spend of approximately $1000
  • We recommend the NUC 14 Pro and the Minisforum UM790 Pro, however it can be any prebuilt setup that has:
    • A modern CPU with:
      • At least 8 cores and 16 threads
      • Single thread rating of 3500 or more
      • Multithreaded rating of 25000 or more
    • 4TB NVMe M.2 storage
    • 32GB RAM (Minimum) 64GB RAM (Recommended)
  • We expect to use these specifications as a reference point when discussing changes to the protocol. A less powerful setup may be sufficient, however it is the user's responsibility to check this at every soft/hard fork. We have plans to create a benchmarking suite that will make this an easy process.

Introduction

This document aims to establish a standardized definition of "commodity hardware", or at the least, recommend hardware specifications that we believe to be suitable for validating on Ethereum mainnet.

A clear hardware specification is crucial for:

  • Ensuring meaningful benchmark comparisons across different implementations
  • Enabling informed decision-making about protocol upgrades and their hardware implications
  • Providing clear guidance for node operators with respects to the future

Without a shared understanding of target hardware specifications:

  • Benchmark results lose significance due to inconsistent testing environments
  • Decision-making becomes challenging for implementation choices, as performance characteristics are heavily hardware dependent
  • Network participants lack clear guidance for hardware investments

Organization

  • Commodity hardware implies hardware that is readily available to buy as a consumer. We first look at what popular CPUs have been bought in the past.
  • The second section contains our recommended NUC build and a rough approximation of the cost, if you were to build it yourself
  • The third section outlines the different roles and whether we think the recommended hardware is suitable for that role.
  • The fourth section outlines miscellaneous changes that may or may not affect the recommended hardware, such as raising the gas limit.

Market Analysis

In order to estimate what popular CPUs people are buying, we did a brief search for popular CPU choices and listed their characteristics. These are not necessarily the most updated models.

  1. High performance Consumer Market

    • AMD Ryzen 7 7800X3D (8 cores/16 threads)
    • CPU benchmarks
    • Target market: Gaming and high-performance desktop computing
    • Represents upper tier of commodity hardware
    • CPU Rating - Single Thread: 3752, Multithread: 34275

  2. Mid-Range Consumer Market

    • Amazon best sellers UK shows the AMD Ryzen 5 5600X (6 cores/12 threads)
    • Popular choice for general computing needs
    • CPU Rating - Single Thread: 3361, Multithread: 21888

  3. Existing Staking Community

    • Intel NUC 11 (4 cores/8 threads)
    • Recommended by EthStaker community
    • Also matches the recommendations from PandaOps
    • Represents minimum viable hardware specification (what is currently being ran in production)
    • CPU Rating - Single Thread: 2800, Multithread: 10164

Overall, we recommend a setup with:

  • 8 cores, 16 threads
  • A single threaded CPU rating of 3500 or more
  • A multi threaded CPU rating of 25000 or more
  • 4TB of NVMe M.2 storage
  • 64GB of memory

CPU rationale

We substantiate the choice of 8 cores, with the steam hardware survey reference. The majority of CPUs here have either six or eight cores. One can view the steam dataset as being biased towards the low/median end of the gaming market.

The CPU rating scores were decided by looking at the high end CPUs on CPU benchmarks that had sixteen or less cores alongside the current consumer trends analysis done above and finding a rough average that we thought were reasonable.

We will not consider AVX512 when conducting benchmarks, however we will consider AVX2(Intel/AMD) and NEON(ARM) since these are widespread.

Storage rationale

The 4TB of storage is due to current history and state growth.

Memory rationale

The 64GB of memory was chosen for two reasons:

  • We set a budget of $1000 and customized a barebones NUC to fit within that budget.
  • Due to benchmarks that we have taken of possible proving systems where they have shown significant memory usage. These are still being prototyped, so there is a future where memory usage does not need to be that high. However, given that extra RAM is cheap comparatively, we recommend 64GB, but 32GB should be sufficient for the foreseeable future.

Prebuilt

NUC

We recommend the ASUS NUC 14 Pro from the NUC series:

  • 16 cores (6 performance + 8 efficiency cores + 2 Low power efficiency cores)*
  • 22 threads
  • Note: The price difference between the Intel NUC 11 (currently recommended by eth stakers) and the ASUS NUC 14 Pro is less than $100
  • When fully customized with 64GB of memory and 4TB of storage, the total cost is $967.
  • The power usage for the NUC 14 pro under load has been reported to be 88W
  • CPU Rating - Single Thread: 3520, Multithread: 25158

*This seems to be the closest NUC model to match the desire to have 8 cores.

Minisforum UM790 Pro

We recommend the Minisforum UM790 Pro with modifications:

  • Minisforum UM790 Pro
  • 8 cores, 16 threads
  • The baseline model costs $400
  • This is the barebones model. We recommend adding 64GB RAM and 4TB storage.
  • The total cost with the RAM and storage is approximately $920.
  • CPU Rating - Single Thread: 3903, Multithread: 30367

Build Your Own

Below, we list out average prices for the main components needed, if you were to build your own server. The total cost is approximately $1000.

CPU

For 8 cores and 16 threads, the average price for a CPU will be $300-400

Storage

  • 2TB SSD, average price tends towards $200
  • 4TB SSD, average price is roughly $300 (recommended)
  • We recommend purchasing NVMe M.2 instead of SATA as NVMe has high throughput. Suggestions: Kingston KC3000, Kingston Fury Renegade, Samsung 990 Pro, Seagate Firecuda 530, Teamgroup MP44, WD Black SN850X.
  • See trends: PCPartPicker storage trends
  • Check out Yorick's guide on buying SSDs
  • Note: Storage requirements have stronger path dependency with items on the roadmap such as history expiry.

Memory

Motherboard

For more resources on building your own setup, see eth-docker's hardware documentation.

Role-Specific Requirements

Attesters

Currently there is no meaningful role separation between an attester and a proposer. Hence the hardware requirements for an attester would follow the same hardware requirements as a proposer.

If there was a meaningful separation, then an attester would be run with weaker hardware since they would no longer need to propose.

Aggregators

An aggregator aggregates BLS signatures. With the introduction of post quantum signatures, the job of the aggregator may become more computationally intensive.

There is also currently no meaningful separation between an aggregator and a proposer.
We believe that our recommended hardware requirements satisfies the desired hardware requirements for an aggregator if they were to be separated, so no meaningful changes would need to be done either way.

Proposers

Proposers are assumed to not be powerful enough to compete with centralized block builders, neither is this a goal.

Key points about proposer requirements:

  • One data point from EthPandaOps(Pari) shows that a 6 core, 12 threads node is using on average 20% of its CPU
  • This average usage doesn't account for peak latency in cryptographic operations
  • More cores would increase peak latency performance and reduce the average
  • With more advanced cryptography being added to the protocol and with the desire to need stakers to update their hardware as least as possible, it makes sense to double the number of cores required.
  • Current hardware provides headroom for anticipated changes (like peerDAS)

Centralized Block Builder

While out of scope for this document, we note some responsibilities:

  • For stateless, these are the entities that would build the stateless witness for the block
  • If a proposer decides to build the stateless witness themselves, they can do it using the recommended hardware requirements according to our benchmarks.
  • For SNARKifying L1, entities with similar power to a block builder will be creating these zk proofs

Stateless Client

Once we have full statelessness, we envision that validators themselves can be stateless.

This:

  • Reduces storage requirements for nodes
  • Increases CPU requirements for witness verification

The stateless verification procedure fits within our recommended hardware requirements, verification is cheap. We also note that our recommended hardware requirements work both for verkle tries and binary trees with stark proofs. The latter requires more benchmarks using traditional hashes.

Miscellaneous changes

Raising the gas limit

Raising the gas limit increases the rate of history growth, which effects the storage requirements. The analysis from paradigm suggests that without any changes, we have 2 to 3 years before we exceed 2TB. This does not include the storage requirements from the consensus layer(CL) however, Pari from DevOps notes that with the CL we have less than six months before we reach the 2TB limit.

Given that the recommended storage space is 4TB, and we plan to implement EIP-4444 in at most two years. This storage requirement should not pose any issues, even in the case that we double the gas limit.

If a user plans to stay at 2TB, then this may be sufficient, given pre-merge files are pruned in less than six months which frees up ~500GB and 4444 is implemented within a year.

Raising the blob limit

As noted in the paradigm post, the additions of blobs, have reduced the history growth caused by the rollup users, since they have switched from calldata to blob data.

It's unclear whether rollups are switching between calldata and blob data currently, which means that it is unclear as to whether raising the blob limit will affect history growth any further.

Increasing validator count

This section will be handwaved the most since it has a lot of path dependencies with Orbit, 3SF and MaxEB.

We know that for SSF, the validator set needs to be reduced, so at the very least, the bump in hardware specifications should be sufficient enough for any aggregation that happens in SSF.

Acknowledgements

With contributions from Parithosh Jayanthi, Kevaundray Wedderburn, Josh Rudolf, Dankrad Feist, Justin Traglia, Ignacio Hagopian and George Kadianakis. We would also like to thank the external reviewers for their feedback: Nixorokish, Yorick Downe, Rémy Roy, Ben Adams, Vitalik Buterin, Lightclient, Andrew Ashikhmin, Marek Moraczyński, Potuz, Joe Clapis, Haurog, Francis(Base), Jimmy(Lighthouse) and Nico Flaig. Feedback does not imply endorsement of this document.

Select a repo