Running Tezos node with small memory (i.e. 8GB)

Note: this document is not to claim that 8GB is the minimum requirement to run current Tezos node.

This is a small memo to explain how to reduce the memory usage of Tezos node, without changing the code at all.

Use --singleprocess option

By default, Tezos node runs 2 processes, the main process and the validator process, to execute the block validation and the other jobs in parallel. Unfortunately this increases the memory usage.

You can run a node with --singleprocess option. This sacrifices the speed but keeps the memory usage lower. This slows down the bootstrapping process. Once bootstrapped, you need not worry about the performance: most of the blocks can be validated in 30 seconds, which is the current block time of Tezos.

Tweak OCaml GC parameter

More compactions

Tezos node is written in OCaml programming language with GC (garbage collection). GC is a mechanism to reclaim "garbage" memory which is no longer used by the program.

If GC is performed frequently, then garbage memory is reclaimed earlier, resulting the lower memory usage. However, GC has a high run time cost. Therefore OCaml programs always runs with some amount of memory garbages.

OCaml GC sometimes performs compactions which clean all the memory garbages and reduce the entire memory usage of the program. Since this is the most expensive GC operation, it is triggered the most rarely. By default, the compaction is executed only if the amount of the garbage memory is more than 500% of the actual (non garbage) data of the program. This means that if an OCaml program is using 6GB of memory, 5GB of it can be garbage.

We can tweak this ratio by setting O=n parameter in OCAMLRUNPARAM environment variable. If we want to set the ratio to 300%, then:

OCAMLRUNPARAM=O=300 ./tezos-node run ...

Note that frequent compaction has performance penalty. Your node validates more blocks with longer time.

Changing allocation policy

You can also change the behavior of OCaml memory allocation policy from the default to best-fit policy to lower the memory usage of Tezos node.

To do this, set a=2 in OCAMLRUNPARAM environment variable:

OCAMLRUNPARAM=a=2 ./tezos-node run ...

Note that this best-fit policy is still experimental in OCaml 4.12.0, which is used to compile the latest Tezos node. (It will be the default in OCaml 4.13.0 or later.)

Setting both

You can specify the both parameters in OCAMLRUNPARAM by separating with ,:

OCAMLRUNPARAM=O=300,a=2 ./tezos-node run ...