Lodestar Setup Guide v2 - Part 3a: Setup Lodestar with Docker Compose

Last Updated: July 3, 2022

Table of Contents

• Lodestar Setup Guide v2 - Part 3a: Setup Lodestar with Docker Compose
• Overview
• Install Docker Engine & Docker Compose
• Download and Configure Lodestar Environment
  • Setup the Lodestar environment file
• Create a new Docker Compose Environment
  • Add an Execution Node, a Lodestar Beacon Node, a Prometheus and a Grafana Service
    • Option 1/3: Nethermind Execution Node
    • Option 2/3: Hyperledger Besu Execution Node
    • Option 3/3: Go-Ethereum (Geth) Execution Node
    • (Optional) Quick Sync your Beacon Node
  • Start the Execution Node Service
  • Start Lodestar Beacon Node Service
    • Remove Quick Sync CLI flags (if required)
  • Start Prometheus and Grafana Services for Lodestar
• Setup Validators for Lodestar
  • Copy the Validator Keys from your USB to your Lodestar keystores directory
  • Copy the Validator Keys from your local machine to your Lodestar keystores directory
  • Configure Lodestar Validator Client
    • Create password file
    • Modify Validator Client Docker Compose file
  • Start the Lodestar Validator Client
• Final Notes

---

This is Part 3a of a step-by-step guide to setup the ChainSafe Lodestar consensus client on the Ethereum Beacon Chain.

This guide is separated into a series of parts, grouped by related steps to setup a full Lodestar consensus beacon node and validator client from start to finish. The topics are separated as follows:

• Lodestar Setup Guide Overview
• Part 1: Generating Staking Data
• Part 2: Set up and Secure your Server
• Part 3a: Setup Lodestar with Docker Compose
• Part 3b: Setup Lodestar with Systemd and NPM Coming Soon!
• Part 4: Funding your Validator Keys
• Final Remarks, Next Steps & Appendix

---

Overview

An Ethereum execution node is required for staking. You can either run a local Ethereum execution (Eth1) node or use a third party node. This guide will provide instructions for running a local Go Ethereum (Geth) node.

OPTIONAL: If you would rather use a third party option then skip this step.

NOTE: Check your available disk space. An Ethereum execution (Eth1) node requires roughly 400GB of space. Even if you have a large SSD there are cases where Ubuntu is reporting only 200GB free. If this applies to you then take a look at Appendix A — Expanding the Logical Volume.

---

Install Docker Engine & Docker Compose

We must install Docker Engine & Docker Compose to run the images on your local machine.

sudo apt install -y docker-compose

sudo systemctl enable --now docker

Verify the installation was successful.

docker-compose --version

Docker Compose should return a non-error stating the version and build.

---

Download and Configure Lodestar Environment

Clone the latest commit of the Lodestar monorepo from Github into your local.

cd ~ && git clone --depth 1 https://github.com/ChainSafe/lodestar.git

Setup the Lodestar environment file

The docker-compose file requires that a .env file be present in this directory. The default.env file provides a template and can be copied. Navigate to the extracted directory and copy the template:

cd ~/lodestar && cp default.env .env

Modify the parameters inside the .env file. Use the nano text editor.

nano .env

TESTNET USERS: If you want to use Lodestar in a testnet, change LODESTAR_NETWORK to the desired network such as LODESTAR_NETWORK=prater.

Then, press CTRL + x then y then Enter to save and exit.

Create a new Docker Compose Environment

We will now generate a new docker-compose.yml which defines 4 main services to run a local Lodestar node:

• Execution node (nethermind_docker | besu_docker | geth_docker),
• beacon_node
• prometheus
• grafana

using the text editor nano. Open the editor:

nano

Add an Execution Node, a Lodestar Beacon Node, a Prometheus and a Grafana Service

An Ethereum execution node is required for staking. You can either run a local Ethereum execution (Eth1) node or use a third party node. This guide will provide instructions for running a local execution node using either Nethermind, Go-Ethereum (Geth), or Besu within Docker containers using Docker-Compose.

NOTE: Check your available disk space. An Ethereum execution (Eth1) node requires roughly 500GB of space. Even if you have a large SSD there are cases where Ubuntu is reporting only 200GB free. If this applies to you then take a look at Appendix A — Expanding the Logical Volume.

---

For the next step, please choose only 1 of the following 3 options. Doing more than 1 will create port conflicts:

Option 1/3: Nethermind Execution Node

If you would like to use Nethermind as your Ethereum Execution Client (eth1), copy and paste the following into the file:

NOTE: YAML files are sensitive to indentations. Copy the lines exactly as seen below.

version: "3.4"
services:
  nethermind_docker:
    image: nethermind/nethermind:latest
    restart: always
    volumes:
      - nethermind_docker:/data
    command: --config mainnet --datadir /data --Network.DiscoveryPort=30303 --Network.P2PPort=30303 --Init.DiagnosticMode=None --JsonRpc.Enabled=true --JsonRpc.Host=0.0.0.0 --JsonRpc.AdditionalRpcUrls \"http://localhost:8545|http;ws|net;eth;subscribe;engine;web3;client|no-auth,http://localhost:8551|http;ws|net;eth;subscribe;engine;web3;client\"
    network_mode: host
    container_name: nethermind_docker

  beacon_node:
    image: chainsafe/lodestar:latest
    restart: always
    volumes:
      - beacon_node:/data
      - logs:/logs
    env_file: .env
    command: beacon --rootDir /data --api.rest.enabled --api.rest.host 0.0.0.0 --metrics.enabled --logFile /logs/beacon.log --logLevelFile debug --logRotate --logMaxFiles 5
    environment:
      NODE_OPTIONS: --max-old-space-size=4096
    network_mode: host

  prometheus:
    build:
      context: docker/prometheus
      args:
        config_file: prometheus.local.yml
    restart: always
    volumes:
      - "prometheus:/prometheus"
    network_mode: host

  grafana:
    build:
      context: docker/grafana
      args:
        DATASOURCE_FILE: datasource.local.yml
    restart: always
    volumes:
      - "grafana:/var/lib/grafana"
      - "./dashboards:/dashboards"
    depends_on: [prometheus]
    network_mode: host

volumes:
  nethermind_docker:
  beacon_node:
  logs:
  prometheus:
  grafana:

TESTNET USERS: If you are running on the Goerli testnet, make sure to modify the command configuration of the execution client you plan to use.

**Example of Nethermind on the Goerli Testnet (Line 8):

    command: --config goerli --Network.DiscoveryPort=30303 --Network.P2PPort=30303 --Init.DiagnosticMode=None --JsonRpc.Enabled=true --JsonRpc.Host=0.0.0.0 --JsonRpc.AdditionalRpcUrls \"http://localhost:8545|http;ws|net;eth;subscribe;engine;web3;client|no-auth,http://localhost:8551|http;ws|net;eth;subscribe;engine;web3;client\"

TESTNET USERS: If you would like to pull the unstable nightly versions of Lodestar, you can change the image: parameter on Line 13 from chainsafe/lodestar:latest to chainsafe/lodestar:next

NOTE: You can also add/change Nethermind CLI Commands to run such as --Network.DiscoveryPort=30303 to change the listening port on command: line 8.

NOTE: You can also add Lodestar CLI commands to run such as --weakSubjectivitySyncLatest to fetch a weak subjectivity state (see Quick sync your beacon node) with a --weakSubjectivityCheckpoint on command: line 19.

When you're done modifying this file, press CTRL+ x to exit and press y, then name the file docker-compose.yml and press y to overwrite the current version. Continue to the next step: (Optional) Quick Sync your Beacon Node.

Option 2/3: Hyperledger Besu Execution Node

If you would like to use Besu as your Ethereum Execution Client (eth1), copy and paste the following into the file:

NOTE: YAML files are sensitive to indentations. Copy the lines exactly as seen below.

version: "3.4"
services:
  besu_docker:
    image: hyperledger/besu:latest
    restart: always
    volumes:
      - besu_docker:/data
    command: --network=mainnet --data-path /data --rpc-http-enabled=true --rpc-http-api=ADMIN,CLIQUE,MINER,ETH,NET,DEBUG,TXPOOL,TRACE --rpc-http-host=0.0.0.0 --rpc-http-port=8545 --rpc-http-cors-origins=\"*\" --host-allowlist=\"*\" --p2p-enabled=true
    network_mode: host
    container_name: besu_docker

  beacon_node:
    image: chainsafe/lodestar:latest
    restart: always
    volumes:
      - beacon_node:/data
      - logs:/logs
    env_file: .env
    command: beacon --rootDir /data --api.rest.enabled --api.rest.host 0.0.0.0 --metrics.enabled --logFile /logs/beacon.log --logLevelFile debug --logRotate --logMaxFiles 5
    environment:
      NODE_OPTIONS: --max-old-space-size=4096
    network_mode: host

  prometheus:
    build:
      context: docker/prometheus
      args:
        config_file: prometheus.local.yml
    restart: always
    volumes:
      - "prometheus:/prometheus"
    network_mode: host

  grafana:
    build:
      context: docker/grafana
      args:
        DATASOURCE_FILE: datasource.local.yml
    restart: always
    volumes:
      - "grafana:/var/lib/grafana"
      - "./dashboards:/dashboards"
    depends_on: [prometheus]
    network_mode: host

volumes:
  besu_docker:
  beacon_node:
  logs:
  prometheus:
  grafana:

TESTNET USERS: If you are running on the Goerli testnet, make sure to modify the command configuration of the execution client you plan to use.

**Example of Besu on the Goerli Testnet (Line 8):

    command: --network=goerli --rpc-http-enabled=true --rpc-http-api=ADMIN,CLIQUE,MINER,ETH,NET,DEBUG,TXPOOL,TRACE --rpc-http-host=0.0.0.0 --rpc-http-port=8545 --rpc-http-cors-origins=\"*\" --host-allowlist=\"*\" --p2p-enabled=true 

TESTNET USERS: If you would like to pull the unstable nightly versions of Lodestar, you can change the image: parameter on Line 13 from chainsafe/lodestar:latest to chainsafe/lodestar:next

NOTE: You can also add/change Besu CLI Commands to run such as --p2p-port=123 to change the listening port on command: line 8.

NOTE: You can also add Lodestar CLI commands to run such as --weakSubjectivitySyncLatest to fetch a weak subjectivity state (see Quick sync your beacon node) with a --weakSubjectivityCheckpoint on command: line 19.

When you're done modifying this file, press CTRL+ x to exit and press y, then name the file docker-compose.yml and press y to overwrite the current version. Continue to the next step: (Optional) Quick Sync your Beacon Node.

Option 3/3: Go-Ethereum (Geth) Execution Node

If you would like to use Go-Ethereum (Geth) as your Ethereum Execution Client (eth1), copy and paste the following into the file:

NOTE: YAML files are sensitive to indentations. Copy the lines exactly as seen below.

version: "3.4"
services:
  geth_docker:
    image: ethereum/client-go:stable
    restart: always
    volumes:
      - geth_docker:/data
    command: --datadir /data --http --cache 2048 --maxpeers 30 --mainnet
    network_mode: host
    container_name: geth_docker

  beacon_node:
    image: chainsafe/lodestar:latest
    restart: always
    volumes:
      - beacon_node:/data
      - logs:/logs
    env_file: .env
    command: beacon --rootDir /data --api.rest.enabled --api.rest.host 0.0.0.0 --metrics.enabled --logFile /logs/beacon.log --logLevelFile debug --logRotate --logMaxFiles 5
    environment:
      NODE_OPTIONS: --max-old-space-size=4096
    network_mode: host

  prometheus:
    build:
      context: docker/prometheus
      args:
        config_file: prometheus.local.yml
    restart: always
    volumes:
      - "prometheus:/prometheus"
    network_mode: host

  grafana:
    build:
      context: docker/grafana
      args:
        DATASOURCE_FILE: datasource.local.yml
    restart: always
    volumes:
      - "grafana:/var/lib/grafana"
      - "./dashboards:/dashboards"
    depends_on: [prometheus]
    network_mode: host

volumes:
  geth_docker:
  beacon_node:
  logs:
  prometheus:
  grafana:

TESTNET USERS: If you are running on the Goerli testnet, make sure to modify the command configuration of the execution client you plan to use.

**Example of Geth on the Goerli Testnet (Line 8):

    command: --datadir /data --http --cache 2048 --maxpeers 30 --goerli

TESTNET USERS: If you would like to pull the unstable nightly versions of Lodestar, you can change the image: parameter on Line 13 from chainsafe/lodestar:latest to chainsafe/lodestar:next

NOTE: You can also add Go Ethereum CLI commands to run such as --port 123 to change the listening port on command: line 8.

NOTE: You can also add Lodestar CLI commands to run such as --weakSubjectivitySyncLatest to fetch a weak subjectivity state (see Quick sync your beacon node) with a --weakSubjectivityCheckpoint on command: line 19.

When you're done modifying this file, press CTRL+ x to exit and press y, then name the file docker-compose.yml and press y to overwrite the current version. Continue to the next step: (Optional) Quick Sync your Beacon Node.

---

(Optional) Quick Sync your Beacon Node

1. Create a free account on Infura.io
2. Create a new Eth 2 project
3. Take note of your Infura Eth2 endpoint for mainnet. It should look like this: https://<project-id>:<project-secret>@eth2-beacon-mainnet.infura.io where <project-id> and <project-secret> are replaced with a bunch of random letters and numbers.
4. Re-open the docker-compose.yml file by using the command:

nano docker-compose.yml

5. Add to the end of Line 19 command: in your docker-compose.yml file:

--weakSubjectivitySyncLatest --weakSubjectivityServerUrl https://<project-id>:<project-secret>@eth2-beacon-mainnet.infura.io

TESTNET USERS: The URL for a Prater endpoint --weakSubjectivityServerUrl should end in @eth2-beacon-prater.infura.io

Example of Lodestar Beacon with quick sync (Line 19):

    command: beacon --rootDir /data --api.rest.enabled --api.rest.host 0.0.0.0 --metrics.enabled --logFile /logs/beacon.log --logLevelFile debug --logRotate --logMaxFiles 5 --weakSubjectivitySyncLatest --weakSubjectivityServerUrl https://1sla4tyOFn0bB1ohyCKaH2sLmHu:[e-mail]

When you're done modifying this file, press CTRL+ x to exit and press y, then name the file docker-compose.yml and press y to overwrite the current version.

---

Start the Execution Node Service

Depending on which execution node you have chosen, start the geth_docker | nethermind_docker | besu_docker service:

NOTE: Replace the following geth_docker commands with the name of the Execution Node client you are running.

Nethermind Example: sudo docker-compose up -d nethermind_docker
Besu Example: sudo docker-compose up -d besu_docker

sudo docker-compose up -d geth_docker

If it has initiated properly, it will download the image and return: Creating geth_docker ... done

Check the logs and ensure it is syncing by using docker logs.

sudo docker logs geth_docker

NOTE: You can follow the output logs live by using the -f flag with the docker logs command.

Example:
sudo docker logs -f geth_docker

When verified that everything is working properly, exit the log by pressing CTRL + C.

Your execution (eth1) node is now running on a http server at http://127.0.0.1:8545. Wait until it is synced and proceed to the next step.

---

Start Lodestar Beacon Node Service

To startup the beacon node, use the following command:

sudo docker-compose up -d beacon_node

If it has initiated properly, it will download the image and return: Creating lodestar_beacon_node_1 ... done

We can confirm if it is running by using the docker ps command:

sudo docker ps

You will see a container running the lodestar_beacon_node_1.

To see the logs from the container, use the command:

sudo docker logs lodestar_beacon_node_1

QUICK SYNC USERS: If you used quick sync you may encounter error: Error onSyncAggregate message=finalityHeader not available when initially starting up. This is NORMAL as your light client server does not have the information it requires yet.

NOTE: You can follow the output logs live by using the -f flag with the docker logs command.

Example:
sudo docker logs -f lodestar_beacon_node_1

When verified that everything is working properly, exit the log by pressing CTRL + C.

When your beacon node is fully synced, the logs will display a line similar to the one below.

info: Synced - block: XXXXXXX

Remove Quick Sync CLI flags (if required)

NOTE: If you used Quick sync your beacon node, follow these instructions. Otherwise, skip to Start Prometheus and Grafana Services for Lodestar.

Once your beacon node has synced, you can remove the weakSubjectivity CLI commands from your docker-compose.yml file.

nano ~/lodestar/docker-compose.yml

Remove the --weakSubjectivitySyncLatest and the --weakSubjectivityServerUrl flags from Line 19. When you're done modifying this file, press CTRL+ x to exit and press y to overwrite and Enter to save the file with the same name.

---

Start Prometheus and Grafana Services for Lodestar

Start the local metrics docker container.

sudo docker-compose up -d prometheus grafana

If it has initiated properly, it will download the image and return: Creating lodestar_prometheus_1 ... done and Creating lodestar_grafana_1 ... done

We can confirm if it is running by using the Docker ps command:

sudo docker ps

As long as the STATUS is not constantly restarting, they are up and running properly.

To check the metrics, you will need to know the IP address of your node in your network. You can check by using:

hostname -I

Use your internet browser and access the Grafana metrics dashboard at port 3000.

Replace <IPaddress> with the address returned from the previous command.

http://<IPaddress>:3000

Log in to your Grafana dashboard and use the default credentials to change your password:

Username: admin
Password: admin

On your left-hand side menu bar, navigate to Dashboards > Browse.

Select Lodestar for our default Lodestar dashboard.

On the top-left corner of the dashboard, make sure you have selected Lodestar to pull information about your local Lodestar node.

When your beacon node is fully synced, the dashboard will display: Sync status: Synced.

---

Setup Validators for Lodestar

OPTIONAL: Skip these following steps and proceed to Final Remarks, Next Steps & Appendix if you are not running a validator.

We will create a keystores folder in our Lodestar directory.

sudo mkdir ~/lodestar/keystores

During Part 1: Generating Staking Data, you generated one or many keystore files that stores your validator signing keys. We will now import those keystores into the Lodestar validator client. If you generated your keystores on another device, make sure to copy them on your staking machine first. Keystore files normally starts with keystore-m and ends with .json.

Your keystore should be in a located in a known directory. Make sure to insert your own path for where your keystore files are located. There should not be any keystore-directory literal.

Copy the Validator Keys from your USB to your Lodestar keystores directory

Configure the Lodestar validator by importing the validator keys.

OPTIONAL: If you created your validator keystores on this local machine, skip this step.

If you generated the validator keystore-m.json file(s) on a machine other than your Ubuntu server you will need to copy the file(s) over to your Lodestar keystores directory. You can do this using a USB drive (if your server is local) or via secure FTP (SFTP) into your Docker container running the validator.

To mount a USB drive into your server, physically insert your USB drive, then find the path to it.

sudo fdisk -l

Look for your USB Drive and confirm the information before proceeding.

You will need the device path before continuing. In this example, the path is /dev/sda1.

Create a new folder under /media called usb-stick then mount your USB drive into it. Ensure you replace <USBDevicePath> with the specific device path to your USB.

sudo mkdir /media/usb-stick

sudo mount <USBDevicePath> /media/usb-stick

Enter your /media/usb-stick path and ensure you can see the contents within it.

cd /media/usb-stick

ls

Locate your validator_keys folder generated from Part 1 and copy it to your Lodestar keystores directory. Insert the path to your validator_keys folder into <ValidatorKeysFolderPath>

sudo cp -a <ValidatorKeysFolderPath>/. ~/lodestar/keystores

cd ~/lodestar/keystores

ls

NOTE: If you have your validator_keys folder on your USB stick, you can use the following commands instead of the ones above:

sudo cp -a /media/usb-stick/validator_keys/. ~/lodestar/keystores

cd ~/lodestar/keystores

ls

The contents inside your validator_keys folder should be now be visible in your Lodestar keystore directory.

If you have your deposit_data files within your Lodestar keystore directory, you must delete it or you will encounter errors when starting up the validator client. You should only have your keystore-m.json files here.

To remove any deposit_data files, you can use the rm command.

NOTE: Replace <NameOfDepositDataFile> with the filename of your deposit_data JSON file.

sudo rm <NameOfDepositDataFile>

Use the ls command to ensure the deposit_data JSON file(s) are removed.

ls

Unmount your USB stick.

sudo umount <USBDevicePath>

You can now physically remove the USB key from your local server. Continue at the next step: Configure Lodestar Validator Client.

Copy the Validator Keys from your local machine to your Lodestar keystores directory

Configure the Lodestar validator by importing the validator keys.

If you generated the validator keystore-m.json file(s) on the local Ubuntu staking server you will need to copy the file(s) over to your Lodestar keystores directory. You can do this using the cp command which follows the standard sudo cp <FileFromDirectoryPath> <ToDirectoryPath>

If you followed Part 1: Generating Staking Data you should be able to find your keystore-m.json file(s) within your validator_keys folder.

Example:

sudo cp /home/user/Downloads/eth2deposit-cli-256ea21-linux-amd64/validator_keys/keystore-m.json /home/user/lodestar/keystores 

If you have your deposit_data files within your Lodestar keystore directory, you must delete it or you will encounter errors when starting up the validator client. You should only have your keystore-m.json files here.

To remove any deposit_data files, you can use the rm command.

NOTE: Replace <NameOfDepositDataFile> with the filename of your deposit_data JSON file.

sudo rm <NameOfDepositDataFile>

Use the ls command to ensure the deposit_data JSON file(s) are removed.

ls

Configure Lodestar Validator Client

Create password file

You will now configure the validator client to import your keystore password. We will make a folder called keystores_password and a text file containing the keystore password.

cd ~/lodestar

sudo mkdir keystores_password

sudo nano ~/lodestar/keystores_password/password.txt

Type your keystore password (the password you used to encrypt the .json files).

When you're done creating this file, press CTRL+ x to exit and press y to save.

Modify Validator Client Docker Compose file

We will recreate the docker-compose.validator.yml file which configures your Lodestar validator docker instance. These parameters will allow you to run the Lodestar validator client through your machine’s host network and connect to your local Ethereum Consensus (eth2) Lodestar beacon node.

Create a new text file with nano.

cd ~/lodestar

nano

NOTE: YAML files are sensitive to indentations. Copy the lines exactly as seen below.

version: "3.4"
services:
  validator:
    image: chainsafe/lodestar:latest
    restart: always
    volumes:
      - validator:/data
      - logs:/logs
      - ./keystores:/keystores
      - ./keystores_password:/keystores_password
    env_file: .env
    command: validator --rootDir /data --importKeystoresPath /keystores --importKeystoresPassword /keystores_password/password.txt --server http://0.0.0.0:9596 --logFile /logs/validator.log --logLevelFile debug --logRotate --logMaxFiles 5
    # A validator client requires very little memory. This limit allows to run the validator
    # along with the beacon_node in a 8GB machine and be safe on memory spikes.
    environment:
      NODE_OPTIONS: --max-old-space-size=2048
    network_mode: host

volumes:
  validator:
  logs:

TESTNET USERS: If you would like to pull the unstable nightly versions of the Lodestar validator client, you can change the image: parameter from chainsafe/lodestar:latest to chainsafe/lodestar:next

When you're done creating this file, press CTRL+ x to exit and press y, then name the file docker-compose.validator.yml and press y to overwrite the current version.

Start the Lodestar Validator Client

To start the validator client, ensure you are in your Lodestar directory in the command line terminal:

cd ~/lodestar

sudo docker-compose -f docker-compose.validator.yml up -d

Once it successfully builds the validator client container you will see the return ... done on your new Docker container containing your validator!

Check that all containers are working properly.

sudo docker ps

If one of the containers continuously reboot, there is an issue and you will need to diagnose them from their logs. You can access these logs with docker logs

sudo docker logs <ContainerId>

NOTE: If you see this within your validator's logs:
info: Node is syncing - Error on getProposerDuties - Service Unavailable: Node is syncing

This is normal if your beacon node has not fully synced to the head of the network yet.

You can also check that you validators are connected through your Grafana dashboard under "Validators connected".

Rebooting your server should autostart all of the containers. Check by rebooting your server, then checking the docker processes.

sudo reboot

sudo docker ps

---

Final Notes

You should now have a functioning Ubuntu based staking server set up with Docker containers.

Continue on with the guide to fund your validator keys in Part 4: Funding Your Validator Keys.