Setting Up a Local FOCIL Devnet with Kurtosis

Special thanks to Parithosh for all the helps

Introduction

This document provides instructions for setting up and running a local devnet of Prysm and Geth prototypes using Kurtosis. We will build Docker images and use them to run Kurtosis. Docker image building scripts and Kurtosis configuration file can be found here.

Prerequisite

Before getting started, ensure that you have Docker and Kurtosis installed:

1. Install Docker and start the Docker Daemon
2. Install the Kurtosis CLI, or upgrade it to the latest version

Build Prysm Docker Images

A Prysm validator client is compatible only with a Prysm CL client. We can build Docker images for both clients at the root of Prysm repository as follows:

For AMD / x86 architecture

# Beacon node
bazel build //cmd/beacon-chain:oci_image_tarball --config=release
docker load -i bazel-bin/cmd/beacon-chain/oci_image_tarball/tarball.tar

# Validator client
bazel build //cmd/validator:oci_image_tarball --config=release
docker load -i bazel-bin/cmd/validator/oci_image_tarball/tarball.tar

# Retag the Docker images for Kurtosis.
docker image tag gcr.io/prysmaticlabs/prysm/beacon-chain:latest prysm-beacon-chain-focil:latest
docker image tag gcr.io/prysmaticlabs/prysm/validator:latest prysm-validator-focil:latest

# (Optional) Remove the intermediate Docker images.
docker image rm gcr.io/prysmaticlabs/prysm/beacon-chain:latest
docker image rm gcr.io/prysmaticlabs/prysm/validator:latest

For ARM architecture (including Apple M1/M2/M3)

# Beacon node
bazel build //cmd/beacon-chain:oci_image_tarball --platforms=@io_bazel_rules_go//go/toolchain:linux_arm64_cgo --config=release
docker load -i bazel-bin/cmd/beacon-chain/oci_image_tarball/tarball.tar

# Validator client
bazel build //cmd/validator:oci_image_tarball  --platforms=@io_bazel_rules_go//go/toolchain:linux_arm64_cgo --config=release
docker load -i bazel-bin/cmd/validator/oci_image_tarball/tarball.tar

# Retag the Docker images for Kurtosis.
docker image tag gcr.io/prysmaticlabs/prysm/beacon-chain:latest prysm-beacon-chain-focil:latest
docker image tag gcr.io/prysmaticlabs/prysm/validator:latest prysm-validator-focil:latest

# (Optional) Remove the intermediate Docker images.
docker image rm gcr.io/prysmaticlabs/prysm/beacon-chain:latest
docker image rm gcr.io/prysmaticlabs/prysm/validator:latest

For your convenience, you can place this script in the Prysm repository and then execute it.

Please note that building Prysm Docker images is not supported on Windows. For more information, you can refer to Prysm Documentation.

Build Geth Docker Image

Building a Geth Docker image is simple:

docker build . -t geth-focil:latest

Similarly, you can use a script instead, if you prefer.

Run a Local FOCIL Devnet with Kurtosis

Now, we can run a local devnet using those Docker images built locally. Incorporating local changes would facilitate more effective testing.

First off, let's clone a repository:

git clone https://github.com/jihoonsong/local-devnet-focil

You can launch a local devnet with Kurtosis:

cd kurtosis
make run

After it's done, you should be able to see this:

Access Dora the Explorer

To access Dora the Explorer, you can go to localhost:65500.

The port of Dora is currently fixed to 65500. This is determined by the order of additional services so if you want to add other services under additional_services in the Kurtosis configuration, you might want to add them below dora to access Dora the Explorer at port 65500 consistently. You can find supported services here.

You can also see the logs of Dora in the Docker app's container. The container name will be something like dora--f95d74646f8744f58f5a09844fa92dc2.

Check Service Logs

It is important and helpful to see the service logs. When something goes wrong, you want to check the logs and find a problem.

To follow logs from all services:

kurtosis service logs focil -f

This might be too verbose to see what each node is doing. We can specify the service name of each node to follow its logs exclusively. To get service names:

kurtosis enclave inspect focil

You can see service names in the red rectangle. If you want to see logs of snooper-engine-1-prysm-geth:

kurtosis service logs focil snooper-engine-1-prysm-geth -f

Stop the Local Devnet

To stop the local devent:

make stop

Using Different Clients than Prysm and Geth

If you want to use different clients other than Prysm and Geth, you can replace clients specified in the Kurtosis configuration file or add new ones. For example, if you want to use Teku and Geth, you can replace participants with:

participants:
  - el_type: geth
    el_image: ethpandaops/geth:prague-devnet-5-a193537
    cl_type: teku
    cl_image: consensys/teku:develop
    count: 2

Miscellaneous

Minimal Config
You can set preset to minimal to use minimal config. It's mainly used for testing API compatibility, spec tests, MEV tests, etc.

However, we would want to use mainnet config for testing as minimal config often suffers from bugs outside of those use cases. Although we cannot change slots_per_epoch with mainnet config, setting slots_per_second to 3-6 seconds achieves most of what we want.

Kurtosis CLI Completion
Configuring Kurtosis CLI Completion could make your life easier.