HackMD
Development updates Week 15-16
- Writing Makefile that automates the process of packaging
- Patching if needed
- offer built packages in the repository (through building them)
Week 15
On week 15 I have finished the build system that allows to build any client based on simple command line argument. As I have detailed this in the previous week, we have 6 steps that we want to make sure happens
- Have the specification files ready (hard requirement)
- Generate the /debian dir based from specification file which will be stored in the gihtub repo along with the specifaction file and all versions of how to build that client.
- Download the source from .tar.gz and extract it. Rename it according to debian naming convention.
- Extract the source, and copy the debian directory to it.
- Build the source.
- Upload the built package to the packaging repository so others can install it.
To this another step was added, which is called patching. Some of the clients Makefile are correct to be built from source, but building through dh-make which is supplied by debian is going to fail in certain scenarios. Also it is always a possibility that a client released and it's not buildable and patching that source after release is not possible. That's why adding and extending the patching system was a very important step.
Week 16 (In-Progress)
Please see each client for the progress. Below you will find all the small issues I encounter. Most likely I will update the beginning of next week, as I still working on fixing these isues (weekend)
Besu
Compiling is failing as precompiled contract throws UnlinkedStatic.
Upgrading to bullseye from buster, solves the LinkingError problem, but results in memory issues. I will need to rebuild the client after upgrading the vm.
- - Compiles
- - Package is built
- - Uploaded and tested on different VM
- - configuration needs to be supplied. Time consuming, but an easy task, though requires rebuilding the client.
Issues:
- - Compiles, but integration tests fail.
- - openjdk 17, requires bullseye to built on
- - gradle must be installed on build
- - sometimes the build runs out of memory, no an issue on VM that spinned up, but if you use the same VM to rebuild
- - besu is installed, but very slow start on vm, most likely because the VM needs to be updated to have the minimal besu requirements
Erigon
Compiles, and run after install.
- - Compiles
- - Package is built
- - Uploaded and tested on different VM
- - configuration needs to be supplied. Time consuming, but an easy task, though requires rebuilding the client.
Issues
- sbuild environment flags_test.go:58: test ~/tmp, got /tmp/sbuild/tmp, expected /home/debian/tmp\n
Geth
Compiles, and runs after install. Install is executed on another debian machine.
- - Compiles
- - Package is built
- - Uploaded and tested on different VM
- - configuration needs to be supplied. Time consuming, but an easy task, though requires rebuilding the client.
Lighthouse
Test fails, both on bullseye and buster. Need to investigate.
- - Compiles
- - Package is built
- - Uploaded and tested on different VM
- - configuration needs to be supplied. Time consuming, but an easy task, though requires rebuilding the client.
Issues
- - Building is extremely slow
- - Release build, test fails
- - Needs at least 50GB space to compile.
- - need to patch the install to pass LD
Teku
Compiles. Package cannot be built yet, running from debian build is results in exception.
- - Compiles
- - Package is built
- - Uploaded and tested on different VM
- - configuration needs to be supplied. Time consuming, but an easy task, though requires rebuilding the client.
Issues:
- There are some issues with Teku being built on buster. Java-17 is not supported on buster. I might consider moving this onto bullseye.
Prysm
Builds
- - Compiles
- - Package is built
- - Uploaded and tested on different VM
- - configuration needs to be supplied. Time consuming, but an easy task, though requires rebuilding the client.
Nimbus-eth2
- - Compiles
- - Package is built
- - Uploaded and tested on different VM
- - configuration needs to be supplied. Time consuming, but an easy task, though requires rebuilding the client.
Lodestar
Compiles. Package needs to built and tested.
- - Compiles
- - Package is built
- - Uploaded and tested on different VM
- - configuration needs to be supplied. Time consuming, but an easy task, though requires rebuilding the client.
Issues:
Node dependency must be installed from custom repo. I need to figure out how to add that to the build-depends field.
Nethermind
Compiles. Package is built. Needs testing.
- - Compiles
- - Need to patch the Nethermind.Runner to ignore git version, and use the the directory tag from the filename
- - Package is built
- - Uploaded and tested on different VM
- - configuration needs to be supplied. Time consuming, but an easy task, though requires rebuilding the client.
Issues
- [] dotnet dependency must be added
cd nethermind/src/Nethermind/Nethermind.Runner dotnet build -c release
You must install the runtime on the client machine
Debian 10
# Install signing key
wget https://packages.microsoft.com/config/debian/10/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
rm packages-microsoft-prod.deb
# Install runtime
sudo apt-get update && \
sudo apt-get install -y aspnetcore-runtime-7.0
# Then you can install
sudo apt install eth-node-nethermind
# Check if nethermind is available
nethermind -v
Issues
- another repo as dependency
- run tests defined in nethermind
- test are failing
Various tasks
In this week I had still a few blocking tasks to be taken care of mainly
- some clients are not buildable (geth)
- some clients are not buildable (erigon)
- some clients are not buildable (lighthouse)
- some clients are not buildable (nimbus-eth2)
- besu (problems with the machine and distro)
- upload must be automated
- the details of configuration must be supplied
- IN-PROGRESS clients actually must be built and checked
- reprepro, packaging server needs to be configured to handle dupload
And many small issues
- add ability to build any version of the package
- add make clean CLIENT=erigon to clean the output
- add github links for patching
- reorganize repository to have a debian_specs folder
- handle multiple distribution (stable/unstable/testing)
- install debcrafter as deps, in the Makefile
- when build fails for a client, subsequent build doesn't apply the patch
- add Standards-Version to the debian/control file
- add ability to copy patches between versions
- geth is building now (had issues with memory)
- lighthouse is building now (missing dependencies, space issues)
- erigon (issues with vm, needed to upgrade to a machine with more CPU)
- reprepro incoming processing is not working
- reprepro cannot add include *.changes
- add license to the project
- updating documentation for build system as I keep working on it
