# Octez Packages
<!-- Put the link to this slide here so people can follow -->
slide: https://hackmd.io/@pietro1234/octez-packages
---
## General overview
We support
- Debian bookworm
- Ubuntu noble
- Ubuntu
- Fedora 39 ( soon to be updated )
- Rockylinux 93 ( to be updated )
Both on amd64 and arm64
We also release a MacOS formula for homebrew
---
## Timeline
- This is the timeline of the packages as in https://gitlab.com/tezos/tezos/-/milestones/322
- v20.3 : we adopt and release Chirs' packages
- v21.4 : we release all packages via apt, deprecate legacy and introduce exp packages
- v22.0 : we remove legacy packages and release the next packages
- this work was constraint by the releases
---
## Debian packages
- All the code is in `scripts/packaging/octez/debian/` .
- We use only standard debian tools and I avoided hacks or workarounds.
- The packages description lives in the `control` file where all packages are declared
- The script part is in the `rules` file where we configure the build system
- Each package has specific files.
- We only build **Binary** packages. Not source packages.
---
### Associated tools
- All packages use systemd to manage their lifeycle
- All packages use the `tezos` user to run and all files belong to this user
- We se logrotate for the logs
- The configuration for the systemd services is in `/etc/default/octez-*`
- Everything else is configured using `octez-* config` commands and stored in `/var/tezos`.
---
### Debconf
- We use debconf to install and configure the packages
- Non-interactive installations are possible via pre-seeding of debconf
- All the debconf question are in the `templates` files
- Debconf is used also to store some default values associated to the package
---
## Rpm packages
- All the spec files are located in `scripts/packaging/octez/rpm/`
- Even if some scripts are shared between debian and rpm packages, there is no automatic process
- Rpm packages do not have a debconf alternative, therefore there is not interactive procedure.
- From the user prospective they offer the same feature set of deb packages.
---
## Homebrew
- It's just a build script that has been tested on MacOS.
- We do not build MacOS binaries
- We do not provide start scripts for MacOS users
- We provide the formula on our release page
---
## CI and testing
- We have a complete parallel/matrix test battery for the packages
- We test the packages build before merging in master.
- Two types of pipelines : full and partial
- Pipelines are triggered automatically if needed + we have manual trigguers
---
## CI jobs deb/rpm
- We build a docker image for each distribution with all the depedendencies
- This image is then used to build data and binary packages
- A job creates the apt/dnf repository
- Each package is tested for installation and for upgrade ( from the last release )
- We also test the scripts that are published in the documentation page.
- For debian, we run `lintian` to make sure packages adhere to the debian policy
---
## CI jobs homebrew
- We run the homebrew formula on a linux container
- We run the homebrew formula on a gitlab server running macOS
---
## Repositories
- We *publish* a test repository for each branch on a GCP bucket.
- This repository is used for the CI tests and can also be used locally
- All repositories are signed with gpg and the key must be imported using apt/rpm
- Master also has a dedicated repository
- Packages are pulished at each release at https://packages.nomadic-labs.com
- We do not offer single packages for download in the release page
---
## URLs for debian
- https://tezos-linux-repo.storage.googleapis.com : dev repo
- https://packages.nomadic-labs.com : releases
- "https://tezos-linux-repo.storage.googleapis.com/abate@refactor-packages-tests/debian bookworm main"
- "https://tezos-linux-repo.storage.googleapis.com/master/debian bookworm main"
---
## Documentation
- All the doc to install the packages is published [here](https://octez.tezos.com/docs/introduction/howtoget.html#installing-packages)
- A [complete manual](https://octez.tezos.com/docs/introduction/services.html) to configure and run octez services shipped with the packages
- A [manual](https://octez.tezos.com/docs/introduction/serokell.html) for the sorekell migration here
- Most scripts in this pages are also run in the CI.
---
## Maintenance and development
- To compile packages locally : `scripts/packaging/build-deb-local.sh`
- There is also a similar script for rpm. The same scripts are also used in the CI.
- To test the packages and the installation scripts locally :
```
DISTRIBUTION=debian \
RELEASE=bookworm \
scripts/packaging/tests/systemd-docker-test.sh \
scripts/packaging/tests/deb/install-bin-deb.sh \
images/packages/debian-systemd-tests.Dockerfile
```
---
## Systemd tests
- We run the test script in a systemd environment ( systemd must run with PID 1 )
- Systemd runs in a docker container and then we spawn a shell to run the script running `docker exec <script>`
- Alternatively **LXD** that is an excellent project to run multiple VMs on the same machine.
- The systemd tests run the installation, upgrade, configuration and run the node and the baker.
---
## Additional details
- The default for the packages are
- history mode: rolling
- network: mainnet
- snapshot: yes
- All commands used to configure the node/client/baker must be run with
`sudo su tezos -c "octez-node config ..."`
---
## Additional details
- The datadir must be in `~tezos/.octez-node` : home directory of the `tezos` user
- To relocate the datadir you must change the home of the tezos user :
`usermod -m -d /custom tezos` and
`echo "DATADIR=/custom/.tezos-node" >> /etc/default/octez-node`
- To download the snapshot `snapshot-import.sh` can be invoked directly
- The Script `wait-for-node-up.sh` can also be used in scripts
---
## Versions
- We use a version schema prefixed with a timestamp and a epoch: this is needed to make sure we respect the versioning order
- For releases, the version is derived by the git tag.
- For debian we use `debchange` to set the version
- The zcash package has a fixed version :`1.0.0`.
- The script ` scripts/ci/octez-packages-version.sh` contains the logic
---
### Packages dependencies
```
$ apt-cache dotty \
octez-baker \
octez-node \
octez-dal-node \
octez-signer -o "APT::Cache::GivenOnly=true"
```
---
## CI flakiness
- Sometimes the GCP bucket we use does not work properly: we have a retry in place
- Sometimes the builds OOM -> we use `very_high_cpu`. The old packages should be removed
- Sometimes systemd tests hang -> This is mostly due to slow network and timeouts
- Sometimes packages installation hang -> This is a network problem. we're setting up a transparent cache
---
## work in progress (for you ?)
- https://gitlab.com/tezos/tezos/-/merge_requests/17387 : about packages purge
- https://gitlab.com/tezos/tezos/-/merge_requests/17393 : teztale package
- https://gitlab.com/tezos/tezos/-/merge_requests/17591 : update rpm releases
- https://gitlab.com/tezos/tezos/-/merge_requests/17607 : remove legacy packages
{"contributors":"[{\"id\":\"e967c8a3-5a48-4899-a235-2737a65d1709\",\"add\":13209,\"del\":5454}]","title":"Octez Packages","description":"slide: https://hackmd.io/p/template-Talk-slide"}