# DeBooting EthOnArm

This document is a proposal for integrating the [EthereumOnArm](https://github.com/EOA-Blockchain-Labs/ethereumonarm) project with [DeBoot](https://github.com/debootdevs/deboot), a set of tools and methods for delivering bootable images and other software from decentralized storage.

## Overview

Given EthereumOnArm's status as a blue chip Ethereum infrastructure project, we might be tempted to take for granted that it will be an early adopter of novel Ethereum-based decentralized infrastructure technologies. Integrating decentralized storage into EthereumOnArm's software hosting and distribution channels is a natural and obvious opportunity.

However, we can do better: it is not hard to pinpoint concrete benefits that would be delivered by integrating with decentralized storage.

1. **Distribution channel diversity.** Decentralized storage offers a novel channel for distribution of EthereumOnArm software with a different set of tradeoffs to traditional webservers.
2. **Trustless guarantee of continuity.** Storage funded in advance by on-chain payments provides a trust-reduced continuity guarantee that the software will remain available, even in the case of project abandonment, for the full support period.
3. **Transparency.** Funders or maintainers of public goods community projects may find it desirable that the use of funds are publicly accounted for. Storage purchased on public blockchains is transparent by default and can be used to provide unfalsifiable audit reports.

### Challenges

Decentralized storage is still in its infancy and hence suffers from the usual issues of software still in development: unreliable backends, user-unfriendly tooling, rapidly changing interfaces and incomplete documentation. These issues can be expected to soften over time.

There are also more fundamental challenges:

* As well as data storage and retrieval, traditional cloud storage providers offer customer support and can be held accountable when something goes wrong (i.e. data cannot be retrieved). Decentralized systems don't necessarily have any clear accountable party, so a much greater share of responsibility rests with the customer --- in this case, the software distributor.
* To implement access-controlled or private repositories, distributors would need to either run their own Swarm network or use a separate access control layer. I am not aware of any options that exist for the latter right now.

### Future directions

The advantages of decentralized storage for software distribution suggests promising avenues of future R&D activity:

* Modular delivery of boot images.
* Network booting a large image means time wasted at waiting for the download to complete when basic services could be starting. This can be accelerated by delivering boot images in stages, starting with core services, and aggressive cacheing.
* Delivering images in layers allows us to use different distribution channels for different components. EthereumOnArm essentially already does this to some extent by using a separate APT repository for its own software, but it could go further --- see below.

* Funding modules. Develop on-chain systems for:
* Atomic and censor-resistant commitment to making a repository available over a support period. For example:
* Storage of binary and source packages;
* Package names in a repository manifest;
* Storage of repository manifest;
* Registration of repository address in a decentralized name service (ENS).
* Connecting storage spend to community governance.

## Booting EthereumOnArm

EthereumOnArm is a flavour of Armbian with a custom configuration script and a custom APT repository serving Web3-related software. It is designed to be run on ARM SBCs, which are generally booted with [Das U-Boot](https://github.com/u-boot/u-boot).

Its boot flow runs as follows:

* Armbian stages
* U-Boot looks for the "compiled" U-Boot script file `/boot.scr` in the boot partition of the boot device.
* Local kernel (`/Image`) and U-Boot initramfs (`/uInitrd`) are loaded from the boot partition.
* EthOnArm stage
* Late in the systemd initialization of userspace (after `network.target`), a shell script `/etc/rc.local` is pulled in by the service manager. This script handles all EthereumOnArm-specific configuration, including creation of the default user `ethereum`, adding the EthereumOnArm APT repository, and installation of default software.

Each stage of the initialization flow could be integrated with decentralized storage, with different tradeoffs incurred in each case. We discuss two of these stages below.[^firmware]

[^firmware]: We omit discussion of integrating decentralized storage into a kernel loader or platform firmware because a lack of supporting libraries makes implementation more difficult.

### DeBooting Armbian

The current DeBoot approach of integrating a Swarm node into an initramfs can be used to boot Armbian userspace from Swarm. A sample implementation can be found in the [aarch64 branch](https://github.com/debootdevs/deboot/tree/aarch64) of the DeBoot repository on GitHub.

Boot flow:

[U-Boot SPL, vmlinuz, swarm-initramfs, extlinux.conf] 
    => [Armbian(init), "root=live:bzz://$HASH"]
    => [Armbian(user)]

Applied to EthereumOnArm, this would give developers a new route to providing updates to its configuration RC scripts separately to the Armbian boot description. In particular, if the image is indexed under the same address (using Swarm's Feed construction), it can deliver updates to these scripts without requiring the end user to write a new image to removable boot media.

This approach could be further accelerated by serving `rc.local` and its associated services as a seperate component to be overlaid on the Armbian userspace.

### APT on Swarm

The other component that could profitably be moved onto decentralized storage is the APT repository. This fairly obvious idea is hardly new; [some inroads](https://github.com/ipfs-shipyard/apt-on-ipfs) to the problem have been made already several years ago in the context of IPFS.

Debian's APT expects a static directory tree available on a webserver. A straightforward implementation suggests itself: upload the target APT repository to Swarm and write the URL of the appropriate call to the Bee API --- (something like http://\$NODE_ADDR:1633/bzz/\$HASH) --- into `sources.list`.

### Proof of concept

I uploaded a version[^apt] of the EthereumOnArm repository to Swarm at the following hash:

apt.ethereumonarm.com    baccb2abf9c2698b5437aa22db17865efeae776c7f17948ebdfe521956a44019

This repo offers bee, geth, reth, lighthouse, and some EthereomOnArm tooling. The storage is prepaid for 28 days at time of writing, i.e. until 2023-11-23.

To use it, we need access to a Bee node. For these instructions I'll assume it's running on `localhost` and exposing its API on the default port.

1. Download the repo keyring from https://apt.ethereumonarm.com/eoa.apt.keyring.gpg or http://localhost:1633/bzz/baccb2abf9c2698b5437aa22db17865efeae776c7f17948ebdfe521956a44019/eao.apt.keyring.gpg and write it to `/usr/share/keyrings/`.
2. Write the following configuration file:

# /etc/apt/sources.list.d/ethereumonarm.sources
Types: deb
URIs: http://localhost:1633/bzz/baccb2abf9c2698b5437aa22db17865efeae776c7f17948ebdfe521956a44019
Suites: focal
Components: main
Signed-By: /usr/share/keyrings/eoa.apt.keyring.gpg

Here's what it looks like when you `apt update`:

Installing `reth`:
Notice that some dependencies (`jq`, `libonig5`) are pulled from default repositories while `reth` itself comes from the EthereumOnArm repo hosted on Swarm!

[^apt]: To get around technical limitations of currently available Swarm tooling, I stripped the repository --- originally nearly 7GB --- down to a size of around 1GB.