# EPF Final Report - Ifeoluwa Oderinde
**Project:** [Add PostgresDB as an optional DB implementation for the Beacon Node](https://github.com/eth-protocol-fellows/cohort-six/blob/master/projects/project-ideas.md#lighthouse-add-postgresdb-as-an-optional-db-implementation-for-the-beacon-node)
**Client:** Lighthouse ([sigp/lighthouse](https://github.com/sigp/lighthouse))
**Cohort:** [6](https://github.com/eth-protocol-fellows/cohort-six)
**Mentor:** [Eitan](https://github.com/eserilev)

## Project Abstract
Implementation of a PostgresDB for Lighthouse Beacon Node as an alternative to LevelDB/Redb in environments where embedded databases are unrealiable (especially NFS).

## Summary
During EPF cohort 6, I implemented a fully functional Postgres backend for Lighthouse's Beacon Node, which included:
- Complete implementation of `KeyValueStore` + sync wrappers for `postgres_impl.rs`, with functional operations (`put/get/delete/exists`), transaction support (`do_atomically`, commit, rollback) and full iterator support.
- Integration into Lighthouse's `BeaconNodeBackend`
- Local testing with [Lighthouse](https://hackmd.io/@uncle-bill/Bywk7BcKxg).
- Partial Kurtosis-based devnet testing for end-to-end verification.

The backend compiles inside Lighthouse, passes local DB tests, serves as a drop-in alternative to Redb/LevelDB for operators who require database durability guarantees.

## Links
[PostgresDB Branch](https://github.com/sigp/lighthouse/pull/8157)

## Project Description
### Problem
Lighthouse currently supports two embedded storage engines:
- **LevelDB**
- **Redb**

Both engines are efficient, but both suffer from a critical problem -- they are unsafe and prone to corruption on NFS-mounted filesystems.

## Solution
I implemented a PostgresSQL backend that provides:
### ACID-safe transactions
Postgres guarantees crash consistency and protect against corruption.
### Remote network storage
Nodes can store their chain database on:
- a managed cloud Postgres instance
- a high-availability cluster
- bare-metal Postgres servers

### Strong durability sematics
Unlike embedded DBs that rely on the underlying filesystem, Postgres has:
- WAL logging
- fsync guarantees
- crash recovery
- vacuuming
- concurrency control

### Identical Lighthouse semantics
The backend behaves as a drop-in replacement for existing backends.

## Status
All core functionality for the first milestone is complete:
### Core PostgresDB Implementation
- `get_bytes`
- `put_bytes`
- `sync`
- `key_exists`
- `key_delete`
- `do_atomically`
- `compact`
- `iter_column_keys_from`
- `iter_column_keys`
- `iter_column_from`
- `compact_column`
- `delete_batch`
- `delete_if`
- sync wrappers via `block_on`

### Backend Integration
- Added `BeaconNodeBackend::Postgres`
- Wired into `interface.rs`
- Added Postgres feature flag
- Added CLI configuration fields

### Testing
- Unit tests and local integration tests pass
- Lighthouse compiles and runs with `--db postgres`

### Kurtosis Testnet Status
- Partial Kurtosis compatibility
- Container builds succeed
- Postgres service launches
- Lighthouse startup sequence requires additional work for full Postgres initialization.

## High-Level Design

## Core Data Structures
1. **`PostgresDB<E>`**
Stores:
- Connection pool
- type phantom
- feature flag activation

`pub struct PostgresDB<E> {
pool: PgPool,
_marker: PhantomData<E>,
}
`

2. **Table Schema**

`CREATE TABLE <table_name> (
key BYTEA PRIMARY KEY,
value BYTEA NOT NULL
);
`

The mirrors Lighthouse's internal KV structure.

3. **Transactions**
Lighthouse's `do_atomically` requires:
- multiple put/delete operations.
- wrapped in a single atomic commit
- with rollback on failure

Implemented using:
`let mut tx = pool.begin().await?;
...
tx.commit().await?;
`

## Code Implementation
### Challenges & Resolutions
**1. Sync + Async Boundary**
Lighthouse still keep all db interactions sync.

**Challenge:**
You cannot call async DB operations from sync trait methods.

**Solution:**
Use:`Handle::current().block_on(...)`
to run async operations in a synchronous context.

**2. SQLx Executor Trait Hell**
SQLx's executor requirements caused type mismatches when passing transactions into internal helper functions.

Resolved by introducing generic bounds:

`where T: sqlx::Executor<Database = Postgres>`

**3. Iterator Sematics**
Lighthouse requires:
- column iteration
- key-only iteration
- range-less scans

Postgres implementation performs
`SELECT key, value FROM {} ORDER BY key;`and streams rows into Rust iterators.

**4. Transaction Context Passing**
`do_atomically` must execute all inner operations within one transaction.
This required refactoring helper functions so they could accept either:
- a PgPool connection
- a PgTransaction executor

## Unfinished Work
**1. Kurtosis End-to-End Testing**
Kurtosis currently assumes embedded DBs.
Remaining tasks:
- Make Postgres service reachable from Lighthouse container
- Validate `--postgres-url` passing
- Ensure DB initializes before Lighthouse boot
- Validate block production + sync Postgres

**2. Performance Benchmarks**
Need to compare:
- write throughput
- read latency
- impact on block processing
- storage growth patterns

**3. Metric Integration**
Useful metics to add:
- DB latency
- rows read/written per slot
- transaction commit time
- error counts

## Pivot / Architecture Changes
**Pivot 1 -- Async trait Implementation**
Initially attempted a sync-only DB; impossible with Lighthouse internals.
**Solution**: implemented full async backend with sync wrappers.

**Pivot 2 -- Kurtosis Adjustments**
Original hope: drop-in with existing EPF Kurtosis setups.
**Reality**: required modifying scripts, flags, and Dockerfiles.

## Journey with EPF
This project stretched me in everyway (still does) -- Rust, database, async runtimes, client internals, Docker/Kurtosis, debugging, and architectural design.

The hardest parts were not the big architectual questions but the hundreds of small, frustrating, invisible problems:
- SQLx lifetime mismatches
- Iterator borrowing rules
- Traits requiring sync sematics
- Kurtosis parsing errors like "unexpected EOF while looking for matching quote"
- Lighthouse initialization paths
- Container networking
- Tokio runtime availability

By the end of the cohort, I was comfortable navigating Lighthouse internals, designing database layers, and reasoning about async Rust at the client level.

This experience has strengthened my confidence as an Ethereum client developer and given me a long-term project, one I’m committed to seeing through beyond the fellowship.

## Acknowledgements
### Mentors
- **Eitan --** For pushing the implementation forward, reviewing designs, merging updates, and giving direct guidance on the best possible way to help the project come to life.
### EPF Program
- **Josh Davis & Mario Havel** -- For running a world-class fellowship that gives real autonomy and real responsibility.
- **All EPF Fellows** -- For discussion and moral support.