EPF4: UPDATE 6

Got very busy with my other project this week, but i was able to dig into ephemery native retention scripts and understand how ephemery reset works manually and rewrote it in typescript. Typescript is the language of choice for Lodestar.
I have written the purpose of this experimental code here, so i will just jump into my own impression of how to modify it within Lodestar here:

import * as fs from 'fs';
import * as path from 'path';
import * as util from 'util';
import { exec } from 'child_process';
import axios from 'axios';
import * as tar from 'tar';

const execPromise = util.promisify(exec);

const genesisRepository = 'ephemery-testnet/ephemery-genesis';
const testnetDir = '/home/ethereum/testnet';
const clDataDir = '/home/ethereum/data-lodestar';
const clPort = 5052;
const genesisResetInterval = 86400; // 1 day in seconds

notes

• most of the constants and modules here are already present in lodestar, which is a good thing. i still want to know how directories are created though!

START CLIENTS

async function startClients() {
    console.log('Start clients');
    await execPromise('sudo systemctl start beacon-chain');
    await execPromise('sudo systemctl start validator');
}

notes

• this is not really neccessary, in the sense that lodestar with ephemery currently starts with ./lodestar beacon --network ephemery. however this is done manually from the terminal by the user.
• is there any implementation code in lodestar in which the client is restarted after a process? If yes, it can be repurposed here.

STOP CLIENTS

async function stopClients() {
    console.log('Stop clients');
    await execPromise('sudo systemctl stop beacon-chain');
    await execPromise('sudo systemctl stop validator');
}

notes:

• same as above.
• stopping the client is probably not semantically correct because the client will still be implementing the following processes below after it has 'stopped'
• what's being stopped actually are the transactions :).

CLEAR DIRECTORIES

async function clearDataDirs() {
    fs.rmdirSync(path.join(clDataDir, 'beacon'), { recursive: true });
    fs.unlinkSync(path.join(clDataDir, 'validators/slashing_protection.sqlite'));
}

notes

• this refers to the management of the directories automatically created in .ethereum. the directory being cleared here is lodestar's which will contain beacon configurations(?) and validator list(?).
• something worth noting here is that the consensus client directories are not recreated as they are done in the original retention scripts for the execution client. i think this is so because once the client is restarted again after the reset, lodestar will reinitialize this directory automatically?
• this process is indicated in the setupGenesis function below:

SETUP GENESIS

async function setupGenesis() {
    console.log('Setup Genesis');
    await execPromise(`~/lodestar/bin/lodestar init --datadir ${clDataDir} ${path.join(testnetDir, 'genesis.json')}`);
}

notes

• this function initializes lodestar while creating the neccessary directories.
• however, in the original retention scripts there is no genesis setup function for the consensus clients, i wonder why it's not available ?
• is it because it's already created by the execution client?
• this function also indicates that the ephemery directory(testnetDir) will be within lodestar's directory(clDataDir)

GET GITHUB RELEASE

async function getGithubRelease(repository: string) {
    try {
        const response = await axios.get(`https://api.github.com/repos/${repository}/releases/latest`);
        return response.data.tag_name;
    } catch (error) {
        console.error('Error fetching GitHub release:', error);
        throw error;
    }
}

notes

• this is basically grabbing the latest release of ephemery (i need to confirm it this is already being done automatically on lodestar)

DOWNLOAD GENESIS RELEASE

async function downloadGenesisRelease(genesisRelease: string) {
    console.log('Download Genesis Release');
    const testnetDirExists = fs.existsSync(testnetDir);
    if (testnetDirExists) {
        fs.rmdirSync(testnetDir);
    }

    fs.mkdirSync(testnetDir, { recursive: true });

    const response = await axios.get(`https://github.com/${genesisRepository}/releases/download/${genesisRelease}/testnet-all.tar.gz`, { responseType: 'stream' });
    response.data.pipe(tar.x({ C: testnetDir }));
    await new Promise((resolve, reject) => {
        response.data.on('end', resolve);
        response.data.on('error', reject);
    });
}

notes:

• if there is a new github release, this function downloads it. more file management here; this release will be downloaded into the .ethereum folder as well.
• lodestar like every other client after initialization has files within the .ethereum. so i suspect something similar to this will be available already.

RESET TESTNET

async function resetTestnet(genesisRelease: string) {
    await stopClients();
    clearDataDirs();
    await downloadGenesisRelease(genesisRelease);
    await setupGenesis();
    await startClients();
}

notes

• this function encompasses all the previous functions and implements them sequentially one after the other.
• (i think) most of these functions already exist in some form on lodestar and i will just need to modify them as neccessary.
• thus, this function will likely end up importing them into the ephemery package within lodestar where the reset can happen.

CHECK TESTNET

async function checkTestnet() {
    const currentTime = Math.floor(Date.now() / 1000);

    const genesisTimeResponse = await axios.get(`http://localhost:${clPort}/eth/v1/beacon/genesis`);
    const genesisTime = genesisTimeResponse.data.genesis_time;

    if (!genesisTime || genesisTime <= 0) {
        console.error('Could not get genesis time from beacon node');
        return;
    }

    const retentionVarsPath = path.join(testnetDir, 'retention.vars');
    if (!fs.existsSync(retentionVarsPath)) {
        console.error('Could not find retention.vars');
        return;
    }

    const { GENESIS_RESET_INTERVAL, ITERATION_RELEASE, CHAIN_ID } = require(retentionVarsPath);

    const testnetTimeout = genesisTime + GENESIS_RESET_INTERVAL - 300;
    console.log(`Genesis timeout: ${testnetTimeout - currentTime} sec`);

    if (testnetTimeout <= currentTime) {
        const latestGenesisRelease = await getGithubRelease(genesisRepository);

        if (!ITERATION_RELEASE) {
            process.env.ITERATION_RELEASE = CHAIN_ID;
        }

        if (latestGenesisRelease === ITERATION_RELEASE) {
            console.error(`Could not find new genesis release (release: ${latestGenesisRelease})`);
            return;
        }

        await resetTestnet(latestGenesisRelease);
    }
}

notes

• this function does the same thing as the previous in that it will be importing the previous functions (including the reset function).
• this function is checking if it's time to reset the testnet so as to proceed towards downloading a new genesis.

MAIN

async function main() {
    const genesisJsonPath = path.join(testnetDir, 'genesis.json');

    if (!fs.existsSync(genesisJsonPath)) {
        const latestGenesisRelease = await getGithubRelease(genesisRepository);
        await resetTestnet(latestGenesisRelease);
    } else {
        await checkTestnet();
    }
}

notes

• this function will end up being an index file in keeping with lodestar best practice of setting up files in directories using typescript.
• this suggests that the code files within ephemery/reset directory will look somewhat like this:
  • checktestnet.ts
  • reset.ts
  • index.ts

Where the index.ts imports all the aforementioned functions to implement ephemery reset automatically.