Try   HackMD

Run the Blobstream X Prover Locally

Overview

There are 3 parts to operating BlobstreamX (or for that matter any ZK-based system):

  • An operator script that is responsible for requesting proofs at a particular cadence (or based on some other condition)
  • A prover that generates proofs based on requests from the operator
  • A relayer that takes generated proofs and submits them onchain to the relevant smart contracts

With the Succinct Platform, once you write your circuit and operator script, the platform will take care of the prover and relayer parts of the system, which is what our current BlobstreamX deployment does (https://alpha.succinct.xyz/celestia/blobstreamx). Our current setup uses the BlobstreamX operator script here, which sends requests to our platform for proving + relaying. The local mode of the operator script will generate proofs locally, which this document covers.

Succinct Platform Concepts

The Succinct Platform has a concept of "releases" and "deployments". A "release" takes a circuit and builds it, storing the relevant artifacts that are used for proof generation. A "deployment" is tied to a specific release and is simply a deployed verifier contract onchain for the particular circuit. Each deployment has a "function_id" and contract address. The contract address is the address of the deployed verifier and the function_id is a unique identifier that the SuccinctGateway (our onchain entrypoint) uses to identify a circuit.

Local Prover Guide

Local proving is run with Docker to ensure that it is replicable on any machine. However, to generate Blobstream X proofs (which are computationally intensive) we recommend a large instance such as AWS EC2's r6a.16xlarge.

Clone the Repository

Clone the BlobstreamX repo. You will use this repository to run the local prover continuously for a given chain. More instructions will be provided in a later section to run the local prover.

git clone https://github.com/succinctlabs/blobstreamx

The operator script run with local mode will generate proofs locally using the downloaded binaries. To configure it, set up the environment variables following .env.example and make sure to set SUCCINCT_RPC_URL=local which will generate proofs locally instead of sending requests to our platform.

Create a folder locally where you will download all the built circuit artifacts and dependencies.

mkdir artifacts; cd artifacts;

Download artifacts

Last updated: 04/03/24 (Ratan)

Download the built circuit binaries from S3 (circuits have compiled binaries to generate proofs). The Blobstream circuit binaries are ~30GB, so downloading might take a bit. You can also build the binaries locally, though this is not recommended (instructions in the Misc section).

Download Verifier (Groth16 Wrapper Circuit)

curl -L https://public-circuits.s3.amazonaws.com/verifier-build13.tar.gz | tar xz

Download Blobstream X Plonky2x Circuits

Get the release ID's corresponding to the Celestia chain you want to prove from the table below.

Note: Header Range with batch size 1024 is used on Arbitrum and Base. Blobstream X on Mainnet (deployment TBA) will use batch size 2048.

Network Type Batch Size Release ID URL
Celestia Mainnet Header Range 1024 c637ee72-f9cb-47ce-ae15-4b0d5d0e15df https://alpha.succinct.xyz/celestia/blobstreamx/releases/33
Celestia Mainnet Header Range 2048 71fc43b9-2803-4f09-9134-84573f972ee1 https://alpha.succinct.xyz/celestia/blobstreamx/releases/34
Celestia Mainnet Next Header - 4992ff5a-9af4-4e71-8591-3cf7b6c07f5b https://alpha.succinct.xyz/celestia/blobstreamx/releases/32
Mocha-4 Testnet Header Range 1024 044ac12a-e42f-406c-b64d-55c09ee5e2a6 https://alpha.succinct.xyz/celestia/blobstreamx/releases/36
Mocha-4 Testnet Next Header - f53a3885-ec02-4be7-bd37-bf70d29cd3c1 https://alpha.succinct.xyz/celestia/blobstreamx/releases/35

Download the Blobstream X artifacts from S3 into your artifacts folder.

curl -L https://public-blobstreamx-circuits.s3.amazonaws.com/{HEADER_RANGE_RELEASE_ID}.tar.gz | tar xz

curl -L https://public-blobstreamx-circuits.s3.amazonaws.com/{NEXT_HEADER_RELEASE_ID}.tar.gz | tar xz

Mark the binaries as executable:

# Depending on which release you used, these commands might be slightly different. 
# Double-check your artifacts folder for the downloaded filenames.
chmod +x header-range-1024/header_range_1024 

chmod +x next-header/next_header

After these steps, you should have the following directory structure:

blobstreamx/
    Cargo.lock
    Cargo.toml
    bin/
    contracts/
    ...
    artifacts/
        verifier-build/...
        header-range-1024/header_range_1024
        header-range-1024/main.circuit
        header-range-1024/{other circuit files}
        next-header/next_header
        next-header/main.circuit
        next-header/{other circuit files}

Environment Configuration

Get the HEADER_RANGE_FUNCTION_ID and NEXT_HEADER_FUNCTION_ID corresponding to the Blobstream X contract you are relaying proofs to. Go to Contract -> Read as Proxy -> {headerRangeFunctionId, nextHeaderFunctionId} to get the function IDs.

  • Example Contract
    • Header Range Function ID: 0x46132c86ed84fdc655528f80f9291dd3116b04902036b96925edc78bbf52b8ca
    • Next Header Function ID: 0x2ce8ca4f509cb09415b5a6ca6afa265571dac0b9f6ddb46f487e017fec71cf25

Set .env in blobstreamx to the following:

PRIVATE_KEY= # Private key for your relayer
RPC_URL= # RPC for the chain you are relaying to.
TENDERMINT_RPC_URL= # Celestia RPC for the chain you are relaying to.
SUCCINCT_RPC_URL=local
SUCCINCT_API_KEY="" # Leave blank for local proving.
CHAIN_ID= # The chain ID you are relaying to.
CONTRACT_ADDRESS= # Address of Blobstream X contract.

# Optional Operator Parameters

# Use the Function ID's for your contract here:
LOCAL_PROVE_MODE=true
LOCAL_RELAY_MODE=false
PROVE_BINARY_{HEADER_RANGE_FUNCTION_ID}="./artifacts/header-range-1024/header_range_1024"
PROVE_BINARY_{NEXT_HEADER_FUNCTION_ID}="./artifacts/next-header/next_header"

WRAPPER_BINARY="./artifacts/verifier-build"

Local Prover

Make sure your Docker daemon is running & pull the latest succinct-local-prover

docker info
docker pull succinctlabs/succinct-local-prover

Then run the following to run the operator with local proof generation:

cargo run --bin blobstreamx --release

Proofs are stored as output_{request_id}.json in the proofs/ folder. The local relayer will read this file to post proofs to the BlobstreamX smart contract.

Proving Time

On an AWS EC2 r6a.16xlarge instance, header range proofs take ~25 mins to generate.

Misc

Relay an Existing Proof

If you want to relay an existing proof in /proofs, run the following command:

cargo run --bin local_relay --release -- --request-id {REQUEST_ID}

First, download the STARK-SNARK wrapper binary:

curl -L https://public-circuits.s3.amazonaws.com/verifier-build13.tar.gz | tar xz

Instead of downloading the Blobstream X circuit binaries from a remote source, you can build them locally by running mkdir -p build && RUST_LOG=debug cargo run --bin header_range --release build && mv ./target/release/header_range ./build/header_range at the top-level of the BlobstreamX repo. It is recommended you download the binaries directly instead of building locally, because you have to be careful that the build must be run with the exact same code as the deployed verifier, otherwise proofs will not generate properly.

Local proof generation inner workings

You can test proof generation with your locally built circuits with the following command: LOCAL_PROVER=true RUST_LOG=info ./build/header_range prove input.json --wrapper-path ./verifier-build.

The input.json should be formatted like this and contain the bytes representation of the inputs to your circuit:

0x00000000001185f047c0c03cbecc6d7d2d40b3f59b440378bf1019fd754e2334a5c186f40bb0c035000000000011871c

The generated proof will be stored at: output.json in the folder you are running this command from. The output.json looks something like this:

{"type":"res_bytes","data":{"output":"0xe909d2c...","proof":"0x219a1b3446b1..."}}

On our platform, a given proof request has all of the information above, including the input bytes and output. An example is here: https://alpha.succinct.xyz/explorer/d3aeb11c-9534-400e-987d-5dbbc195b0d0. You can get the output by clicking View in the Output section.

Relay lifecycle

You should think of the SuccinctGateway as a really simple router contract that uses deployed verifiers to verify proofs and also handle callbacks on contracts that use the proofs.

When you submit this transaction, the SuccinctGateway does the following steps:

  • Uses the function_id to get the deployed verifier address and uses this address to verify the proof
  • Calls the callback address that will use the verified inputs/outputs of the proof through verifiedCall (logic here) to update its own internal state.
Last changed by 
 
Succinct Labs Uma + John
0
734
Published on HackMD