# Run the Blobstream X Prover Locally ## Overview There are 3 parts to operating BlobstreamX (or for that matter any ZK-based system): * An **operator script** that is responsible for requesting proofs at a particular cadence (or based on some other condition) * A **prover** that generates proofs based on requests from the operator * A **relayer** that takes generated proofs and submits them onchain to the relevant smart contracts With the Succinct Platform, once you write your circuit and operator script, the platform will take care of the prover and relayer parts of the system, which is what our current `BlobstreamX` deployment does (https://alpha.succinct.xyz/celestia/blobstreamx). Our current setup uses the BlobstreamX operator script [here](https://github.com/succinctlabs/blobstreamx/blob/main/bin/blobstreamx.rs), which sends requests to our platform for proving + relaying. The local mode of the operator script will generate proofs locally, which this document covers. **Succinct Platform Concepts** The Succinct Platform has a concept of "releases" and "deployments". A "release" takes a circuit and builds it, storing the relevant artifacts that are used for proof generation. A "deployment" is tied to a specific release and is simply a deployed verifier contract onchain for the particular circuit. Each deployment has a "function_id" and contract address. The contract address is the address of the deployed verifier and the `function_id` is a unique identifier that the `SuccinctGateway` (our onchain entrypoint) uses to identify a circuit. # Local Prover Guide Local proving is run with Docker to ensure that it is replicable on any machine. However, to generate Blobstream X proofs (which are computationally intensive) we recommend a large instance such as AWS EC2's `r6a.16xlarge`. ## Clone the Repository Clone the BlobstreamX repo. You will use this repository to run the local prover continuously for a given chain. More instructions will be provided in a later section to run the local prover. ``` git clone https://github.com/succinctlabs/blobstreamx ``` The operator script run with `local` mode will generate proofs locally using the downloaded binaries. To configure it, set up the environment variables following `.env.example` and make sure to set `SUCCINCT_RPC_URL=local` which will generate proofs locally instead of sending requests to our platform. Create a folder locally where you will download all the built circuit artifacts and dependencies. ``` mkdir artifacts; cd artifacts; ``` ## Download artifacts Last updated: 04/03/24 (Ratan) Download the built circuit binaries from S3 (circuits have compiled binaries to generate proofs). The Blobstream circuit binaries are ~30GB, so downloading might take a bit. You can also build the binaries locally, though this is not recommended (instructions in the Misc section). ### Download Verifier (Groth16 Wrapper Circuit) ``` curl -L https://public-circuits.s3.amazonaws.com/verifier-build13.tar.gz | tar xz ``` ### Download Blobstream X Plonky2x Circuits Get the release ID's corresponding to the Celestia chain you want to prove from the table below. Note: Header Range with batch size 1024 is used on Arbitrum and Base. Blobstream X on Mainnet (deployment TBA) will use batch size 2048. | Network | Type | Batch Size | Release ID | URL | |------------------|--------------|------------|------------|----------------------------------------------------------| | Celestia Mainnet | Header Range | 1024 | c637ee72-f9cb-47ce-ae15-4b0d5d0e15df | https://alpha.succinct.xyz/celestia/blobstreamx/releases/33 | | Celestia Mainnet | Header Range | 2048 | 71fc43b9-2803-4f09-9134-84573f972ee1 | https://alpha.succinct.xyz/celestia/blobstreamx/releases/34 | | Celestia Mainnet | Next Header | - | 4992ff5a-9af4-4e71-8591-3cf7b6c07f5b | https://alpha.succinct.xyz/celestia/blobstreamx/releases/32 | | Mocha-4 Testnet | Header Range | 1024 | 044ac12a-e42f-406c-b64d-55c09ee5e2a6 | https://alpha.succinct.xyz/celestia/blobstreamx/releases/36 | | Mocha-4 Testnet | Next Header | - | f53a3885-ec02-4be7-bd37-bf70d29cd3c1 | https://alpha.succinct.xyz/celestia/blobstreamx/releases/35 | Download the Blobstream X artifacts from S3 into your artifacts folder. ``` curl -L https://public-blobstreamx-circuits.s3.amazonaws.com/{HEADER_RANGE_RELEASE_ID}.tar.gz | tar xz curl -L https://public-blobstreamx-circuits.s3.amazonaws.com/{NEXT_HEADER_RELEASE_ID}.tar.gz | tar xz ``` Mark the binaries as executable: ``` # Depending on which release you used, these commands might be slightly different. # Double-check your artifacts folder for the downloaded filenames. chmod +x header-range-1024/header_range_1024 chmod +x next-header/next_header ``` After these steps, you should have the following directory structure: ``` blobstreamx/ Cargo.lock Cargo.toml bin/ contracts/ ... artifacts/ verifier-build/... header-range-1024/header_range_1024 header-range-1024/main.circuit header-range-1024/{other circuit files} next-header/next_header next-header/main.circuit next-header/{other circuit files} ``` ## Environment Configuration Get the `HEADER_RANGE_FUNCTION_ID` and `NEXT_HEADER_FUNCTION_ID` corresponding to the Blobstream X contract you are relaying proofs to. Go to `Contract` -> `Read as Proxy` -> {`headerRangeFunctionId`, `nextHeaderFunctionId`} to get the function IDs. - [Example Contract](https://basescan.org/address/0xa83ca7775bc2889825bcdedffa5b758cf69e8794#readProxyContract) - Header Range Function ID: `0x46132c86ed84fdc655528f80f9291dd3116b04902036b96925edc78bbf52b8ca` - Next Header Function ID: `0x2ce8ca4f509cb09415b5a6ca6afa265571dac0b9f6ddb46f487e017fec71cf25` Set `.env` in `blobstreamx` to the following: ``` PRIVATE_KEY= # Private key for your relayer RPC_URL= # RPC for the chain you are relaying to. TENDERMINT_RPC_URL= # Celestia RPC for the chain you are relaying to. SUCCINCT_RPC_URL=local SUCCINCT_API_KEY="" # Leave blank for local proving. CHAIN_ID= # The chain ID you are relaying to. CONTRACT_ADDRESS= # Address of Blobstream X contract. # Optional Operator Parameters # Use the Function ID's for your contract here: LOCAL_PROVE_MODE=true LOCAL_RELAY_MODE=false PROVE_BINARY_{HEADER_RANGE_FUNCTION_ID}="./artifacts/header-range-1024/header_range_1024" PROVE_BINARY_{NEXT_HEADER_FUNCTION_ID}="./artifacts/next-header/next_header" WRAPPER_BINARY="./artifacts/verifier-build" ``` ### Local Prover Make sure your Docker daemon is running & pull the latest `succinct-local-prover` ``` docker info docker pull succinctlabs/succinct-local-prover ``` Then run the following to run the operator with local proof generation: ``` cargo run --bin blobstreamx --release ``` Proofs are stored as `output_{request_id}.json` in the `proofs/` folder. The local relayer will read this file to post proofs to the BlobstreamX smart contract. #### Proving Time On an AWS EC2 `r6a.16xlarge` instance, header range proofs take ~25 mins to generate. # Misc ## Relay an Existing Proof If you want to relay an existing proof in `/proofs`, run the following command: ```shell cargo run --bin local_relay --release -- --request-id {REQUEST_ID} ``` ## Local Circuit Builds [NOT RECOMMENDED] First, download the STARK-SNARK wrapper binary: ``` curl -L https://public-circuits.s3.amazonaws.com/verifier-build13.tar.gz | tar xz ``` Instead of downloading the Blobstream X circuit binaries from a remote source, you can build them locally by running `mkdir -p build && RUST_LOG=debug cargo run --bin header_range --release build && mv ./target/release/header_range ./build/header_range` at the top-level of the BlobstreamX repo. It is recommended you download the binaries directly instead of building locally, because you have to be careful that the build must be run with the exact same code as the deployed verifier, otherwise proofs will not generate properly. ## Local proof generation inner workings You can test proof generation with your locally built circuits with the following command: `LOCAL_PROVER=true RUST_LOG=info ./build/header_range prove input.json --wrapper-path ./verifier-build`. The `input.json` should be formatted like this and contain the bytes representation of the inputs to your circuit: ``` 0x00000000001185f047c0c03cbecc6d7d2d40b3f59b440378bf1019fd754e2334a5c186f40bb0c035000000000011871c ``` The generated proof will be stored at: `output.json` in the folder you are running this command from. The `output.json` looks something like this: ``` {"type":"res_bytes","data":{"output":"0xe909d2c...","proof":"0x219a1b3446b1..."}} ``` On our platform, a given proof request has all of the information above, including the `input` bytes and `output`. An example is here: https://alpha.succinct.xyz/explorer/d3aeb11c-9534-400e-987d-5dbbc195b0d0. You can get the output by clicking `View` in the `Output` section. ## Relay lifecycle You should think of the `SuccinctGateway` as a really simple router contract that uses deployed verifiers to verify proofs and also handle callbacks on contracts that use the proofs. When you submit this transaction, the SuccinctGateway does the following steps: * Uses the `function_id` to get the deployed verifier address and uses this address to verify the proof * Calls the callback address that will use the verified inputs/outputs of the proof through `verifiedCall` ([logic here](https://github.com/succinctlabs/blobstreamx/blob/main/contracts/src/BlobstreamX.sol#L148-L152)) to [update its own internal state](https://github.com/succinctlabs/blobstreamx/blob/main/contracts/src/BlobstreamX.sol#L167-L169).
Last changed by  
 
Succinct Labs Uma + John
363
Published on HackMD