zk-wave (now joined the EIP-X team)

Project Proposal

Initial objective: improve zkWASM test coverage.

EIP-X objectives: is contributing to EIP-X

Motivation

TBC

Project Description

Specification

TBC

Roadmap

TBC

gantt
    title Projects timeline
    dateFormat  YYYY-MM-DD
    
    section A
    ??                             :2023-08-14, 23d
    ??                             :2023-09-03, 30d
    ??                             :2023-10-08, 7d
    ??                             :2023-10-08, 7d
    ??                             :2023-10-15, 15d
    ??                             :2023-10-29, 7d
    
    section B
    ??                             :active, 2023-08-14, 23d

Possible Challenges

TBC

Goals

Initial objective:

  • Work on zkOracle open-source and open-protocols.
  • Building prototypes of zkOracle using a decentralized p2p networks.
  • Improving client architecture to incorporate zkOracle
  • Developing and improving low-level EVM mechanics where possible
  • Contributing to the public good.
  • Improve knowledge in weak areas and share the journey.

EIP-X objective:

Collaborators

Fellows

See https://github.com/sogolmalek/EIP-x#collaborators

Mentors

Resources

TODO

Schedule

Common

W0

  • Note: Start days of the week are shown below
  • 20230707 W0 Dev update & Office hours
    • Deep dive into identified areas
    • Learn and familiarize with previous work and current solutions.
    • Connect with chosen mentors and work towards defining a deliverable project scope of work
    • Write and submit this project proposal on the selected problem(s), suggested solutions, and a roadmap for working on the issue throughout the program.
    • Report presentation to other participants and select mentors to gather final feedback.

W1

  • 20230715 W1 Dev update & Standup call & Office hours
    • Contacted Suning Yao who confirmed that zkOracle that was mentioned at ethresear.ch has a solution being developed by @HyperOracle
    • Reviewed the whole zkWasm-C codebase but did not really understand it since I had not yet read the Hyper Oracle documentation https://docs.hyperoracle.io/. I was not able to run it because I could not find the project/ directory, so raised this issue https://github.com/DelphinusLab/zkWasm-C/issues/5, however more recently this week I discovered the project/ directory in a specific commit, so if time permits I will revisit that repository.
    • Read through the Hyper Oracle documentation here https://docs.hyperoracle.io/ and summarised it with hand-written notes to try to understand how it works so when I read more code implementations then hopefully it will make sense.
    • Initial thoughts on issues to contribute to add to the roadmap for working on throughout the program is to create examples, which should help reinforce how it works and help the community, and since they are open issues in the zkGraph repository here https://github.com/hyperoracle/zkgraph/issues that were only opened in the past few weeks, and since it is written in TypeScript it should be an achievable goal.

W2

  • 20230722 W2 Standup call & Office hours
    • Sick with flu from 20th July until Standup on 25th July so unfit to progress.
    • Participated in Encode ZK Bootcamp day 1-4 on 25-28th July.

W3

  • 20230731 W3 Dev update & Standup call & Office hours & Demo day
    • Participated in Encode ZK Bootcamp day 5 on 1st Aug.
    • Tried to use the Hyper Oracle zkgraph implementation. I created a PR here where I added dotenv and updated the readme file to show how to check if the network supported debug_getRawReceipts method and how to configure it to use Goerli network, however I encountered an error and created an issue in the repository here

W4

W5

W6

  • 20230822 W6 Standup call & Office hours
    • Sick still
    • Meeting to discuss EIP-X project by Sogol with Michael Elliot also in attendance
    • Created a PR https://github.com/a16z/helios/pull/262 for Helios project thanks to help from @ncitron from Helios to allow users to run it in a Docker container and improved the accessibility of understanding how to use the examples including configuration and documentation
    • Reading Axiom ZK
    • Requested to be early partner of Axiom https://docs.axiom.xyz/

W7

W8

Prototype P1 Plan Draft

  • EXIST - Axiom submits a request to Helios Light Client to query Ethereum (axiom-quickstart)
  • EXIST - Light Clients try to get canonical chain of blocks, which is the tip of the chain that most agree on through consensus
    • Light Clients use Light Ethereum Subprotocol (LES) communication protocol https://github.com/ethereum/devp2p/blob/master/caps/les.md and only download block headers as they appear and fetch other parts of the blockchain state on-demand from Full and Archive nodes
    • Light Clients follow the chain by asking for state from other peers and executing it as part of their own EVM
    • But if a Light Client user wants to do something like interact with a smart contract that has not been in the last 100 blocks, then they will not have the proof or the state for it, so then they additionally need to request the state for that smart contract, which are in contract state as mentioned here https://docs.axiom.xyz/developers/sending-a-query/finding-storage-slots by burdening a Full or Archive node, and that is where the Portal Network prototype comes in to allow them to request it from another Light Client, since Portal Network allows Light Clients to act as both client and server (instead of only client) and interact in a distributed fashion, but Portal Network is still a few years from completion.
    • Full and Archive nodes already have the state, they do not need the ZK proofs, since they sync the latest state with their peers.
    • Portal Network is solving Light Client problem since at the moment it is expensive for them to acquire the state from Full and Archive nodes because a lot of them have to ask them all the time, which is a heavy burden
    • Pre-state Example:
      • Given a transaction, where the state change is a change of account x balance by -100, and account y balance by +100. Assuming the pre-state account x balance is 100 ETH, and account y balance is 0 ETH that Full and Archive nodes know from genesis then they may provide that information to Light Clients. Light Clients are able to calculate the post-state of account balances themselves i.e. post-state account x balance is 0 ETH, account y balance is 100 ETH, and Full and Archive nodes do same thing. To be certain they are correct the Merkle Trie with the state is commited when they execute the block themselves in EVM knowing the state, and then in the next block the root block hash has a block header field called state root, and if that matches the state they already have then it means consensus agreed on the new state, but they may only be certain when it is a finalised block
  • EXIST - Light Client (Helios or similar) or similar provides block headers of a finalised block to Axiom
  • TODO P1 - Axiom's implementation that generates the ZK proof is https://github.com/ltfschoen/axiom-eth but although the Axiom team runs it as a server, the repository does not appear to include the server implementation
  • EXIST - Axiom Prover backend server generates ZK proof (axiom-quickstart)
  • TODO P1 - Axiom Prover backend server implementation logic should be modified to:
    • Determine the pre-state from the block headers of a specific latest finalised block, which includes only the state and list of account addresses that performed transactions in it necessary to execute that block.
    • Filter the pre-state to only include specific Merkle Trie leaves that contain a batch/list of accounts that we may specifically request (such as specific account abstraction smart contracts that may have emitted events)
    • Only generate ZK proof for the filtered part of account addresses that performed transactions in the pre-state of a block that was requested
      • Note: Do not need to generate ZK proofs for historical state accounts and transactions, only need the state for the current latest finalised block. By pre-state aka "witness" we mean all the accounts we need to execute the transaction in this specific block
  • TODO P1 - Axiom Prover backend server broadcasts the generate ZK proof of the pre-state for each finalised block header of each block so it may be received by other Light Client (Helios or Portal Network), as this is much for efficient, and is the opposite of how it currently works where they are all having to separately request from servers the same information.
    • Axiom Prover backend server(s) may already have a Light Client integrated
  • TODO P1 - Embed Axiom's proving and verifying logic into Light Clients
  • EXIST - Axiom Prover submits ZK proofs on-chain via p2p to Verifier smart contract on Ethereum (axiom-quickstart)
  • TODO P1 - Axiom should only submit ZK proof on-chain for parts of pre-state of that block to an Axiom Verifier smart contract.
    • Note: Axiom serves as the trustless state provider
  • TODO P1 - Light Client (Helios or Portal Network) to be modified to include a verifier function that listens for broadcast events containing the latest finalised block's ZK proof that is in the form of a Wire Protocol Standard to be received. Upon receipt handles the ZK proof using Axiom to get the pre-state verified using the Axiom Verifier, then take parts of the state that come with the ZK proof to execute the block themselves in EVM knowing the state transition is correct. See https://ethresear.ch/t/beacon-chain-light-client-networking/11063
    • Trin may already already use the Wire Protocol Standard to receive propogated messages, and have an EVM implement that we may reuse
  • EXIST - Axiom Verifier smart contract verifies the ZK proof
  • TODO P1 - Axiom Verifier smart contract https://github.com/axiom-crypto/axiom-v1-contracts/blob/main/contracts/AxiomV1.sol should only have to verify a ZK proof with only the parts of the pre-state that is necessary to execute the block. See https://docs.axiom.xyz/protocol-design/architecture-overview#fulfilling-queries-in-axiomv1query
  • EXIST - Light Client (Helios or Portal Network) then has the necessary pre-state and may execute it with their own EVM to put data on-chain on Ethereum
    • Note: They will already have the state transition and post-state as Light Clients are able to calculate the post-state themselves from the pre-state and the state transition. Therefore since Light Clients can do the state transition part themselves they do not need Axiom to include the state transition in the ZK proof. Axiom only proves historical data and historical state of accounts at certain block numbers. If we had to generate ZK proofs for state transitions (instead of just the pre-state) we would be doing the same as zkEVM and that is of a higher complexity and unnecessary
  • TODO - create Sequence Diagram https://mermaid.js.org/syntax/examples.html

W9

W10

W11

TODO Summary of Axiom Docs:

  • Steps

    • Configure Axiom SDK (JavaScript) to use JSON-RPC provider (i.e. local node, Alchemy, or Infura) where instantiating the Axiom class enables interactions with the Axiom SDK API and to interact with on-chain contracts

    • Obtain latest block headers. Specify query requests that filter the block headers and generate witness data that we need. Batch multiple query requests to Axiom.

      • EIP-X does not need to generate ZK proofs for all historical state accounts and transactions, we only need the state for the current latest finalised block. By pre-state aka "witness" we mean all the accounts we need to execute the transaction in this specific block

    • TODO Create front-end dApp (similar to https://demo.axiom.xyz, which uses smart contracts deployed using https://github.com/axiom-crypto/axiom-apps) instead of JSON-RPC queries using cURL

    • Query request (using cURL or front-end dApp) is encoded and sent as on-chain transaction by user for historic data from Ethereum block headers, accounts, and account storage via Axiom by building a query using the Axiom SDK (JavaScript) using the Axiom Query Format and submitting it with sendQuery on-chain to the AxiomV1Query smart contract. Payment of ETH is required for gas fees and proving fees.

      • Query allows access up to 8 pieces of historic information from any historic block, by querying information including historic blockNumber up to current block, Ethereum address, and storage slot in contract storage
      • Batch Query into Axiom allows a user to access up to 64 pieces of historic on-chain information simultaneously in a smart contract, where Batch Queries are identified by a serialized form of the query, and keccakQueryResponse as the Merkle-ized form of the query (may be uniquely determined from the query but contains additional on-chain history info)
      • Resources:

    • Query response. Wait for query fulfillment by monitoring its status using the Axiom Explorer.

      1. Off-chain prover of a proving server will index the query, generate the query response result using the Query Response Format, and generate a ZK validity proofs (proving its validity) in response to user queries
      2. Off-chain prover verifies that the ZK proof that was generated was correctly put on-chain. It does this by interacting with on-chain smart contracts, and generates the resulting keccakQueryResponse (the primary way to access results provided for each query along with poseidonQueryResponse) is written/committed on-chain to the AxiomV1Query smart contract storage in a Merkle-ized format (allowing Axiom to fulfill queries into large amounts of data without paying to store each piece in contract storage), allowing downstream applications to trustlessly access the query response results that were fulfilled on-chain

    • Read (using cURL or front-end dApp) and fetch witnesses to prepare an on-chain transaction to read the final value we are proving and verify query response results against smart contract storage trustlessly.

    • Submit the fetched witness data on-chain to execute a transaction to prove the final value. ETH is required for gas fees.

      • Note: The smart contract storage data has been previously verified using ZK proofs on-chain with the Axiom SDK (JavaScript) and may contain any historical Ethereum on-chain data that is encoded in block headers, states, transactions, and receipts. So we may access any historic Ethereum on-chain data trustlessly just like an archive node.

    • View the on-chain transaction once confirmed using Axiom Explorer.

    • Compute (verifiable compute) the ingested data by Axiom then apply verified compute primitives on top, where the validity of each computation is verified in a ZK proof.

      • Note: The verified compute primitives may include diverse operations from basic analytics (sum, count, max, min) to cryptography (signature verification, key aggregation) and machine learning (decision trees, linear regression, neural network inference).

    • Verification by Axiom accompanying each sendQuery result with a ZK proof of validity that is verified on-chain in the Axiom smart contract and the final query response results (with security cryptographically equivalent to that of Ethereum) is then trustlessly available for use in customer user smart contracts. The ZK proof proves:

      1. Requested input data was correctly fetched from the chain
      2. Compute was correctly applied

    • Operate on and use the final query response results that have been ZK-verified on-chain in our user application and custom smart contracts

  • References

W12

W13

W14

W15

W16

  • 20231107 W16 Final Dev update
    • Created offer_manual.rs script in https://github.com/ltfschoen/trin/pull/2
    • Meeting with Sogol and Noah on 4th Nov for 2 hours
      • Noah focussing on generating and verifying ZK, highlighted that the ZK is just a base64 format string that has a size of about 300kB
      • Luke to focus on creating PR to Trin that:
        • Test fixture with a large string to represent a ZK that has been generated by https://github.com/sogolmalek/zkevm-circuits/tree/main/mpt-witness-generator
        • Write tests using peertest to represent each Trin node, since get network timeouts when trying to send offers between actual Trin nodes at the moment
        • Wrap the ZK in a new message type to be transferred between Trin nodes, and to be used for verifying the ZK. It needs to store the associated finalised block number, the ZKP from the sending Trin node (which may not have been verified yet). Refer to Sogol's https://github.com/sogolmalek/EIP-x/tree/main/my_discv5_app and Luke's examples in the example/ folder at https://github.com/ltfschoen/trin/pull/2
        • Obtain latest block headers from History Network
        • Use State Network instead of History Network (i.e. StateContentKey) for dealing with the latest state since we are interested in the latest block and otherwise using cache, to to use associated ZKPs to verify the correctness of data across Trin nodes. See https://ethereum.github.io/trin/developers/architecture/workspaces.html#trin-state
        • Offer and check ZKP sent to other nearby Trin nodes
        • Find other Trin nodes with latest block ZKP
        • Verify the ZK by each Trin node that receives a ZK from another Trin node
        • Future allow user to adjust how often to send ZKP

W17

  • Meeting on 7 Nov 9:15-10am with Sogol and Noah
    • Find where to put our code in Trin and share with team
    • ZK proof to be provided from an external source via a Trin API endpoint to all the Trin nodes
      • So it will not be necessary to obtain it from the trin-history package block headers
      • Possibly also allow Trin nodes to broadcast or find a ZK proof directly from other nearby nodes since that may be a faster way to receive the ZK proof and obtain their consensus
    • Each Trin node would verify the ZK proof they have received
    • Each Trin node may not need to store the ZK proof in its DB, it may only need it long enough to verify the ZK proof, and then emit an on-chain event that indicates its approval or otherwise
    • Consensus vote amongst the majority of Trin nodes to determine whether to accept a specific ZK proof or not
    • trin-state package of Trin to verify the correctness of the ZK proof

Final Development Update

Link to separate document here

Ethconnect Conference

Devconnect Satellite event in Prague (for those not going to Istanbul) 15-21 Nov

Progress

Other Links

Select a repo