Grant Proposal

**Project Abstract**
uZumaKi aims to develop an open-source, community-driven benchmarking suite to address the critical need for standardized evaluation in zero-knowledge proof systems. Our comprehensive framework will improve the protocol to drive the wider adoption of ZK-based technologies, particularly in but not limited to the Ethereum Ecosystem. By harnessing data-driven insights and advocating for hardware-neutral metrics, uZumaKi can foster adoption, enhance the protocol and drive the adoption of ZK technologies more accessible.

**Objectives**
- **Establish Industry Standards**: Develop well-defined benchmarks and metrics for consistent evaluation of ZK-proof systems.
- **Deliver Data-Driven Insights**: Generate detailed UI/UX(Status and Alerts), reports and comprehensive analysis from the performance metrics data which can be helpful for protocol optimization and developer's choices. Reference for statistics and data to be parsed from performance metrics: https://shorturl.at/DISZ6
- **Foster Community-Centric Development**: Build a collaborative framework where the ZK community actively contributes and receives incentives for their contributions leading to continuous improvement and evolution of the benchmarking suite, keeping pace with rapid ZK innovations.
- **Promote Hardware-Agnostic**: To fairly compare STARK and SNARK systems across hardware platforms, focus on their core operations: FFT, polynomial operations, cryptographic hash functions for STARKs; and modular multiplication for SNARKs, crucial due to their reliance on elliptic curve cryptography.
- **Establish a ZK community learning platform**: To provide one stop learning resource for new developers entering the ZK ecosystem. The benchmarking suite will provide insights into the inner workings of different ZK systems, helping developers understand their performance characteristics and trade-offs along with detailed documentation, tutorials and educational materials accompanying the benchmarking results.

**Outcomes**
- **Comprehensive Insights and Analysis**:
- The suite's data-driven insights and comprehensive performance analysis will directly contribute to faster innovation cycles, leading to more robust, efficient, and secure ZK technologies.
- **Actionable UI/UX insights**:
- With detailed reports, analysis, graphs, charts, and track performance trends over time provided by uZumaKi, developers and organizations will have access to empirical data necessary for making informed decisions and optimizing both performance and resource utilization.
- **Enable community-driven development**:
- Open source development will bring essential standardization which will involve developers, protocol designers as well as the community to get aligned and efficiently perform over fragmented data.
- **Accelerate ZK Adoption**:
- By providing a comprehensive benchmarking suite and learning resources, uZumaKi will lower the barriers to entry for developers and encourage more projects to integrate ZK technologies and accelerate their adoption across a wide range of applications, from privacy-preserving DeFi to scalable Layer 2 solutions.

**Grant Scope**
- **Benchmark Design**: Research and development of a broad range of benchmarks covering:
- Proof Types: STARKs, SNARKs, etc.
- HLL/DSL(Rust, Noir), Intermediate Representation(VAMP-IR, Miden-IR), Circuit/Assembly/ByteCode - Miden Assembly, Risc-V
- Hardware: CPUs, GPUs, Hardware Providers(Ingonyama, Cysic) & cloud systems.
- Circuit Complexities: Simple to advanced, reflecting real-world scenarios.
- **Suite Development**: Creation of software tools and infrastructure to:
- Automate benchmark execution periodically.
- Collect, parse and visualize relevant performance data.
- Generate detailed reports and analyses.
- **Community Engagement**: Build a community of contributors through:
- Clear documentation.
- Collaboration channels (e.g., GitHub Discussions, Discord).
- Targeted outreach to protocol and industry.
- Incentivization mechanisms for external contributors (bounties, grants, etc.)
- **Collaboration with projects**:
- Work with multiple projects/protocols to parse, analyse and derive metrics from the custom circuits benchmarked by the protocol.
- Integrate benchmarking results into the uZumaKi learning platform
- Establish partnerships with hardware providers like ingonyama and cysic to benchmark on their hardware using Icicle and hardware native libraries for support proof systems.

**Current Benchmarking Stats**
*System Info:*
**CPU** : 8vCPU
**RAM** : 16 GB

We have parsed below metrics for variations of Fibonacci and SHA-256 hash program as input and have generated charts/graphs for comparison and visualization.
| System | Prover/Verifier Metrics |
| -------- | -------- |
| **Miden** | cycles, mean, median, stddev, memory, proof size, prover/verifier time
| **RiscZero** | mean, median, stddev, memory, prover/verifier time
| **Stone** | mean, median, times, stddev, user, system, min, max, prover/verifier time

Link to the graphs derived from benchmarking data: https://github.com/zk-bankai/uZumaKi/tree/main/benchmarks/graphs

**Project Team**
The uZumaKi benchmarking suite is being developed by a core team of 2 ZK researchers/engineers and 1 Community Lead:
- Aniket Prajapati - 160 hrs/mo
- Arun Jangra - 80 hrs/mo

**Background**

"Uzumaki is the community chosen zkBankai Hacker House winning project which was participated by industry leaders: Sandeep Nailwal, Bobbin Threadbare, Jordi Baylina, Remi Gai(Inko), Danilo(Kakarot) and many more"

Aniket Prajapati:
- Extensive experience developing backend solutions for nearly a decade.
- Active open-source contributor to multiple projects for Starknet ecosystem.
- https://github.com/keep-starknet-strange/madara/pull/1362
- https://github.com/paradigmxyz/reth/pull/4323
- https://github.com/kkrt-labs/kakarot-rpc/pull/471
- Recently completed an internship with LambdaClass as a Cryptography engineer where I contributed to cairo prover and cryptographic primitives.
- https://github.com/aniketpr01/elliptic-curve-cryptography
- https://github.com/aniketpr01/naive-rsa
- GitHub: https://github.com/aniketpr01/

Arun Jangra:
- I have been in the ZK space for 5 months and have been deep-diving into the core fundamentals of ZK.
- I am working as an intern in a Defi company which operates on Starknet.
- I started my journey as a cyber security enthusiast participating in international CTFs and getting into the top 10 ranks and eventually got interested in blockchain and I have been in blockchain space since 1+ year.
- Projects :
- https://github.com/AniDhumal/Yggdrasil
- https://github.com/ocdbytes/IPFS_DUMP_REPO
- https://github.com/ocdbytes/HyperSDK
- https://github.com/ocdbytes/ZK
- GitHub: https://github.com/ocdbytes

**Methodology**
1. Write circuits for Stark systems.
2. Start benchmarking the performance for Starks provers with different programs and input loads.
3. Measure and extract all the data required for efficient comparison with different provers, and provide a comprehensive in-depth report and analysis of the performance metrics data.
4. Add the benchmarking data to a SQL database, which can be later used for deriving time-series metrics and can help find anomalies and trends.
5. Create a CI/CD pipeline and automate using Github Actions, such that each run will be stored along with version number for efficient tracing of the system evolution over time.
6. Engage with community, work and collaborate with protocols, and create tasks that be assigned and tracked for task management.
7. Build a UI/UX that be helpful to visualize all the performance metrics, pros/cons and trade-offs in a single pane of view, which will be a bird's eye view of all the important metrics for the ZK system.
8. Publish results and gather community feedback.
9. Iterate and extend benchmarks to address new ZK innovations.
10. Contribute to standardization efforts and best practices.
- The same cycle will be followed for all the benchmarking lined up.

**Timeline**

Milestone 1: Initial Benchmarking Framework (Q2 2024)
- Modular Benchmark design for defining and running benchmarks.
- Develop open source STARK prover & Verifier Benchmarks.
- Develop simple comparative reports and analyses.
- Build feedback channels and actively involve core contributors to the community, and with the help of zkBankai, we can expedite the development from the initial stages by creating new tasks/issues, documentation, reviews and more.
- Deliverables:

| No. | Deliverable |
| -------- | -------- |
| 1 | Benchmarking framework |
| 2 | Initial STARK benchmarks |
| 3 | Generate Reports and Analyses |

Milestone 2: Expanded Benchmarks and Insights (Q3 2024)
- Deploy suite in diverse environments, collect data, and refine benchmarks based on feedback
- Expand STARK benchmarks to cover more proof systems and circuits
- Eastablish Partnership with Ingonyama, cysic for benchmarking on supported proof systems.
- Develop basic SNARK prover/verifier benchmarks
- Generate polished comparative reports and insights
- Grow community engagement and contributions
- Deliverables:
| No. | Deliverable |
| -------- | -------- |
| 1 | Complete STARK and initiate SNARK benchmarks |
| 2 | Detailed Insights |
| 3 | Increased Community Involvement |

Milestone 3: Standards Contributions and Maintenance (Q4 2024)
- Conduct hardware diversity and power efficiency analysis
- Benchmark primitives for fair comparison between STARK and SNARK hardware
- Engage with community, incorporate feedback, and conduct community call
- Develop and release library for generic STARK/SNARK benchmarking
- Contribute to ZK benchmarking standards and best practices
- Deliverables:
| No. | Deliverable |
| -------- | -------- |
| 1 | Cross-platform benchmarks |
| 2 | Community feedback integration |
| 3 | Build standards proposals |

Future Goals(Q1 2025):
- Integrate benchmarking results into ZK learning platform.
- Continuously update suite to track latest ZK research and innovations.
- Additional systems to be benchmarked like HLL/DSL, Compilers, Circuit/Assembly, ZKVM's, Sequencers, Lookup tables, Arithmetization, ZK-LLVM, FHE and more.

**Budget**
We are requesting a total of $75,000 in grant funding, allocated as follows:

| Engineer's Compensation | UI/UX Development | Compute costs | Conference expenses | Incentivization for community contributors |
| -------- | -------- | -------- | -------- | -------- |
| $30,000 | $5000 | $20,000 | $10,000 | $10,000

- Engineer compensation covers the core team's development and research efforts
- UI/UX development costs cover the creation of intuitive interfaces for the benchmarking suite and ZK learning platform
- Compute costs cover the hosting and runtime expenses for the suite, including specialized hardware as needed
- Conference expense to reach and present uzumaki at different events for better visibility and potential collaborations with protocols.
- External contributor incentives will be allocated towards offering grants, bounties, and rewards for community members who contribute significantly to the benchmarking suite

After the grant period, we will ensure the sustainability and maintenance of uZumaKi through a combination of community support, protocol partnerships, and dedicated funding from aligned organizations interested in advancing ZK benchmarking standards. By establishing uZumaKi as the go-to platform for ZK performance insights, we aim to attract long-term backing and contributions from the wider ecosystem.