Cranelift ZisK backend

# Cranelift ZisK backend Author: [aborg](https://github.com/aborg-dev) Created on: 8 November 2024 Status: Implementation This document outlines the scope of work to build a new Cranelift backend that can lower WASM to ZisK. ## Plan ### 1. Prototype Rust -> WASM -> ZisK end-to-end conversion As a first step, we will build an end-to-end prototype to identify all critical decisions that we will need to make in a full solution (see "Open Questions" section below). The easiest way to do this would be to leverage existing Cranelift RISC-V backend and RISC-V -> ZisK conversion. This will allow us to evaluate the end-to-end solution without needing to build the most expensive part - a new Cranelift backend. I expect this to take 2-4 weeks of work. #### Code Changes Current WASM -> CLIF lowering expects Wasmtime VM environment, we need to change this to ZisK VM environment: - [wasmtime_environ::Compiler](https://github.com/bytecodealliance/wasmtime/blob/66910067642ce2ddf5509845306508f89a24fc9e/crates/cranelift/src/compiler.rs#L223) - Top-level class that coordinates translation of WASM functions into CLIF IR - Changes: Function signatures, call conventions, libcalls, trampolines - Size: 1k LoC - [FuncEnvironment](https://github.com/bytecodealliance/wasmtime/blob/66910067642ce2ddf5509845306508f89a24fc9e/crates/cranelift/src/func_environ.rs#L86) and [Call](https://github.com/bytecodealliance/wasmtime/blob/66910067642ce2ddf5509845306508f89a24fc9e/crates/cranelift/src/func_environ.rs#L1216) - Concrete implementation of translation of different WASM features into CLIF IR - Changes: Globals, Tables, Memory - Size: 3k LoC ### 2. Prototype direct CLIF -> ZisK backend that demonstrates performance gains Next, we want to demonstrate that the new backend can bring sizable performance gains. At this stage, it's fine to cut corners and support only a subset of functionality needed for our benchmarks. The primary benchmark will be [eth_block](https://github.com/aborg-dev/isa_benchmarks/blob/main/src/bin/eth_block.rs), but other examples from [isa_benchmarks](https://github.com/aborg-dev/isa_benchmarks) would work as the stepping stones. We are targeting 50% performance improvement compared to existing RISC-V backend. This will be a good point for go/no-go decision. I expect this to take 4-8 weeks of work. ### 3. Productionize CLIF -> ZisK backend In this stage, we will focus on turning the prototypes above into a production-ready, maintainable solution. This will likely require extending the implementation to support more programes, designing proper interfaces, writing user-facing documentation. I expect the bulk of the work to be in this stage. This would be a good point to onboard more people to this project. ## Open Questions - Which parts of WASM VM do we need to implement in ZisK? - WASM Cranelift programs [expect](https://bytecodealliance.zulipchat.com/#narrow/channel/217117-cranelift/topic/zkASM.20backend/near/392154720) to be executed in the VM - How will the Rust interface to access external (e.g. input and output) functions look like? - We can follow existing [pattern of `ziskos`](https://github.com/0xPolygonHermez/zisk/blob/develop/ziskos/entrypoint/src/lib.rs) to define these functions with inline assembly (what would it mean for WASM?) - Alternatively, we can leverage WebAssembly System Interface (WASI) and [implement](https://blog.dkwr.de/development/wasi-load-fd-write/) methods like `fd_read` and `fd_write` that will allow native Rust constructs like `io::stdin()` and `println!()` - Can we build the backend outside of `wasmtime`? - See [Externally maintained cranelift backends](https://github.com/bytecodealliance/wasmtime/issues/7124#issuecomment-1742766469) - We want to be able to use ISLE ## Prior work/Reference - [ZKWASM](https://hackmd.io/IqQMg6I6Qz-AEc8MSEU4eg) project - Took 2 engineer-months to run Fibonacci and 6 engineer months to run SHA256 - [Discussion about creating riscv32 backend](https://bytecodealliance.zulipchat.com/#narrow/channel/217117-cranelift/topic/Adding.20a.20RISC-V32.2032-bit.20Cranelift.20backend) and [project time estimates](https://github.com/bytecodealliance/wasmtime/issues/1183#issuecomment-1684159806) - Estimated to take 3 months with reviews and 20k lines of resulting code - [Steps to create a new backend](https://gist.github.com/nihalpasham/8301c865df690f5f293d93643f2efe8) - [Wasmtime architecture](https://github.com/bytecodealliance/wasmtime/blob/main/docs/contributing-architecture.md)