# LLVM Zisk backend

This is a summary of work done for building the LLVM backend for Zisk.

Assumes knowledge on how Zisk works.

For detailed log, see [CraneZisk Sync](/KRXifdQBTLmlQo-iQ_jXeQ) and [Status updates](/cr4cdKSKT4OZ_rKAPzK-wg)

The code with Zisk-related experiments is available at:
- https://github.com/aborg-dev/llvm-project/commit/315c4c94b2875b49fdbc60c8a462fbb7aea73525
- https://github.com/0xPolygonHermez/rust/tree/aborg/zisk_llvm

## High-level overview and goals

Zisk already uses LLVM through Rustc. The target `riscv64ima-polygon-ziskos-elf` corresponds to [LLVM RISCV backend](https://github.com/llvm/llvm-project/tree/main/llvm/lib/Target/RISCV).

The goal of the project is to generate Zisk bytecode directly from LLVM to bypass a few limitations of RISC-V:
- Move from limited 32 registers to *infinite* number of registers
- Allow direct operations with immediates up to 64 bits compared to only 12 bits in RISC-V

## High-level design and execution plan

LLVM backends are quite large and complex:
- RISC-V: 80k lines of code (45k C++, 30k TableGen)
- WebAssembly: 20k lines of code (17k C++, 3k TableGen)

So writing one from scratch is a lot of work. Instead we decided to fork RISC-V backend that can already be used to generate Zisk and iterate from there:
- Fork LLVM RISC-V backend (in reality just start modifying it in-place)
- Build Rust compiler with modified LLVM and use it as Zisk toolchain
- Modify the backend to emit bytecode that leverages Zisk for performance gains (but deviating from RISC-V ISA)

## Lessons learned

Onboarding to LLVM is a massive time commitment, at least 2 months.

Moving away from established conventions is very hard, a lot of interdependent code.

For any non-trivial idea, consulting with an active LLVM developer is highly recommended.

Best starting points seem to be existing "simpler" backends like RISC-V and WebAssembly.

## Ideas

### RISC-V

Zisk backend is based on RISC-V.

The code for the backend is located at https://github.com/aborg-dev/llvm-project/tree/rust-llvm/rustc/19.1-2024-12-03/llvm/lib/Target/RISCV.
A good overview of the backend is given in https://www.youtube.com/watch?v=AFaIP-dF-RA.

### WebAssembly

WebAssembly is a stack machine with a lot of similarities to Zisk

LLVM can target WebAssembly as a backend:
- https://llvm.org/devmtg/2015-10/slides/BastienGohman-WebAssembly-HereBeDragons.pdf
- https://github.com/aborg-dev/llvm-project/tree/rust-llvm/rustc/19.1-2024-12-03/llvm/lib/Target/WebAssembly

In particular, the ideas relevant for Zisk are
- Definition of registers: https://github.com/aborg-dev/llvm-project/blob/rust-llvm/rustc/19.1-2024-12-03/llvm/lib/Target/WebAssembly/WebAssemblyRegisterInfo.td
- Translation from virtual registers to WebAssembly stack slots: https://github.com/aborg-dev/llvm-project/blob/rust-llvm/rustc/19.1-2024-12-03/llvm/lib/Target/WebAssembly/WebAssemblyRegNumbering.cpp

## Development

Below is the set of technical recipies for building and using Zisk LLVM backend.

Codebase is available at:
- https://github.com/0xPolygonHermez/rust/tree/aborg/zisk_llvm
- https://github.com/aborg-dev/llvm-project
- For fresh start, use upstream https://github.com/rust-lang/llvm-project

### Setting up the project

The first step to set up the development environment is to:
- Fork Rust compiler https://github.com/0xPolygonHermez/rust
- Fork Rust LLVM fork https://github.com/rust-lang/llvm-project/tree/7e8c93c87c611f21d9bd95100563392f4c18bfe7

Relevant article: https://sourcecodeartisan.com/2020/09/13/llvm-backend-1.html

### Building Rust compiler with new LLVM

Rust needs to be built with the config that enables local LLVM build:

config.toml:

[llvm]
download-ci-llvm = false
optimize = false
static-libstdcpp = true
link-shared = true

[build]

configure-args = []

[install]
prefix = "rust/build/local"
sysconfdir = "rust/build/etc"

Then run:

DESTDIR=build ./x.py build && ./x.py install

Note, that this step will take from 30 minutes to 2 hours, depending on how powerful your machine is.

You can speed it up in the future by only rebuilding the LLVM library:

cd rust/build/x86_64-unknown-linux-gnu/llvm/build
DESTDIR="" cmake --build . --target install --config Release -- -j 32

cp ../lib/libLLVM.so.19.1-rust-1.83.0-nightly .../rust/build/host/stage2/lib

### Register new Rust toolchain

rustup toolchain link custom_zisk .../rust/build/host/stage2

### Build and run Zisk program using a new toolchain

Uses Zisk hello_world project:

cargo clean && cargo +custom_zisk build -Zbuild-std=panic_abort,std -Zbuild-std-features=compiler-builtins-mem --target riscv64ima-polygon-ziskos-elf --release
cargo run --bin ziskemu -- -i ../hello_world/build/input.bin -xm -e ../hello_world/target/riscv64ima-polygon-ziskos-elf/release/sha_hasher

For debug mode, use:

LLVM_DEBUG=1 RUST_BACKTRACE=full cargo clean && cargo +custom_zisk build -Zbuild-std=panic_abort,std -Zbuild-std-features=compiler-builtins-mem --target riscv64ima-polygon-ziskos-elf --release