---
# System prepended metadata

title: General Overview

---

# General Overview

Discussions in [#188](https://github.com/near/wasmtime/issues/188) [#189](https://github.com/near/wasmtime/issues/189) [#191](https://github.com/near/wasmtime/issues/191) [#184](https://github.com/near/wasmtime/issues/184)




All our tests are generated from wasm spectest. 

They go through next pipeline:
- Compile to zkasm.
- Run the generated zkasm fill table with results.
- Track changes.
- Alert in GitHub PRs in case of degradation.

We use the snapshot-testing pattern in many places (TODO:link).

## Test Creation and Generation

**This is the most difficult part I guess. Maybe should be suspended before we create a interpreter.**

We use a snapshot-based pattern for generating Wasm from spectest wast files. We want automatically add tests if wasm spectest was updated.



## Wasm Compiling
**This should be migrated to filetest infra.**

We aim to have a single script that compiles all `.wat` files to zkasm and writes results to a CSV file in a snapshot-based pattern. This script should be capable of:
- Compiling a single file with verbose compiler messages?
- Recompiling all files and saving them.
- Compiling all files and checking if the results are the same as the committed files (used in CI).

I think script here should be a rust function with #[test] annotation.

## Running Compiled zkasm

We intend to implement a single command to run all zkasm and update the state table in a snapshot-based pattern. This script should be capable of:
- Running a single file with verbose zkasm processor messages.
- Rerunning all files and updating state files.
- Running all files and checking that the state is the same as in the state files (used in CI).

I think it should be bash or js script somewhere in `/ci`

For this stage I suggest to do only small improvements, it's quite ok already.

## Developer Tooling
We aim for easy, intuitive, and fixed commands for the following tasks:
- Recompile a single `.wat` file.
- Recompile all `.wat` files and update zkasm files.
- Compile all `.wat` files and check if the results are the same as in the zkasm files.
- Run a single zkasm file.
- Run all zkasm files and check the diff with the state.
- Run all zkasm files and update the state.
- Perform the entire CI pipeline locally.

## Testing Wasm
We have a script that tests all tests and benchmarks to ensure they are correct programs. It sets up the environment, runs Wasm, and checks if all programs finish with code 0.

I'm okay with it, but it should be refactored and placed in separate file.

## Accepting PRs
Our CI checks that tests and benchmarks are correct, that committed zkasm files correspond to files generated by the compiler, and that state files contain actual data.

We plan to implement a tool that automatically checks for any degradation in state files, for example test which passed in main branch and fails in feature branch.

I suggest to just write a gh action which writes to the PR if something was degradated.

Than, human read this information and makes a desicion to approve PR or not.

Pros:
- When human make final decision it automatically resolves all corner cases, such as adding\removing test, even corner cases we don't expect now

Cons:
- Human make decision == less automation & human factor. But, if bot write's diff only in case of quality degradation ('pass' => 'runtime error' or 'runtime error' => 'compilation error'), it's output will relatively small. We have 105 closed PR's now. As I remember, it was 1 or 2 PR's where we intentionaly commit code which reduce amount of passing tests. Thus, in majority of cases we expect bot output be empty. In very few cases we will still need to do manual review of failed tests, and it is possible that between 100 intentionally failed tests will be 1 unintentionally failed test

# Steps to Achieve "Finalized" Infrastructure
- [ ] Rewrite wast parsing to a generic approach.
- [ ] Reorder `.wat` and zkasm files: move all state.csv to one directory, consider removing hand-written tests, and possibly merge some tests into a single file.
- [ ] Introduce asserts as host functions in zkasm.
- [ ] Separate different parts of the infrastructure into different files.
- [ ] Add npm caching in CI.
- [ ] Disable unused tests in CI.
- [ ] Create a developer CLI tool and aim to stabilize it.
- [ ] Implement a GitHub action that alerts if any of our state.csv files degrade.

# Technical Details
Here we will provide a description of the implemented infrastructure and instructions on how to add new hand-written tests or benchmarks. TODO: This section will be updated as the major part of the infrastructure will be rewritten.
