HackMDOverview
This memo summarizes my preliminary work on supporting WebAssmebly-based pre-compiled contracts in Flashbots' SUAVE implementation. Two separate features are provided. These are
- (Demo 1) demonstrating the ability to compile an existing contract (MEV-Share's
ExtractHint) to WASM and execute it (see commit be61c5) , and; - (Demo 2) demonstrating ability of compiled WASM code to call a method on
suave.ConfidentialStoreBackend(see commit 9fdf66).
I review each of these features below, and include a short anaylysis of current capabilities, limitations, design choices and future work. These features should be considered fit for demonstration purposes only, and are not production-ready.
Demo 1: Compile ExtractHint to WASM and call in unit test
At a high level, there are two parts to this demo:
- Compiling
ExtractHint's business logic to WASM - Instantiating a Wazero runtime and using it to run the bytecode from step 1
This demo can be run by executing tests in the e2e package.
Compiling & Embedding WASM
In the spirit of a dead-simple demo, ExtractHint is implemented as a Go application in core/vm/internal/main.go. It is then compiled to core/vm/internal/main.wasm with the following command:
Note that this requires Go 1.21 to perform the compilation step, but that the host application (i.e. the one running the Wazero runtime) can be compiled with older versions. The bytecode is made available to the host application through the "embed" package, which allows the bytecode to be direclty embedded in the compiled binary. This strategy is recommended for builtin precompiled contracts. Other distributions strategies are discussed below.
Executing the Embedded WASM
The extractHint.runImpl method is modified to instantiate a Wazero runtime, configure the WebAssembly System Interface (WASI), load and compile the embedded guest code (i.e. the WASM produced in the previous step), and execute the main function. Input is provided by serializing call parameters to JSON and feeding these into stdin. Output is written to stdout and errors are written to stderr. A nonzero exit code signals that stderr contains data, which should be interpreted as an error string.
Demo 2: Calling Host-Supplied Method of ConfidentialStoreBackend.
This demo uses Wazergo) to define a WASM "host module", i.e. a collection of built-in functions that the guest code can call at will (conceptually similar to a syscall). The suavexec host module exports the retrieve method to the guest module, which accesses the confidential data-store.
To run this demo, ensure you have checked out the lthibault branch and run:
It is worth nothing that this is a low-level very interface. WASM/WASI programming is effectively systems programming; the abvailable APIs correspond to a subset of standard POSIX. The bulk of the work involves translating complex data types like []byte into the the WASM machine stack (implemented as []uint64). Variable-length data-types like strings are passed to host functions as (uint32, uint32) pairs representing an offset and size in WASM's linear memory. This linear memory is accessed as a raw []byte array by the host runtime, and can be manipulated directly. Crucially, because there is no reflection, performance approaches native speeds (though there is usually an O(n) copy involved). Zero-copy implementations are sometimes possible.
Next Steps & Possible Improvements
Re-Use of Runtime & Module Instances
The wazero.Runtime instances and modules can be instantiated ahead of time and kept in a free list to avoid setup costs. Note that with the compilation cache that is currently used, these costs are relatively small–less than 1ms for small contracts–but obviously compound over a large number of calls.
Define error types that satisfy interface{ Errno() uint32 }
This will allow us to print informative messages in host logs when a guest call returns an error. Currently, we are restricted to uint32 status codes.
Compile Multiple Contracts into a Single WASM Binary
A more aggressive strategy would be to compile multiple precompiles into the same WASM binary, and to export their respective function names. This achieves two things:
- Precompile A can call Precompile B directly (and with support for re-entrancy). This avoids the overhead of calling a host function (i.e. a native Go function that is exported from the host to the WASM guest). This latter point is conceptually equivalent to avoiding a syscall in a native POSIX process.
- Management of pre-compiles is simplified since only one "fat" blob of bytecode needs to be maintained and pooled. Various strategies can be employed to optimize the trade-off between binary size and performance.
One thing to note about re-using module instances is that we must be careful not to introduce global state in the WASM code, else this is potentially buggy/exploitable.
Distribute WASM Bytecode with BitSwap (or similar)
As @dmarzzz no doubt mentioned, I also have interest in expertise in P2P systems, and my prior work on Wetware is focused on the management of WASM processes in large P2P clusters. In this vein, I have been working on an approach to distributing WASM code that leverage's Protocol Labs' BitSwap protocol to distribute WASM code based on its content hash. A similiar strategy could be applied to publish, fetch and execute user-provided precompiles. In short: a precompile could be called by its "content identifier" (effectively, a crytpographic hash of its contents), whereupon BitSwap would fetch the corresponding bytecode from peers. Once downloaded, it can be executed normally.
Add support for byte-streams to wasmexec host module
It may be worth considering an approach similar to Wetware's for Suave. In essence, we define a low-level ABI that corresponds to Go's net.Conn, and then use this as a basis for more complex RPC. This would have the benefit of speeding up development and facilitating interaction with complex datatypes that may not have built-in Wazergo types (e.g. custom structs). I am a strong proponent of Cap'n Proto for such applications, as the inline encoding and memory layout keep the overhead extremely low. Incidentally, I am one of the core maintainers of the Go implementation.
Misc
A note about compiler support
Demo 2 uses a fixed-size buffer to move private store bytes in and out of the guest. While not ideal, it has the advantage of avoiding the need to export a malloc function, which maintains compatibility with Go 1.21. Exported functions are not yet supported in mainline Go, but are supported by the TinyGo compiler, which works very well with WASM.
Demos 1 & 2 can be compiled with either TinyGo or Go 1.21.
TinyGo:
Go 1.21:
