## The Problem: CGO breaks stuff and is slow Currently, CosmWasm is implemented on cosmos chains through a relatively complex stack: * [x/wasm](github.com/CosmWasm/wasmd) is imported to a chain and integrated into the chain's app wiring and lives in a repository containing a sample chain called wasmd. * wasmd imports [wasmvm](github.com/CosmWasm/wasmvm) * wasmvm contains bindings to allow go applications to call out to [wasmer](wasmer.io). * In order for the chain written with the [cosmos-sdk](github.com/cosmos/cosmos-sdk) to interface with [cosmwasm](github.com/cosmwasm/cosmwasm), wasmvm uses [cgo](https://dave.cheney.net/2016/01/18/cgo-is-not-go) Because it uses cgo, the chain->x/wasm->wasmvm->cosmwasm stack is needlessly complex. These complexity issues are in my opinion behind a number of performance and stability issues that have been seen in CosmWasm over time. At Notional, we've held a symposium of sorts for the past 8 months to move towards a pure Go wasm virtual machine that can run CosmWasm contacts without the performance and stability disadvantages that come with using cgo. Thus far, our efforts have failed, but here's the work that's been done so far: https://github.com/notional-labs/wasmd/tree/allinone The code I linked to above does not work right yet, and I am going to refresh it. The template for starting the work looks like: ```bash git clone https://github.com/cosmwasm/wasmd cd wasmd/x/wasm git clone https://github.com/cosmwasm/wasmvm rm -rf wasmvm/.git ``` ...and then doing a find and replace to change every import of wasmvm to your new repository. There are frankly still some questions about binary blobs, which are out of scope for this document, because our goal is to eliminate them permanently. This branch is where I'm merging things to. The rule is that this branch must always pass tests. ## Implementation paths ### Re-use wasmvm * https://github.com/notional-labs/wasmd/tree/wazerod-template - [x] remove the no cgo files, because we will make them unnecessary - [x] remove go.sum and go.mod, so it is no longer an independent module - [x] use github actions instead of circleci - [x] lint the repository and ensure that all tests pass, without changing over to wazero - [ ] swap out to wazero - [ ] refactor any logic with dependencies on cgo types ### Start fresh in the x/wasm keeper If we're starting fresh in go, then we would likely want to start from here: https://github.com/notional-labs/wasmd/blob/9a99aa21af31cbffa011d6fbac0c854b41bb6922/x/wasm/keeper/keeper_cgo.go#L38 ## The task So, the task at hand here is to remove calls to [wasmer](wasmer.io) and replace them with calls to [wazero](https://github.com/tetratelabs/wazero). There are also possible issues along the way, such as support for something like the cranelift compiler found in wasmer, but for now the goal should be to make a working prototype, as it is likely that we can enable different methods of compilation upstream in wazero if needed, since it is open source and completely written in go. ------- ## Case study: PebbleDB vs RocksDB in chains using CosmWasm The current default databse used in Cosmos is GoLevelDB, a leveldb that is written in pure Go. Ethereum used to use it, and has now shifted to PebbleDB as default. PebbleDB is RocksDB in pure Go. When Notional was in its early days relaying, we gained a signficant advantage by using RocksDB, but we noticed that this would disappear on CosmWasm chains. **Why did RocksDB provide no benefit when using a CosmWasm chain?** Well, basically cgo borks the garbage collector in go, and every cal out to cgo also messes with the concurrency features of Go. Pebble provided us with all of the advantages of RocksDB, without the need for CGO, and the performance was easily twice as good as using rocks, weather or not chains were using CosmWasm. Every CosmWasm chain is much more resource intensive than its pure Go equivalent (much higher cpu usage, 2-3x RAM usage) because of the effects that CGO has on two really critical features of Golang: * Garbage Collection * Concurrency In addition to this, CGO does not follow the Go type system, so types in CGO are easy to get slightly wrong.