CosmWasm Events State-Breakage

Currently, there is a bug that leads to gas being charged for newly added events and attributes in Cosmos SDK messages that are called from CosmWasm contracts

As a result, this prevents chain developers from adding useful events in a state-compatible manner.

For example, these events in the Osmosis concentrated liquidity module lead to a state-break due to diverging gas numbers in the v23.x release line.

Source 1

The first source of the state-break stems from the fact that wasmd does not correctly filter out events for StargateMsg here. As a result, the ReplyCosts function ends up charging gas for the newly added event attributes.

Source 2

The second source of the gas differences is affected by one of the following components:

• wasmvm
• cosmwasm
• Quasar CL vault contract

The problem starts when we call wasmvm.Reply() from x/wasmd. When called with new events, it returns a higher gasUsed value, causing a state-break. This call gets propagated all the way to wasm VM.

• The above log shows the gas differences between Osmosis run with the new events (left) and witout (right)
• Note that it also prints a structure called gasReport that internal wasmvm and cosmwasm logic operates on.

Below is the definition of the printed structure:

type GasReport struct {
	Limit          uint64
	Remaining      uint64
	UsedExternally uint64
	UsedInternally uint64
}

Tracing the executes down the stack, we see that
the divergence originates in the gasReport here.

The gasReport that differs is created by the call to instance.create_gas_report().

Which is implemented here

With the newly emitted event, the Remaining gas is smaller while UsedInternally is larger (see log diff screenshot.)

This is how we compute these values in CosmWasm to return back to Go:

GasReport {
    limit: state.gas_limit,
    remaining: gas_left,
    used_externally: state.externally_used_gas,
    // If externally_used_gas exceeds the gas limit, this will return 0.
    // no matter how much gas was used internally. But then we error with out of gas
    // anyways, and it does not matter much anymore where gas was spend.
    used_internally: state
        .gas_limit
        .saturating_sub(state.externally_used_gas)
        .saturating_sub(gas_left),
}

We can see that used_internally is computed from externally_used_gas, gas_limit and remaining.

Since remaining is the only one that changed based on logs, it is the cause of the divergence.

Next, we should evaluate why gas_left that is set to remaining differs.

This function returns the gas_left value from the cosmwasm layer of abstraction.

That in turn taps into the wasmer runtime APIs here. As a result, the gas difference origin due to newly addded events is confirmed to be from the cosmwasm runtime.

The following are the open theories:

• The submessage event response difference affects the cosmwasm gas numbers.
• A Quasar contract might be reading the response events and persisting them in state.

Based on the initial investigations of the affected message in the Quasar contract execute flow, the first theory is more likely since I do not see the contract storing any responses/chain events in state.

I would appreciate any pointers on this matter from Quasar or CosmWasm experts.

March 4 Update

The cause of the second source of the event state breakage stems from the fact that we push all events into the reply to cosmwasm contracts. Some internal logic charges gas for these additional events.

This is where it is charged. Stepping inside that call, we see instance.allocate() that makes wasmer runtime to consume internal gas.

I propose to avoid submitting any events to the smart contracts via reply

The proposal is non-trivial since there are already existing users relying on these events.