Introduction to ABCI++ and Vote Extensions

Overview of ABCI++

With the release of v0.50 of the Cosmos SDK we get the latest version of CometBFT and the brand new ABCI++ interface.

Previously with ABCI, the consensus engine (CometBFT) would interact with the app (Cosmos SDK) at decision time when the block has already been created and its contents set. ABCI++ allows the application to intervene at three key places of consensus execution:

PrepareProposal: At the moment a new proposal is to be created, the proposer can incorporate application-dependent logic/work into the proposal creation process.

ProcessProposal: As a proposal is being validated, validators can perform application-dependent work and checks. This enables validators to evaluate the proposal's compatibility with the application's state and requirements.

ExtendVote and VerifyVoteExtensions: Applications can extend the voting process by requiring their validators to perform additional actions beyond simply validating blocks. The application can provide specific instructions or checks that validators must execute before casting their votes.

ABCI++ step-by-step

It's important to understand that the flow of block production is cyclical, given that some parts use information from the previous cycle (block). We are also going to ignore the fact that InitChain exists given that it's a method called only once and not part of the usual block production flow.

PrepareProposal

The first method being called is the one creating the block proposal and is called only on the selected validator as "block proposer" (selected by CometBFT, read more about this process here: https://docs.cometbft.com/v0.38/spec/consensus/proposer-selection).
This validator now is in charge of creating the next block by selecting the transactions from the mempool and possibly adding some extra transactions that the application needs.

An "extra transaction" could be the result of the vote extensions from the previous block, which are only available on the next height at which vote extensions were enabled. More on this in the practical example below.

This step may be non-deterministic.

ProcessProposal

This method is called on all validators, and they can verify if the proposed block is valid. In case an invalid block is being proposed validators can reject it, causing a new round of PrepareProposal to happen.
Rejecting a proposal has strong liveness implications for CometBFT so applications should accept proposals even if it, for example, contains some invalid transactions which will be ignored at execution time.

This step MUST be deterministic.

If your application has Optimistic Execution enabled, it will start processing blocks at this stage. Read more about it here: https://medium.com/the-interchain-foundation/optimistic-execution-landing-in-the-cosmos-sdk-a28fc72af650

ExtendVote

This method is called on all validators and in this step is when validators can do "extra work" on top of the usual block producing. For example, providing information from external APIs or key shares for a threshold encryption scheme. This information will be available on the next height (see PrepareProposal).

This step may be undeterministic, meaning that each validator can provide different inputs.

VerifyVoteExtension

During this method call validators will validate the vote extension provided by other validators. Here a vote can be rejected but CometBFT calls to have extra care when doing this and suggests that invalid vote extensions should be accepted at this step and ignored when processing them (in PrepareProposal). Similarly with ProcessProposal, if too many votes are rejected it will call for a new block to be proposed (thus delaying or even stopping block production).

This step must be deterministic.

FinalizeBlock

In this last step is where the execution of transactions happen (and in the context of Cosmos SDK, BeginBlock and EndBlock too).

This step must be deterministic.

Prerequisites

We expect the reader to have a chain project already working as we won't go through the steps of creating a new chain/module.

We also assume you are already familiar with the Cosmos SDK, if you are not we suggest you start with [CREATE MODULE/CHAIN tutorial] as ABCI++ is considered an advanced topic.

Overview of the project

We'll go through the creation of a simple price oracle module focusing on the vote extensions implementation, ignoring the details inside the price oracle itself.

We'll go through the implementation of:

1. ExtendVote to get information from external price APIs.
2. VerifyVoteExtension to check that the format of the provided votes is correct.
3. PrepareProposal to process the vote extensions from the previous block and include them into the proposal as a transaction.
4. ProcessProposal to check that the first transaction in the proposal is actually a "special tx" that contains the price information.
5. PreBlocker to make price information available during FinalizeBlock.

Implement ExtendVote

First we'll create the actual Vote Extension struct, this is the object that will be marshaled as bytes and signed by the validator.

In our example we'll use JSON to marshal the vote extension for simplicity but we recommend to find an encoding that produces a smaller output, given that large vote extensions could impact CometBFT's performance. Custom encodings and compressed bytes can be used out of the box.

// OracleVoteExtension defines the canonical vote extension structure.
type OracleVoteExtension struct {
	Height int64
	Prices map[string]math.LegacyDec
}

Then we'll create a VoteExtensionsHandler struct that contains everything we need to query for prices.

type VoteExtHandler struct {
	logger          log.Logger
	currentBlock    int64                            // current block height
	lastPriceSyncTS time.Time                        // last time we synced prices
	providerTimeout time.Duration                    // timeout for fetching prices from providers
	providers       map[string]Provider              // mapping of provider name to provider (e.g. Binance -> BinanceProvider)
	providerPairs   map[string][]keeper.CurrencyPair // mapping of provider name to supported pairs (e.g. Binance -> [ATOM/USD])

	Keeper keeper.Keeper // keeper of our oracle module
}

---

Finally, a function that returns sdk.ExtendVoteHandler is needed too, and this is where our vote extension logic will live.

func (h *VoteExtHandler) ExtendVoteHandler() sdk.ExtendVoteHandler {
    return func(ctx sdk.Context, req *abci.RequestExtendVote) (*abci.ResponseExtendVote, error) {
        // here we'd have a helper function that gets all the prices and does a weighted average using the volume of each market
        prices := h.getAllVolumeWeightedPrices()

        voteExt := OracleVoteExtension{
            Height: req.Height,
            Prices: prices,
        }
        
        bz, err := json.Marshal(voteExt)
        if err != nil {
            return nil, fmt.Errorf("failed to marshal vote extension: %w", err)
        }

        return &abci.ResponseExtendVote{VoteExtension: bz}, nil
    }
}

As you can see above, the creation of a vote extension is pretty simple and we just have to return bytes. CometBFT will handle the signing of these bytes for us. We ignored the process of getting the prices but you can see a more complete example here: https://github.com/facundomedica/oracle/blob/main/abci/vote_extensions.go

Implement VerifyVoteExtension

Here we'll do some simple checks like:

• is the vote extension unmarshaled correctly?
• is the vote extension for the right height?
• some other validation, for example, are the prices from this extension too deviated from my own prices? Or maybe checks that can detect malicious behavior.

func (h *VoteExtHandler) VerifyVoteExtensionHandler() sdk.VerifyVoteExtensionHandler {
    return func(ctx sdk.Context, req *abci.RequestVerifyVoteExtension) (*abci.ResponseVerifyVoteExtension, error) {
        var voteExt OracleVoteExtension
        err := json.Unmarshal(req.VoteExtension, &voteExt)
        if err != nil {
            return nil, fmt.Errorf("failed to unmarshal vote extension: %w", err)
        }
        
        if voteExt.Height != req.Height {
            return nil, fmt.Errorf("vote extension height does not match request height; expected: %d, got: %d", req.Height, voteExt.Height)
        }

        // Verify incoming prices from a validator are valid. Note, verification during
        // VerifyVoteExtensionHandler MUST be deterministic. For brevity and demo
        // purposes, we omit implementation.
        if err := h.verifyOraclePrices(ctx, voteExt.Prices); err != nil {
            return nil, fmt.Errorf("failed to verify oracle prices from validator %X: %w", req.ValidatorAddress, err)
        }

        return &abci.ResponseVerifyVoteExtension{Status: abci.ResponseVerifyVoteExtension_ACCEPT}, nil
    }
}

Implement PrepareProposal

For these methods we'll have a separate handler:

type ProposalHandler struct {
    logger   log.Logger
    keeper   keeper.Keeper // our oracle module keeper
    valStore baseapp.ValidatorStore // to get the current validators' pubkeys
}

And we create the struct for our "special tx", that will contain the prices and the votes so validators can later re-check in ProcessPRoposal that they get the same result than the block's proposer. With this we could also check if all the votes have been used by comparing the votes received in ProcessProposal.

TODO: Ask CometBFT team if this is valid. Check if the ExtendedVotes from PrepareProposal are from the same validators from ProcessProposal

type StakeWeightedPrices struct {
    StakeWeightedPrices map[string]math.LegacyDec
    ExtendedCommitInfo  abci.ExtendedCommitInfo
}

Now we create the PrepareProposalHandler. In this step we'll first check if the vote extensions' signatures are correct using a helper function called ValidateVoteExtensions from the baseapp pacakge.

func (h *ProposalHandler) PrepareProposal() sdk.PrepareProposalHandler {
    return func(ctx sdk.Context, req *abci.RequestPrepareProposal) (*abci.ResponsePrepareProposal, error) {
        err := baseapp.ValidateVoteExtensions(ctx, h.valStore, req.Height, ctx.ChainID(), req.LocalLastCommit)
        if err != nil {
            return nil, err
        }
...

Then we proceed to make the calculations only if the current height if higher than the height at which vote extensions have been enabled. Remember that vote extensions are made available to the block proposer on the next block at which they are produced/enabled.

...
        proposalTxs := req.Txs

        if req.Height > ctx.ConsensusParams().Abci.VoteExtensionsEnableHeight {
            stakeWeightedPrices, err := h.computeStakeWeightedOraclePrices(ctx, req.LocalLastCommit)
            if err != nil {
                return nil, errors.New("failed to compute stake-weighted oracle prices")
            }

            injectedVoteExtTx := StakeWeightedPrices{
                StakeWeightedPrices: stakeWeightedPrices,
                ExtendedCommitInfo:  req.LocalLastCommit,
            }
...

Finally we inject the result as a transaction at a specific location, usually at the beginning of the block:

...
            // NOTE: We use stdlib JSON encoding, but an application may choose to use
            // a performant mechanism. This is for demo purposes only.
            bz, err := json.Marshal(injectedVoteExtTx)
            if err != nil {
                h.logger.Error("failed to encode injected vote extension tx", "err", err)
                return nil, errors.New("failed to encode injected vote extension tx")
            }

            // Inject a "fake" tx into the proposal s.t. validators can decode, verify,
            // and store the canonical stake-weighted average prices.
            proposalTxs = append([][]byte{bz}, proposalTxs...)
        }

        return &abci.ResponsePrepareProposal{
            Txs: proposalTxs,
        }, nil
    }
}

Implement ProcessProposal

Now we can implement the method that all validators will execute to ensure the proposer is doing his work correctly.

Here, if vote extensions are enabled, we'll check if the tx at index 0 is an injected vote extension

func (h *ProposalHandler) ProcessProposal() sdk.ProcessProposalHandler {
    return func(ctx sdk.Context, req *abci.RequestProcessProposal) (*abci.ResponseProcessProposal, error) {
        if req.Height > ctx.ConsensusParams().Abci.VoteExtensionsEnableHeight {
            var injectedVoteExtTx StakeWeightedPrices
            if err := json.Unmarshal(req.Txs[0], &injectedVoteExtTx); err != nil {
                h.logger.Error("failed to decode injected vote extension tx", "err", err)
                return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT}, nil
            }
...

Then we re-validate the vote extensions signatures using
baseapp.ValidateVoteExtensions, re-calculate the results (just like in PrepareProposal) and compare them with the results we got from the injected tx.

            err := baseapp.ValidateVoteExtensions(ctx, h.valStore, req.Height, ctx.ChainID(), injectedVoteExtTx.ExtendedCommitInfo)
            if err != nil {
                return nil, err
            }

            // Verify the proposer's stake-weighted oracle prices by computing the same
            // calculation and comparing the results. We omit verification for brevity
            // and demo purposes.
            stakeWeightedPrices, err := h.computeStakeWeightedOraclePrices(ctx, injectedVoteExtTx.ExtendedCommitInfo)
            if err != nil {
                return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT}, nil
            }
            
            if err := compareOraclePrices(injectedVoteExtTx.StakeWeightedPrices, stakeWeightedPrices); err != nil {
                return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT}, nil
            }
        }

        return &abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_ACCEPT}, nil
    }
}

Important: In this example we avoided using the mempool and other basics, please refer to the DefaultProposalHandler for a complete implementation: https://github.com/cosmos/cosmos-sdk/blob/v0.50.1/baseapp/abci_utils.go

Implement PreBlocker

Now validators are extending their vote, verifying other votes and including the result in the block. But how do we actually make use of this result? This is done in the PreBlocker which is code that is run before any other code during FinalizeBlock so we make sure we make this information available to the chain and its modules during the entire block execution (from BeginBlock).

At this step we know that the injected tx is well-formatted and has been verified by the validators participating in consensus, so making use of it is straightforward. Just check if vote extensions are enabled, pick up the first transaction and use a method in your module's keeper to set the result.

func (h *ProposalHandler) PreBlocker(ctx sdk.Context, req *abci.RequestFinalizeBlock) (*sdk.ResponsePreBlock, error) {
    res := &sdk.ResponsePreBlock{}
    if len(req.Txs) == 0 {
        return res, nil
    }

    if req.Height > ctx.ConsensusParams().Abci.VoteExtensionsEnableHeight {
        var injectedVoteExtTx StakeWeightedPrices
        if err := json.Unmarshal(req.Txs[0], &injectedVoteExtTx); err != nil {
            h.logger.Error("failed to decode injected vote extension tx", "err", err)
            return nil, err
        }

        // set oracle prices using the passed in context, which will make these prices available in the current block
        if err := h.keeper.SetOraclePrices(ctx, injectedVoteExtTx.StakeWeightedPrices); err != nil {
            return nil, err
        }
    }
    return res, nil
}

Conclusion

ABCI++ and Vote Extensions IDK, write something here lol