# Aleo consensus scoping

## Design proposal

TL;DR: Implement an MVP of a blockchain that:

* Uses tendermint as a library for the consensus protocol
* Uses the intermediate layers of SnarkVM for program execution and verification
* Generates transactions with a basic local command-line program
* Only has validator nodes (no proving pools, delegators)
* Uses a basic implementation of proof of stake (no coinbase puzzle)
* Can be easily deployed, run and monitored in AWS or similar
* Doesn't implement finalize nor storage.

Risks/known limitations:
- no current reference use case to drive the blockchain implementation
- limited functionality will be exposed to implement programs at the beginning
- non-obvious implementation of staking for validators
- no mechanism to punish bad behavior from validators

The proposal is based in the questions and answers at the bottom of this doc, read on for more details.

## Short-term roadmap

Note: we will be moving these to our [trello board](https://trello.com/b/HrNY1Yd8/bft-consensus) and track progress there, this list will likely be outdated.

* [DONE] minimal prover/verifier code using SnarkVM
* [DONE] minimal local tendermint core cluster setup
* [IN PROGRESS] integrate minimal SnarkVM deploy/execute with a tendermint setup
* Research how the public credits model would work and whether it's feasible for us to adapt SnarkVM to it
* Document the MACI reference use case
* Document the user-defined tokens reference use case
* Document how converting public credits to private tokens would work
* Add transaction query capabilities to the client CLI
* Add disk persistence of the ledger (using tendermint built-in feature)
* Improve transaction validations
* Setup infrastructure for a non-local deploy of the cluster (aws or nomad)
* Write a cli helper to generate genesis blocks
* Add credit generation on blocks for validator rewards
* Add transaction fees
* Research how stake could be implemented and linked to the tendermint voting power
* Implement Proof of Stake / validator staking
* Add basic ledger info endpoints (may be thin wrappers around tendermint queries)
* Implement a non-trivial aleo application that can run on this setup
* Consider forking the explorer to work with this blockchain
* Automate deploys
* Add logging and monitoring (e.g. with newrelic or datadog)
* Add ledger backups
* Deploy a large-ish amount of nodes and stress them

This should be enough for an MVP, after having this in place and testing it (hopefully with some users), we should assess what can and can't be done with the features that we have, considering pending features (prooving pools, delegators, coinbase puzzle, storage, finalize, etc.) and decide what the best next steps would be.

## Questions

- [x] What is the user-facing functionality/use case we want to provide?
In other words, what problem are we solving? How exactly the decentralized and private building blocks that we provide could be used to implement a real-world application.
It's important to be able to lay out what the actors would be in each case, how they would interact with aleo (what programs they'd need to implement, what records to emit, etc.) in order to identify wether they are actually feasable, and which features are required to be implemented in the blockchain and in the vm to enable them.

Can we for example describe how we would implement the use case of one account proving something like age or salary range to another party without revealing the actual number?
Can we describe a financial use case, assuming aleo credits?

Coming up with concrete real-world examples is harder than originally anticipated, in part because it requires a mindshift, requires some assumptions about data and actors that we can expect to participate in the blockchain and also because we would need to spend some time designing aleo programs given a specific set of supported features. After discussing with the team, we conclude that we need to build an MVP and then see what can we build with it.

That being said, these are some potential applications that we could explore:
- privacy-preserving cash transactions (something like zcash, but also with arbitrary program logic)
- voting/MACI/tally
- proof of identity/age/residency/income, assuming some trusted authority issues the data as records.
- videogame tokens
- proof of honest videogame client logic

- [x] What are the main issues with the current (Aleo's) approach?
The Aleo consensus proposal as described [here](https://hackmd.io/iZanoXjITjy0KlKNHbrUAw) has a number of drawbacks:
* It's very complex
* It requires many components to be developed together to make it work: the bft consensus, the coinbase puzzle, the different types of nodes.
* the consesus algorithm is based on DiemBFT which is a somewhat recent protocol that hasn't been battletested in production.
* DiemBFT also doesn't have reusable implementations (our implementation would either need to reverse-engineer diembft code base or work our own implemenation from the papers).
* It would also not be a "vanilla" implementation of DiemBFT but also has some innovations that would make it harder to implement and test (see for example the [timeout scenarios](https://hackmd.io/iZanoXjITjy0KlKNHbrUAw#Scenario-1-The-target-is-reached-before-the-timeout) to consider).

We would prefer instead to have an MVP that can be constructed iteratively (the simplest thing that could possibly work first), and make it better, ideally based on lessons learned in a testnet with users. In this regard the approach is similar to the one in [this other proposal](https://hackmd.io/uPr85wbPQpirYLFxFOL43A).

- [x] What sort of transactions will the clients make?
We assume similar transactions as planned for SnarkOS and supported by SnarkVM:
* **Deploy**: sends a program to the blockchain, including the keys needed for the verification of the program function executions. If the program is valid and the deploy transaction gets committed to the ledger, the program can be subsequently be executed.
* **Execution**: the client runs a program locally taking inputs and generating outputs, and with a side-effect of generating an execution proof. The proof is sent to the blockchain as part of an Execution transaction. If the transaction is valid it will be committed to the ledger (which will include applying side-effects as changing records).

We assume that other operations such as solving a coinbase puzzle, making transfers and managing records can be implemented as program executions.

- [x] What sort of queries will the clients make?
i.e. once transactions are committed, what kinds of questions the system will need to answer about them?

* Assuming the deploy and execute transaction return a transaction ID, the clients should be able to query by the transaction id and learn if the transactions was committed (e.g. if the execution was accepted as valid and it's now in the ledger).
* Given a record stored off-chain, an account may query if it exists in the ledger as spent or unspent
* The node should expose endpoints for exploring the ledger, eg get latest N blocks

Assuming on-chain record storage, we may want to also support the case where a user provides a view key and the node uses it to return the list of records associated with the account (with the known caveat of the security issues of sending a key to an external server --the user could do this on a trusted environment, e.g. a locally running node).

- [X] What makes a transaction valid?
We can just take whatever SnarkVM is doing as a reference and use the same or a subset of those validations. For example: basic sanitization of inputs, deployment view key validity based on the associated ceritificates, execution proof validity according to view key, records spent actually existed and were unspent. See [here](https://github.com/Entropy1729/snarkVM/blob/2c4e282df46ed71c809fd4b49738fd78562354ac/vm/compiler/src/ledger/mod.rs#L674).

- [x] What vm will be used and which parts of it?
We will start by using SnarkVM:
- We will use the Entropy1729 fork, mainly to work against a fixed version and not get disturbed by ongoing breaking changes. Also to have freedom to make changes to it should it be necessary.
- We will not rely on SnarkOS nor the current ledger-related components of SnarkVM.
- We will likely not use the current SnarkVM high-level definition of a Transaction. We will, at least at the beginning, use the lower-level Execution and Deployment structs, as well as the "Process" struct which currently containes most of the execution and verifying logic. We may eventually need to break these abstractions and customize them.

See [this document](https://hackmd.io/7Yca2ef8Ro2uAPEI5fkeow) for a reference on the SnarkVM components.

On the mid-term we expect to switch from SnarkVM to an arkworks based VM that will be developed separately (we may consider supporting both if it's not too much of an effort).

- [ ] What sort of information will be stored in the ledger?
- [ ] what are it's privacy features, e.g. who can read it?

for example, will execution inputs and outputs live in the ledger?
can they be retrieved from it? are they ciphered? can they be deciphered only by the account that generated the execution?

TODO we need to dive a bit more in snarkvm to understand how this is currently designed

- [X] How will the ledger be persisted?
Tendermint Core provides functionality to index transactions in order to be able to query them accordingly. By default, it does so in a KV store but it also has built-in PostgreSQL functionality. It defines a way to index them by default events, but custom events can be created. This allows you to query at the transaction level or at the block level, and also subscribe to transactions as well. Reference: https://docs.tendermint.com/v0.34/app-dev/indexing-transactions.html and https://docs.tendermint.com/v0.34/tendermint-core/running-in-production.html
If we were to use our own Tendermint implementation, we could build specific functionality.
In terms of how the transactions actually get recorded, committed blocks contain a list of them, and they are arbitrary byte arrays: https://github.com/tendermint/spec/blob/8dd2ed4c6fe12459edeb9b783bdaaaeb590ec15c/spec/core/data_structures.md

- [ ] How are records generated, used and consumed?
TODO research current snarkvm design for this

- [X] What are the system components?

For the first iteration, the system will be composed by off-chain proving clients (which can be just simple command-line Rust applications) and Tendermint blockchain validators aka verifiers.

A longer-term implementation can incorporate proving pools (which we can assume are just beefy clients from the point of view of the blockchain), delegators (that don't participate in as validators in the consensus protocol vote delegate stake to validators and gain a portion of the reward), read-only clients and maybe others. We won't worry about these for now.

Note that these conceptual framework doesn't account for the specific nodes supported/required by tendermint. In practice we may have other types of nodes, for example Seeds for discovery or Sentry nodes for security of the tendermint cluster.

- [x] How will the consensus be implemented?
We will start by using tendermint core as a library and implementing the hooks that implement the aleo-specific logic in Rust, calling SnarkVM.
Once that is working we can consider moving customizing the consensus implementation according to lessons learned by running with users and limitations we may encounter.
- [x] Does the blockchain have an associated currency?
Yes, this is necessary at the very least for making the consensus protocol run (for nodes to have stake), validators to have an incentive and likely to support financial use cases. We assume we will support the aleo credits/gates as currently exposed in Aleo instructions.
- [x] Do transactions cost a fee? who keeps it? what's the mechanism for clients to pay it?
We assume that transactions can include a fee, which should be variable and optional. It shouldn't be a requirement to include a fee for the basic consensus mechanism to be exercised, but we can add it eventually to the MVP.
We will likely add a faucet for testing fees and credits-related programs.
- [x] Are there coinbase transactions or similar rewards for proposer nodes?

We need some way of generating credits as part of the protocol and giving it to the proposer or to the entire set of validators as a transaction included in each block is a way to do it. But we won't be implementing a PoW-like coinbase puzzle at least in the short term. We assume that should this be necessary later on it can be added as an external program.

Note that not having a puzzle and not having slashing leaves room for the validator to break the protocol without a cost.

- [X] How do we implement proof of stake?
In Tendermint Core, all validators have an associated voting power. There is no explicit requirement that forces applications to use PoS, but it can be implemented. Docs do not include examples of how it could be implemented (there could be initial token offerings and transactions to stake/unstake tokens).

From https://docs.tendermint.com/v0.34/tendermint-core/validators.html:
> Tendermint has a different approach to block creation. Validators are expected to be online, and the set of validators is permissioned/curated by some external process. Proof-of-stake is not required, but can be implemented on top of Tendermint consensus. That is, validators may be required to post collateral on-chain, off-chain, or may not be required to post any collateral at all. Validators have a cryptographic key-pair and an associated amount of "voting power". Voting power need not be the same.

Cosmos (which is built around tendermint) has a staking module that may serve as a reference: https://docs.cosmos.network/v0.46/modules/staking/

Note that it's not obvious how aleo validator nodes can be made to stake, so this will need some research time. We need to make the Tendermint voting power proportional to the validator staked credits, so we need some mechanism for a validator account to stake credits (which are presumably private) and make the network aware of this transaction, perhaps using a proof of some program execution.

As an initial implementation, we can have a fixed set of validators and give them equal voting power.

- [x] Do we need to consider how to bootstrap the network?
We may need to worry about this for a long term mainnet, but to get started we can just have a tool to generate arbitrary genesis blocks (e.g. assigning credits to a fixed set of initial accounts)

- [x] With the minimal assumptions described above, do we have the proper incentives for a testnet (and eventually mainnet) to work?
Assuming we will only have validator nodes participating in consensus, they will have an incentive to validate transactions (to gain the coinbase transactions --even without puzzle-- and the transactions fees). As noted, though, there may not be enough incentives to honor the protocol (no slashing and no cost in proposing multiple blocks), but we consider this is not something that we need to solve in the initial MVP.

- [X] Will this be a permissioned or permissionless blockchain?
e.g. can third parties with the proper hardware run a node, join the network, and eventually participate in the staking?

Tendermint Core provides functionality for third parties can join in. There is a discovery protocol (peer exchange):
- Peer discovery: https://github.com/tendermint/tendermint/blob/v0.34.x/spec/p2p/node.md
- Peer docs: https://github.com/tendermint/tendermint/blob/v0.34.x/spec/p2p/peer.md
- PEX message spec (peer exchange/discovery): https://github.com/tendermint/tendermint/blob/v0.34.x/spec/p2p/messages/pex.md
- PEX reactor (for some reason this was deleted so take it with a grain of salt, but it contains valuable info): https://github.com/tendermint/tendermint/blob/1b2b24055c5e44c41347cd524a713f0eddb7881c/spec/reactors/pex/pex.md

- [X] How do nodes get added or replaced? how do they learn the ledger history and setup its state?
Some of the docs above detail some of the functionality related to discovery and sync. Furthermore, Tendermint Core provides a couple different ways for nodes to sync the blockchain: https://docs.tendermint.com/v0.34/tendermint-core/fast-sync.html

- [X] Do we need extra security measures or components to make it permissionless?
does tendermint already give us support for this?
is there any drawback with this approach?

Tendermint has functionality to detect faults, and gossip them through the network. It seems that when enough evidence is collected, it will be committed to the chain. This Evidence data is also sent to the ABCI app as well. https://github.com/tendermint/tendermint/blob/v0.34.x/spec/consensus/evidence.md
> It is worth noting that the evidence system is designed purely to detect possible attacks, gossip them, commit them on chain and inform the application running on top of Tendermint. Evidence in itself does not punish "bad actors", this is left to the discretion of the application. A common form of punishment is slashing where the validators that were caught violating the protocol have all or a portion of their voting power removed. Evidence, given the assumption that 1/3+ of the network is still byzantine, is susceptible to censorship and should therefore be considered added security on a "best effort" basis.

Based on that, if we want to make the network secure for production we will eventually have to implement slashing ourselves. The cosmos implementation (that's built around tendermint) may serve as a reference:
https://docs.cosmos.network/v0.46/modules/slashing/

Slashing won't be part of the scope of the short-term MVP.

- [x] Do we have on-chain storage?

Fore the initial implementation, we assume that we won't support on-chain storage, and see how far we can go in terms of application usability with that. We can consider adding storage later on.

Note that we assume that at least we can store record commitments in the ledger. Which should be provide a basic degree of state, while the clients are in charge of tracking their records, they cannot just made them up, the ledger will be able to validate them. In addition, if it's not a big extra effort to actually store the entire record values in the ledger (not just their commitments) we will do that.

- [x] Do we need to keep current aleo notion of record, transaction and transition?
At least at the beginning we will, since they are already available and they seem to work to some degree. We will revisit if when find limitations with the model whil we're implementing the ledger, or when we decide to move to a different VM.
- [x] What is finalize? do we need it? how does it affect the implementation of the blockchain?
Finalize refers to an aleo instructions feature to instruct parts of a program to be executed on-chain. Because transactions and records are private by default, this is necessary for programs that have shared state between accounts (for example a swap).
We won't support this in the short-term implementation.

- [x] What would current applications (wallet, roulette, explorer) look like with this implementation?
- Some sort of explorer should be easily supported almost by definition: it will expose the ledger to whatever degree available.
- The Roulette should be easily supported (assuming we continue to be lenient about the randomness) since it doesn't require storage.
- We can assume deploy and execute, as well as transfers, so we should be able to support some form of a wallet, assuming the state is stored in the client.