Rosetta Review

This page describes my initial findings after reading the Rosetta specifications, docs, looking at the examples, and coding a little bit.

What is Coinbase's Rosetta?

Rosetta is an open-source specification that makes integrating with blockchains simpler, faster, and more reliable. If a blockchain provides the Rosetta implementation, different services, and apps that can "talk" Rosetta would be able to integrate with that blockchain.

The main selling point is the Coinbase integration. Coinbase would like the blockchain developers to integrate with them by providing a Rosetta implementation. Coinbase isn't going to integrate with a specific blockchain implement. The blockchain devs (us) are doing the dirty work for them.

Coinbase integration is not the only selling point, once Symbol/Nis1 implements Rosetta, other (or our own) explorers/wallets/apps could easily integrate (in theory).

What is a Rosetta Implementation?

A Rosetta implementation is a docker container image that runs a rest server implementing the Rosetta Open API specification.

Some bullet points:

• The Open API is a POST-only RCP style specification. There are 23 endpoints to implements. The RCP style makes code generation simple.
• There is an online and offline mode. If the container runs in online mode, blockchain data can be queried and transactions can be announced. If the container runs in offline mode, stateless SDK-related operations like public key to address transformation, payload processing, and static configuration loading are executed. No connection to a node in offline mode.
• There are specific requirements about how to build and run the Docker container. The container itself can contain multiple internal services but they must all be started when the container starts. Only one mounted volume is allowed.

Implementation languages

Rosetta is just an open-source specification and has some docker (packaging) requirements. It doesn't force us to use any language. The Coinbase team does provide an official Golang SDK. The SDK provides utility functions and services to create Rosetta rest servers and clients. From what I'm seeing, most teams have used the GO SDK but there are also examples of Javascript and Rust.

Possible symbol implementations:

Rosetta implementation talking to an external node

The initial idea was to have a Rosetta Symbol service talking to an external node, like one of our Testnet or Mainnet nodes (or a Symbol node running in the exchange infrastructure). But it seems like the node needs to be built-in. Talking to an external node is actually a common mistake.

Some quotes:

To reiterate, any external services must be wholly contained in your single Dockerfile (should be started and stopped automatically)!

Our security team considers it an “integration blocker” to rely on some external party (some external node) for connectivity to the network

I'm not 100% about if external nodes are completely banned. Maybe Coinbase allows a Rosetta container to talk to a separate node hosted by Coinbase. I have asked that question.

Rosetta implementation with a built-in node

If having a separate container/compose running a Symbol node is not an option, we would need to squash a Symbol node next to the Rosetta implementation in the same Docker file/container.

These Rosetta examples do that, running a node and a Rosetta server next to each other.

https://github.com/coinbase/rosetta-bitcoin/blob/master/Dockerfile
https://github.com/coinbase/rosetta-ethereum/blob/master/Dockerfile

These are some initial options for built-in node:

Full dual node:

Squashing Catapult, Broker, RocksDB, Mongo, Recovery, Rest, and the Rosetta implementation (new) into one large container. Too big, too many points of failure. Rest needs to be extended adding or modifying some endpoints as they are not Rosetta friendly. e.g. Block by hash, or transaction hashes of a block.

Api only node:

Squashing Catapult, RocksDB, Recovery, and the Rosetta implementation (new) into one large container. The Rosetta implementation would query RocksDB and talk to Server API directly. We most likely need a RocksDB query/indexer service/component (new). Unsure how hard is to query and index RocksDB.

Api only node + light rest:

Squashing Catapult, RocksDB, Recovery, Ligh Rest (new), and the Rosetta implementation (new) into one large container. The Rosetta implementation would talk to a Light Rest service. The Light Rest implementation would query RocksDB and talk to the Server API. A light rest service could be used for regular node deployment, not just Rosetta. We most likely need a RocksDB query/indexer service/component (new).

Catapult extension:

A catapult C++ extention that exposes the Rossetta implementation by querying the RocksDB directly.

Custom/Go catapult client:

A brand new Catapult client that exposes the Rosetta endpoints. E.g. a GO catapult client that uses the Rosetta GO SDK.

Endpoint mappings.

TODO. In this section, I would like to talk about how the 23 Rossetta endpoints can be mapped to Symbol data and operations.

Future research, questions, TODOs:

• How symbol specific transactions like metadata, restrictions, etc can be implemented in Rossetta?
• How Symbol specific entities like restriction, metadata, etc can be used in Rosetta?
• Can a built-in node vote, harvest and collect harvesting rewards?
• How to create a single image for both testnet and mainnet nodes. How to setup the node configuration in the image, especially around dynamic/provided keys or customizations.
• Can a Symbol/Nis1-Rosetta be a first-class citizen? Would regular node admins want to run those kinds of nodes?
• What if we create explorers and wallets that understand Rosetta for our networks or any other Rosetta compatible subchain?
• How can Rosetta be extended by adding extra Symbol/Nis1 specific endpoints (like /node/unlockedaccount)?
• Understand how mempool and subchain works in Rosetta.
• Storage prunning and how can we delete old blocks.

Links

• Docs
• Community Not super active lately?
• Rosetta Go SDK

Examples:

• Bitcoin - Go
• Ethereum - Go
• Celo - Go
• Vechain - Javascript
• Diem - Rust
• Lots of examples you can see in the community form.

Blogs:

• Bitcon Rosetta
• Harmony

Test:

• Rosetta CLI To perform integration tests by hitting the different endpoints.
• LunarDev Cloud service hosting different Rosetta implementations. It can be used to see how other protocols behave.
• E2E test requirements

Docker:

• Multiple service question
• Why to run all in one container