HackMDHOW THE CERAMIC ANCHOR WORKS
Brief Introduction
Ceramic is a decentralized network for composable data. It helps make application building with Web3 data very easy. Ceramic makes data composable and reusable across all application by decentralizing application database.
Ceramic Anchor Service is a proof of concept implementation of an anchor service according to Ceramic specifications. The implementation currently uses the Ethereum block chain but is built in order to be blockchain agnostic. It is also easier to add more modules in order to support other blockchains.
Ceramic Anchor Network (CAS) is a hosted “layer-2” solution for generating anchor commits for many different stream transactions in a scalable, low cost manner. Ceramic nodes are responsible for sending anchor requests containing a Stream ID and Commit ID to a Ceramic Anchor Network, which then batches these transactions into a merkle tree, and includes the merkle root into a blockchain platform in a single transaction (currently Ethereum). After the transaction makes its way onto a blockchain, a Ceramic nodes creates an anchor which includes a reference to the blockchain transaction for every anchored stream. A Ceramic Anchor Service eliminate the need for each stream transaction to have its own corresponding blockchain transaction, which would be slower and more expensive.
CERAMIC ANCHOR SERVICE NETWORK
Networks are collection of Ceramic nodes that share specific configurations and communicate over a dedicated libp2p topic. Networks are discrete from each other and streams that exist on a network cannot be found on or moved to another network. It is not permitted to create personal Ceramic Anchor Service but use the free community operated CAS which requires no additional setup
There are three (3) main types of Ceramic Anchor Service network
(1) MAINNET: It is used for production deployment.
Nodes connected to Mainnet will automatically use this CAS which generates anchor commits using the Ethereum Mainnet blockchain (eip155:1) and communicates over the /ceramic/mainnet libp2p topic.
Mainnet usage is currently restricted to projects in the Early Launch program (ELP). You need to sign up for the ELP waitlist.
ETH Mainnet Address:
0xaf65E45F4C0BD388F91EeB23cFCd52F4fCdd6Ee2
To view Mainnet CAS transactions, see Etherscan.
(2) CLAY TESTNET: It is used by the community for application prototyping, development and testing official protocol release today
Nodes connected to Clay Testnet will automatically use this CAS which generates anchor commits using the Ethereum Ropsten blockchain
ETH Ropsten Address:
0x1C124c86f7fc22e67974337E889a513b16a5703f
To view Clay Testnet CAS transactions, see Etherscan.
(3) DEV UNSTABLE: It is used by Ceramic Core Protocol developers for testing new protocol features
Nodes connected to the Dev Unstable network will use this CAS which generates anchor commits using the Ethereum Rinkeby blockchain and communicates over the /ceramic/dev-unstable libp2p topic.
ETH Rinkeby Address:
0x41Ee0C359D95970A83229D8e9801cc2672390217
IPFS
Ceramic relies on a system called IPFS to connect and share data in ceramic networks. These IPFS runs as a separate process from the Ceramic Node itself, with each node connected to its dedicated IPFS node over HTTP. The Ceramic Daemon can launch an IPFS process automatically (referred to as “running IPFS in bundled mode” in the ceramic configuration file) which is designed for testing and local development only. For production deployment you’re meant to run your IPFS process manually and point your Ceramic node at it (referred to as “running IPFS in remote mode” in the ceramic configuration file). Thos gives room for more config options for your IPFS node allowing more controlled resource allocation as well as improved maintenance, debugging and observability.
Running IPFS in remote mode
Ceramic and IPFS will not automatically restart if they crash “You should configure your own restart mechanism and ensure DATA PERSISTENCE in between restarts.
To run a Ceramic node in production it is critical to persist the Ceramic State Store and IPFS datastore . The selected storage choice should also be configured for disaster recovery with data redundancy and backups or snapshotting because whenever these data is lost it might cause a permanent displacement of Ceramic Streams which will corrupt the node.
CERAMIC STATESTORE: It holds the state for pinned streams and act as a cache for the Ceramic Streams that your nodes create. You must pin streams you care about and also ensure that the statestore doesn’t get deleted to prevent the data created with your ceramic node from lost
IPFS DATASTORE: It stores the raw IPFS blocks that make up Ceramic streams to prevent data corruption, use environment variables written profile file or otherwise injected into your environment on start so the datastore location doesn’t change between reboots/restarts
NOTE: Before running the Ceramic Daemon always configure it to use IPFS in remote mode
How to setup a well-connected Ceramic Node
1) Connecting to Ceramic: The Ceramic daemon serves an HTTP API that clients use to interact with your Ceramic node. The default API port is 7007. Make sure this port is available to all clients you plan to use for your application.
2) Staying connected to IPFS: Ceramic nodes rely on IPFS for networking. IPFS nodes connect to each other using a Libp2p module called "switch" (aka "swarm"). This module operates over a websocket, on port 4011 by default. The websocket port must be accessible to the internet so your Ceramic node can be connected to the network.
Additionally, when running IPFS the IPFS API port must be accessible by the Ceramic node. The default API port is 5001. The IPFS node address will then be passed to Ceramic with the ipfs.host option in the Ceramic daemon config file.
3) Connect to the Ceramic anchor service: There are three anchor service and either the Eth mainnet or clay testnet can be connected to for projects as the dev unstable is only meant for testing new protocol features
(i) Eth Mainnet: For nodes that wish to connect to Ceramic mainnet, the node's IP address will have to be added to the allowlist for the Ceramic Anchor Service node operated by 3BoxLabs. Once you have fully configured your Ceramic node with this guide and have a way to persist its configuration and state, open an issue in the Ceramic Anchor Allowlist Repo with the public, static egress IP address for your Ceramic node, and a brief description of the data persistence setup for the multiaddress, Ceramic State Store, and IPFS Repo. Once your issue is closed, you will be connected to the Ceramic network and the Ceramic Anchor Service.
(ii) Clay Testnet: Nodes that wish to connect to other Ceramic networks, such as the clay testnet, do not need to do anything special to gain access to the Ceramic Anchor Service. The Anchor Service for the testnet-clay network is open to everyone and does not have an IP allowlist like the mainnet service does.
Mainnet nodes will not run immediately after start up until your pull request is reviewed and your IP address is added to the allow list for the 3Box Labs hosted anchor service.
4) Observability: Ceramic has a debug mode that you can enable using the –debug flag. This will allow you to see all logs printed to your console, including debug logs, API requests, events, and errors.
For observability, it is best to have these logs written to files to debug any issues and to generate metrics. Logging to files can be enabled with the logger.log-to-files config file option. The default location for logs is ~/.ceramic/logs but this path can be configured with the logger.log-directory config file option. Even without debug mode enabled you will still get critical logs and metrics written to files.
Request and event logs are written in logfmt. This makes them easy to import into Grafana dashboards using a log scraping agent like Promtail and a log aggregator like Loki, which can be used as a data source for Grafana.
