## **:warning: THIS IS A MOVING AND INCOMPLETE DOCUMENT! EXPECT IT TO BREAK!** :warning: The following is a ~~concise~~ tutorial which will cover the following: - An overview of the Polkadot SDK - more specifically, its **templates** - An overview of coretime, and what it means to *deploy* on a core - The development pipeline is going from a template to a core looks like - How and where to procure a core - Configuring, compiling, and using a template - Deploying on our core > Please note: **Agile Coretime** is only available on the following networks for now: > > - ***Kusama*** - The Canary Network > - ***Rococo*** - Testnet. > > View more on Polkadot's networks [here](https://wiki.polkadot.network/docs/build-network-overview). This tutorial will use **Rococo** for testing purposes. **It is advised to start syncing your Rococo DB as soon as possible**, using the `fast-unsafe` option. Your parachain will need to run a **collator**, which passes data from your parachain to the relay chain to be verified. The collator is really like your parachain's validator, only it moreso plays the role of producing blocks and sending state changes to the relay chain. It is possible for it to take some time to sync Rococo (12 hours - 1 day), depending on your internet connection, unless you can get your hands on a snapshot of the DB. However, there is quite a bit of setup you can do meanwhile until you have to sync your node! Without further ado, let's get started. (if you don't care about the SDK or Coretime and want to jump straight to the tutorial, [click here](#Going-from-SDK--gt-Core)). ## A Primer on the Polkadot SDK At first glance, the Polkadot SDK can be rather overwhelming, and in a way it is - it packs a lot of tech into one place. The Polkadot SDK used to be overarching **three** respositories: - **Polkadot** - Which for a time, included both the client implementation and runtime, until the runtime was moved to the Polkadot Fellows organization. - **Substrate** - The underlying, core primitives and libraries for building blockchains (any blockchain, not just one for Polkadot). Much of Polkadot is built with Substrate! - **Cumulus** - A set of libraries and tools which pertain specifically with connecting blockchains to Polkadot, known as parachains. For our purposes, we're really only interested in a couple of specific parts: - [FRAME](https://paritytech.github.io/polkadot-sdk/master/polkadot_sdk_docs/polkadot_sdk/frame_runtime/index.html) - a framework which makes using Substrate easy. Using FRAME and Substrate, we can create a *runtime*, which is made of runtime modules called *pallets*, and boom, we can build a distributed ledger with custom business logic. - The templates within [`templates/`](https://paritytech.github.io/polkadot-sdk/master/polkadot_sdk_docs/polkadot_sdk/templates/index.html). > For an in-depth dive into the monorepo, I highly recommend you look into the [Polkadot SDK Docs, which explains everything.](https://paritytech.github.io/polkadot-sdk/master/polkadot_sdk_docs/polkadot_sdk/index.html) These are our tools; using them, we can build a completely custom blockchain, which we can then deploy on Polkadot. ## What in the Coretime!? Before explaining what **Agile Coretime** is, a little history: - On Polkadot, one can utilize the security of the Polkadot network by deploying a **parachain**. - **Polkadot** then verifies the state transitions of each parachain. This means that Polkadot collectively represents the states of many blockchains. Other perks, such as interoperability, made this an attractive option for those wanting to build layer one blockchains. - However, in order to participate, each parachain had to go through an auction process to secure a **slot**. This proved to have a barrier to entry for those wanting to build. - Parachain slots also proved to be costly - as you effectively had to pay for an amount of *blockspace* to use. Most parachains chose to have a 2 year slot, meaning at the end of 2 years, they have to renew their slot. Agile Coretime changes this paradigm completely. Cores in agile coretime can be thought of an abstraction over what once was a parachain slot. Cores allow one to upload and run code to the Polkadot Supercomputer. This paradigm completely changes the way we think of running "L1" blockchains, as it is really as simple as uploading the code and genesis to the relay chain, almost akin to a smart contract. Here is an excerpt from the Polkadot Wiki, which explains in more detail: >Agile Coretime enables efficient utilization of Polkadot network resources and provides economic flexibility for builders, generalizing Polkadot beyond what was initially proposed and envisioned in its whitepaper. > > In Polkadot 1.0, the only way for a parachain to be secured by Polkadot was to rent a lease through an auction, which guaranteed parachain block validation for up-to two years. This involved locking significant amount of DOT, leading to a high barrier of entry for small and medium scale blockchain projects. The parachains produced blocks at a regular interval of 12 seconds, irrespective of the network activity. This led to inefficient resource allocation and misplaced economic incentives while producing almost empty blocks under lighter traffic and being unable to accommodate heavier traffic when it exceeded block limits. Agile Coretime resolves all of these drawbacks. - [*Intro to Agile Coretime, Polkadot Wiki*](https://wiki.polkadot.network/docs/learn-agile-coretime) You may also see the term "task" being used in the context of coretime. In theory, a task does not have to be a blockchain/parachain. So long as it's able to fulfill the requirements of the Polkadot protocol, then it can be a task. ## Going from SDK -> Core The development pipeline from the Polkadot SDK to a Core is actually quite simple: 1. **Reserving** a `ParaId`, where you will upload your runtime and genesis state. 2. **Compiling** the runtime (written in Rust) to a WebAssembly blob, thereby defining how your state transitions from one state to the next. This runtime is created using the Polkadot SDK. 2a. **Ensure** your chain spec is viable and ready to be deployed as a live, working parachain. 3. **Generating** your genesis state and wasm. 4. **Obtaining** a core, most likely through a [Coretime marketplace](https://wiki.polkadot.network/docs/learn-guides-coretime-marketplaces). 5. **Assigning** that core to your `ParaId`. 6. **Ensuring** you have at least one honest, synced collator for your task By the end of this process, you should have a fully syncing, and working blockchain that is secured by the overarching security of whatever relay chain (Kusama, or Rococo) it is running on. The following diagram is a visualization of this process, although the order may be slightly different (i.e., one should have a ParaId reserved before starting): ```mermaid --- title: SDK to Core --- flowchart TD subgraph GA["Generate Artifacts"] direction LR A["Creating a runtime"]-->B["Compiling to Wasm"]-->C["Generate Genesis State"] end subgraph PC["Procure ParaId & Core"] direction LR PARAID["Reserve ParaId"] PARAID-->D["Buy Bulk Coretime"] PARAID-->E["Issue On-Demand Coretime Extrinsic"] end subgraph DEP["Deploying"] direction LR F["Register artifacts to ParaId"]-->assign["Assign Core"]-->G["Sync collator"]-->H["Begin generating blocks!"] end GA-->PC PC-->DEP ``` ## Getting ROC and Reserving a ParaId Head over to Polkadot.js to reserve a ParaId. We'll need a ParaId to upload our parachain's code: 1. Get the [Polkadot.js Web Extension.](https://polkadot.js.org/extension/) 3. Create a wallet, and get some [ROC via the faucet.](https://faucet.polkadot.io/) with your new address 2. Make sure you select **Rococo** as your network via the tabs on the side 4. Head to [Network > Parachains > Parathreads (the tab)](https://polkadot.js.org/apps/#/parachains/parathreads) ## Creating our Chain & Generating Artifiacts We can now move on to actually starting our project. Some important prerequisites are: 1. **Have** Rust and its associated tooling installed. 2. **Install** the nightly version. 3. **Have** all a command line, git, and other common development tools at your disposal. If the above list sounds confusing, make sure to visit the [Build](https://wiki.polkadot.network/docs/build-guide) section of the Wiki, where We need to clone the Polkadot SDK. This guide is using release [`polkadot-v1.10.0`](https://github.com/paritytech/polkadot-sdk/releases/tag/polkadot-v1.10.0), which we can easily clone using the tag: ```shell git clone git@github.com:paritytech/polkadot-sdk.git --branch polkadot-v1.10.0 --single-branch ``` Now, navigate to `polkadot-sdk/templates/parachain`. Open this in your code editor of choice, and your ready to get started with developing and compiling your chain. ### Using the Parachain Template This tutorial won't go into the specifics of the template, nor will it go into the specifices of FRAME and Subtrate. All you need to know is the following: - `runtime/` - Runtime and busiess logic. This is your all of your pallets (runtime modules) are configured. The runtime is what gets uploaded to the core as a WebAssembly blob. - `node/` - The node implementation, which takes care of networking and the like. The genesis configuration (`chain_spec.rs`) is also located here. - `pallets/template` - A handy, locally available template pallet, which we can modify and include in our runtime. > Pallets are essentially just Rust crates, which are imported as dependencies as seen in `runtime/Cargo.toml` When we compile our template, we can extract the runtime code as a `.wasm` blob, which is one of the key artifacts for our core. For the sake of this example, we won't go into adding or modifying any pallets. However, this is definitey a next step after you get used to deploying your parachain on Rococo! ### Getting our parachain live-ready > *Note to future self*: It is possible that how we configure a chain spec may change, watch this PR. Before we generate our parachain's code, we have a bit of prep to do to our node inside of `node/src/chain_spec.rs`. Namely, there are a few main factors: 1. Make sure that `relay_chain` is set to the target relay chain (`rococo`, in our case) 2. Make sure that `para_id` (right below `relay_chain`) is set to your reserved ParaId 3. Make sure that our `ChainType` is set to `ChainType::Live` 4. Remove all collators except for **one**, Alice. See the FAQ for why we do this (tldr; for simplicity's sake) 5. Be sure to also set the para id in `testnet_genesis`! 6. Optionally, change the name and id of your chain (mine is called "SomeChain" for the name, and "some_chain" for the id). My ticker symbol for the default token is called "SOME". If you fail to do one of these, there is a large chance that your chain may fail to produce blocks. Feel free to copy the below config, and use it to ensure everything is in place for a Rococo deployment. This function should replace the `local_testnet_config` function within `node/src/chain_spec.rs`: > **:warning: WARNING! :warning:** Make sure you replace the ParaId with the one you reserved earlier! ```rust pub fn local_testnet_config() -> ChainSpec { // Give your base currency a unit name and decimal places let mut properties = sc_chain_spec::Properties::new(); properties.insert("tokenSymbol".into(), "SOME".into()); properties.insert("tokenDecimals".into(), (12).into()); properties.insert("ss58Format".into(), (42).into()); #[allow(deprecated)] ChainSpec::builder( runtime::WASM_BINARY.expect("WASM binary was not built, please build it!"), Extensions { relay_chain: "rococo".into(), // You MUST set this to the correct network! para_id: YOUR_PARA_ID_HERE, } ) .with_name("SomeChain") .with_id("some_chain") .with_chain_type(ChainType::Live) .with_genesis_config_patch( testnet_genesis( // initial collators. vec![ ( get_account_id_from_seed::<sr25519::Public>("Alice"), get_collator_keys_from_seed("Alice"), ), ], vec![ get_account_id_from_seed::<sr25519::Public>("Alice"), get_account_id_from_seed::<sr25519::Public>("Bob"), get_account_id_from_seed::<sr25519::Public>("Charlie"), get_account_id_from_seed::<sr25519::Public>("Dave"), get_account_id_from_seed::<sr25519::Public>("Eve"), get_account_id_from_seed::<sr25519::Public>("Ferdie"), get_account_id_from_seed::<sr25519::Public>("Alice//stash"), get_account_id_from_seed::<sr25519::Public>("Bob//stash"), get_account_id_from_seed::<sr25519::Public>("Charlie//stash"), get_account_id_from_seed::<sr25519::Public>("Dave//stash"), get_account_id_from_seed::<sr25519::Public>("Eve//stash"), get_account_id_from_seed::<sr25519::Public>("Ferdie//stash") ], get_account_id_from_seed::<sr25519::Public>("Alice"), (YOUR_PARA_ID_HERE).into() ) ) .with_protocol_id("template-local") .with_properties(properties) .build() } ``` Once this is in place, you are ready to compile your parachain. ### Generating the Runtime and Genesis Be sure to first build the node using the following (assuming you're within `polkadot-sdk/templates/parachain`): ```shell cargo build -p parachain-template-node --release ``` ```shell ../../target/release/parachain-template-node export-genesis-state genesis ``` ```shell ../../target/release/parachain-template-node export-genesis-wasm genesis-wasm ``` ## Starting to Sync - Running Your Collator You should now start syncing your collator. Keep in mind that you will need to sync Rococo first - this could take some time (12 hours to a day - depending on your download speed), so best to get started ASAP. In order to avoid the storing the full state of the relay chain, be sure to run with the appropriate pruning flags: ```shell ../../target/release/parachain-template-node --collator \ --alice \ --base-path <path-for-your-chain> \ --force-authoring \ -- \ --chain=rococo \ --sync fast-unsafe \ --blocks-pruning 256 \ --state-pruning 256 ``` > **Fun fact**: This command really spins up *two* nodes: your collator node for your parachain, along with an embedded relay chain node, hence the seperation in the command via `--`. You should now see your relay chain ## Create the Parathread With your genesis and genesis-wasm created, you can now create your parathread. Head back to [Network > Parachains > Parathreads (the tab)](https://polkadot.js.org/apps/#/parachains/parathreads) in PolkadotJS, and click *+ Parathread* in the top right: 1. Upload `genesis-wasm` in the `code` field 2. Upload `genesis` in the `initial state` field Pay the deposit, and you're good to go! Keep in mind the parathread takes around **~2 hours** to onboard. ## Procuring Coretime > **Ideally, your node should be synced with the relay before this step.** Luckily, procuring a core is easy, as there are coretime marketplaces already available. Visit [The Coretime Marketplaces](https://wiki.polkadot.network/docs/learn-guides-coretime-marketplaces) page on the Polkadot Wiki to learn more about them, and purchase a core! We have two options: 1. Bulk - pay upfront to use Rococo's validation for our parachain. 2. On-demand - pay as we go for our block production. With bulk coretime, we assign a core to our ParaId, and as long as that core is valid, our parachain will produce blocks and finalize via Rococo, the testnet. Until we have to renew the core, we shouldn't have to touch the It's worth noting that you can easily experiment using on-demand extrinsics, then later switch to a bulk coretime model. For now, let's start with on-demand coretime to get our first blocks going. ### On-Demand Coretime Provided your collator is synced, you can try creating a block using an on-demand extrinsic: 1. Head to [PolkadotJS > Extrinsics](https://polkadot.js.org/apps/#/extrinsics) 2. Issue the `onDemandAssignmentProvider.placeOrderAllowDeath` extrinsic 3. Set the `max_amount` to at least `1000000000000` and `paraId` to your ParaId. 4. As long as your collator is running, you should see your block height increase from 0 to 1! ### Bulk Coretime: Assigning your Core If everything is working as intended, you can now choose to assign bulk coretime to your parachain for persistent block generation. For my assignment, I will use RegionX's CoreHub, a nice UI for both purchasing and assigning cores. 1. Connect your wallet (button in the top right corner) 2. Click **Purchase a Core** on the left 3. In the lower right, click "Purchase Core" 4. Go to **My Regions**, and click to select your region 5. Click **Assign** on the right side 6. Click **Add Task**, and input your ParaId along with a name for your chain 7. Select your task, and select **Provisional Assignment** 8. Provided everything is synced and your collator is running, you should see blocks being persistently created: ![GMcdatDW8AEC8kh](https://hackmd.io/_uploads/r12b0mxMC.jpg) ## What's Next? Now that you have the ability to launch a layer one blockchain, the possbilities are endless: - Create a pallet - Upgrade your network - Setup a local development environment for your parachain ## Alternatives! There are a number of alternatives, take a look: - [Tansii Dancebox](https://docs.tanssi.network/builders/tanssi-network/testnet/dancebox/) - an even easier way to utilize Polkadot compute with your own runtimes. - This tutorial not working for you? [OpenZeppelin's tutorials should be up to date!](https://docs.openzeppelin.com/substrate-runtimes/1.0.0/) ## FAQ / Troubleshooting **Q: Why do I have to sync Rococo locally? Can't I just use a remote, trusted node and connect to that?** **A:** You can actually, via the `--relay-chain-rpc-urls` flag, which can be passed to your collator. Unfortunately, the caveat is you can't use this for collation at this time - meaning if you intend on being a collator/validator for your blockchain and intend to create blocks, you need to sync locally. --- **Q: Is there a faster way to sync Rococo? Why not warp sync?** **A:** Warp sync is currently not possible on Rococo or Westend. [See this answer for more context.](https://substrate.stackexchange.com/questions/9730/rococo-cant-warp-sync-stuck-at-16mb-finality-proof-download) --- **Q: My collator is not producing blocks!!** **A:** Check these sanity checklists: - https://substrate.stackexchange.com/questions/178/how-can-i-get-my-parachain-to-produce-blocks-sanity-checklist - https://substrate.stackexchange.com/questions/1394/our-parachain-doesnt-produce-blocks-checklist --- **Q: I want to run more than one collator, how do I do that?** **A:** Ideally, you would want to run these on seperate machines/servers, but you could as long as you ensure you can provide different RPC/WebSocket and P2P ports for each collator. You also may need to sync a seperate instance of Rococo for each collator on the same machine. --- **Q: Why do we only have one collator? Isn't it better to have more?** **A:** Mostly for simplicity. If we have more than one collator, we would have to also spin it up, which would be a hassle on a single machine (it is possible though). Of course, if you had an actual network with multiple collators, it is assumed you'd have seperate VPS/servers for each. --- **Q: Why are we registering parathreads and not parachains?** **A:** *Parathreads* are a bit of an outdated term by now. They refer to what are now known as on-demand parachains. Although they be references in various places through PolkadotJS, docs, or other UIs, really we only have two types of parachain: on-demand parachains, and parachains which use bulk coretime. --- **Q: What is a good amount to put as the `max_amount` for my on-demand assignment?** **A:** todo