Guide on manually setting up a parachain with a [collator](https://wiki.polkadot.network/docs/learn-collator) using custom keys (not using dev accounts such as `Alice` & `Bob`).

### Prerequisites
You should be comfortable with the following tutorials:
* https://docs.substrate.io/tutorials/build-a-parachain/prepare-a-local-relay-chain/
* https://docs.substrate.io/tutorials/build-a-parachain/connect-a-local-parachain/

# Setup

## Build the Polkadot and Parachain binaries.

Make sure they are compiled based on the same release version. If you want to connect the parachain to an already running Relay chain you can skip the Polkadot steps.

* Build Polkadot:

git clone https://github.com/paritytech/polkadot
cd polkadot
git checkout <release>
cargo b -r

* Build Parachain:

git clone https://github.com/substrate-developer-hub/substrate-parachain-template
cd substrate-parachain-template
git checkout <polkadot-release>
cargo b -r

## Generate chainspec, runtime wasm, genesis state
### Chainspec

plain:

./target/release/parachain-template-node build-spec --disable-default-bootnode > parachain-chainspec.json

**Important**: edit the `parachain-chainspec.json` where necessary. For example; set the correct `para-id`, `protocolid` or `relay_chain`.

The `para-id` needs to be reserved on the Relay Chain. Check out the ["Connect a local Parachain"](https://docs.substrate.io/tutorials/build-a-parachain/connect-a-local-parachain/) tutorial and look for the step *"Reserve a unique identifier"*

raw:

./target/release/parachain-template-node build-spec --chain parachain-chainspec.json --disable-default-bootnode --raw > parachain-chainspec-raw.json

### Runtime wasm

Required to register the Parachain on the Relay Chain. The Relay Chain needs the runtime code to validate blocks.

./target/release/parachain-template-node export-genesis-wasm --chain parachain-chainspec-raw.json parachain-wasm

### Genesis state

Required to register the Parachain on the Relay Chain. Both the Relay Chain and the Parachain need to start from the same starting state.

./target/release/parachain-template-node export-genesis-state --chain parachain-chainspec-raw.json parachain-genesis-state

# Launch

## Relay Chain
Either follow the tutorial ["Prepare a local Relay Chain"](https://docs.substrate.io/tutorials/build-a-parachain/prepare-a-local-relay-chain/), use [Zombienet](https://github.com/paritytech/zombienet) to setup a local environment or connect to Rococo.

**Important**: we need the same chainspec that is used to start the validators for the collator(s). In case of connecting to Rococo, we need the [Rococo chainspec](https://github.com/paritytech/polkadot/blob/master/node/service/chain-specs/rococo.json).

## Parachain

### Register Parachain

If necessary, register the parachain to the Relay Chain with the `parachain-genesis-state` and the `parachain-wasm` files generated above. In addition, you need a `parachain manager account` and the `para-id`. Check out the ["Connect a local Parachain"](https://docs.substrate.io/tutorials/build-a-parachain/connect-a-local-parachain/) tutorial and look for the step *"Register with the local Relay Chain"*

### Launch Collator

Before we launch a Collator for the Parachain, make sure that its database is purged from any previous attempts, as any leftover state can cause syncing issues. In other words, the genesis state of the collator won't match the genesis state that has been given to the Relay chain.

./target/release/parachain-node-template purge-chain --base-path /tmp/parachain/<collator> --chain parachain-chainspec-raw.json

Launch the collator:

./target/release/parachain-node-template --collator \
--name C1 \
--base-path /tmp/parachain/collator1 \
--chain parachain-chainspec-raw.json \
--force-authoring \
--rpc-port 10001 \
--ws-port 10002 \
--listen-addr /ip4/0.0.0.0/tcp/10003/ws \
-- \
--execution wasm \
--chain /{PATH_TO_RELAYCHAIN_CHAINSPEC} \
--port 10004 \
--ws-port 10005

1. Ensure the collator is peering with the relay chain (and the other collator(s) if present) by checking the collator output logs.
* No peers with the Relay Chain: if you made changes to the relay chainspec, make sure `bootnodes` are provided. In addition, [NAT](https://github.com/paritytech/substrate/issues/2772) can also be a problem.
* No peers with the Parachain: see **Important**.
2. No blocks being produced should be expected until the session key is added to the [keystore](##Keystore) - This is specific to a custom chain spec and would probably work if using the parachain-node-template default spec as it would probably just use alice/bob as collators.
`./target/release/parachain-node-template --help`:

**Important**:
- the `force-authoring` flag is only necessary when you have one collator (e.g. for testing).
- the `listen-addr` flag is necessary if you want other nodes to be able to connect to you to join the network. An additional `bootnodes` flag is necessary for this other collator (when the chainspec doesn't provide `bootnodes`). In addition, you need the `[parachain] local identity` of the collator you want to connect to (you can find this in the logs when you start your node).

An example of launching a second collator that you want to connect to the first collator shown above:

./target/release/parachain-node-template --collator \
--name C2 \
--base-path /tmp/parachain/collator2 \
--chain parachain-chainspec-raw.json \
--force-authoring \
--rpc-port 10006 \
--ws-port 10007 \
--listen-addr /ip4/0.0.0.0/tcp/10008/ws \
--bootnodes /ip4/127.0.0.1/tcp/10003/ws/p2p/{INSERT_COLLATOR_1_PARACHAIN_NODE_IDENTITY} \
-- \
--execution wasm \
--chain /{PATH_TO_RELAYCHAIN_CHAINSPEC} \
--port 10009 \
--ws-port 10010

**The `force-authoring` can still be provided for the case where you want it to build blocks if it is the only collator in the network.*

Last, depending on the chain and the syncing method, the time it takes to be completely synchronized and build your first block can vary. The flag `--sync=warp` enables the node to make use of the [warp sync protocol](https://spec.polkadot.network/chap-sync#sect-sync-warp). This decreases the synchronization time significantly by only validating / applying full blocks at the end of the initial synchronization process.

# Session Keys

The session key needs to be set for a collator to start producing blocks. It is advised to use a different keypair than the collator keypair. This is to minimize exposure of the collator keypair.

## Generating keys
There are multiple ways to generate keys, such as:
- Polkadot JS extension or any other custodial
- [PolkaVault](https://wiki.polkadot.network/docs/polkadot-vault)
- [Subkey](https://docs.substrate.io/reference/command-line-tools/subkey/)

## Session-pallet
Session keys are set in [session-pallet](https://github.com/paritytech/substrate/tree/master/frame/session). Session keys can be added to [genesis state](https://github.com/paritytech/substrate/blob/cb450b626ac8e8848db76933e114e57e7cce3e8d/frame/session/src/lib.rs#L417-L421), otherwise [set_keys](https://github.com/paritytech/substrate/blob/4d9d9116b5f65f4a7f3e70c1f4ab74d9fed60c0c/frame/session/src/lib.rs#L589) needs to be called by the collator.

In order to change your session keys you'd have to call `set_keys` with the new public key.
## Keystore

In order for a collator to be able to sign e.g. its produced block, the session keypair needs to be added to the `keystore`. The keystore is a file that provides a secure and encrypted storage solution for your private keys. It ensures that only authorized processes can access and use the private keys when needed.

1. Connect to your collator via Polkadot JS: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A10002 (`ws-port` flag set to 10002)
2. Go to `Developer` and `RPC calls` and click **author**, **insertKey**.
3. Enter "aura" as the **keyType**
4. Enter the seed/mnemonic of the collator's session key as **suri**.
5. Enter he hex value of the public key of the collator's session key for **publicKey**.

The Collator's session keys are added to the keystore and should now be building blocks.

# Invulnerables & Candidates

As for cumulus' out-of-the-box implementation, the set of collators that are allowed to build blocks are coming from the pallet [`collator-selection`](https://github.com/paritytech/cumulus/blob/7d51356009b652aca3fa1dbba73b7c729003cd6c/pallets/collator-selection/src/lib.rs#L16-L60), more specifically the `invulnerables` and `candidates`.

If we want to add a new collator we either [add it to `invulnerables`](https://github.com/paritytech/cumulus/blob/7d51356009b652aca3fa1dbba73b7c729003cd6c/pallets/collator-selection/src/lib.rs#L490) through `root` or we can [register as a candidate](https://github.com/paritytech/cumulus/blob/7d51356009b652aca3fa1dbba73b7c729003cd6c/pallets/collator-selection/src/lib.rs#L424).

**Important**: `invulnerables` will always be chosen to build blocks. As for `candidates`, there is a [`desired_amount`](https://github.com/paritytech/cumulus/blob/7d51356009b652aca3fa1dbba73b7c729003cd6c/pallets/collator-selection/src/lib.rs#L205) that caps the amount of candidates that can be registered. In other words, if a collator wants to register as candidate but the `desired_amount` is met, the collator has to wait until a collator [`leave_intent`](https://github.com/paritytech/cumulus/blob/b3c98894621b317e07af5bd32f2b5a5e12bfc327/pallets/collator-selection/src/lib.rs#L469) or the desired_amount [is increased](https://github.com/paritytech/cumulus/blob/b3c98894621b317e07af5bd32f2b5a5e12bfc327/pallets/collator-selection/src/lib.rs#L389).

# Additional information
- [How to make a substrate node publicly accessible?](https://substrate.stackexchange.com/questions/1787/how-to-make-substrate-node-publicly-accessible/1798#1798)
- [What is an RPC node?](https://substrate.stackexchange.com/questions/2246/what-is-an-rpc-node/2250#2250)
- [Secure validator / collator](https://wiki.polkadot.network/docs/maintain-guides-secure-validator)
- [Is it ok to combine RPC node with collator together?](https://substrate.stackexchange.com/questions/925/is-it-ok-to-combine-rpc-node-with-collator-together)
- [What's the easiest way to set up a relay chain locally with a couple of parachains?](https://substrate.stackexchange.com/questions/33/whats-the-easiest-way-to-set-up-a-relay-chain-locally-with-a-couple-of-parachai)
- [What are the predefined accounts (ALICE, BOB, etc)?](https://substrate.stackexchange.com/questions/41/what-are-the-predefined-accounts-alice-bob-etc)