# How to create a symbol testnet network
## Introduction
**Experimental/Draft!**
There are several articles about how to create Symbol nodes connecting to Symbol Testnet or Mainnet networks. But, what if you want to create your own fully decentralized symbol network for fun/testing?
If you want to test a symbol network locally you can just use symbol-bootstrap. Run:
```bash=
symbol-bootstrap start -p bootstrap
```
This command will create a symbol network bundled in a unique docker-compose. The compose runs 2 peers nodes, 1 API node, and a rest gateway.
This private network is great when testing new features, like a new transaction or new rest endpoint before the feature is deployed into a shared testnet. Private local networks are also great for e2e testing, when you want to start a network in a given or "empty" state, run some operations, assert the new states and reset the network.
A local private network is not production-ready. Running multiple symbol nodes in one machine is not recommended, it's not decentralized, other people cannot connect to the network easily.
In this guide, I'm going to explain how you can create your symbol network, how you can create multiple 'seed' nodes, and how you can share the network configuration so other people can connect to it. The name of this testnet is **Fernet**.
This guide uses an experimental version of [`symbol-bootstrap`](https://github.com/symbol/symbol-bootstrap) and a new cli tool called [`symbol-network`](https://github.com/fboucquez/symbol-network). The `symbol-network` cli scales `symbol-bootstrap` to create and manage multiple nodes, for new or existing networks.
## Installation
Before starting, let's install the required tooling. The tolling required is the same as the standard `symbol-bootstrap` but the experimental distribution (for now) and `symbol-network`.
This guide assumes an Ubuntu distribution, other OSs would be similar.
1) Node, Docker, and Compose
```bash=
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash -
apt-get update
apt-cache policy docker-ce
apt-get install -y unzip gcc g++ make nodejs unzip docker.io
docker --version
curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
docker-compose --version
```
2) Install the experimental symbol-bootstrap
```bash=
npm install -g symbol-bootstrap@network
symbol-bootstrap version
symbol-bootstrap verify
```
3) Install the experimental symbol-network
```bash=
npm install -g symbol-network@network
symbol-network version
symbol-network verify
```
## Network Layout
Before running symbol-network, it's good to think about the network layout we want initially for our network. The **Fernet** network would be:
* Based on Symbol Tesnet
* Single currency, named, `fernet.branca`, same initial supply.
* Network domain `fernet.wtf`. No hostname prefix.
* Network Type `Testnet`
* Explorer, Web Wallet and Faucet
* HTTP and HTTPS nodes
* 2 Voting Peers - 3M balance
* 1 Voting Dual - 3M balance
* 2 Harvesting Dual Nodes - 1M balance
## Network Configuration
Now that we know how the network should look, let's run the tool!
### Init:
The first step is the init wizard:
```bash=
mkdir symbol-fernet
symbol-network init --ready
```
The first questions are about the basic network configuration. These questions are the most common, advance tunning can be manually done later by changing the yml files this command creates.
This command asks for a 'master' password. The master password is used to encrypt the generated network and node keys.
The command skips this section if you are creating nodes for an existing network. For example, you want to create and maintain dozens of testnet/mainnet nodes.
The second part asks how many nodes you want to create for a new or existing network. We are going to fill in the numbers defined above.
Once the `init` command finishes, you can see that 3 files have been created:
- **network-input.yml**: the not-expanded network layout. It defines how many of each node we want. If you pre-create this file, `symbol-network init` won't ask you any questions and just run.
- **key-store.yml**: The file that contains the network keys, nodes keys, and voting files the tool generates. The private values are encrypted using the provided `master` password. You can decrypt the file using `symbol-bootstrap decrypt` command.
- **custom-network-preset.yml**: The network preset your nodes would use when running. These files are like the '-p mainnet' and `-p bootstrap` for your network. This is NOT the custom preset of a given node. You can tweak it, for example, change the network supply or inflation.
The cli tool will recommend the different commands you can run next to continue the network creation process.
### Expand nodes:
The second automatic step 'expands' the nodes so each node can be customized independently.
```bash=
symbol-network expandNodes
```
This command creates the following file:
- network.yml: It contains each node configuration, including their custom preset. It's possible to tune it by changing the default hostnames, friendly names or adding new custom preset properties like `logLevel: Debug`
### Nemesis generation:
The third automatic step is to 'generate' the nemesis block for your new network. The nemesis block includes all the seed nodes accounts, link transactions, faucet, and 'founder' accounts. The founder account is the account that keeps the remaining supply after the node and faucet account distribution. All these accounts can be found encrypted in the **key-store.yml**.
```bash=
symbol-network generateNemesis
```
This command generates:
- nemesis-seed: a folder containing the nemesis block of your network.
- nemesis-target: a folder containing a temporary node used to generate the nemesis block
The `nemesis-seed` block along the `custom-network-preset.yml` could be shared so other people can connect to the network after launch.
### Try the Nemesis Node:
Before creating the nodes to be published in your VPC infrastructure, you can try how everything looks by running:
```bash=
symbol-bootstrap start -t nemesis-target --noPassword --detached --healthCheck --report
```
Go to the local explorer at `http://localhost:90`.
Explorer shows the nemesis block, the seed accounts, the nodes' link transaction, etc.
Once you are happy, stop the node:
```bash=
symbol-bootstrap stop -t nemesis-target
```
### Node configuration:
Finally, to create the nodes' target folders and zip distributions ready to be deployed in your infrastructure, run:
```bash=
symbol-network configureNodes --offline --zip
```
This command will ask for a new password, the 'node' password. The node password is used to encrypt each node individually. It's the password symbol-bootstrap asks when running the different commands. Use a different password than the master password.
This last command generates:
- nodes: the folders ready to use for each node
- distribution: the zips of the node ready to be pushed to the VPC machine.
### Configuration backups:
It's recommended you version the different created files. I use git to commit and backup the files.
```bash=
git init
```
My .gitignore file:
```
nemesis-target/databases/
nemesis-target/nodes/node/logs/
nemesis-target/nodes/node/data/
nemesis-target/gateways/rest-gateway/logs/
logs.log
```
Then commit all the other files for future reference. You may or may not want to backup in a remote github repository. Node and network private keys are encrypted in the `key-store.yml` file.
## Network Deployment
The `symbol-network` doesn't handle the deployment. It is outside the current scope of the tool. The possible infrastructures, cloud providers, and options are limitless.
This is up to the developer to script it. You can use bash and scp, you can use terraform and s3 buckets or ansible scripts, etc.
The required process is fairly simple though:
- Install symbol-bootstrap@network and its dependencies on each node VPC.
- SCP or Pull the zip to each node VPC, unzip.
- Update firewalls opening ports 80, 7900, 3000, 3001, 443 depending on the node type.
- Update the DNS values linking nodes' hostnames with the VPC IPs.
The deployment process is similar to this [guide](https://dev.to/fboucquez/the-23-symbol-node-3c41) although the zips for all the nodes are precreated, you just need to deploy and `symbol-bootstrap run -d` them.
### Network launch:
To run the network, connect to each box and run:
```bash=
symbol-bootstrap run -d
```
### Node update:
The general process to upgrade a node is installing a new version of the `symbol-bootstrap`, and probably a `custom-network-preset.yml` (global change) or a `custom-preset.yml` (node change) upgrade. Then run:
```bash=
symbol-bootstrap stop
symbol-bootstrap start -c custom-preset.yml --upgrade
```
### Allow new nodes to connect:
If you want another person to connect to your network, share with him the `custom-network-preset.yml` and `nemesis-seed` folder. Then this person can run:
```bash=
symbol-bootstrap start -p custom-network-preset.yml -a dual -c my-new-node-custom-preset.yml
```
Note that official Symbol networks have the network preset and seed bundled on `symbol-bootstrap`. That's why you can just run `-p testnet` or `-p mainnet`. The process for the custom testnet network is the same, just that the network prest and nemesis seed are externally provided.
### Contabo deployment
I have deploed FerNet to [contabo's](https://contabo.com/) VPCs. These are the running nodes and services:
- [dual-001.fernet.wtf](https://dual-001.fernet.wtf:3001/chain/info)
- [dual-002.fernet.wtf](https://dual-002.fernet.wtf:3001/chain/info)
- [dual-003.fernet.wtf](https://dual-003.fernet.wtf:3001/chain/info)
- beacon-001.fernet.wtf
- beacon-002.fernet.wtf
- [Faucet](https://faucet.fernet.wtf)
- [Explorer](https://explorer.fernet.wtf)
Contabo [costs](https://contabo.com/en/vps) around 12 USD per node. The cost is quite lower than [Hetzner](https://www.hetzner.com/cloud) although Contabo's console is not nearly as nice.
You may need to upgrade storage or processing power depending on how much the network is used.
## Conclusion
I've created Symbol Network from our experience creating the initial Mainnet and current Testnet nodes. It bundles the original private tooling in a user-friendly and open-source cli that uses and scales Symbol Bootstrap.
This new cli tool allows you to create and maintain new Symbol networks quickly as well as nodes for existing networks without losing the configurability
Have a try! Feedback is always appreciated!
## TODOs
- Remove the 'experimental' flag from the CLIs! There are still some features to be reviewed and merged before getting there.
- Add Statistic Service to the `demo` assembly. This Statistic Service wouldn't be production-ready as it doesn't have HA. It could be good enough for different Testnets though.
- Explain how to add additional accounts at nemesis block (duable, just need to explain and maybe update the wizard).
- Update Explorer, Faucet to the latest with Statistic Service. It may not be trivial as the statistic service is essentially empty when launching a new network. These services may need to fall back initially to a static list?
- Add a `services` assembly. An assembly without a symbol node that includes Explorer, Faucet, Statistic Service (and Web Wallet?) fully HTTPS 443 support. I have created this manually (for gurush.dev), I need to bundle it in a `symbol-bootstrap` assembly and `symbol-network` node type.
- Add nemesis block multisig modification to symbol-network, so nodes' Mainnet accounts, faucet, and founder accounts are multisig from block 1. I had that for Mainnet/Testnet configuration, but I need to make it nice for the cli tool. This feature depends on `symbol-signer-cli` to create, cosign and merge them into nemesis block transactions. Of course, multisig can be done after network launches too!
Google Drive
Gist
Clipboard
Sign in with Wallet
