# Iroha Multiple Instances Setup

That note explores common pitfalls of setting up the Iroha network.

Let's assume that the starting size of the network is 3 nodes.

## Genesis Block

All the nodes should be launched with **the same** genesis block.
The best way to ensure this - just copy over your `genesis.block` from the first node to the rest nodes.

Since our initial network configuration contains three nodes - all of them should be listed in the genesis block.

{
  "block_v1": {
    "payload": {
      "height": "1",
      "prevBlockHash": "0000000000000000000000000000000000000000000000000000000000000000",
      "transactions": [
        {
          "payload": {
            "reducedPayload": {
              "commands": [
                {
                  "addPeer": {
                    "peer": {
                      "address": "192.168.1.10:10001",
                      "peerKey": "bddd58404d1315e0eb27902c5d7c8eb0602c16238f005773df406bc191308929"
                    }
                  },
                  "addPeer": {
                    "peer": {
                      "address": "192.168.1.10:10002",
                      "peerKey": "b27902c5d7c8eb0602c16238f000602c16238f005773df406bc1913089291231"
                    }
                  },
                  "addPeer": {
                    "peer": {
                      "address": "192.168.1.11:10001",
                      "peerKey": "05773df406bc1913088eb0602c16238f0052131111234773df406bc44451616f"
                    }
                  }
                },

The listing shows the beginning of the genesis block with a transaction that contains three AddPeer commands.

Here we should note that nodes can run on the same host but with different internal ports specified.

Addresses should be achievable to each other among the nodes (even for "self" - the node should be able to contact itself via its public address). Mentioned ports should also be opened for incoming external connections.

All the nodes should have different public keys (and their different private counterparts correspondingly).

## Configuration File

{
  "block_store_path" : "/tmp/block_store/",
  "torii_port" : 50051,
  "internal_port" : 10001,
  "pg_opt" : "host=localhost port=5432 user=postgres password=mysecretpassword",
  "max_proposal_size" : 10,
  "proposal_delay" : 5000,
  "vote_delay" : 5000,
  "mst_enable" : false,
  "mst_expiration_time" : 1440,
  "max_rounds_delay": 3000,
  "stale_stream_max_rounds": 2
}

Torii port is a port where Iroha users send their transactions or queries. It should be exposed and achievable from the outside.

The internal port should match the port specified in the genesis block for that particular node.

If two nodes running on the same host, then their block store path, torii port, internal port and Postgres connections' options must be different.

TO BE CLARIFIED with official [iroha.readthedocs.io](https://iroha.readthedocs.io): pg_opt could also contain dbname option. If so, then dbname only can be different for "the same host" nodes' configs.

## Launch Command

Whether Docker is used or not, every iroha instance execution starts from the command like the following:

./irohad --genesis_block genesis.block --config nodeX.config --keypair_name nodeX

# --overwrite_ledger is a commonly-used option for tests, just add it if needed

For the example above it is assumed that:
* `genesis.block` is the same on all nodes
* `nodeX.config` is differs for each node
* working directory for each `irohad` contains own pair of `nodeX.priv` and `nodeX.pub` files that represent the keypair, which public key should match the one from the genesis block for the particular node

## Adding 4th Node at Runtime

All you need for network expanding is:
1. copy over the genesis block to the 4th host (**nothing should be modified inside!**)
1. generate a new keypair for the new peer
1. compose its individual config file
1. launch the peer
1. issue a transaction with add peer command and send it to one of peers from the existing network

As soon as the transaction with the AddPeer command gets committed, the 4th peer will be considered as a legal consensus participant. Then 4th peer will download all the blocks that were committed and become able to process any input in its desired way.

Let's assume our 4th host has `192.168.1.13` ip address.

Its config file may look like:

{
  "block_store_path" : "/tmp/block_store/",
  "torii_port" : 50051,
  "internal_port" : 10001,
  "pg_opt" : "host=localhost port=5432 user=postgres password=mysecretpassword",
  "max_proposal_size" : 10,
  "proposal_delay" : 5000,
  "vote_delay" : 5000,
  "mst_enable" : false,
  "mst_expiration_time" : 1440,
  "max_rounds_delay": 3000,
  "stale_stream_max_rounds": 2
}

Its keypair files might have the following contents:

node4.priv

cc5013e43918bd0e5c4d800416c88bed77892ff077929162bb03ead40a745e88

node4.pub

bddd58404d1315e0eb27902c5d7c8eb0602c16238f005773df406bc191308929

Then the AddPeer command should specify 4th peer address as `192.168.1.13:10001` with its public key `bddd58404d1315e0eb27902c5d7c8eb0602c16238f005773df406bc191308929`.

For example, you may use Iroha Python library, then the code of command creation going to look like:

peer = primitive_pb2.Peer()
peer.address = '192.168.1.13:1001'
peer.peer_key = b'bddd58404d1315e0eb27902c5d7c8eb0602c16238f005773df406bc191308929'
tx = iroha.transaction([
    iroha.command('AddPeer', peer=peer)
], creator_account='admin@test')

**NOTE** *I did not launch the code above. This should be considered as an iroha-python-like pseudocode. Actually working sample might look slightly different.*

Actual Iroha Python library implementation accessible via pip: https://pypi.org/project/iroha/

Its source code: https://github.com/hyperledger/iroha-python

A bit outdated example of AddPeer command usage: https://github.com/hyperledger/iroha/blob/master/example/python/permissions/can_add_peer.py

## Contacts

Any questions are welcome in our [Iroha Community Chat](https://t.me/hyperledgeriroha) in Telegram.