What are we measuring

Codex: v0.1.6
Log message "Received blocks from peer" indicates that a node received a block. It identified the peer from which the block was received and which block was received. (dataset + index)
We are counting per node, how many times a specific block is received. We are disregarding which peer it was sent from.

Test

In this test we create a swarm in which 1 node contains a file. The other nodes simultaneously begin downloading it.
File size: 1, 5, 10, 20 MB
Number of nodes: 3, 5, 10, 20

Data

Data (csv)

filesize,numNodes,recv1x,recv2x,recv3x,recv4x,recv5x,recv6x,recv7x,recv8x,recv9x,recv10x,recv11x,recv12x,recv13x,recv14x,recv15x,recv16x,recv17x,recv18x,recv19x
1,3,34,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
1,5,60,6,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
1,10,126,13,4,3,1,2,3,0,1,0,0,0,0,0,0,0,0,0,0
1,20,202,28,13,12,12,10,5,8,12,12,5,1,1,1,0,0,1,0,0
5,3,141,21,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
5,5,208,82,29,5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
5,10,397,154,86,40,36,12,2,2,0,0,0,0,0,0,0,0,0,0,0
5,20,433,286,185,130,104,83,62,53,46,30,21,10,13,20,7,5,10,4,37
10,5,391,183,59,11,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
10,10,761,321,187,93,45,26,11,4,1,0,0,0,0,0,0,0,0,0,0
10,20,906,553,430,283,217,131,105,77,58,43,38,19,21,28,28,35,63,20,4
20,3,555,87,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
20,5,813,298,140,33,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
20,10,1490,631,361,208,96,55,36,11,1,0,0,0,0,0,0,0,0,0,0
20,20,1584,1225,857,517,371,267,212,157,138,92,71,86,80,83,91,180,59,24,5

Download Flow

Inspecting the v0.1.6 implementation of Codex, we find the following sequence of events when download is initiated.

1. Fetch manifest

Manifest encoded as a single block is fetched using the download-CID. We proceed only after the manifest is received and decoded successfully. We will not consider erasure-coding because the tests have been performed without it.

2. Initialize stream

A stream object is initialized. This stream will transmit data to the downloading application.

3. Read loop

While the bytes streamed is less than the total bytes in the dataset, we use the bytes-sent counter to determine the block-index in the dataset. We ask for this block from the network-block-store. When received, we write the bytes from the block to the stream and advance the current position. Only after the block has been copied to the stream entirely do we ask the network-block-store for the next block.

Requesting a block

We will take a closer look at the sequence of events occuring in the block exchange engine of Codex when a block is requested. This section considers only the retrieval of a single block.

1. Pending block

A pending-block handle is created to represent the block retrieval process. If one already exists, the existing one is used instead.

2. Cheapest, known peer

From already-connected peers, we select the ones we know have the block, and sort them by the price for the block. If none are available, DHT discovery is started asynchronously.

3. Peer selection

If we have any peers from the previous step, we select one semi-randomly. If we don't, we select one from all the peers known to the block exchange engine. If we don't have any of those either, we exit the process assuming that DHT discovery will yield peers some time in the future.

4. Once peer

Once a peer has been selected, we send a wantBlock to that peer. And we send a wantHave message to all other peers.

5. Response to the wantBlock message

When receiving a wantBlock message, the peer will send the block back if it has the block in local storage.

6. Response to the wantHave message

When receiving a wantHave message, the peer will send a blockPresence response. It contains for the blocks in the wantHave message an indicator if the block is present or not. Additionally, it contains all the block CIDs currently wanted by this peer.