Short intro to IPFS
===

What is IPFS?
---
IPFS is a decentralized file distributing system based on peer-to-peer protocol, adding any content on it is like broadcast a unique self-describe torrent file to every peer connected and start to seed the file.
Files will remain on the internet until there's no node seeding it.
IPFS could be seen as a single BitTorrent swarm, exchanging objects within one Git repository.

Easy instruction
---
### Useful command

ipfs init //initialize
ipfs id //view node id
ipfs add <path to Files/Folders> [-r] [-w] //add file to ipfs //-r for recursive -w for keeping directory structure
ipfs cat <ipfs-path> //view file 
ipfs get <ipfs-path> //download file
ipfs ls <ipfs-path-of-directory> //list file in directory

### Going online

ipfs daemon //run ipfs online
ipfs name publish <ipfs-path> //using ipns feature, public files to your nodeid

after running daemon, you can use web console
http://localhost:5001/webui
### Using official gateway to view files
https://ipfs.io/ipfs/<ipfs-path>
https://ipfs.io/ipns/<nodeid>/[path-to-file]

### Official Demo
{%youtube 8CMxDNuuAiQ %}

IPFS design
---
The IPFS Protocol is divided into a stack of sub-protocols:
- Identities
- Network
- Routing
- Exchange
- Objects
- Files
- Naming

reference:
https://github.com/ipfs/specs

### Identities
Nodes are identified by NodeId, which is a hashed public key(default RSA2048),
>key generation 的計算
>[color=red]

type PrivateKey []byte
// self-describing keys
type Node struct {
    NodeId NodeID
    PubKey PublicKey
    PriKey PrivateKey
}

and required PoW while establishing it.(In order to prevent sybil attack)
>https://www.twcert.org.tw/assets/pdf/SybiAttack.pdf

difficulty = <integer parameter>
n = Node{}
do {
    n.PubKey, n.PrivKey = PKI.genKeyPair()
    n.NodeId = hash(n.PubKey)
    p = count_preceding_zero_bits(hash(n.NodeId))
} while (p < difficulty)

>p = count_preceding_zero_bits(hash(n.NodeId))
>why?[color=red][name=7/12討論]

Upon first connecting, peers exchange public keys, and check:
**hash(other.PublicKey) equals other.NodeId**.
If not, the connection is terminated.

### Network
>firewall 可以過嗎? ->多分おｋ
>[color=red][name=7/12討論]

12:42:06.468 DEBUG     engine: wants QmaBCkhaeaaCzEKK1K9xzL39vMaXjA6QU1aERqSoXfVJmm - 2147483617 engine.go:241                         
12:42:06.468 DEBUG     engine: peer <peer.ID SoLju6> wants QmaBCkhaeaaCzEKK1K9xzL39vMaXjA6QU1aERqSoXfVJmm ledger.go:80                 
12:42:19.996  INFO    bitswap: Sending block [Block QmaBCkhaeaaCzEKK1K9xzL39vMaXjA6QU1aERqSoXfVJmm] to <peer.ID SoLju6> wantmanager.go:117                              
12:42:23.881 DEBUG     engine: <peer.ID SoLju6> cancel QmaBCkhaeaaCzEKK1K9xzL39vMaXjA6QU1aERqSoXfVJmm engine.go:237                    
12:42:39.681 DEBUG        dht: <peer.ID boMjby> adding <peer.ID SoLju6> as a provider for 'QmaBCkhaeaaCzEKK1K9xzL39vMaXjA6QU1aERqSoXfVJmm'                              
12:42:39.681  INFO        dht: received provider <peer.ID SoLju6> for QmaBCkhaeaaCzEKK1K9xzL39vMaXjA6QU1aERqSoXfVJmm (addrs: [/ip4/127.0.0.1/tcp/4001 /ip4/104.236.151.122/tcp/4001 /ip4/10.12.0.5/tcp/4001 /ip4/172.17.0.1/tcp/4001 /ip6/::1/tcp/4001 /ip6/2604:a880:1:20::1d9:6001/tcp/4001 /ip6/fc3d:9a4e:3c96:2fd2:1afa:18fe:8dd2:b602/tcp/4001 /ip4/127.0.0.1/tcp/8081/ws /ip4/104.236.151.122/tcp/4001]) handlers.go:291

IPFS can use any network, it stores addresses as **multiaddr** formatted byte strings for the underlying network to use.
It can also connect to local network, and have local swarm peers.
For example :
> SCTP/IPv4 connection
/ip4/10.20.30.40/sctp/1234/

> SCTP/IPv4 connection proxied over TCP/IPv4
/ip4/5.6.7.8/tcp/5678/ip4/1.2.3.4/sctp/1234/

### Routing
IPFS use a DSHT(Distributed Sloppy Hash Table) based on **S/Kademlia** and **Coral**.
>Coral 的特性？ V.S. bitTorrent DHT
>routing 的意義？ => Searching for node or data
>[color=red][name=7/12討論]

DSHT:
>A sloppy insert is akin to an append in which a replica pointer appended to
a “full” node spills over to the previous node in the lookup path.
A sloppy retrieve only returns some randomized subset
of the pointers stored under a given key.

http://iptps03.cs.berkeley.edu/final-papers/coral.pdf
IPFS DHT makes a distinction for values stored based on their size,
small values (equal to or less than 1KB) are stored directly on the DHT,
and for values larger, the DHT stores references, which are the NodeIds of
peers who can serve the block.
useful refs:https://github.com/ipfs/examples/blob/master/examples/data/readme.md
(1KB的小檔案快取目前並沒有實做)
>1KB 是檔案大小還是 block? -> 檔案大小
>一剛開始決定的 block 大小？
>var BlockSizeLimit = 1048576 //1 MB
>[name=7/19會議][color=green]

type IPFSRouting interface {
    FindPeer(node NodeId)
    // gets a particular peer's network address
    SetValue(key []bytes, value []bytes)
    // stores a small metadata value in DHT
    GetValue(key []bytes)
    // retrieves small metadata value from DHT
    ProvideValue(key Multihash)
    // announces this node can serve a large value
    FindValuePeers(key Multihash, min int)
    // gets a number of peers serving a large value
}

### Block Exchange
>清楚的示意圖
>[color=red][name=7/12討論]

IPFS use a Bittorrent inspired protocal: **BitSwap** to exchange blocks.

It use a **credit-like system** to incentivice nodes to seed and punish the leechers.
Let the ***debt ratio*** **r** between a node and its peer be:
$$r = \frac{bytes\_sent}{bytes\_recv+1}$$
and the *probability* of sending to a debtor be:
$$P(send|r) = 1-\frac{1}{1+exp(6-3r)}$$
The relation between $P$ and $r$:
So, the more bytes a peer send to you, the more probability you send bytes to him.

In the case that a node has nothing that its peers want (or nothing at all),
**it seeks the pieces its peers want, with lower priority** (cache) than what the node wants itself.
This incentivizes nodes to cache and disseminate rare pieces, even if they are not interested in them directly.
BitSwap nodes also keep **ledgers** accounting the transfers with other nodes.
>懲罰機制?
>[name=7/12討論][color=red]

type Ledger struct {
    owner NodeId
    partner NodeId
    bytes_sent int
    bytes_recv int
    timestamp Timestamp
}

Bitswap protocol specification :

// Additional state kept
type BitSwap struct {
    ledgers map[NodeId]Ledger
    // Ledgers known to this node, inc inactive
    active map[NodeId]Peer
    // currently open connections to other nodes
    need_list []Multihash
    // checksums of blocks this node needs
    have_list []Multihash
    // checksums of blocks this node has
}

type Peer struct {
    nodeid NodeId
    ledger Ledger
    // Ledger between the node and this peer
    last_seen Timestamp
    // timestamp of last received message
    want_list []Multihash
    // checksums of all blocks wanted by peer
    // includes blocks wanted by peer's peers
}

// Protocol interface:
interface Peer {
    open (nodeid :NodeId, ledger :Ledger);
    send_want_list (want_list :WantList);
    send_block (block :Block) -> (complete :Bool);
    close (final :Bool);
}

>上方 Peer 程式碼 interface & struct？ -> 僅為示意,語法是錯的
>[color=green][name=7/19會議]

Sketch of the lifetime of a peer connection:
1. Open: peers send ledgers until they agree.
2. Sending: peers exchange want_lists and blocks.
3. Close: peers deactivate a connection.
4. Ignored: (special) a peer is ignored (for the duration of a timeout)
if a node's strategy avoids sending

### Objects
IPFS builds a **Merkle DAG**[^first], a directed acyclic graph where links between
objects are cryptographic hashes of the targets embedded in the sources.
This is a generalization of the **Git data structure**.
[^first]:確保交易的順序

It provides some useful propoties, including:
1. **Content Addressing**: all content is uniquely identied
by its multihash checksum, including links.
2. **Tamper resistance**: all content is verified with its
checksum. If data is tampered with or corrupted, IPFS
detects it.
3. **Deduplication**: all objects that hold the exact same
content are equal, and only stored once. This is par-
ticularly useful with index objects, such as git trees
and commits, or common portions of data.

### Path to Files
After add file into IPFS, one can view the content by browsing gateway with
#### Format
> /ipfs/<hash-of-object>/<name-path-to-object>
#### Example
> /ipfs/XLYkgq61DYaQ8NhkcqyU7rLcnSa7dSHQ16x/foo.txt
> /ipfs/<hash-of-foo>/bar/baz.txt
> /ipfs/<hash-of-bar>/baz.txt
> /ipfs/<hash-of-baz>

### Naming
By using **IPNS**, one can access a mutable file with same link.
Example:
> /ipns/XLF2ipQ4jD3UdeX5xp1KBgeHRhemUtaA8Vm/
> /ipns/XLF2ipQ4jD3UdeX5xp1KBgeHRhemUtaA8Vm/docs
> /ipns/XLF2ipQ4jD3UdeX5xp1KBgeHRhemUtaA8Vm/docs/ipfs

It also provide **Peer link** feature let users can link other users' objects directly into their own objects. **[NOT implement]**
> Alice links to bob Bob
> ipfs link /<alice-pk-hash>/friends/bob /<bob-pk-hash>

> Eve links to Alice
> ipfs link /<eve-pk-hash/friends/alice /<alice-pk-hash>

>Eve also has access to Bob
/<eve-pk-hash/friends/alice/friends/bob

>access Verisign certified domains
/<verisign-pk-hash>/foo.com

>上傳的文件會存在多久？
>有 url，有無可能完全存取不到？
>[color=green][name=7/19討論]