# IPLD Explorers

**Abstract**: IPLD is the underlying data structure for IPFS. It is a huge hash-linked directed acyclic graph, with all files, git repos, blockchains, etc., within it. IPLD is the heart of IPFS. The tooling for manipulating IPLD directly has recently landed into both go-ipfs and js-ipfs under the API/commands `ipfs dag`. What we need next is to be able to make it nicer for humans to interact with IPLD directly. For this purpose, we have a few different "graph explorers" in mind. You can think of them similar to a visual explorer for a git repo (eg rendering the commit graph), or file explorers, or blockchain explorers. We aim to create several graph explorers, (1) one based on a traditional programming REPL, (2) another based on JSON tree explorers, (3) another based on a column-based tree viewer, (4) and a graphical version using d3 graph plotting, and (5) one in 3D which allows VR exploration.

## Introduction

To learn more about IPLD, please check out:

- https://github.com/ipld/ipld
- https://github.com/ipld/cid
- [IPLD: Enter the Merkle Forest](https://www.youtube.com/watch?v=Bqs_LzBjQyk) - conceptual talk from @jbenet
- [IPFS, IPLD, Blockchain Fun](https://www.youtube.com/watch?v=bi-4YGZXxwA&t=333s) - demo talk from @whyrusleeping
- http://ipld.io/
- website is outdated and will change soon to reflect important API changes, but explains the ideas ok. Talks may be better.

**The ["Relevant API" section in this document](#relevant-api) gives examples on ipfs node calls to make.**

## Explorers

### Explorer 1. IPLD REPL


Examples:
- `node`
- https://repl.it/languages/javascript
- https://binary-studio.com/wp-content/uploads/2015/05/node-repl.png

The first idea is to part from a very simple REPL. The REPL would be primarily about hashes and content. It would have a few "builtin" functions:

- `resolve(<path>)` - walk down a path and return the object found there
- `ls(<path>)` - list the entries in a path
- `tree(<path>, <depth>)` - list all entries starting from path, and up to `<depth>` graph depth. (may be useful)
- `get(<path>)` - walks down a path and returns that path

Entering `<path>` is the equivalent of `get <path>`, and every non-path value is added to ipfs, hashed, and its CID is returned.

A more advanced version would allow:

- `eval(<path>)`, which would load the value at `<path>`, interpret it as javascript, and eval it, adding it to the current context \o/.

### Explorer 2. tree explorers


Examples:
- https://shrimpworks.za.net/assets/projects/json-explorer/jsonex.png
- http://www.webandsay.com/images/json-diff-released/jsondiff.png
- http://visualizer.json2html.com/
- http://www.jsoneditoronline.org/

Structured explorers:
- https://blockexplorer.com/block/00000000000000000156c68693b097733b83084f327a10a591948b9dec54032e
- https://explorer.zcha.in/blocks/000000006263c2e8a35c0e950f5d139be1b6060554895f4327c81db4b1f00813

The tree explorer builds a very similar editor to traditional JSON tree explorers. This may be overkill for now.

### Explorer 3. column-based viewer


File browsers / explorers:
- https://en.wikipedia.org/wiki/File_manager
- https://tawus.files.wordpress.com/2011/08/filebrowser.png
- https://i.stack.imgur.com/j1v4A.jpg <-- columns

The column-based file system exploration fits our graphs very well, as well. This sould perhpas be the easiest to get working well. There's many examples and libraries out there.

### Explorer 4. graphical version using d3 graph plotting


This version visualizes the IPLD graph using d3. The tool at https://github.com/ipfs/dataviz may still work. it aims to-- the demo is just a way to go fix it.

- ipfs dataviz: https://github.com/ipfs/dataviz is currently working, and showing _files_. It should also show _the raw data_. Now that IPLD is out, the project could be updated. Though because having access to the file graph is useful, we may want to copy the visualization and remix it to navigate on nodes, instead of files.

- https://camo.githubusercontent.com/23d4bbfccd8aeb42dc9374827d364707c0e17981/68747470733a2f2f7261772e6769746875622e636f6d2f6b6573736c65722f7374617469632f6d61737465722f6e6f64652d6a736f6e2d6578706c6f7265722e706e67

### Explorer 5. 3D VR Explorer

Blockchain explorers:
- in 3d: https://www.youtube.com/watch?v=3ujUIz9hQ7c
- http://www.bitcoinlinks.net/files/styles/card_image__320x240_/public/img/bitbonkers_0.png?itok=2vKUm481

This explorer is a native way to look at the bitcoin blockchain; it's just done on a 3D environment! This is possibly super cool, but we'd love to let users navigate and explore, jumping from block to block.




## How to Implement

All of these explorers do one thing: traverse the graph. Therefore, we only need a couple of api calls to do this: `ipfs.dag.ls`, and `ipfs.dag.get`. See more in the [Relevant API](#relevant-api) section below.

### Path in the anchor and history

All explorers have to do with a given path. It is important to keep this path in the hash/anchor of the html page, as we want to be able to link to exact locations in the graph.

### Navigation path

Another thing all explorers track is navigation path. While this may be multiple nodes open in some explorers, it is enough to keep track of a single string path for the others. It would be useful if this navigation path is also given through the URL bar, so that people can link each other to the exact same location and "path used to get there".

## Relevant API

All of the explorers here will use the following api calls:

- `ipfs.dag.ls`
- `ipfs.dag.get`

See [the DAG API](https://github.com/ipfs/interface-ipfs-core/tree/master/API/dag#dagget), which works in both js-ipfs and js-ipfs-api.

### Example: `ipfs.dag.ls`

var IPFSAPI = require('ipfs-api')
var ipfs = IPFSApi('/ip4/127.0.0.1/tcp/5001')

// given an object such as: 
//
// var obj = {
//     "a": 1,
//     "b": [1, 2, 3],
//     "c": {
//         "ca": [5, 6, 7],
//         "cb": "foo"
//     }
// }
// 
// ipfs.dag.put(obj, function(err, cid2) {
//     assert(cid == zdpuAkxd9KzGwJFGhymCZRkPCXtBmBW7mB2tTuEH11HLbES9Y)
// })

ipfs.dag.ls('zdpuAkxd9KzGwJFGhymCZRkPCXtBmBW7mB2tTuEH11HLbES9Y', function(err, result) {
    for (var i in result.entries) {
        console.log(result.entries[i])
    }
})
// Returns:
// a
// b
// c

ipfs.dag.ls('zdpuAkxd9KzGwJFGhymCZRkPCXtBmBW7mB2tTuEH11HLbES9Y/c', function(err, result) {
    for (var i in result.entries) {
        console.log(result.entries[i])
    }
})
// Returns:
// ca
// cb

ipfs.dag.ls('zdpuAkxd9KzGwJFGhymCZRkPCXtBmBW7mB2tTuEH11HLbES9Y/c/ca', function(err, result) {
    for (var i in result.entries) {
        console.log(result.entries[i])
    }
})
// Returns:
// 0
// 1
// 2

### Example: `ipfs.dag.tree`

var IPFSAPI = require('ipfs-api')
var ipfs = IPFSApi('/ip4/127.0.0.1/tcp/5001')

// given an object such as: 
//
// var obj = {
//     "a": 1,
//     "b": [1, 2, 3],
//     "c": {
//         "ca": [5, 6, 7],
//         "cb": "foo"
//     }
// }
// 
// ipfs.dag.put(obj, function(err, cid2) {
//     assert(cid == zdpuAkxd9KzGwJFGhymCZRkPCXtBmBW7mB2tTuEH11HLbES9Y)
// })

ipfs.dag.tree('zdpuAkxd9KzGwJFGhymCZRkPCXtBmBW7mB2tTuEH11HLbES9Y', function(err, result) {
    for (var i in result) {
        console.log(result[i])
    }
})
// Returns:
// a
// b
// b/0
// b/1
// b/2
// c
// c/ca
// c/ca/0
// c/ca/1
// c/ca/2
// c/cb

ipfs.dag.tree('zdpuAkxd9KzGwJFGhymCZRkPCXtBmBW7mB2tTuEH11HLbES9Y/c', function(err, result) {
    for (var i in result) {
        console.log(result[i])
    }
})
// Returns:
// ca
// ca/0
// ca/1
// ca/2
// cb

### Example: `ipfs.dag.get`

var IPFSAPI = require('ipfs-api')
var ipfs = IPFSApi('/ip4/127.0.0.1/tcp/5001')

// given an object such as: 
//
// var obj = {
//     "a": 1,
//     "b": [1, 2, 3],
//     "c": {
//         "ca": [5, 6, 7],
//         "cb": "foo"
//     }
// }
// 
// ipfs.dag.put(obj, function(err, cid2) {
//     assert(cid == zdpuAkxd9KzGwJFGhymCZRkPCXtBmBW7mB2tTuEH11HLbES9Y)
// })

function errOrLog(err, result) {
    if (err) console.error('error: ' + err)
    else console.log(result)
}

ipfs.dag.get('zdpuAkxd9KzGwJFGhymCZRkPCXtBmBW7mB2tTuEH11HLbES9Y/a', errOrLog)
// Returns:
// 1

ipfs.dag.get('zdpuAkxd9KzGwJFGhymCZRkPCXtBmBW7mB2tTuEH11HLbES9Y/b', errOrLog)
// Returns:
// [1, 2, 3]

ipfs.dag.get('zdpuAkxd9KzGwJFGhymCZRkPCXtBmBW7mB2tTuEH11HLbES9Y/c', errOrLog)
// Returns:
// {
//   "ca": [5, 6, 7],
//   "cb": "foo"
// }

ipfs.dag.get('zdpuAkxd9KzGwJFGhymCZRkPCXtBmBW7mB2tTuEH11HLbES9Y/c/ca/1', errOrLog)
// Returns:
// 6