# Brave support for dat:// ecosystem

## Milestone 1. Access to the dat:// network API

### Deliverables

* Any website (http, ipfs) can use client-side JavaScript to fetch data from the dat:// network and create custom app logic to display and modify data.
* This would work similar to WebTorrent, as an API used in websites, rather than dat:// URLS in the browser exactly. This milestone is a prerequistite to viewing dat:// URLS.

### Roadmap

1. Make an extension, using browerify to replace net and dgram with Chrome APIs.
2. Initialize hyperswarm-proxy server.
3. Inject JS into pages which talks to extension over a stream speaking hyperswarm-proxy.

### JS APIs provided by Brave extension

- Hyperswarm
- This creates a 'swarm' of peers using a Kademlia DHT maintained by the dat:// community.
- This will be compatible with the next version of Beaker.
- App developers will be able to see peerId for each peer, which are randomly generated hashes. User ip:port will be hidden inside the extension to protect user privacy.

### Required Input from Brave team
- Help design how best to get Hyperswarm injected from extension (globals, imports, etc? also naming considerations)

### Example
Simplified example in a typical website

1. Share content

$ dat share my-cool-cats
Created new dat in my-cool-cats
dat://fbab844afd3c2f4ad7a736fbf7e4479683f199610a82db54005ff9d4a3dd7f99
Sharing dat: 782 files (5.5 MB)

2. Fetch content & seed to peers

import { Hyperdrive } from '@dat/sdk'; // or include dat.min.js in script tag 

const drive = Hyperdrive();

drive.join('dat://fbab844afd3c2f4ad7a736fbf7e4479683f199610a82db54005ff9d4a3dd7f99');

const catVideo = drive.readFile('/mycat.mp4')

Under the hood, this expands to the following

const Hyperswarm = ...
// TODO: design how to get Hyperswarm injected from extension

const dat = require('@dat/sdk')
const raw = require('random-access-web')
const swarm = Hyperswarm()

const drive = dat.Hyperdrive(raw)

// `socket` is duplex Node stream. `details` specifies topic (if possible), and if connection is incoming
swarm.on('connection', (stream, details) => {
    // Raw connection here, can use own multiplexing connection
    stream.pipe(drive.replicate()).pipe(stream)
})

// Support EventTarget? Stream is a WHATWG stream?
swarm.addEventListener('connection', (stream, details) => {})

// Topic should be an ArrayBuffer with 32 bytes or dat:// url, else it'll be hashed into one.
swarm.join('dat://fbab844afd3c2f4ad7a736fbf7e4479683f199610a82db54005ff9d4a3dd7f99')

### Supporting Context
Replication

- Hyperswarm & hypercore-protocol
- Forward extensions as-is, to support custom apps
- https://www.npmjs.com/package/hyperswarm-proxy

Storage

- In-browser (duplicated between app origins)
- Use random-access-web in apps using hyperswarm directly

JavaScript API
- dat-sdk: https://github.com/datproject/sdk

Compatibility
- Uses same replication protocol and crypto as `dat` CLI

## Milestone 2. Viewing dat:// URLs

### Deliverables
- Visiting `dat://<KEY>` or dat://dat.foundation should start a hyperswarm instance and connect to the swarm, downloading the archive
- If present in the archive, an index.html is rendered, otherwise show a list of top-level files and directories. This resolution is recursive for sub-directories according to [resolve-dat-path](https://github.com/RangerMauve/resolve-dat-path)
- This will support streaming media like videos from Dat using regular `<video>` elements with no JS

### Example

<video src="dat://.../mycat.mp4" />
<a href="dat://example.com/cats">My Link</a>

### Roadmap
1. Implement Hyperdrive into the extension. This is a Node.js library.
* This could be implemented either SOCKS5 proxy with native Node.js daemon, or in JavaScript. We'd prefer to run in the extension, but this is open for discussion
* Use sparse mode so that feeds are fetched when requested by the user browsing the webpage, ensuring that users only download what they have requested.
4. Domain is considered secure like https:// (access to "secure" web APIs) because the contents are protected by transport encryption and content signing
5. Integrate [dat-dns](https://github.com/datprotocol/dat-dns) so that users can use DNS to request websites.

### Compatibility

* Communicates through RPC layer to extension.
* Uses same networking code as Beaker and CLI

### Needs from Brave team

* Daemon over SOCKS5 proxy or JavaScript land similar to js-ipfs? Since dat is implemented in Node.js, JS implementation in the extension is how it works in Cliqz.
* What are the performance/integration/security considerations? Curious to hear what Brave team thinks is a better approach.

## Milestone 3. APIs for creating Archives/Feeds for full compatibility with Beaker

### Deliverables
* Websites can use the built in Hyperdrive and Hypercore JavaScript APIs to create archives within the Dat extension
* The SDK will auto-detect their presence for added performance
* Beaker compatibility

### Roadmap
- Define RPC protocol for Hyperdrive and Hypercore
- Wire up RPC to SDK instance within extension
- Inject JS APIs into pages using same method as Hyperswarm
- Automatically seed created archives
- Agressively garbage collect unused archives and feeds

### Hyperdrive API over RPC

- Hyperdrive API data and replication managed by extension
- In-app you'll need to remember what archives you created (for deleting)
- Future feature: control over replication `local: true`, `dht: true`, `announce: true`, `lookup: true`

const archive = window.Hyperdrive('some key')

// Use node-style CBs for legacy
archive.readFile('example', (err, data) => whatever)

// Use promises for new apps
await archive.readFile('example')

// Use extensions same way as in Node
archive.registerExtension('example', {
    encoding: 'json'
    onMessage: (message, peer) => peer.send('Hello World')
})

### Hypercore API over RPC

- Use similar API to hypercore-promise or the SDK
- Could be adapted to existing apps with minimal effort
- Easier to use for new users just getting into Dat

const feed = window.Hypercore('some key')

// Existing node style API for legacy
feed.append('Hello', (err) => whatever)

// Promsie API for new
cosnt data = await feed.get(0)

// Listen on custom messages over replication connections
feed.registerExtension('multifeed', {
    encoding: 'json'
    onMessage: (message, peer) => saveKeys(message)
})

// Listen on peer event using EventEmitter
feed.once('peer-add', (peer) => peer.send(getKnownKeys()))

// Also support EventTarget
feed.addEventListener('peer-add', ({peer}) => peer.send(getKnownKeys()))