Symbiosis protocol

Purpose & impact

Background & intent

Describe the desired state of the world after this project? Why does that matter?

There is an inherent asymmetry in node capabilities imposed by the constraints of the operating environment. Attempts to overcome these limitations often lead to ad-hoc stop-gap solutions that introduce centralization into the system. Teams building products on IPFS are required to fill capability gaps with (unintentionally) incompatible infrastructure of their own.

IPFS HTTP API provides inadequate solution for constrained environments as it is designed to give a single client an absolute control of the host node as opposed to coordinating access of multiple tenants.

Here we propose design which embraces asymmetry among node capabilities. Instead of traditional client-server model, symbiotic network is formed through a protocol that enables IPFS nodes with limited capabilities and/or lifespan (from now on referred as symbionts) to partner with IPFS nodes that posses greater capabilities and/or lifespan (from now on referred as hosts).

Through the symbiosis protocol a host can announce services that symbionts can request to overcome own limitations.

Protocol defines m:n relationship implying that a symbiont can have multiple partner hosts and a host could partner with multiple symbionts.

Assumptions & hypotheses

What must be true for this project to matter?

  • Neither embedded IPFS node nor IPFS client provide an adequate solution for building products in constrained environments.
  • OS level node sharing with existing building blocks is impractial.
  • Lack of generalized delegation protocol between nodes leads ad-hock stop-gap solutions.
  • Teams are unable to plug domain specific extensions without having to fork our stack.
  • Proposed solution avoids accidental centralization (0 cost in host symbiont switching)
  • Provides a seamless capabilities upgrade and downgrade experience.
User workflow example

How would a developer or user use this new capability?

Example: Symbiont leveraging multiple hosts

A symbiont on a phone may be paired with a host on a laptop to leverage improved capabilities when the two are on a same wifi network. At the same time a symbiont on the phone can be paired with a remote host over the internet to allowing all user devices to sync on the go.

Example: Web content leveraging IPFS Desktop

Multiple symbionts that are embedded in web apps (on different origins) can request services from the host embedded in an IPFS Desktop. IPFS Desktop would surface a permission prompt, if user grants it, the host will enable the requested services to the requesting symbiont. (e.g allowing it to have local area network access to discover and connect to other IPFS nodes in the same wifi)

Example: Cloud computing

Cloud computing provider exposes IPFS in their serverless platform. To do so they spin up IPFS symbionts, pre-paired with a multi-tenant always-on host at the edge.

Example: IPFS as an OS service

Desktop application embeds a node which can operate as a symbiont and/or a host. At startup it attempts to run as a symbiont that leverages an existing host (E.g exposed through a Unix Domain Socket). If host is not running, it starts one, which can be leveraged by itself and others.

Example: Filecoin based cold storage service

Host A provides AWS Glacier as "cold storage" service. Host B abstracts filecoin markets & deals as "cold storage" service. Symbiont can leverage either or both via single interface.

Impact

How directly important is the outcome to our top-level mission?

🔥🔥🔥

  • Enables teams to plug domain specific extensions without having to fork our stack.
  • Enables IPFS nodes to leverage local infrastructure.
  • Enables short-lived IPFS node use cases (e.g. mobile devices, serverless computing).
Leverage

How much would nailing this project improve our knowledge and ability to execute future projects?

🎯🎯🎯

  • Enables IPFS to expand into areas like serverless computing where both full IPFS node and IPFS Client are impractical.
  • Lays foundation for multi tenanant nodes that are needed to realize full potential of embeded IPFS in web browsers (e.g brave) and/or OS.
  • By enabling domain specific protocol extensions we grow the network instead of fragmenting it.
Confidence

How sure are we that this impact would be realized? Label from this scale.

Level 3

There is enough evidence supported by our own ad-hock stop-gap solutions combined with solutions our collaborators need to resort for to justify this level of confidence.

Project definition

Brief plan of attack
  1. Specificy symbiosis wire protocol.
  2. Implement domain service that host announces (will figure out the right one by talking to collabs).
  3. Implement permission handling flow on the host.
  4. Make symbiont leverage announced service through the protocol.
  5. Collaborate with teams to validate design with differnt use cases.
  6. Take learnings and implement zeroconfig service discovery.
What does done look like?

What specific deliverables should completed to consider this project done?

  • Have at least one collab adopt symbiosis protocol.
  • IPFS Desktop providing services that enhances in-browser node capabilities.
What does success look like?

Success means impact. How will we know we did the right thing?

  • Collabs are adopting a protocol to extend a stack instead of forking it.
Counterpoints & pre-mortem

Why might this project be lower impact than expected? How could this project fail to complete, or fail to be successful?

  • It does not adequately solve problems of our users.
  • Solution is to generic to be practical.
  • Service discovery is not reliable and still requires node configuration.
Alternatives

How might this project’s intent be realized in other ways (other than this project proposal)? What other potential solutions can address the same need?

  • Implement permission and authentification layer to existing IPFS HTTP API.
  • Allow users to extend HTTP API with domain specific routes.
Dependencies/prerequisites
  • none
Future opportunities
  • IPFS in serverless compute.
  • IPFS node sharing across applications.
  • Domain specific protocol extensions as services (e.g. replication optimization based on DAG structure knowledge like Textile does)

Required resources

Effort estimate
  • Large
  • Uncertainty on discovery implementation until several services are implemented.

Roles / skills needed

  • Ecosystem team member for coolab coordination.
  • At least on go engineer for host implementation.
  • At least on JS engineer for symbiont implementation.

Appendix

Constraints of the operating environment

There is an inherent asymmetry in node capabilities imposed by the constraints of the operating environment

Networking

  • Browsers can not be dialed

    Other than throughs WebRTC, which has bunch of problems:

    1. Implementations in browsers (as of this writing) use significantly more resources (than http connections ).
    2. Require centralized signalling
    3. Require TURN servers to rely traffic when direct connections can't be establish.
    4. Are not available in worker threads.

    In practice resulting in significant overhead, additional limitations while still relaying traffic through TURN server.

  • Browsers allow only a handful network connections.
  • Browsers can only dial addresses with SSL certificates.

Lifetime / Availability

  • Mobile operating systems make long lived connections impractical (backgrounding, battery usage)
  • Mobile phones oftes switch networks and there for network addresses
  • Web platform does not support long lived background tasks
    • service workers are deactivated as soon as fetch is complete
    • workers are terminated as soon as tab is closed
    • shared workers are not implemented in Apple webkit.
  • Cloud computing provides short lived execution model
  • Circuit Relay allows nodes to overcome some networking contraint by tunneling communication.
  • Pinning Service API standardizes a way to leverage some nodes in the system for data persistence & availabality.
  • Preload nodes allows data to remain available after browser node is gone.

Ad-hoc stop-gap solutions

Attempts to overcome inherent platform limitations often leads to stop-gap solutions that introduce centralization into the system:

Preload Service

Preload nodes augment JS-IPFS, exposing API endpoints for IPFS that aren't otherwise available in web platform. In practice, that means JS-IPFS clients can add some content locally, then use a preload node to request that CID, effectively caching the data and allowing the browser tab to be closed without the data instantly becoming unavailable.

The addresses are hardcoded in js-ipfs and need to be tied to their specific peerid.

Delegated content routing

(Ab)uses IPFS HTTP API (specifically /api/v0/dht/findprovs and /api/v0/refs endpoints in order to leverage more capable node in the network to perform content routing calls.

Unfortunately it is not part of the protocol. So routing node discovery happens out of band (In practice that is hard coded address in the configuration). It also implies setting up damain name and SSL/TLS certificates on the routing node.

Pinning Services API

Custom HTTP API enabling local IPFS node to manage pins on remote node(s).

This is great solution enabling different services to be plugged in. Still, if nodes were able to develop symbiotic relationships delegated pinning would just be part of it.

Centralized Signaling Services

Most discovery schemes supported in web environment requires centralized signalling service

(Unintentionally) Incompatible infrastructure

Teams building products on IPFS resort to filling capability gaps with custom (unintentionally) incompatible infrastrcuture.

  • http://fission.codes has a server component that enables (TODO: Verify details)
    1. User to associate multiple IPFS nodes (across user devices) with an account/identity.
    2. Update DNSLink records to reflect current state of user data.
    3. Data persistence to facilitate cross device sync.
  • Textie threads requires a deamon (TODO: verify)
    1. JS library is a client of the deamon.
  • 3Box maintains own infrastructure (TODO: Verify)
    1. To replicate user data.
    2. Facilitate pubsub.

HTTP API provides inadequate solution

IPFS HTTP API provides inadequate solution to the constrained environments

  • It gives client an absolute control of the host node

    • This is inpractical even with a localhost as clients (apps) need to be isolated and their access needs to be revocable.

      • Browser with embeded IPFS node would need to provide each app restricted & isolated access to the node
      • An OS level IPFS service would need to implement similar sandboxing to prevent data races.
      • If malicious web app tricking user into giving access, assume full control of node and all the user data in it.
      • Well intentioned clients can run into race conditions
        • Unintentionally overwrite path in MFS.
        • Unpin content other clients needs.
  • It provides inpracitcal path to incremental enhancements.

    • By default requests from web browsers are blocked (Due to CORS vialoation) which requires:

      1. Stopping a host node.
      2. Writing site origin into IPFS config file.
      3. Starting a host node back up.

      In the process host will drop all other clients. They will need to keep trying to reconnect and on success recreate local state

  • Fallback mechanism (like ipfs-provider) will needs to switch between JS-IPFS node and IPFS Client. This results in an observable difference:

    • IPFS Node with different peerid / address(es).
    • Different states of (M)File System and Pinset.

    Making it impractical when building competitive product experience.

  • Not a practical option on mobile

    • It is not viable to run a local IPFS host node on mobile device, due to backgrounding.
    • Remote IPFS host node would require
      • SSL/TLS certificates
      • Some form of authentification
      • A host per modile device.

Similar problems are manifested in desktop operating systems

  • Applications may want to use different node configurations

    This was pressing issue when we had IPFS Desktop, Textile Desktop Photos and Radicle.
    They all end up embedding differnt IPFS nodes to avoid this problem.
    // TODO: Check how does IPFS Desktop, and Space Daemon get along with one another

  • Using IPFS HTTP API for sharing would still inherit all the issues.

Symbiosys protocol

Protocol draws it's inspired from DNS Service Discovery (DNS-SD) a zero-configuration networking technique.

Wire protocol that is agnostic of transport and representation (message encoding) (This makes it a good fit for cross thread communication and a cross network interface).

At the high level protocol allows IPFS nodes to announce services they can provide to other nodes on the network, which can be requested by connected nodes to overcome their limitations.

Protocol defines:

  1. Interface enabling host node to announce services it can provide / lease.

    How announcement gets delivered is not specified, it can be over local mDNS over pubsub off band etc

  2. Interface enables connected symbiont to request an access to set of capabilities. Request encodes:

    1. Set of requested services / capabilities.
    2. A public key to associate permission / sandbox with.

Protocol assumes:

  1. Host provides service(s) to multiple unrelated symbionts.
  2. Host persists granted permissions across multiple sessions.
  3. Host Isolates and enforces access by a public key.

    Public key cryptography is used to enforce access level (Typically public key would correspond to peer id, but protocol does not mandate this to enable various use cases).

  4. Host is free to revoke permission

    Revokation reason can be arbitrary. E.g.

    • Payment for service was not received.
    • In IPFS Desktop user revokes permission to specific web page).

Services

Protocol does not prescribe specific set of services with an assumbtion that they could be developed as needed. That said below are couple of examples:

  • Enhanced dialability (tunneled through the host)
  • Local area network access
  • Remote pinning
  • Content routing
  • Content caching
  • Content naming (IPNS, DNSLink)
    • fission.codes updates DNSLink when user MFS changes
  • Domain specific replication
    • Textile threads sync without having to go back and forth
    • CRDT specific replication scheme
  • Querying
    • IPLD selectors
  • Inbox / Message queue
    • Textile allows queueing messages so you can get them when you're back online