# IPFS Client W3C WICG Proposal

notes/questions/todo
- link to relevant specs implemented
- discuss and decide on a name, eg "ipfs client"

## Process / Status

Steps
- [x] draft explainer document, which provides the more context on the problem, intro to solution, open challenges, proposed scope, pointer to prior/current work
- [x] publish explainer document - https://little-bear-labs.github.io/ipfs-chromium/md_doc_explainer.html
- [x] draft proposal issue text - short summary of problem on the web today, along with proposed solution, should be a couple of paragraphs, and pointer to the longer explainer
- [ ] file issue [here](https://github.com/WICG/proposals/issues/new?assignees=&labels=remote-explainer&projects=&template=proposal-with-explainer.md&title=PROPOSAL+NAME)
- [ ] if incubation proposal is accepted, migrate the rest of the work below into the incubation repo eg github.com/wicg/ipfs-client or something like that

## Proposal Issue Text

Summarized version of below, into 2-3 paragraphs matching the requirements quoted below from [github.com/wicg/proposals](https://github.com/WICG/proposals?tab=readme-ov-file#what-does-a-proposal-look-like):

> What does a proposal look like?
>
> A proposal outlines a particular problem or challenge on the web and offers a potential concrete solution. Without being too prescriptive, you know you've got a proposal when you can articulate a specific way (procedurally, algorithmically, declaratively) that a new or current web technology solves an existing problem or challenge. If the problem is unclear, or the potential solution is too abstract you're not quite there yet.

And the issue template uses this text:
> *Describe the challenge or problem on the web you are proposing we address.*

DRAFT ISSUE OUTLINE

- problem + solution summary (1para)
- gateway statistics (scale of usage + problem)
- point to explainer, w/ context of chromium work so far (1para)
- links to previous works and resources wrt ipfs in browsers (list of references)

DRAFT ISSUE TEXT

The web today uses locations (URLs) as the sole resolver for content. This fails to meet user needs when content changes, websites are down, or companies close. While these constraints are features for many use cases, other use cases require content to be accessed independently of a single and/or original location. Examples are [web archives](https://arstechnica.com/gadgets/2024/02/google-search-kills-off-cached-webpages/) for mitigating mis/disinformation, content shared across/between websites, and web applications which need to work on networks where DNS resolution is not possible.

The [InterPlanetary File System (IPFS)](https://ipfs.io) meets these user needs in various amounts depending on how and where it is implemented. A common pattern is to access IPFS-addressed data via a single HTTP gateway, without verifying that the content is unmodified. This is problematic for many reasons, from privacy to data manipulation to scaling.

This is a proposal to discuss in the WICG how web browsers might integrate a subset of IPFS such that it is safe for users, aligns with the web's origin security model and does not require paradigmatic changes to browsers' HTTP-centric code base. Examples could be [IPFS CIDs for SRI](https://github.com/ipfs/in-web-browsers/issues/214), Fetch integration, native addressing support, and verifiable retrieval of content-addressed resources from IPFS through multiple HTTP gateways as has been prototyped thus far, and detailed in this [explainer](https://little-bear-labs.github.io/ipfs-chromium/md_doc_explainer.html).

The goals of incubation discussion at the WICG are:
- Identify and specify feature integration points
- Review privacy threats and solutions / mitigations
- Review security threats and solutions / mitigations
- Draft recommendations for user agent communication in UI to end users
- Scope and prioritize future IPFS features could work on the web, using more of its transport-agnostic nature for more user benefit

__IPFS Use Today__

The IPFS DHT has ~200,000 unique peers on average at the time of this writing. There are many client-only IPFS implementations which are not represented in this number.

HTTP use of the IPFS network through gateways is not represented in the DHT. There are many IPFS gateways, large and small. Some are listed at the [IPFS Public Gateway Checker](https://ipfs.github.io/public-gateway-checker/). The ipfs.io and dweb.link gateways are operated by Protocol Labs, and are averaging ~1.25 million requests per day.

There are many implementations and integrations of IPFS in non-browser contexts, from support in tools like [curl](https://curl.se/docs/ipfs.html), [ffmpeg and mpv](https://blog.ipfs.tech/2022-08-01-ipfs-and-ffmpeg/) to [implementations in various programming languages and platforms](https://docs.ipfs.tech/concepts/ipfs-implementations/).

The metrics above are all available on an ongoing basis at https://probelab.io.

__Browsers and Extensions__

- Chromium: Schemes are [allow-listed](https://chromestatus.com/feature/4776602869170176). Deeper support in Chromium is an ongoing project by Protocol Labs, Little Bear Labs, and Igalia on everything from better non-HTTP address handling to experimental IPFS client support. [Various issues filed for Chromium](https://issues.chromium.org/issues?q=ipfs), [blog post](https://blog.ipfs.tech/2023-05-multigateway-chromium-client/).
- Brave Browser: Brave has had IPFS features for many years, from bundling IPFS Companion to native address support to running a full Kubo node. See their [product documentation](https://brave.com/blog/ipfs-support/) for more on their IPFS features.
- Opera: Native IPFS address support in desktop, iOS and Android, redirected to an HTTP gateway as [announced here](https://blog.ipfs.tech/2021-02-08-opera-ios-and-ipfs/).
- Firefox: Has allow-listed the protocol schemes for browser extensions.
- IPFS Companion: Browser extension available for Chrome (and Chromium based browsers) and Firefox, adds a suite of features to pair your browser with a locally-running IPFS node. Available in [Chrome web store](https://chromewebstore.google.com/detail/ipfs-companion/nibjojkomfdiaoajekhjakgkdhaomnch) and [Mozilla addons for Firefox](https://addons.mozilla.org/en-US/firefox/addon/ipfs-companion/).

END DRAFT ISSUE TEXT

### Dependencies

An IPFS Verifying Client would likely implement all or part of the following IPFS specifications:

- CIDs: The Actual Specification, Not Code Which Implements it in a Github Repo Somewhere
- Addressing: The URI formats for IPFS addresses in web user agents. [link](https://github.com/ipfs/in-web-browsers/blob/master/ADDRESSING.md)
- Trustless Gateway: The HTTP API for interacting with IPFS HTTP gateway implementations without delegating any trust to them. [link](https://specs.ipfs.tech/http-gateways/trustless-gateway/)
- Delegated Routing: A mechanism by which IPFS nodes can offload routing-related work to another process or server. [link](https://specs.ipfs.tech/routing/http-routing-v1/)
- Web Pathing: A set of content paths for IPFS HTTP gateway implementations that are optimized for maximal HTTP and web standards alignment. [link](https://github.com/ipfs/specs/pull/453/files)

## Introduction

The InterPlanetary File System ([IPFS](https://ipfs.tech)) is a protocol for addressing data by its contents to ensure enduring and verifiable integrity and availability.

There are multiple ways of addressing IPFS content, many of which are listed in [Addressing IPFS on the web](https://docs.ipfs.tech/how-to/address-ipfs-on-web/#ipfs-addressing-in-brief).
However, the commonly used method of addressing IPFS resources in web browsers via HTTP URLs has varying and significant drawbacks. They permanently encode a single gateway at the point of addressing, for the lifetime of usage of the URL, amongst other challenges. In contrast, the `ipfs` scheme's addressing approach provides the resilient and verifiable addressing which IPFS was designed for.

Please read the Chromium IPFS Embedded Client [explainer] for a deep dive into what this looks like in practice.

## Goal

This proposal is to standardize `ipfs:` and `ipns:` schemes for addressing IPFS resources, how to validate them in web browsers, and any other features necessary to support IPFS websites.

## Compatibility requirements

### Minimum subsets of base ([multibase](https://github.com/multiformats/multibase#multibase-table))

Implementations should accept as input URLs encoded with any of:

* f - base16
* F - base16 (uppercase)
* b - base32
* B - base32 (uppercase)
* k - base36
* K - base36 (uppercase)
* z - base58btc (case-sensitive!)
* u - base64url (case-sensitive!)

For discussion on why this subset, see [this issue](https://github.com/multiformats/multibase/issues/120)'s discussion thread.

### Convert every address origin to base32 during canonicalization

This minimizes an origin not comparing equal to itself for a trivial reason (wrong base for digits). Some considerations as to which base to choose:

* Digits are Latin-1 only (maximum tool compatibility)
* Case-insensitive - Preferably avoid any base that uses uppercase letters at all
* Familiarity due existing popularity
* Performance - conversion speed of an optimized implementation
* Pre-existing standard proposal (i.e. [RFC4648](https://datatracker.ietf.org/doc/html/rfc4648))
* Compactness of encoding

These criteria lead us to believe base32 is the best choice as a canonical multibase target.

### Minimum subset of CIDv1 codecs

Relevant entries in the [multicodec table](https://github.com/multiformats/multicodec/blob/master/table.csv) are those tagged `ipld`.

* DAG-PB
* RAW
* libp2p-key (for IPNS names)
* DAG-CBOR
* DAG-JSON

### Minimum subset of multihashes

Relevant entries in the [multicodec table](https://github.com/multiformats/multicodec/blob/master/table.csv) are those tagged `multihash`.

At this stage in prototyping, support is limited to those most commonly found in existing DAGs and documents.

* sha2-256
* sha2-384
* sha3-512
* blake2b-256
* blake3
* Identity (i.e. the data itself inlined in place of a hash)
- Identity CIDs should not generate requests to gateways, since the data is always available by definition

### Validation

Implementations must implement the [Trustless Gateway](https://specs.ipfs.tech/http-gateways/trustless-gateway/#accept-request-header) specification for fetching data.

The specific choices of `Accept:` request header and/or query parameters are left as an excercize in choosing QoI trade-offs.

IPNS records must have their [signature validated](https://specs.ipfs.tech/ipns/ipns-record/#record-verification). Their expiration date must not have passed. If a record with a higher sequence number is discovered, all earlier records must be discarded.

IPFS blocks must have their hashes validated.

IPNS records which only have [V2 fields](https://github.com/ipfs/specs/pull/428) should be fully supported and considered [valid](https://specs.ipfs.tech/ipns/ipns-record/#record-verification).

### Caching

IPNS records should be cached according to their TTL field.

Immutable data (/ipfs/X/...) can be safely cached indefinitely, and should be cached as if `Cache-Control: public, max-age=29030400, immutable` (~12 months) was received. This may be done to either the blocks necessary for assembling responses to resource requests, or the final responses themselves.

### Features necessary to support existing websites

Although an implementation of this will not be running a gateway, existing sites are written and deployed with [subdomain gateways](https://specs.ipfs.tech/http-gateways/subdomain-gateway/) in mind, so that is a good source of comparison.

Some features required (MUST) for compatibility:

* Content-Type sniffing
* [_redirects](https://specs.ipfs.tech/http-gateways/subdomain-gateway/#ref-web-redirects-file)
* generating HTML to display preview of CBOR
* for a requested directory, load index.html if present
- otherwise generate HTML to display listing of the directory

### User configuration

A user must be able to control which gateway(s) their browser connects directly to. This would include, at a minimum:

* A list of gateways to always include
* Being able to toggle on/off any discovery mechanisms used
- If an implementation uses discovery, it may have a user-configurable deny list.
* Information/education on the implications of these choices.

An implementation MAY allow a user to configure/influence the algorithm chosen for assigning which requests to which gateways (or under which conditions).

## Recommendations

### Parallelism

In cases where multiple gateways are considered available and multiple unfulfilled but required CIDs are known, an implementation should request blocks in parallel. This most commonly occurs when requesting a multi-block file.

An implementation may make redundant requests to different gateways, cancelling extraneous requests once one validates properly, to reduce the risk of an especially slow response from a particular gateway.

### Feature detection

An implementation may consider using [Feature Detection API](https://github.com/ipfs/specs/pull/425), depending on its maturity, to consider which gateways get sent which _kinds_ of requests.

Failing that, one should respond to consistent failing status codes (or more likely) failing validation to deprioritize that gateway for that kind of request.

### Partial CAR

Optimize verifiable responses

### /routing/v1 API

Though a user should be able to opt out, the routing API provides a useful approach to gateway discovery

### Gateway affinity

A gateway which contains some blocks of a DAG are more likely to contain other blocks from the same DAG than another gateway is.

An implementation may take advantage of this pattern.

## Security & Privacy considerations

The Group will try to identify all the privacy and security issues of the features it produces, addressing them when possible or suggesting actions to mitigate them.

In-browser validation is critical for security and gateway selection is important for privacy. On the latter point, some degree of user education should be available through the settings UI. See the [explainer] for more discussion of this.

## Feedback

Please provide all feedback below.

[explainer]: https://little-bear-labs.github.io/ipfs-chromium/md_doc_explainer.html "IPFS Embedded Client Explainer"