---
# System prepended metadata

title: CAPER — Content-Addressed Protocol for Efficient Retrieval

---


# CAPER — Content-Addressed Protocol for Efficient Retrieval

:::success
**TODO**
- [ ] re publishing on the top-level specs site: could the site be repurposed so as to *only* contain the minimal viable specifications set? does that mean that it would import that multi* specs so as to be essentially complete?
:::

CAPER is a minimalist iteration over IPFS, focused on providing a bedrock interoperability layer
for typed content-addressed data in a way that is efficient, well-defined, and reasonably easy to 
implement.

*Note: The name is intended solely to be used as a convenient handle as this discussion progresses.*

## Motivation

IPFS is a wonderful family of technologies that compose to expose a set of operations on
content-addressed data. That what IPFS provides is desirable is undeniable: the 
[ecosytem directory](https://ecosystem.ipfs.tech/) speaks for itself.

Over the years, the set of protocols that make up IPFS have undergone many iterations and 
supported much experimentation. Existing implementations have developed features organically 
and progressively solidified around some backwards compatibility requirements.

This diversity and variability has taught us a lot and has produced beautiful results. However,
it has also turned IPFS into a sprawling and complex stack. Sitting under IPFS, libp2p alone 
is already an ecosystem unto itself, with a high multiplicity of approaches to peer-to-peer
communication.

As a result, it has become hard to tell what IPFS "is" or whether a given piece of software is
an IPFS implementation. We lack a shared specification of what a minimal implementation of 
IPFS is that is *guaranteed* to interoperate with the rest of the IPFS Cinematic Universe. 
Composability is wonderful, optionality isn't.

This document therefore lays out requirements for an IPFS bedrock ruthlessly aimed at 
interoperability and implementability. The underlying thesis is that a robust and well-defined
foundation will be better able to support a diverse and composable ecosystem at much higher
scales than today.

## Semantics

This section details the semantics that IPFS CAPER supports. A protocol concretizing these semantics
can then be developed, and other functionality layered on top of it. This approach is (loosely) 
modelled on [RFC 9110: HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110).

CAPER provides a uniform interface for interacting with content-addressed data — regardless of 
its type, nature, or implementation. This interface includes identifying, retrieving, and
providing data.

### Identify

The heart of content-addressing is content addresses. The fundamental substrate of IPFS is 
content-addressed blocks identified by CIDs.

CAPER imposes stricter restrictions compared to the broader CID world. Only CIDv1 is accepted. 
CIDv0 is fully deprecated. Systems that have CIDv0s need to convert them to CIDv1 before using them. 
(CIDv2 and CIDv3 are mentioned in various places, but they are currently reserved as imaginary
creatures.)

:::danger
**issue-cid-content-type**

The [CID spec](https://github.com/multiformats/cid) is vague about which multicodec values are 
valid in a CID:

> **What formats (multicodec codes) does CID support?** \
> We are figuring this out at this time. It will likely be a subset of multicodecs for secure 
distributed systems.

Based on the [decoding algorithm](https://github.com/multiformats/cid#decoding-algorithm), the
content type could be literally anything in the 
[multicodecs table](https://github.com/multiformats/multicodec/blob/master/table.csv), including
another CID or the multicodec for a hash.

One way to look at the content-type part of the CID is that it is similar to the media type used
in MIME and HTTP. That makes it essentially an open-ended set of values, with no guarantees that
any given client will understand the resource, only that it will be able to access the bytes.
However this approach may run up against what might *reasonably* (?) be expected from an IPFS
implementation, notably in terms of graph traversal. It also assumes that CID, as an open-world
typed content address format, may feature an arbitrary number of different content types and that
those are only meaningful if you can resolve them against the multicodecs table. Is this desirable?

Could this mean to divergent implementations on aspects that we care about (eg. CID → CAR of the
DAG)? Basically, the question here is: what do we *expect* that an IPFS implementation *must* be
able to do, and what can be delegated to another layer?

Alternatively, we could consider that only a small, fixed set of CID content types are relevant and
restrict CAPER CIDs to those. All other CIDs would be considered unresolvable over IPFS CAPER. Such
a list might look like:

* raw
* dag-cbor
* dag-pb
* car
* dag-json (or just CBOR?)

For everything else we could consider standardizing an IPLD structure with a media type field + 
bytes. No traversal would be expected at this layer.

Note that if the list includes CAR, then we would have to also address 
[unresolved issues with CAR](https://ipld.io/specs/transport/car/carv1/#unresolved-items) if we
want to provide interoperability guarantees.
:::

### Verify

> *Trustless, but verify.*

Every component that receives bytes matching a CID, be it because it retrieved the data or 
because it is a writable recipient for it, must validate the bytes against the CID hash
component.

Without verification we lose the decentralized, trustless nature of IPFS. We could consider
surfacing verification failure as a standard message similar to 404.

### Retrieve

An IPFS CAPER process can interact with another system (eg. a local store, another process) to
verifiably retrieve a CID.

:::danger
**issue-traversal**
- Do we want to expect of CAPER implementations that they support DAG traversal, arguably limited
  to a strict subset of traversable formats?
- If so, do we leave bounding graph traversal up as an implementation detail?
:::

:::danger
**issue-pathing**
- Do we want to expect of CAPER implementations that they support path traversal (which requires
  the ability to traverse graphs) on top of CIDs?
- If so, can we align on a *very* strictly limited set of pathing methods?
- We could have layers, with path retrieval sitting atop block retrieval, but for that to be 
  practical we would probably still want to limit pathing methods (ie. we shouldn’t end up with 
  twenty *FS variants).
:::

:::danger
**issue-private-retrieval**
- How private can we make retrieval in the bedrock?
:::

### Provide

An IPFS CAPER implementation must be able to indicate to other implementations what CIDs it is
able to provide.

A CAPER implementation may not be able to serve as a provider or receiver of content, it might be
restricted to connecting to IPFS processes for verified retrieval and writing. Such 
implementations are "clients" in the sense that they are peers that cannot be read from or written
to.

### Receive

An IPFS CAPER implementation must be able to be written to by other processes. Requirements for
writability are common across the stack.

Being writable likely means that the system should also support deletion (which it may or may 
not abide by, and without any guarantees that the CID resource will actually become unavailable if
it does).

### Advertize

An IPFS CAPER implementation should be able to tell other processes that interact with it which 
features on top of the required minimum it supports. (Think `OPTIONS`.)

A clear way of advertizing additional functionality makes it possible to select specific peers based
on what they support. It can also serve as a form of handshake mechanism when optimizations can be
applied.

## Further Reading

- [Defining IPFS and IPFS integrations](https://www.notion.so/Defining-IPFS-and-IPFS-integrations-1fc599790aa1424889c47d968e2db9d1)
- [IPFS User Intents](https://www.notion.so/d2138b8548fa43cea81f4a281be88c69?v=0c1c2d0d70834842a290ace89d3f4fbb&p=de2e0211274046dd8b8e1069929c4966&pm=s)