<style>
.ui-infobar, #doc.markdown-body { max-width: 1024px; }
</style>

# Accelerate container image innovation with Oras Artifact and Referrer API

In the past few years, as the rapid adoption of container technology in many scenarios, the container images become bigger and bigger. We often see our customers bring their gigabytes-sized images and hope to start their workloads on thousands of machines instantaneously.

A container image consists of multiple read-only layers. Each layer consists of multiple directories and files. To distribute the image, the files in each layer are usually packed and compressed into a single tar.gz file and pushed into a remote registry. To start the container on a new machine, the client needs to first download and decompress each layer tar.gz file. This process is known as `image pull`. For a gigabytes-image, the `image pull` can take minutes which largely depends on the network and disk conditions.

To address the cold-start problem, the container community has started several initiatives, for example, [Stargz](https://github.com/containerd/stargz-snapshotter), [DADI](https://github.com/alibaba/accelerated-container-image), [Teleport](https://github.com/Azure/acr/blob/main/docs/teleport/README.md) and etc.

All of the solutions need to convert the original tar.gz layer files to an optimized format which allows the corresponding client to only fetch the minimal required data to quickly start the container.

On the other side, tar.gz is still the de-facto layer format and supported by most image build tools as the default.

As an open-source community, we are thrilled to see the fast innovation on the image format. Realistically I don't expect any of these new image format can dominate the container world soon. I'd hope to see all these image formats can work side by side with the common tar.gz layer format. The client can query the compatible image format and decide how to fetch the layer to start the container. From the end user perspective, they can use the same image reference (eg, myregistry/myimage:mytag) to deploy the container on different environments.

To enable the above experience, we need to resolve two problems:

1. Have a way to define the relationship between different image artifacts for the same image.
2. Have a way to query the associated artifacts of an image.

**Oras Artifact and Referrer API come to rescue.**

[Oras Artifact project](https://github.com/oras-project/artifacts-spec) is a community initiative to extend [OCI image artifact](https://github.com/opencontainers/artifacts). It is built on top of [OCI distribution-spec](https://github.com/opencontainers/distribution-spec). It allows the artifact publisher to define the reference between the artifacts in their manifests. It also introduces a new [Referrer API](https://github.com/oras-project/artifacts-spec/blob/main/manifest-referrers-api.md) to query the associated artifacts.

Let's take an example to see how it can support various image artifacts for the same image. In this example, I will use DADI optimized image format.

The following is the manifest for a `redis:latest` image based on tar.gz layer format. The manifest digest is **`sha256:820582b05253c2b968442b8af31d791ae64478bcc18e04826c5ce42f974d3272`**.

{
    "schemaVersion": 2,
    "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
    "config": {
        "mediaType": "application/vnd.docker.container.image.v1+json",
        "size": 7698,
        "digest": "sha256:de974760ddb2f32dbddb74b7bb8cff4c1eee06d43d36d11bbca1dc815173916e"
    },
    "layers": [
        {
            "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "size": 27139373,
            "digest": "sha256:f7ec5a41d630a33a2d1db59b95d89d93de7ae5a619a3a8571b78457e48266eba"
        },
        {
            "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "size": 1730,
            "digest": "sha256:a36224ca8bbdc65bae63504c0aefe0881e4ebdf1ec362830b347a2ab5120a8de"
        },
        {
            "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "size": 1417921,
            "digest": "sha256:7630ad34dcb2a98e15daa425b6e076c06f9f810c7714803a3bdbcf750a546ea0"
        },
        {
            "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "size": 10097197,
            "digest": "sha256:dd0ea236b03be5127b0b7fb44f1d39f757c5381a88d650fae0a0d80b0c493020"
        },
        {
            "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "size": 133,
            "digest": "sha256:ed6ed4f2f5a63e3428ddd456805d71dc55f68b763ae8d5350f29b8cea46373f2"
        },
        {
            "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "size": 409,
            "digest": "sha256:8788804112c6c0d3e91725e26d741c9b8e4ad910e63136e129043220eb8d11d4"
        }
    ]
}

After we convert the image to DADI optimized image format, we can push it to the registry as an Oras artifact and let it reference the original image manifest (**`sha256:820582b05253c2b968442b8af31d791ae64478bcc18e04826c5ce42f974d3272`**).

{
    "mediaType": "application/vnd.cncf.oras.artifact.manifest.v1+json",
    "artifactType": "dadi.image.v1",
    "subject": {
        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
        "digest": "sha256:820582b05253c2b968442b8af31d791ae64478bcc18e04826c5ce42f974d3272",
        "size": 16724
    },
    "blobs": [
        {
            "mediaType": "application/vnd.docker.container.image.v1+json",
            "digest": "sha256:8e3bd5ee258431fd6bb25652afc666f7582777229c5147e096ccf9b20c06ac1c",
            "size": 3688,
        },
        {
            "mediaType": "application/vnd.oci.image.layer.v1.tar",
            "digest": "sha256:a8b5fca80efae55088290f3da8110d7742de55c2a378d5ab53226a483f390e21",
            "size": 4739584,
            "annotations": {
                "containerd.io/snapshot/overlaybd/blob-digest": "sha256:a8b5fca80efae55088290f3da8110d7742de55c2a378d5ab53226a483f390e21",
                "containerd.io/snapshot/overlaybd/blob-size": "4739584"
            }
        },
        {
            "mediaType": "application/vnd.oci.image.layer.v1.tar",
            "digest": "sha256:87763befd4f3289905d709bd03c969db43e512502be7e1132b625bdef487d01f",
            "size": 43458048,
            "annotations": {
                "containerd.io/snapshot/overlaybd/blob-digest": "sha256:87763befd4f3289905d709bd03c969db43e512502be7e1132b625bdef487d01f",
                "containerd.io/snapshot/overlaybd/blob-size": "43458048"
            }
        },
        {
            "mediaType": "application/vnd.oci.image.layer.v1.tar",
            "digest": "sha256:5bf55fa8550c47a1054c7a02138b9f79b5f574f040b1e444ad717d320d3afc67",
            "size": 25600,
            "annotations": {
                "containerd.io/snapshot/overlaybd/blob-digest": "sha256:5bf55fa8550c47a1054c7a02138b9f79b5f574f040b1e444ad717d320d3afc67",
                "containerd.io/snapshot/overlaybd/blob-size": "25600"
            }
        },
        {
            "mediaType": "application/vnd.oci.image.layer.v1.tar",
            "digest": "sha256:62a999219eb529a2403f2b5849869d3253bf1014721293333f7f66be54308b94",
            "size": 2610688,
            "annotations": {
                "containerd.io/snapshot/overlaybd/blob-digest": "sha256:62a999219eb529a2403f2b5849869d3253bf1014721293333f7f66be54308b94",
                "containerd.io/snapshot/overlaybd/blob-size": "2610688"
            }
        },
        {
            "mediaType": "application/vnd.oci.image.layer.v1.tar",
            "digest": "sha256:f2d33f598db59a8a4fcb490764cdfca3157ec6a742870378154cbef93acefce9",
            "size": 17303040,
            "annotations": {
                "containerd.io/snapshot/overlaybd/blob-digest": "sha256:f2d33f598db59a8a4fcb490764cdfca3157ec6a742870378154cbef93acefce9",
                "containerd.io/snapshot/overlaybd/blob-size": "17303040"
            }
        },
        {
            "mediaType": "application/vnd.oci.image.layer.v1.tar",
            "digest": "sha256:8d77203e222f30ab4b8ba2e232fd9d71880dd80f6f24fa18e45d1d578e40eb57",
            "size": 8192,
            "annotations": {
                "containerd.io/snapshot/overlaybd/blob-digest": "sha256:8d77203e222f30ab4b8ba2e232fd9d71880dd80f6f24fa18e45d1d578e40eb57",
                "containerd.io/snapshot/overlaybd/blob-size": "8192"
            }
        },
        {
            "mediaType": "application/vnd.oci.image.layer.v1.tar",
            "digest": "sha256:8bdb50d0eb5ec766ba235c06ac8c8a6f44ab1beeed756efa532e73b79786e36a",
            "size": 11776,
            "annotations": {
                "containerd.io/snapshot/overlaybd/blob-digest": "sha256:8bdb50d0eb5ec766ba235c06ac8c8a6f44ab1beeed756efa532e73b79786e36a",
                "containerd.io/snapshot/overlaybd/blob-size": "11776"
            }
        },
        {
            "mediaType": "application/vnd.oci.image.layer.v1.tar",
            "digest": "sha256:c32be80bd739307433858c79994ffc050393691d77453bd93bee1cc89c7473c1",
            "size": 11264,
            "annotations": {
                "containerd.io/snapshot/overlaybd/acceleration-layer": "yes",
                "containerd.io/snapshot/overlaybd/blob-digest": "sha256:c32be80bd739307433858c79994ffc050393691d77453bd93bee1cc89c7473c1",
                "containerd.io/snapshot/overlaybd/blob-size": "11264"
            }
        }
    ]
}

Optionally we can also sign the Dadi image artifact and push the signature as another Oras artifact. The diagram below shows the reference relationship between the artifacts.


When a Dadi compatible client tries to run the redis image on a fresh machine, it will follow the steps below to query and pull the optimized image artifact.

1. Call Manifest API to get the image digest of `redis:latest` which is `sha256:820582b05253c2b968442b8af31d791ae64478bcc18e04826c5ce42f974d3272`.
* GET `/v2/redis/manifests/latest`
3. Call Referre API to query the referrer artifact manifest whose type is "dadi.image.v1".
* GET `/oras/artifacts/v1/redis/manifests/sha256:820582b05253c2b968442b8af31d791ae64478bcc18e04826c5ce42f974d3272/referrers?artifactType=dadi.image.v1`
3. [Optional] If policy required, call Referre API again to qeury the signature artifact of the Dadi image artifact. Verify the signature.
4. Pull the layers (blobs) defined in the Dadi image artifact manifest and start the container.