owned this note
owned this note
Published
Linked with GitHub
# Notary v2 Fallback Registry Support
## Goals
- Able to push a signature to existing registries without server-side modifications
- Works with registries that restrict media types on uploaded manifests
- Works with registries that enforce immutable tags
- Does not break existing container runtimes
- Able to copy signatures to other repositories and registries
- Able to pull signature by tag
- Able to pull signature by target image digest
- Able to deploy by digest to a signed image (looser version of above requirement, allowing the digest to be an index containing the signature and image)
- Able to push multiple signatures for the same image
- Able to copy a subset of signatures
- Local signatures coexist with upstream signatures that are periodically synced
- Avoids race conditions
- Able to attach a signature to an already pushed image
- Does not modify the manifest/digest of the image
- Able to sign individual images
- Able to sign a manifest list
- Avoids cluttering the tag listing
- Signatures are deleted with the signed artifact
## Suggested Format Change
### Definitions
- **Artifact** - definition
- **Reference Type** - definiton
### Requirements
| # | Item | Description |
| - | - | - |
| 1 | Push a single level reference without registry changes | Fallback support for existing registries, one level deep. Signature on an image, sbom on an image, but may not support signature on sbom for an image |
| 2 | Push multi-level references, wihtout registry changes | Similar to above, with n-depth support |
| - | add above items | |
| n | Support multiple blobs in a reference type | An artifact may have multiple blobs that make up the artifact. A helm chart is a reference type, which has two layers |
| n1 | Lifecycle management, based on the subject tag | Users that wish to delete the root object can delete the graph of references |
### Comparison of Options
Same table #'s above, with the green/yellow/red checkboxes, with footnotes for any details.
## Possible Implementations
### Pushing OCI Artifact with a digest specific tag
This model follows the [cosign design](https://github.com/sigstore/cosign/blob/main/specs/SIGNATURE_SPEC.md#storage) where the digest of the signed object is used to create a tag in the form:
<digest-alg>-<digest>.<suffix>
So if the digest for the image to be signed is `sha256:123456` then the signature may be pushed as an OCI artifact with the tag `sha256-123456.sig`.
Potential issues:
- Clutters the tag listing
- Not all registries allow OCI artifacts, but this can be worked around by reusing container image media types the registry permits
- Pushing multiple signatures requires one of:
- Adding some kind of uuid to the tag which slows retrieves (extra tag listing API call)
- Wrapping the signatures in an index (introducing race conditions and making copies with more/fewer signatures difficult to maintain)
- Changing the OCI artifact to have an array of signatures (same issues with the index above)
- Registries that enforce immutable tags may not be able to update or add sigatures without using some kind of uuid on the tag
- When deleting a signed artifact, signatures may persist
### Pushing OCI Artifact as a separate manifest in an OCI Index
The resulting index may look like:
```json=
{
"mediaType": "application/vnd.oci.image.index.v1+json",
"schemaVersion": 2,
"manifests": [
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:111111111111111",
"size": 1152,
"platform": {
"architecture": "386",
"os": "linux"
}
},
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:aaaaaaaaaaaaaa",
"size": 1152,
"platform": {
"architecture": "386",
"os": "linux"
},
"annotations": {
"org.notaryproject.sig.ociartifact": "true"
}
},
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:222222222222222",
"size": 1152,
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:bbbbbbbbbbbbbbb",
"size": 1152,
"platform": {
"architecture": "amd64",
"os": "linux"
},
"annotations": {
"org.notaryproject.sig.ociartifact": "true"
}
}
]
}
```
Runtimes looking for the correct image should pull the first matching index entry for their platform (`sha256:111111111111111` and `sha256:222222222222222`) (see [distribution-spec PR #880](https://github.com/opencontainers/image-spec/pull/880)).
Signature verification would look for entries with the appropriate annotation (e.g. `org.notaryproject.sig.ociartifact` found on `sha256:aaaaaaaaaaaaaa` and `sha256:bbbbbbbbbbbbbbb`).
Potential issues:
- Not all registries allow OCI artifacts, but this can be worked around by reusing container image media types the registry permits
- Runtimes need testing to ensure they work with multiple matching index entries and pull/run the image and not the signature
- Unable to query the digest of the image, must either pin to the digest of the index or fall back to digest tags
- Signing an index (manifest list) would result in an index pointing to an index which may break some runtimes
- Adding signatures to an image will change the digest on the index wrapper, making signature copies difficult to maintain if they are either a subset or extended with local signatures
- Race conditions exist when making multiple concurrent changes to the index wrapper
- Registries that enforce immutable tags cannot update or add sigatures
### Attaching signature as annotation to descriptor in an OCI Index
The resulting index may look like:
```json=
{
"mediaType": "application/vnd.oci.image.index.v1+json",
"schemaVersion": 2,
"manifests": [
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:111111111111111",
"size": 1152,
"platform": {
"architecture": "386",
"os": "linux"
},
"annotations": {
"org.notaryproject.sig": "U2lnbmVkIHNoYTI1NjoxMTExMTExMTExMTExMTEK"
}
},
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:222222222222222",
"size": 1152,
"platform": {
"architecture": "amd64",
"os": "linux"
},
"annotations": {
"org.notaryproject.sig": "U2lnbmVkIHNoYTI1NjoyMjIyMjIyMjIyMjIyMjIK"
}
}
]
}
```
In this case the signature is base64 encoded as the annotation.
A signature for a single platform image would be packaged as an index with a single manifest.
This Index may be pushed as the image tag, or using separate [digest specific tags](#Pushing-OCI-Artifact-with-a-digest-specific-tag)
Potential issues:
- Testing needed to verify registries do not block annotations
- As annotations are added, the size of the manifest grows
- Unable to push multiple signatures without adding a counter to the annotations or pushing multiple wrappers (separate digest tags)
- With digest specific tags:
- Clutters the tag listing
- When deleting a signed artifact, signatures may persist
- Without digest specific tags:
- Signing an index (manifest list) would result in an index pointing to an index which may break some runtimes
- Adding signatures to an image will change the digest on the index wrapper
- Race conditions exist when making multiple concurrent changes to the index wrapper
- Registries that enforce immutable tags may not be able to update or add sigatures without using digest specific tags and uuids on those tags
## Comparison of options
| | Digest specific tag | Separate manifest in index | Inline as annotation on same tag | Inline as annotation with digest tag |
|-----|-----|-----|-----|-----|
| Able to push a signature to existing registries without server-side modifications | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Works with registries that restrict media types on uploaded manifests | :white_check_mark: | :white_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Works with registries that enforce immutable tags | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
| Able to copy signatures to other repositories and registries | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Does not break existing container runtimes | :heavy_check_mark: | :grey_question: | :heavy_check_mark: | :heavy_check_mark: |
| Able to pull signature by tag | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Able to pull signature by target image digest | :heavy_check_mark: | :x: | :x: | :heavy_check_mark: |
| Able to deploy by digest to a signed image | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Able to push multiple signatures for the same image | :white_check_mark: | :heavy_check_mark: | :white_check_mark: | :white_check_mark: |
| Able to copy a subset of signatures | :heavy_check_mark: | :white_check_mark: | :white_check_mark: | :heavy_check_mark: |
| Local signatures coexist with upstream signatures that are periodically synced | :heavy_check_mark: | :white_check_mark: | :white_check_mark: | :heavy_check_mark: |
| Avoids race conditions | :white_check_mark: | :x: | :x: | :white_check_mark: |
| Able to attach a signature to an already pushed image | :heavy_check_mark: | :white_check_mark: | :white_check_mark: | :heavy_check_mark: |
| Does not modify the manifest/digest of the image | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Able to sign individual images | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Able to sign a manifest list | :heavy_check_mark: | :x: | :x: | :heavy_check_mark: |
| Avoids cluttering the tag listing | :x: | :heavy_check_mark: | :heavy_check_mark: | :x: |
| Signatures are deleted with the signed artifact | :x: | :heavy_check_mark: | :heavy_check_mark: | :x: |
Key:
- :heavy_check_mark: : supported
- :white_check_mark: : requires workarounds to support
- :grey_question: : research needed
- :x: : unsupported