Open Containers Initiative - October 24, 2022

Location

Room 335 & Online
Same Zoom as the OCI Dev Call!
https://zoom.us/my/opencontainers?pwd=S2tJVGVra0dYdlZCRjJwdXdPdGRQQT09
Passcode: 77777

Recording

https://youtu.be/8lPr9cbLSmA

Date + Time

Monday, October 24
10am to 12pm Eastern

Attendees

  • Phil Estes 👩‍💻
  • Brandon Mitchell 💻
  • Jason Hall 💻
  • nisha 💻
  • josh dolitsky 💻
  • Lachlan 💻
  • Akihiro 👩‍💻
  • vbatts 👩‍💻
  • Jesse Butler 👩‍💻
  • cra 👩‍💻
  • amye 👩‍💻
  • Mike Brown 👩‍💻
  • Michael Brown 👩‍💻
  • Sebastiaan van Stijn 👩‍💻
  • tianon 💻
  • Jimmy Zelinski 👩‍💻
  • Alexander 👩‍💻
  • Markus Lehtonen 👩‍💻
  • Ramkumar Chinchani
  • 👩‍💻

Notes

  • artifact support/v1.1 release candidates available; yay! Now, how do we finish the work - Mike Brown (IBM) leading discussion on topics
    • distribution spec conformance tests - update with referrers API
    • artifacts repo: what to do with it? Jason - suggested migrating the useful content into image-spec/example of artifact types, etc.
      • One less thing to go stale is a plus (vbatts)
      • Image-spec has a list of implementations where adding media types used by other projects has precedent (Brandon)
      • Distribution-spec for API implementations will likely use conformance, with a 1.1 conformance section (Brandon)
    • registry implementation with distribution/distribution:
    • Will there be more maintenance of the 1.0 branch?
    • client support of new artifact types
      • image runtimes don't need this to continue running images
      • Higher level clients, tooling, and admission controllers will likely first add support to check image signing
      • Signing is the only potential use case that might early adopt this for runtimes
    • Is there a need for any KEPs?
      • May come into play if we move from admission controller to scheduler and runtime integration (e.g. pod security policy)
    • How to lead to standardizing similar mediaTypes e.g. SBOMs
      • (nisha) there are multiple SBOM types, and they can link to each other. It would be a lot for clients to handle
      • (brandon) push, pull, verify
      • (vbatts) there are some assumptions here to acknowledge: there will always be additional mediaTypes and iterations on this workflow. Ideally implementors can gain momentum on generic types like application/sdpx-vX and with some +zstd or whatever compression suffix. Use Annotations or similar to differentiate the nature of that document. Additionally, this sounds like a good place for a library to make life easier for implementors to not re-create the wheel over and over. Similar to some of the other OCI ``*-tools` repos.

Proposed Agenda Items

  • Update on Artifacts - How to Move to GA (Mike Brown - IBM)
  • Standardizing artifact media types and annotations - allowing multiple SBOM generators to work with multiple SBOM consumers (Nisha Kumar)
  • Standardizing references - common string for referencing manifests on a registry, in an OCI Layout, or some other future storage (Brandon Mitchell)
  • Authentication and authorization - standardize how clients login to registries (Brandon Mitchell)

Update on Artifacts - How to Move to GA (Mike Brown - IBM)

Status: The state of the Open Containers Initiative (OCI) is strong.

Big Shout-Out to the reference-types WG, their project is finished, the work group disbanded. Their content, wrt. spec changes has been merged into the image-spec and distribution-spec repositories.

Release candidates have been published:

A map of what needs to happen next:

  • Spec project updates to the various conformance test buckets to cover the changes.

    • conformance and docs (examples for clarity) for new and changed manifests in the Image spec.
      • Improvements to artifactType definition description (merge or update and link artifacts repo content)
    • conformance for referrers api in the distribution spec and modifications to existing Pull/Push buckets to cover optional subject references
    • Image Spec Milestone
    • Distribution Spec Milestone
    • artifacts repo -> image and artifacts spec docs
    • image spec 1.0 refresh request prior to 1.1 ga will 1.0 continue to receive fixes (Sebastian)
  • registries

    • distribution/distribution needs a PR for updating to the 1.1 OCI specs *many registries start from this CNCF project
    • registries and whether based on distribution/distribution
      • go-containerregistry: open issue Jason is working on this PR
      • Docker Hub: ? (based on distribution, but would need to accept upstream changes)
      • ACR: ?
      • ECR: Michael (WIP)
      • GCR/GAR: ?
      • quay: ?
      • bundle-bar: ?
      • django-oci: ?
      • keppel: ?
      • openregistry: ?
      • Zot: Ram (WIP)
      • ICR: Mike WIP
      • OCR (Oracle): Nisha will engage
      • name: ____________ status: _____________
      • name: ____________ status: _____________
      • name: ____________ status: _____________
  • clients for build, resolve, push, pull, cache, need to do their thing updating to the new specifications

    • Status of clients
      • How does this affect runtimes today? Image runtimes may not need to update.
      • umoci, buildah, moby, buildkit, containers/image, containerd, cri-o,
      • regclient includes support
      • crane will likely add support soon, based on go-containerregistry
      • sigstore will likely update after go-containerregistry updates
    • build tool invention may be needed for:
      • new subject refers field in image
      • new artifact manifest and it's subject refers linking to other manifests
      • image index inclusion of artifacts
    • container runtimes and OCI image libraries need to update to the new image-spec and also need to map out:
      • where the artifacts will be stored
      • how artifacts will be made available to containers, snapshotters, and container runtime plugins
      • what cli updates will be made to support the referrers api
      • changes needed for resolve, push, and pull to support the distribution-spec changes
  • higher level spec changes are needed to build on / take advantage of new artifacts and the referrers API

    • while not a "GA" requirement the process is long for k8s KEPs and thus should start earlier rather than later.
    • k8s admission controller handling of artifacts for pods/containers
      • sigstore/policy-controller should coordinate with the rest of sigstore
    • pod spec policy changes
      • CRI handling of mounts/validations related to artifacts
      • example: image pull policy modified to support patch artifact (see Sajay's Patch Manifest Proposal)
  • A flushing out of the release candidate spec changes needs to occur by the registry providers and client tooling, receiving feedback from the implementors

    • Example: Relaxing referrers API requirements #357 && #968
  • Artifact Providers

    • New Artifacts need to be proofed out with OCI friendly integrations and do their IANA registration process.
    • These new artifacts provide use cases and are a forcing factor to get client tools, registries, and users engaged and moving forward with us.
    • Artifact type and blob media type should be standardized (see next section).
  • Other projects that were on an earlier trajectory wrt using legacy OCI specs for new artifacts.. and are hopefully moving to the r.next OCI specs

    • ORAS: open issue
    • HELM: depends on distribution/distribution

Standardizing artifact media types and annotations - allowing interoperability among clients (Nisha Kumar/Brandon Mitchell)

  • Use case: How to support multiple SBOM generators and consumers?
  • If the same tool that creates an artifact is pulling it, there's no issue
  • When multiple tools can create an artifact, with multiple consumers, what media types are used should to be standardized
  • Possible ways to define an artifact:
    • Single manifest, multiple blobs:
      ​​​​{
      ​​​​  artifactType: "application/spdx", // undefined IANA media type
      ​​​​  blobs: [
      ​​​​    {
      ​​​​      mediaType: "application/spdx+json",
      ​​​​      // or "application/json", possibly compressed
      ​​​​      // ...
      ​​​​    },
      ​​​​    {
      ​​​​      mediaType: "application/spdx+xml",
      ​​​​      // ...
      ​​​​    }
      ​​​​  ]
      ​​​​}
      
    • Single manifest, single blob:
      ​​​​{
      ​​​​  artifactType: "application/spdx+json",
      ​​​​  blobs: [
      ​​​​    {
      ​​​​      mediaType: "application/spdx+json",
      ​​​​      // same media type for both blob and artifactType?
      ​​​​      // ...
      ​​​​    }
      ​​​​  ]
      ​​​​}
      
    • Single manifest, compressed blob:
      ​​​​{
      ​​​​  artifactType: "application/spdx+json",
      ​​​​  blobs: [
      ​​​​    {
      ​​​​      mediaType: "application/gzip",
      ​​​​      // or "application/spdx+json+gzip"
      ​​​​      // ...
      ​​​​    }
      ​​​​  ]
      ​​​​}
      
    • Single manifest with generic type:
      ​​​​{
      ​​​​  artifactType: "application/sbom", // undefined IANA media type
      ​​​​  blobs: [
      ​​​​    {
      ​​​​      mediaType: "application/spdx+json",
      ​​​​      // compressed?
      ​​​​      // ...
      ​​​​    },
      ​​​​    {
      ​​​​      mediaType: "application/spdx+xml",
      ​​​​      // ...
      ​​​​    },
      ​​​​    {
      ​​​​      mediaType: "application/vnd.cyclonedx+json",
      ​​​​      // ...
      ​​​​    },
      ​​​​    {
      ​​​​      mediaType: "application/vnd.cyclonedx+xml",
      ​​​​      // ...
      ​​​​    },
      ​​​​  ],
      ​​​​  annotations: {
      ​​​​    // indicate what kinds of blobs are included for filtering?
      ​​​​  }
      ​​​​}
      
  • (vbatts) Assume there will always be a next implementation. How to facilitate clients. Artifact type standardized and annotations do more filtering. Start maintaining libraries that support this and allow implementers to use it.
  • (Jason) would rather have communities evolve on their own rather than recommend.
  • (Brandon) Push and pull with various tools - need to standardize around method
  • (Nisha) No standardized workflow
  • (Brandon) What are the OpenSSF tools verifying?
  • Takeaways
    • Need to see a workflow that works
    • Similar to using Artifacts as general guidance. Add use

Standardizing references - common string for referencing manifests on a registry, in an OCI Layout, or some other future storage (Brandon Mitchell)

  • https://github.com/opencontainers/tob/pull/114
  • Examples for registries:
    • alpine -> docker.io/library/alpine:latest
    • docker.io/library/alpine:3@sha256:bc41182d7ef5ffc53a40b044e725193bc10142a1243f395ee852a8d9730fc2ad
  • Examples for OCI Layout:
    • skopeo: oci:dirname
    • anchore: oci-dir:dirname
    • regclient: ocidir://dirname:v3
    • buildkit: oci-layout://dirname
    • Would be useful to define this before every implementation solves it differently
    • skopeo and anchore can be confused with registry references (docker.io/library/oci:dirname)
  • Referencing tags and digests within a Layout
    • Appending a :tag and/or @sha256:...
    • Paths containing those fields must explicitly specify a tag
  • RFC has no support for relative paths
    • RFC 1738
    • <scheme>://<host or empty>/<path>
    • Some implementations use a . for the host to indicate a relative path (<scheme>://./<path>)
    • Others treat everything after the <scheme>:// as the path so <scheme>:///<path> is an absolute path and <scheme>://<path> is relative
  • (Jason) would also like to have an OCI maintained parser for these
  • (Akihiro) Would also like to do immutable tags for reproducibility, but this shouldn't be tied to this initiative because it needs experimentation on where it should live
  • Why not eBNF grammar?
    • Corner-cases like localhost or dockerhub
  • Should this be a working group, or addition to image-spec?
    • Jason: add to image-spec

Authentication and authorization - standardize how clients login to registries (Brandon Mitchell)

Select a repo