# CNAB Community Meeting **Meeting time:** Weekly Wednesday 9:00 AM - 10:00 AM US Pacific Time **Zoom Link:** https://zoom.us/j/653255416 **GitHub Repo:** https://github.com/deislabs/cnab-spec **Group slack channel:** #cnab @ cloud-native.slack.com (Get an invite from: https://slack.cncf.io) **Mailing list:** https://groups.google.com/a/opencontainers.org/forum/#!forum/dev **Document Link:** https://aka.ms/cnab/meeting **YouTube:** https://www.youtube.com/playlist?list=PLL6BzOBDywQeaaKFZkdt10JTZr5BxjQvQ ## Meeting Minutes and Agenda ## **March 20 Agenda** | | | | -------- | -------- | | Recording | | | Attending | Lachlan Evenson, Ryan Moran, Silvin Lubecki, Nuno do Carmo, Jason Stevens, Simon Ferquel, Radu Matei, Urvashi Reddy, Josh Dolitsky, Vaughn Dice, Jeremy Rickard, Darren (Intel), Sameer Advani, Glyn Normington | |Note Taker | Jeremy Rickard | **Agenda** * New Folks * Demos * Summary of Changes: https://hackmd.io/s/SJxpDvTLN * Please Review and Comment: * https://github.com/deislabs/cnab-spec/pull/137 * https://github.com/deislabs/duffle/issues/668 * Should custom extensions be exposed to invocation images? * What are the goals or use cases for libcnab-rust? * JSON schema support? **Notes** * libcnab-rust: * Partially becuse we want to learn RUST * We are trying to exercise parts of the spec in other languages * Re: Should custom extensions be exposed to invocation images? - Can we expose extension metadata to the invocation images? - Simon suggests that more generally, it would be good to add additional properties across the spec (friendly names for parameters and what not) - Glyn will raise an issue to cover this, add to meeting notes: CNAB spec [issue 143](https://github.com/deislabs/cnab-spec/issues/143) raised. * JSON Schema Support: * Urvashi - would it be possible to update cnab spec to adopt JSON Schema wholey * Part of effort to enrich parameter definition to allow JSON Schema definition for more complicated type * Butcher: currently missing object and array types * Simon: suggest that we embed validation into invocation image and define a way to report validation errors * parameters - they look very much like JSON schema, but does not adhere to the JSON schema - difference between valid string in the JSON schema and a valid string in the CNAB parameter - in the runtime, there is a JSON API that sends parameters -- painful experience, where there are regular JSON payloads that need to be converted into the String type of the CNAB parameter - is the bundle.json format supposed to be human readable, exposed to users? - thick bundles and air-gapped environments - Docker Compose example -- invocation image embeds the logic to provision the images on the cluster - at least for OCI images, we should have a registry in the air-gapped environment, and the images would be re-located there - do we need thick bundles at all for this scenario? We can copy things from one registry to another - this works for OCI images, no scenario for other images (VMs) - until the VM story can be satisfied, we still need thick bundles - great to copy the images during relocation (in a DMZ), but other locked down scenarios don't have internet at all, PLUS long term support - the duffle implementation of image relocation could handle both thin and thick bundles **Action Items** - Glyn open issue for #143 - Urvashi/Ryan: open PR with the json schema stuff - butcher: list all air-gapped scenarios and decide if we need thick bundles ## **March 13 Agenda** | | | | -------- | -------- | | Recording | https://youtu.be/rss-Npo9h8A | | Attending | Chris Crone, Caroyln Van Slyck, Ryan Moran, Karen Chu, Urvashi Reddy, Matt Butcher, Radu Matei, Nuno do Carmo, Chris Green, Josh Dolitsky, Silvin Lubecki, Michelle Noorali, Swapnil Bawaskar, Vaughn Dice, Gareth Rushgrove, Jeremy Rickard | | Note Taker | Jeremy Rickard | **AGENDA** * New Folks * Demos * Carolyn: Porter + JSON Schema * Please Review and Comment: * https://github.com/deislabs/cnab-spec/pull/133 * https://github.com/deislabs/cnab-spec/pull/137 * Summary of Changes: https://hackmd.io/s/SJxpDvTLN * Proposal: Relax MUST use Canonical JSON to SHOULD use Canonical JSON * Plans for duffle rewrite or similar? * Should custom extensions be exposed to invocation images? **NOTES** * Porter and it's mixins present JSON schema now, that drives IntelliSense for VS Code - If you install other mixins, will they get incorporated? - Mixins *can* expose schema but they aren't required right now - How can you generate the mixin? - Right now we did it by hand, but there is a wide range of tooling but different versions aren't compatible with different tooling * We'd like to merge 133 and 137 soon. Additive but useful. * Relax MUST use Canonical JSON? * Collectively discovering Canonical JSON isn't terribly well supported * Might limit the support of libraries in other languages * Chris: sounds reasonable, checking signature after decoding binary feels wrong * From chat: should we change images back to array from a map * would be a breaking change need to decide soon * check with Silvin or Simon from Docker as to why that is the form (check with them in cncf slack) * tooling can have their own manifest files that represent in different ways * Start a discusison in #cnab channel in CNCF slack * Plans for Duffle Rewrite: * Pivotal asked * Current plans probably should be in the invocation images themselves * Follow up from last week docker socket: docker is making that change, we should add to best practices for runtimes * Consider the spec in _slushy_ state and try not to make breaking changes between now and June. **ACTION ITEMS** Jeremy: * Add _Should custom extensions be exposed to invocation images?_ to next week's agenda ## **March 6 Agenda** | | | | -------- | -------- | | Recording | https://www.youtube.com/watch?v=yFOfyX7BSyg | | Attending | Carolyn Van Slyck, Michelle Noorali, Karen Chu, Josh Dolitsky, Chris Chrone, Glyn, Jeremy Rickard, Nuno do Carmo, Radu Matei, Steve Lasker, Urvashi Reddy, Gabrielle, Sameer Advani, Daniel Fein, Ryan Moran | | Note Taker | Jeremy Rickard | **AGENDA** * New Folks * Demos: * CNAB push -> promote -> pull demo (simon aka Chris :) * Authoring Porter bundles with Lua & MoonScript (Josh) * Source code: https://github.com/jdolitsky/porter-moon-demo * Please review: * https://github.com/deislabs/cnab-spec/pull/123 (breaking change) * https://github.com/deislabs/cnab-spec/pull/131 * Summary of This Week's Changes: https://hackmd.io/s/SJxpDvTLN * Running invocation images as privileged containers with Duffle -- [deislabs/duffle#651](https://github.com/deislabs/duffle/pull/651) (Radu) * Add Maintainers * Require 2 LGTMs? * (If time) Discuss next steps for https://github.com/deislabs/cnab-spec/issues/95 ** NOTES ** - Chris Crone from Docker demoed cnab push -> promote -> pull - [Docker App](https://github.com/docker/app) - Push Pull with [CNAB-To-OCI](https://github.com/docker/cnab-to-oci) - Apache 2 license, feel free to use it - Question from Michelle: Do you calculate the hash locally or on push - Answer from Chris: On Push because it depends on the layers being archived in registry - Steve Lasker: still a burder on registry to understand how they are stored, registries may not have all info they need - Chris: We need to define annotations to define how to identify, later we can make a better solution - Question (from Radu): Is there a signed bundle story yet - Answer: We aren't doing signing yet - Question (from Radu): How do the artifacts arrive? - Answer: push referenced images first, then push invocation image, then push oci index - Josh Dolitsky demoed Lupo and Moopo - Build porter bundles using [Lua]([https://www.lua.org/) and [Moonscript](https://moonscript.org/) - Created issue in Porter [#207](https://github.com/deislabs/porter/issues/207) to investigate how to integrate lupo and moopo more tightly with porter. - Privileged invocation images - DO WE DO IT OR NOT - Duffle used to mount the socket all the time - We've removed that - Question: other than demo scenarios, real world scenarios where you'd want to run privlidged invocation images (i.e. modifying the same docker engine that you're running in) - Handle we support that with --privledge? - Chris: Don't do it by default, but you should have an option to tell the driver to run it as root, mount socket. Leave it up to the tooling. - Radu: do we have clear guidance on how to provide Docker URL and Certs for how to do production like things. - Chris: we've added docker context support for Docker CLI. It requires latest Docker app and docker CLi right now. You can specify contexts and it passes stuff in. - Chris: Hope to contribute this upstream to Duffle (PR is up: https://github.com/deislabs/duffle/pull/661) - Maineriners and LGTM+2 - Michelle suggests the Helm policy for nominating maintainers - 2 LGTMs seems like a good idea now - Issue 95: - Follow up conversation needed - Michelle might propose adding an artifacts manifest for thick bundles so tooling can know where to place things ** ACTION ITEMS ** * Michelle will PR a maintainers governance document based on Helm to nominate new maintainers ## **February 27 Agenda** | | | | -------- | -------- | | Recording | https://youtu.be/wlkoi5ga6V8 | | Attending | Jeremy Rickard, Ryan Moran, Nuno do Carmo, Urvashi Reddy, Gareth Rushgrove, Glyn Normington, Radu Matei, Lachie Evenson, Matt Butcher, Steve Lasker, Michelle Noorali, Carolyn Van Slyck, Karen Chu, Jason Stevens, Josh Dolitsky, Vaughn Dice, Matt Fisher, Gabrielle, Sameer Advani, Chris Green | | Note Taker | Lachlan Evenson | **AGENDA** * New Folks * Demos: * CNAB/porter wsl demo (Nuno) * CNAB push -> promote -> pull demo (simon) [Please schedule for next week instead] * [lupo](https://github.com/jdolitsky/lupo) - PoC of using Lua to build Porter bundles (Josh) [maybe next next week] * Tooling request for help (are you working on a CNAB tool and need help?) * CNAB core spec 1.0 progress * Release notes / summary of spec changes (jeremy)? * Spec proposal for JSON on STDIN - [deislabs/cnab-spec#114](https://github.com/deislabs/cnab-spec/issues/114) * Self-contained documentation for bundles (Radu) - [deislabs/cnab-spec#118](https://github.com/deislabs/cnab-spec/issues/118) **NOTES** * Should we run things outside the invocation image but rather directly on the host operating system? We could build a rkt driver potentially that unbundles and image. (based off the details in Nuno's demo) * Have CNAB core-spec to stabilize by the end of the week * There are four specifications in cnab-spec repo. Core, Repository, Security, Claims. Also non-normative has been moved out to the 800 section. * Question - Core is the only mandatory specification. We want people to be able to say, we are cnab-spec compliant. * Are image drivers in or out of the spec? The drivers are part of the duffle implementation and are not part of the spec. * Spec proposal for JSON on STDIN - parse data over STDIN directly in to the run tool via "Stdin: true". What happens if you have that on two parameters? How do you know what parameter is what? * Discussion around support object, or array with declared schema and have JSON schema validate. * Documentation discussion - having a directory within the bundle that can be stored as an OCI layer and then parse by the repository. Updates to core-spec would be a refence to a directory and a description field. https://github.com/deislabs/cnab-spec/issues/118 ## **February 20 Agenda** | | | | -------- | -------- | | Recording | https://youtu.be/jG1N8QnYUjE | | Attending | Lachlan Evenson, Jeremy Rickard, Josh Dolitsky, Matt Butcher, Daniel Fein, Carolyn Van Slyck, Karen Chu, Chris Crone, Adnan Abdulhussein, Gabrielle, Nuno do Carmo, Radu Matei, Sameer Advani, Swapnil Bawaskar, Urvashi Reddy, Vaughn Dice, Simon Ferquel, Ryan Moran, Atlas | | Note Taker | Lachlan Evenson | **AGENDA** * Demos: * Short update on TUF/In-Toto design * alpha/beta releases of spec in run up to 1.0 * Storage of CNABs in registries * Feedback on [CNAB to OCI](https://github.com/docker/cnab-to-oci/pull/19) update * Naming of common credentials * May not age well in the spec * Align tool builders on common credential names (i.e.: kubeconfig, etc.) * Would CNAB be an appropriate place to host a spec related to storing multiple content types over OCI? (Josh Dolitsky) * OCI maling list discussion: https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/idUW9KWQsBo * Porter + Lua (Josh Dolitsky) * https://github.com/deislabs/porter/issues/173 **NOTES** * Security currently direction switching from open pgp to [TUF](https://github.com/theupdateframework/notary) and [In-Toto](https://in-toto.github.io/)? * Proposal of third specification on security. Optionally implmentable without affecting cnab core compliance * Matt Butcher is working on the draft hopefully finished by the end of the week * Questions? * Will this tie into registries? Yes and with notary * Will all the specs go v1 at the same time? Currently not sure. * Identified security issue in Duffle "duffle docker driver mounts the docker socket for no apparent reason. That gives CNAB invocation images a wide open door to your machine.
(Will write an issue on Docker repo for this one)
" * We shouldn't have aplha and beta of the releases to stop fragmentation. When will the core part of the spec be v1. * Move fast on cnab core and get it to v1 * What is the current rate of change and how many tools implement the specificiation * Chris proposing CNAB to OCI tool be used to store bundles in registries * Where should the CNAB registry storage conversation be had? * Discussion around tight coupling of image spec and distribution spec * Challenging to make changes to OCI distribution to make it a generic store because every tool has different use-cases * Annotations are the best way forward with custom types * Start with standard agreed upon key then move forward with the implementation * DECISION: Move forward with CNAB to OCI and continue discussion in OCI distribution. Agree on annotation across communities. Simon to help with pulling CNAB to OCI into Docker-app * CNAB to OCI doesn't currently support thin-bundles.Simon and Chris to raise issue to hash out the detail * Thin bundle = main OCI Index + config blob * Thick bundle = Thin bundle + deep copy of everything linked by the main OCI index
 * Well known custom actions - https://github.com/deislabs/cnab-spec/blob/master/805-well-known-custom-actions.md * Porter mixin for CNAB that makes a single bundle from multiple bundles * As soon as the spec has any way to communicate outputs between bundles then we could compose bundles as you suggest. Until then I think it would be a bit clunky. Like you could run a bunch of bundles, but without being able to pass data between them, it wouldn’t be as useful as I would want, no? * Jeremy has proposal and will open PR **ACTION ITEMS** * https://github.com/deislabs/cnab-spec/blob/master/101-bundle-json.md#the-image-map -> refs and image map injection at runtime is redundant * https://github.com/deislabs/cnab-spec/issues/113 ## **February 13 Agenda** | | | | -------- | -------- | | Recording | https://youtu.be/d3EOrtsCd7o | | Attending | Jeremy Rickard, Radu Matei, Matt Butcher, Ryan Moran, Urvashi Reddy, Chris Crone, Karen Chu, Lachlan Evenson, Carolyn Van Slyck, Vaughn Dice, Simon Ferquel, Swapnil Bawaskar, Silvin Lubecki, Adnan Abdulhussein, Nuno do Carmo, Sameer Adveni, Daniel Fein, Josh Dolitsky, Simon, Gabrielle, Glyn Normington| | Note Taker | Jeremy Rickard | **AGENDA** * Demo: Porter * What's chang(ed|ing) in the spec since last week * Metadata: License * Namespacing rules * Canonical JSON * Proposed: JSONSchema subset for params * Proposed: Remove CNAB_P_* * Commit on storing CNABs in registries * Explanation of chosen method * How and when should this be added to the spec? * Short update on TUF/In-Toto design **NOTES** **Lots of new people again, thanks for joining us!** * Porter Demo * Links to Porter * https://porter.sh * https://github.com/deislabs/porter * Question from nuno: * currently testing the exec mixin, but will the other cnab “clients” be able to run it too? (and yes I would totally understand if I’m out of scope)
 * Answer: Hey Nuno, right now they are porter concepts. So they work w/ the Porter build / porter run flow. Once you build the bundle, you could run that in other tooling, but right now the porter runtime knows how to invoke them
 * Question from glyn: * How do people know how to use mixins * Answer: Mixins provide a schema that returns JSON schema * Canonical JSON * Moving to Canonical JSON for CNAB is preferred * One pro per Radu: "And a lot more language implementations for canonical JSON serializers..""
 * Proposed: JSONSchema subset for params * Ryan Moran has been working on a proposal. No PR yet. * Has been chatting with Matt about it. * Would introduce more strcture to parameters * CNAB in Registries * see repo below for example of what is the current thought * CNAB to OCI: https://github.com/docker/cnab-to- oci/blob/13c4adaaf091e6f996116aff5ecc3f0eb6a3dd20/README.md#example * Matt Fisher: thinks its a good idea * Chris Crone [10:51 AM] For those interested in how CNAB and other artifacts are stored in registries, there's a #artifact-registry channel now * Did not cover the TUF/In-Toto Design updates * We decided to extend the meeting for next week. The invite will get updated. **ACTION ITEMS** ## **February 6 Agenda** | | | | -------- | -------- | | Recording | https://youtu.be/_E1R3mip6aY | | Attending | | | Note Taker | | **AGENDA** * Spec decomposition (claims, runtime, etc) * Storage of CNABs in registries * Common repository for test data (client conformance) - https://github.com/deislabs/cnab-spec/issues/90 * Strategies for keeping the spec, bundles, and implementations in sync (for example https://github.com/deislabs/cnab-spec/issues/92) * Tooling overview * [Duffle](https://github.com/deislabs/duffle/), [Porter](https://github.com/deislabs/duffle/) * [Docker App](https://github.com/docker/app) * [Python Client](https://github.com/garethr/pycnab), [.NET Client](https://github.com/deislabs/cnab-netstandard) * If you are working on something, please submit a PR and list it on https://cnab.io/community-projects/ **NOTES** * Introduction - everyone's awesome, again * spec decomposition * different stabilities for different parts * should x be part of the spec? * storage of CNAB bundles in registries * Gareth gives a quick background for the registry discussion * Steve L - it is agreed that we want to store additional artifacts in registris with the OCI folks, we are at the point where details are discussed. * how to let specific teams that are in charge of different artifacts own the specific artifact schema * anyone interested in this discussion, join the weekly OCI meeting, today (Wednesdays) at 2pm pst: https://calendar.google.com/calendar/ical/linuxfoundation.org_i0sado0i37eknar51vsu8md5hg%40group.calendar.google.com/public/basic.ics * OCI mailing list conversation: https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/oguystbnnw4 * additional repo with test data * we want to have a common place for test data to use with various implementations * how to validate that a bundle is valid with a version of the spec? * the Python implementation already started some work on validation **ACTION ITEMS** * create a repo for a bundle validator * create a common repo for test data * Rename the bundles repo to bundle-examples or something to indicate its purpose better (that they show what a bundle looks like with the raw CNAB spec, not duffle or porter, etc) --- ## **January 30 Agenda** | | | | -------- | -------- | | Recording | https://youtu.be/a7cqHpTrDTY | | Attending | Matt Butcher, Michelle Noorali, Jeremy Riickard, Brain Redmond, Adnan Abdulhussein, Miguel Martinez, Radu Matei, Chris Crone, Gareth Rushgrove, Silvin Lubecki, Carolyn Van Slyck, Matt Fisher, Lachie Evenson, Phil Estes, Augustine Correa, Michele Buccarello, Peter Benjamin, nalla | | Note Taker | Jeremy Rickard | **AGENDA** * Introductions * tldr; Everyone is pretty cool * The current proposals for storing CNAB in registries * Two approaches * Chris Crone is attending OCI meetings * Status report on spec * Made changes to how to specify images in the spec * Makes it easier to switch registry information * Merged * Part of larger story for getting OCI registry support * Spec evolution defined in [CNAB Spec Section 900](https://github.com/deislabs/cnab-spec/blob/master/901-process.md) * The current proposal for switching from OpenPGP to TUF * The changes to the `images` section on the spec. **NOTES** * OCI Meeting today will discuss OCI spec adoption * https://github.com/opencontainers/runtime-spec#meetings * https://www.uberconference.com/opencontainers * Can OCI dev open containers mailing list be used or will we use another * As spec changes, we need a way to have example bundles verified * Currently checks that duffle can install bundles * Can/should add schema validation * Should Claims be in the spec? * Possibly move toward a more componetized spec * Add to agenda next week: discuss decomposition of spec * Michelle gave an update on Duffle * Doing some refactor of builder, split image and bundle * We've created a [waffle.io board](https://waffle.io/deislabs/duffle) * Move TUF discussion to next week * If you have tooling demos, feel free to add to the agenda for upcoming weeks * Michelle: A useful thing as people start proposing/doing conference talks, we should make a repo somewhere that we can store collateral/images/etc for everyone to reuse * Should we put it in deislabs? * Let's make a community repo **ACTION ITEMS** * Matt Fisher and Radu Matei to start working document on how to store CNAB in OCI registries * Lachie add action item for Agenda next week for spec decomposition * Michelle and Carolyn to make a community repo in deislabs ---