# CNAB Community Meeting **Meeting time:** Every other Wednesday 9:00 AM - 10:00 AM US Pacific Time **Zoom Link:** https://zoom.us/j/653255416 with passcode 77777 **GitHub Repo:** https://github.com/deislabs/cnab-spec **Group slack channel:** #cnab @ cloud-native.slack.com (Get an invite from: https://slack.cncf.io) **Mailing list:** https://groups.google.com/a/opencontainers.org/forum/#!forum/dev **Document Link:** https://aka.ms/cnab/meeting **YouTube:** https://www.youtube.com/playlist?list=PLL6BzOBDywQeaaKFZkdt10JTZr5BxjQvQ ## Meeting Minutes and Agenda ## Future Agenda Items * Installation state https://github.com/cnabio/cnab-spec/pull/411 (Radu) * non-normative section for naming dependencies installation name, e.g. myinstall-redis * upgrade path for dependencies * Installation Lockfile The resolved dependencies' installations e.g. Wordpress required MySQL, the lockfile has the installation name of mysql stored * unused dependencies, ownership tracking * Detecting unused dependencies * Query for any lockfiles that reference an installation ## September 15, 2021 - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Carolyn Van Slyck, Hadrien Patte, Vaughn Dice, Matt Butcher, Brian DeGeeter, Ralph Squillace | | Note Taker | Vaughn Dice | * Reminder: Anyone can add items to the agenda * Carolyn: Demo how Porter is using parameter sources and outputs to support a bundle "state bag" * Demo link: https://github.com/carolynvs/tabbycat-demo * Slated to arrive in an upcoming v1 alpha release * Background: Had been authoring bundles with the Terraform mixin, but anytime install failed, the .tfstate file wouldn't be persisted, causing all sorts of issues * Solution: combine two existing concepts: Parameter Sources and Outputs. Treat the .tfstate file as state that shouldn't/needn't be managed by the bundle author. Rather, Porter does the plumbing in the background in the form of its new 'state bag', e.g. `state` section of the Porter manifest * Anything in this `state` bag will be persisted regardless of the result of an action (install, etc.) * Additionally, entries won't show up in the Parameters (or Outputs) section when viewing/explaing a bundle (they were in the past) * Vastly simplifies managing state. Recall that users of the Terraform mixin can still configure/use a remote backend (using a cloud provider, etc.) but this brings more complexity/overhead. Still perhaps recommended for production, however. * (Tangent): Recall that a K8s Service IP value can be acquired via the Helm3 mixin. Handy! * How is this implemented? * Although this is a Porter feature, it uses concepts that are in CNAB Spec today (i.e. other implementations can do similar!). Main spec concept used is [Parameter Sources](https://github.com/cnabio/cnab-spec/blob/762e461046786f14090ba60d029d68c98d004c08/810-well-known-custom-extensions.md#parameter-sources) * Question (Ralph): How does this state feature interact with bundle version? * Clarification: Bundle version represents the semver of the bundle itself, App version represents the semver of the bundled app (in demo, this value went under the `custom` section), Installation revision represents a particular instance+modifying action+result * Example: Say the same/unmodified bundle is 1. installed (revision 1) then 2. upgraded (revision 2). The state used on upgrade is from the previous revision (1, install). * Ralph: This looks great. Understand that bundles are getting more and more diverse. Does this feature sufficiently cover the growing array of use cases for tracking state? * Hello from Brian DeGeeter! (Terraform mixin contributor!) Q: Is more metadata necessary in the `state` section? For example, knowing what version of the bundle is associated with state. * Carolyn: Recall that we track both the previous and current version of the bundle.json within the installer image when running an action. * Follow-up idea: Do we really want bundle authors to always create standard state sections (say, tfstate and/or tfvars for a Terraform-based bundle)? Perhaps we can streamline it such that this isn't needed. * Needs attention (reviewers!): * [Add CODEOWNERS file](https://github.com/cnabio/cnab-spec/pull/421) * [Bug when injecting optional parameters](https://github.com/cnabio/cnab-go/issues/266) * [Renamed cancelled to canceled](https://github.com/cnabio/cnab-spec/pull/420) ## August 18, 2021 - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Carolyn Van Slyck, Radu Matei, Matt Butcher, Vaughn Dice | | Note Taker | Vaughn Dice | * Carolyn: Demo v1 changes in Porter Operator that are part of the new installation state spec * Carolyn has been working on functionality to store additional information around installations * An installation record will now be a first-class resource -- vetting in Porter first, CNAB Spec next! * Demo in Porter... * Support for namespaces (for installations as well as cred and parameter sets) * Empty namespace = global * Support for labels (helps with organization, mgmt of installations) * Other items tracked on an installation: * Bundle repository * Bundle version * Bundle digest * Credential sets (named mapping of cred values) * Parameter sets * Parameter values * Setting things up so users can edit the installation record to reflect what they want it to look like; then Porter can apply those changes. (Shift towards imperative approach) * Discuss lessons learned about what should go in the spec, and what should stay in the tooling * Data access layer that existed in cnab-go was not very performant; did not support rich querying * Moving towards removing this logic from spec libraries like cnab-go and putting the control into the implementations * Porter now uses mongo for storage backend * The default is the 'mongo-docker' plugin which runs mongo in a Docker container with an associated/persisted Docker volume * Can also use the 'mongo' plugin to provide a mongo connection string to Porter to connect to * Questions * Radu: What are the items that are anticipated to land in CNAB Spec * Carolyn: In general, only things related to the runtime * Installation State Spec is first up * A lot of this work is being done to lead up to a more robust dependencies spec * Group (present attendees) consensus agrees this is a good approach ## August 4, 2021 - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Matt Butcher, Radu Matei, Vaughn Dice, Aries | | Note Taker | | * Deferred dependency discussions and operator demo until a time convenient for Carolyn. * Update on signy "It makes sense and we are happy with the current plan" * Verification process is nearly complete as well * Butcher/Vaughn: Carolyn is working on storing installation records (claims and labels) in MongoDB. * No major or minor spec changes between now and September ## June 23, 2021 - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Carolyn Van Slyck, Jacob LeGrone, Matt Butcher, Aries, Radu Matei, Ralph Squillllace | | Note Taker | | Agenda (Notes inline): * Carolyn: Mistakes were made, taking storage out of the CNAB spec and cnab-go. * Here's what we have now: installation states, using namespaces and labels * Current interface is CRUD for one item * No current need to query the data * Would be nice to say "list all items in a namespace" or "list all things with labels X, Y, Z" * Started to feel like creating an ORM w/o knowing the storage layer * Suggestion: Claims store should not be in `cnab-go`. Only the format should. Storage is implementation-specific. * Jacob: Agree... interop between runtimes is a non-goal of CNAB * Butcher: I agree with Jacob, it is a non-goal to have interop between different runtimes at the claim level. * Carolyn: So we can remove it from `cnab-go`, and things will be backed out of the spec so that the spec is just about the format. * Jacob: Could add it back in the future. See also Temporal's storage interface. * Carolyn: Any new agenda items? Signy? * Ralph: Trishank, Radu, Ralph have been chatting about the remaining four issues to achieve 1.0 * Main issue was to handle situation where notary can be flooded with work * Need to document what Notary thinks it can handle * Impl works, though * Radu: Scenario: You have been pushing in-toto md for a few versions. The actual TUF target JSON for your repo (version/tag) has been growing to lots and lots of MD. All of that data attaches to the same file as notary (shared w/ in-toto). There is an (internal) limit to how much MD Notary can handle. So somehow we should mention that the MD can be stored externally. But... (a) this was not originally a requirement, and (b) it exists regardless of in-toto. So probably we should just test and document rather than coming up with an external storage. That would be a spec bump. * Ralph: Agree. Document the Notary issue (what is the max size?) and declare 1.0. * Radu: Workaround is trivial: When you approach the limit, create a new tag * Jacob: Is this actually likely in practice? * Ralph: We don't know * Clarification about what sort of data we are talking about, and that there are definitely limits. Don't know how high those limits are. And this impacts _all_ MD for Notary, not just signatures. * Carolyn: So there are _three_ other issues? * Radu: Hackathon maybe next week to solve remaining 3 * Jacob: Is there a way to remove old MD to reduce the size? * Radu: You'd have to look into Notary. Not sure. * Discussion about a few ways to delete or overwrite * Aries: introduction: Cloud Cover as a PM. Have a similar tech to CNAB, but would like to observe. * https://github.com/cldcvr/ (particularly codepipes) * FIN ## June 9, 2021 - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Carolyn Van Slyck, Sam Boyer, Vaughn Dice | | Note Taker | Vaughn Dice | Agenda (Notes inline): * Discuss progress with installation spec POC * https://github.com/cnabio/cnab-spec/pull/411 * Seeing limitations with the default filesystem installation data store (which is only intended for dev anyways) * Options: drop support for local filesystem backing store, move to query-able store (SQLite?) * Would rather support a more production-friendly default (query/filter/etc.) * Data Storage Access patterns * Slow due to how we pull documents individually. * Can we ditch the filesystem store for sqlite? * Add claim document cache? * How do we handle migrations when it comes to changes in the claim documents? * Thinking this will/should be the responsibility of tool/impls as opposed to cnab-go. * Pull Requests * https://github.com/cnabio/cnab-to-oci/pull/109 - Use existing relocation map when fixing up a bundle * https://github.com/cnabio/cnab-to-oci/pull/110 - Upgrade to Go1.15 * Hey Silvin & Radu! * May want to add CNABy maintainers to this project * Updates from Sam around po11y + CNAB * https://github.com/pollypkg/polly * For (at least) distribute, maybe more * Question on how po11y's concept of Signals fits in w/ CNAB approach * https://github.com/pollypkg/polly/discussions/6 * Abstraction around a query, intended to be re-usable across config objects * Carolyn mentioned a couple key points that Sam found interesting: * Execution of bundles don't necessarily need to result in installations (ephemeral actions) * Custom actions can be used for generic diff and/or other introspection purposes * video from DockerCon: [Rethinking Application Delivery With Cue and Buildkit](https://docker.events.cube365.net/dockercon-live/2021/content/Videos/TamRA3F2zT8fycvq5) ## May 26, 2021 - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Carolyn Van Slyck, Vaughn Dice | | Note Taker | Carolyn Van Slyck | Agenda (Notes inline): * (carolynvs): Release a new patch version of the spec with https://github.com/cnabio/cnab-spec/pull/414 * Pull requests: * allow numbers in cnab-go: https://github.com/cnabio/cnab-go/pull/248 * add ValidateSchema function: https://github.com/cnabio/cnab-go/pull/256 * add output validation: https://github.com/cnabio/cnab-go/pull/255 * validate parameter default against its definition: https://github.com/cnabio/cnab-go/pull/254 * validate param regardless of if override or default: https://github.com/cnabio/cnab-go/pull/252 * add support for params of object type in ConvertValue: https://github.com/cnabio/cnab-go/pull/251 * Fix storage bug for outputs: https://github.com/cnabio/cnab-go/pull/257 * Propose changing to only requiring 1 review for cnab-go pull requests * https://github.com/cnabio/cnab-go/issues/258 ## May 19, 2021 - Security Meeting | | | | -------- | -------- | | Recording | | | Attending | Carolyn Van Slyck, Scott Buckel, Ralph Squillace, Radu Matei, Trishank Karthik Kuppusamy, Carlos O Kieffe | | Note Taker | Trishank | Agenda (Notes inline): * Pull requests: * https://github.com/cnabio/signy/pull/92 * Doesn't block on Notary, so we are good to go with signing * Concerns about being able to verify previously signed bundles, because the canonicalization might be different now * Are there security (availability) concerns? For example, can different tools verify bundles signed by other tools? What about backwards-compatibility? Decision is that we can break it for now because no production use yet, but we do need to advertise it explicitly (perhaps as a MAJOR SemVer bump). * https://github.com/cnabio/signy/pull/89 * https://github.com/cnabio/signy/pull/80 * Trishank to break up issues and put into a project * Scott and others will pick up issues as seen fit ## May 12, 2021 - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Carolyn Van Slyck, Vaughn Dice, Sam Boyer, Matt Butcher, Jacob LeGrone, Ralph Squillace | | Note Taker | | Agenda (Notes inline): * Revisit canonical json and non-integer numbers: https://github.com/cnabio/cnab-spec/issues/413 * Cancelled is official canceled: https://github.com/cnabio/cnab-spec/issues/412 * Carolyn will open a PR to fix the spec * Dependencies meetings starting next week! * Is anyone blocked or waiting on comment/reviews? * Issue Queue Spluenking * https://github.com/cnabio/cnab-spec/issues/405 * https://github.com/cnabio/cnab-spec/issues/382 * Sam: Grafana, Prometheus monitoring mixins https://groups.google.com/g/mixins * Working on a cue base system for writing schemas, helps create migrations across versions as breaking changes are introduced. * https://groups.google.com/g/mixins/c/uE1A3gXQ6vs * some parameters are known at the beginning of the process vs template variables which are deferred to the end * can't even assume that you are deploying to k8s, must work on any platform/installation * three stage lifecyle of observability objects: produce, distribute, consume * produce loop: user creates and refines the object, determine parameters * distribute: perhaps backed by oci registries, git vcs, prometheus endpoint to distribute their mixins, TBD * consume: some parameters aren't known until the consume phase * mixin package is a tree, some parameterized, includes grapha dashboards and prometheus rules * expand parameters (explode), filter out some objects, transform object into format for deployment tool * needs to fit into anyone's toolchain * grafana today: use mixins to power 1 click integrations (monitor a go app, wasmy stuff, javascript, etc). No parameterization atm * https://grafana.com/blog/2021/01/14/how-prometheus-monitoring-mixins-can-make-effective-observability-strategies-accessible-to-all/ * jsonnet based mixins, has standard labels that are attached to all scraped metrics. A generic mixin can assume some template variables are available, such as environment. Right now you just have to hope that the mixin has a replaceable parameter, and when you use it you have to manually wire up that parameter. * Need a preview, what is the impact of changing a parameter * would like to track the origin of the parameter (the resolution of the parameter) * Jacob: We could provide an extension to validate parameters with something other than jsonschema, such as cue ## April 28, 2021 - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Carolyn Van Slyck, Jacob LeGrone, Vaughn Dice | | Note Taker | Vaughn Dice | Agenda (Notes inline): * Alt week meeting - Can we use it for dependencies? Sounds like the security meeting may be starting up again? * Showdown! We'll all just show up and sort it out :) * Migrating Docker App to Porter - Call for contributors and ideas! * Docker App is no longer going to be maintained * Carolyn is working on a migration blog post * ... and also trick Chris into helping with the mixin :) * https://github.com/getporter/porter/discussions/1548 * Inconsistent spelling of cancelled/canceled: https://github.com/cnabio/cnab-spec/issues/412 * cnab-go/impl uses 'canceled'; spec 'cancelled' * Leaning towards an errata fix in the spec (keeping 'canceled') * Installation State Specification * https://github.com/cnabio/cnab-spec/pull/411 * Calling all CNABers: Needs eyes! * Carolyn can set up a branch using these changes in Porter to see what it'd look like in the Real World * Hey! Check out Carolyn's KubeCon talk!!!! ## March 31, 2021 - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Carolyn Van Slyck, Jacob LeGrone, Vaughn Dice, Simon Davies, Matt Butcher | | Note Taker | Vaughn Dice | Agenda: * Sync our spec maintainers teams * (GitHub) Maintainers out of sync w/ OWNERS * (carolynvs): Declarative bundles and a v2 spec ideas * Porter Proposal for Mixins as Bundles: https://github.com/getporter/porter/discussions/1499 * Problem: Porter packs all mixins in a bundle in the bundle's invocation image, leading to big images and other issues * Idea: What if we distro'd mixins as bundles themselves (as opposed to separate binaries, as they are in Porter now) * (Jacob): We thought of the workflow interface to be a bundle itself; decided against it. Fairly happy with this decision. They are similar in many ways, though. * (Jacob): They are fairly simplistic, e.g. "I want installations x,y,z installed in order a, b or c" * (Carolyn): In this case, none of the mixin bundle executions would be tracked as separate installations -- only the one installaion for the main bundle using them. * (Carolyn): Efficiency gains: multiple bundles that use the same mixin/version combos (helm, terraform, etc.) can now re-use the same cached mixin bundles * (Carolyn): see other Pros in issue above (further provenance assurances, optimized for layer cacheing, performance boost, etc.) * (Simon): Q: Considered having a CNAB mixin in Porter? * (Carolyn): Perhaps, open to discussing this in a more Porter-centric context. Here we're trying to guage if this model might fit well in CNAB * (Jacob): One difference with what we use internally and this approach: We like the ability to track mixin installations separately for dashboarding/debugging. In this approach, that same visibility may not be present * (Carolyn): Right. It could be a matter of what makes sense to the bundle author. If sub-installations need to be tracked separately, this would be a good use case for a dependency (which does get its own installation) vs if they do not (maybe go for these mixin bundles) * (Carolyn): Realize the origins of this idea are/were Porter-specific, but wanted to present it here before considering a full-fledged proposal * (Carolyn): This would represent a shift from the CNAB runtime tool knowing how to run *one* bundle to knowing how to run *multiple* bundles and track via a graph (still possibly representing just one tracked installation) * (Jacob): Q: How to organize creds/params/outputs to/from these internal/mixin bundles * (Carolyn): Thinking this can/will be defined in the workflow section (either new section in bundle.json or maybe under well-defined custom extension) * (Carolyn): Do we want to setup a working group meeting? * (Jacob): Currently more interested in the Dependencies proposal, but open to discussing this as bandwidth allows * Item: (Jacob) Question around the Dependencies proposal. For the Exclusive Resource case, we need to figure out how to name the dependency according to the parent bundle name. (We utilize installation names heavily for tracking/filtering/displaying installations) * (Future agenda item added) * Needs Review: * Installation State Specification - https://github.com/cnabio/cnab-spec/pull/411 * Originated from wanting to add label and namespace support to the Claims Spec (think K8s labels/namespaces) * Evolved into a new spec * Introduces a new document dedicated to the installation itself, which also features label and namespace fields prominently * Also tracks bundle reference and version to know if a newer version might be available, etc. * (Jacob) Very interested in this; Use similar approach for internal tool. Ex: dashboard showing all installations for a bundle, also filtering by other factors (k8s cluster, datacenter, etc.) * (Jacob) Does seem like a good idea to normalize this approach * Application version (as label) discussion: * (Jacob) we interrogate the bundle used for the installation to get this info (often stored separately in a graph db; in general if certain metadata changes, they don't want it to change/update the digest) * (Carolyn) there are use cases where bundle authors may want to specify a non-semver app version * (Carolyn) this is a good opp to look at how we might want to change how/where we query data * Pod Affinity for K8s Driver - https://github.com/cnabio/cnab-go/pull/245 ## March 17, 2021 - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Carolyn Van Slyck, Ralph Squillace, Vaughn Dice | | Note Taker | | Agenda: * (carolynvs): Declarative bundles and a v2 spec ideas ## March 3, 2021 - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Carolyn Van Slyck, Vaughn Dice, Matt Butcher, Jacob LeGrone | | Note Taker | Vaughn Dice | Agenda: * carolynvs: Add `apply to` field for credentials was merged and is in Porter. * Scope creds to a particular set of actions; will be omitted for others * In Porter, `porter explain` takes an optional `action` arg to show what creds/etc. apply to a particular action * Parity with Parameters * Eventually will be included in the next 1.x spec release * Optionally save logs as an output * https://github.com/cnabio/cnab-go/pull/243 * Save them via a persistent storage option * Consider if the bundle execution happens in a remote environment, would want the ability to reference logs after the action * Mech: logs are captured/persisted like other outputs * Jacob: What about streaming logs? * Carolyn: Different effort... this isn't 'real' streaming of logs * Jacob: Consider a long-running action and we'd like to see streaming status/progression. Maybe the runtime can read concurrently from the output file as its being written? * Brainstorming a strategy of communication between the driver and the invocation image... * If we had a well-known output file, the driver could 'watch'/tail as its being written * Trying to come up with a more general solution... maybe a custom extension... 'io.cnab.somethin'.streamable' * (As written currently, the output file logs only available after the action has run/completed) * Carolyn going to look at drafting up something for this streamable scenario * carolynvs: Support installation level data storage * https://github.com/cnabio/cnab-spec/issues/410 * carolyn: Walk through dependencies proposal for Porter * https://github.com/getporter/proposals/pull/8 * https://github.com/cnabio/cnab-spec/pull/409 * Carolyn: Overview: How can we utilize k8s approach to resolving instances of software (via namespaces and labels) for dependency resolution * See proposal for scenarios intended to be supported * Discussion on Polymorphic Dep section: * Butcher: Gentoo did something similar; this is a good problem to solve * Carolyn: Experience w/ K8s Service Catalog which attempted a rigid enforcement of service interfaces. We'd like to take a less-strict approach. * Jacob: Could the API be extended to allow for differing output names (e.g. 'root-connection-string' vs 'connection-string')? * Carolyn: Ah, also tried to solve in Service Cat. Really hard! Had the sys admin setup mappings... no one was really keen to do this. Much preferred to come to an agreement at the bundle level, e.g. authors agree that to impl the mysql bundle interface, the output needs to be named 'connection-string', etc. * Butcher: Say we needed a key/value store. Redis vs etcd, etc. Both fulfill the interface. But maybe differ on driver used, etc. How do we support this finer-grained specification? * Carolyn: Could utilize parameters (already in spec), or maybe labels (e.g. "supports-driverx=true", etc.) on top of using the interface * Jacob/Carolyn: Lockfiles. When we resolve deps per requires/interface, we should keep the records of each in a dep graph lockfile. * Butcher: How crucial would a lockfile be? Is the dep resolution system already repeatable/dependable? * Carolyn: (segues into the 'Register a Bundle' section of the proposal) e.g. 'helm-mysql' is my preferred impl of the mysql iface * Discussion on shared dep resources... similar concerns/exp in Helm, e.g. https://github.com/helm/helm/issues/7699 * On to selectors: namespace, label, name, interface * e.g. I rely on Wordpress where label = "app=myapp" * ## February 17, 2021 - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Carolyn Van Slyck, Vaughn Dice, Matt Butcher, Chris Crone | | Note Taker | Chris Crone | Agenda: * carolynvs: Our current jsonschema library doesn't support concurrency. * The alternative library also doesn't support it, https://github.com/xeipuuv/gojsonschema/pull/323. I'm not sure if it is still maintained and will accept the patch. * Unless we get lucky we are going to have to maintain a fork. * I reached out to Marwan to see what projct is using his patch in case other projects need to support a fork as well. * K8s Driver Updates: https://github.com/cnabio/cnab-go/pull/238 * Support delegation in Signy: https://github.com/cnabio/signy/pull/80 Notes: * carolynvs: Our current jsonschema library doesn't support concurrency. * Chris: PR looks good to me * Reached out to Sebastiaan at Docker to see if he knows what's up as he's contributed to the library before * carolynvs: K8s Driver Updates * Need some eyes on this * Biggest change is on volumes * carolynvs: Support delegation in Signy * Could use more eyes on the PR * carolynvs: Moved all repos from master to main branch :tada: * carolynvs: Proposed Dependencies Change for Porter that we will want to evaluate for the CNAB Dependency spec: https://github.com/getporter/porter/discussions/1441 * Better model for dependency management with impacts on storage layer ## February 3, 2021 - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Matt Butcher, Ralph Squillace, Jacob LeGrone, Chris Crone, Carolyn Van Slyck, Vaughn Dice | | Note Taker | Vaughn Dice | Agenda: - Media Type: Just need to formalize this: https://github.com/cnabio/cnab-spec/issues/381 - Just need to decide on the appropriate media type and update the spec - Two things: 1. Need clarification of wording, 2. Assemble in correct order aka "Everyone's learning the Registy spec Yay!" - Ralph is stoked to grab the IANA stuff - ? Do we already spec what goes in the config blob itself? - We just need the config media type - ? Do we want to say where annotations show up (e.g. title, version)? - Depends on Docker v OCI manifest spec - Some registries don't allow a mix - Suggestion: annotations could be optional for now - Differentiation between applying at root vs config object would be helpful (makes sense to be the latter, this is what Jacob does) - Jacob will file an issue to track - ? How is the v1 version used on the mediaType? - Is it the overall manifest version or bundle.json? - It seems like we should tie it to the bundle.json spec version - Leaning towards just tracking the major version - and make sure we only allow backward-compatible v1.x releases (can add fields, etc. but can't change existing schema fields) - Full spec version is encoded in bundle.json, so if tools need finer-grained validation, they can extract/inspect - We could add a semver annotation to the manifest itself to check w/o fetching the config object - carolynvs: I am going to rename our default branches in the cnabio org to main unless someone objects. - No objections - Pull Requests: - Make validating a bundle thread-safe https://github.com/cnabio/cnab-go/pull/237 - Radu suggested that we use a different library. Carolyn is looking into it. - How'd we discover? In Porter, we are keen on doing things in parallel (dependencies, running tests, etc.) Porter is using the current patch. - Carolyn has reached out to see if maintainers would want to integrate the patch(es)... have not heard back - Consensus: if alternate library thread-safe, let's do that way, otherwise we're comfortable with the patched lib ## January 20 2021 - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Chris Crone, Simon Davies, Matt Butcher | | Note Taker | Matt Butcher | - What is missing from the registry spec? - Last time, we were wondering if we have sufficient info for "as a CNAB runtime, I can tell the difference between artifacts" - Chris gave background about what the issues where, what facilities are available today, and what OCI features might be coming in the future. - Chris: Some registries don't allow mixing OCI types and Docker types, so that prevents us from taking some approaches. So we took the approach of making it work with compatibility with today's registries, thus CNAB-to-OCI - Issue https://github.com/cnabio/cnab-spec/issues/382 - Chris: Annotations don't exist on Docker registry indexes, so... Do we want to fix (have partial solution) as part of the spec now? And have v2 when this gets sorted? - Matt: I think the question is do we write a specification with today's capabilites, or do we completely defer writing a spec? - Simon: Enable success today, and that means writing a spec for today, accepting that we may need to do a breaking change for a v2 registry spec - Some discussion about switching to ORAS. Doing so would give up on a number of off-the-shelf tools that do image layer scanning, etc. None of us found a compelling reason to justify rewriting. ## January 14th - Registry and Security meeting Canceled due to the lack of response. ## January 6, 2021 - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Matt Butcher, Carolyn Van Slyck, Simon Davies, Trishank Kuppusamy, Jacob LeGrone | | Note Taker | Matt Butcher | - Issue Backlog Review - https://github.com/cnabio/cnab-spec/issues/363 (yes we would like to do this) - Proposed change to section 810 - Bundle should be able to declare "needs Docker socket" or "needs privilege" - Carolyn: Porter does something like this with extension. We could make this official.Uses a custom extension. - Because this is critical for a runtime to be able to work correctly, we should standardize this - Simon: Does this tightly couple us to containers? - Carolyn: With the extension, we just mark that the bundle has Docker required, and has a custom extension. - Simon: There are a number of niggly edge cases with Docker being more flexible in execution backends. - Matt: Do we want to do this in core or in 810-well-known-extensions - Carolyn: We probably need two extensions in 810. One is for remotes, one is for tighter requirements for a Docker daemon - Simon: And a design that recognizes that this is driver-specific. - https://github.com/cnabio/cnab-spec/issues/366 (yes we would like to do this) - Use `applyTo` for credentials - Proposed change to CNAB Core - Hoping to do this as a minor patch - Simon: What is the usecase? - Carolyn: Install command interacts with cloud and creates a new cluster. To do that, it needs credentials for the cloud provider itself. But after that, it does not need cloud provider creds, just the new KubeConfig. This prevents us from having to give creds that are unnecessary for subsequent commands (like getting logs, etc.). - https://github.com/cnabio/cnab-spec/issues/385 - Created https://github.com/cnabio/cnab-spec/issues/401 to track state concerns as a whole - Proposed change to 810 (parameter-sources) - Carolyn: We can go this way or talk about workflows - Jacob and Carolyn: The use cases are pretty similar. Should look at both and decide between the two. - Jacob: There are cases where we want params that are disjoint from bundle. This might help with that. - Simon and Jacob: Similar to Teraform. NB: The notetaker lost track of this conversation. :-D - Discussion over whether there are still gaps between the state discussion and the workflow discussion - Simon: May be information in a claim that is useful in subsequent action, but that is not explicit input to the further action. State storage (data that is generated but not fed as input) might be modeled as output, but right now we are restricted in that it has to be a param for a subsequent action. Workflow or task-level dependencies might help here. - Carolyn: Porter was starting to use parameters to store state, and that was problematic because we don't want that exposed to the user. Hid those params in the explain feature. So it's not advertised, but still doesn't feel like the right way to solve it - Simon: Hard for the runtime to extract claim information on behalf of the bundle. - Carolyn: Explicit outputs and parameters -- we can condsider a new "state" concept to the bundle author and have the runtime automatically attach state per the bundle author's definition (e.g. tf state). This makes it clear that this is an internal concern of the bundle. - Jacob: TF state definitely is a good use case for this - Carolyn: Will make a new issue covering all of these three. We can then dive in on that issue. - https://github.com/cnabio/cnab-spec/issues/338 - Extension for Declarative K8S Installers - Carolyn: This was originally for VMWare. Are we still interested? - Nobody seems too interested in this. Shall we close-won't-fix for now? - Status on Registry Spec - https://github.com/cnabio/cnab-spec/issues/382 - Some discussion on whether this is already handled by OCI. - Jacob: This probably does not need to be part of our spec. We could make a suggestion to use the OCI annotations... - Tishank and Carolyn: If we have some specific labels we use, we probably should define them in the spec - Other things - Jacob: We should make sure the tools can figure out how to resolve a bundle from an OCI reference - Need issue if there is not one already ## November 4th - Registry and Security meeting Canceled due to the lack of agenda. ## October 28th - General meeting | | | | -------- | -------- | | Recording | | | Attending | Matt Butcher, Chris Crone, Vaughn Dice, Radu Matei, Carolyn Van Slyck | | Note Taker | Radu Matei | ### Agenda - final approval for the Claims specification: - Chris: I approved before, and there have been no changes in the meantime. - Matt: I also approve. - Matt: we need create the two tags (for the Claims spec and 1.1 core) and we should be done with the release of both Claims and 1.1 - update on the registry specification: - Chris: Radu created issues on - Radu: we merged an initial draft of the specification, and we have issues for the remaining work items. - Matt: a blog post about the claims and 1.1 Core? - Chris: sure, we never over-communicate - Matt: I volunteer for a blog post. - Chris: Jacob's talk about CNAB: https://www.youtube.com/watch?v=H67uuwVO1tc - Chris: how is Porter coming along, when do we see another demo? - Carolyn: we can show another demo! We showed an air-gapped demo for SIG App Delivery, and we can it. - Chris: Any KubeCon talks we know about? - Carolyn: now that Porter is CNCF, we should have a talk next conference. ## October 21st - Security and Registry meeting | | | | -------- | -------- | | Recording | | | Attending | | | Note Taker | Radu Matei | ### Agenda - review / questions about `signy`#80 - [document](https://hackmd.io/@radu/SJHDDDpPP) - `"intoto"` vs `"in-toto"` as the JSON key for the in-toto custom metadata field (see [comment](https://github.com/cnabio/signy/pull/80#discussion_r509029313)) - other Signy issues: - [`signy` #81](https://github.com/cnabio/signy/issues/81) - Use length from Notary to cap bundle download size - [`signy` #70](https://github.com/cnabio/signy/issues/70) - Disallow pushing trust collection to a repository without a tag ## October 14th - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Carolyn Van Slyck, Radu Matei, Simon Davies, Ralph Squillace | | Note Taker | Radu Matei | ### Agenda - maintainer vote on Claims spec - [`cnab-spec#390`](https://github.com/cnabio/cnab-spec/pull/390) - we tried to have unanimity in the past from spec maintainers - :tada: claims spec 1.0.0 has now been merged - review [`cnab-go-#232`](https://github.com/cnabio/cnab-go/pull/232) - Fix invocation image digest validation (Carolyn) - when multiple repo digests were present for an image, `cnab-go` would previously fail; but this is guaranteed to happen for `cnab-to-oci`. - example use case for this scenario in [this Porter blog post](https://porter.sh/distribute-bundles/#image-references-after-publishing) - we need one more person to have a look at the PR before merging, ideally one of `cnab-to-oci`'s maintainers - the fix has been tested (and is currently used by Porter), but we want to make sure there are no edge cases that haven't been considered yet. ## Oct 7th - Security and Registry meeting | | | | -------- | -------- | | Recording | | | Attending | | | Note Taker | | ### Agenda - No meeting ## September 30th - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Jacob LeGrone, Matt Butcher, Carolyn Van Slyck, Radu Matei, Vaughn Dice, Simon Davies | | Note Taker | Carolyn Van Slyck | ### Agenda 🚨 New zoom link: https://carolynvs.com/zoom - Welcome new CNABBERS! 🦀 - CNAB_LAST_REVISION https://github.com/cnabio/cnab-spec/pull/386 - This has been merged as of yesterday - Could be useful for stateless invocation images, e.g. doesn't have config maps but does add annotations. Maybe it does a diff action if it had the revision. - DD doesn't do that today because they use helm instead. - It's not clear which revision for the runtim to grab, due to custom actions. - We removed to pave the way for a final release of CNAB Core 1.1.0 and CNAB Claims 1.0.0. For now let's remove and wait for a solid use case. - CNAB Core 1.1.0 Release - https://github.com/cnabio/cnab-spec/pull/388 (approved + merged) - It's not approved by the working group yet. We need approval from the WG: - radu, butcher, jeremy, silvan, Jacob, carolyn (from OWNERS) - Need a majority vote from the above people before we can create a branch - Then the executive team can approve and cut a tag from the branch - We just need to cut the tag to formalize the release (process above) - CNAB Claims 1.0.0 Release - https://github.com/cnabio/cnab-spec/pull/390 - Same process as the core release above - We need change logs for both releases. Vaughn volunteers to create them. - Let's add a checklist item to the PR template to add to the change log. - Vaughn will follow-up and add an issue for it. - Not sure if we have a PR template? - Demo at a later time the Data Dog Worflow and trick them into upstreaming it into the spec - Is it cloud native if it's not YAML? 🤔 ## September 23rd - Security and Registry meeting | | | | -------- | -------- | | Recording | | | Attending | | | Note Taker | | ### Agenda - Welcome new CNABBERS! 🦀 - Chatting about Signy and registries ## September 15th - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Trishank Kuppusamy, Matt Butcher, Radu Matei, Chris Crone, Vaughn Dice, Ralph Squillace | | Note Taker | | ### Agenda - Welcome new CNABBERS! 🦀 - Porter was accepted into the CNCF Sandbox! ([link](https://github.com/cncf/toc/issues/533)) - review [`cnab-spec`#384](https://github.com/cnabio/cnab-spec/pull/384) - clarification for `contentDigest` - vote for the claims spec? - [`signy`#80](https://github.com/cnabio/signy/pull/80) - finish the verification implementation - quick update on cnab in GH, in-toto, and signing work ## Notes - review [`cnab-spec`#384](https://github.com/cnabio/cnab-spec/pull/384) - this was a blocker to one of Jacob's PR, and a follow-up adding verification for the content digest on the Docker driver. One more comment to address, but mostly follow-up issues. It needs one more reviewer. - Matt: If we merge #384, it is a change in the spec - do we need to release a new minor release, or just an erratum? - defer to Chris on this PR - Matt: I think it requires a minor release. - Chris: I think it requires a minor - Voting for the claims spec - Vaughn: Carolyn has implemented it in Porter - Vaughn: Reading through spec to see if it is ready. Otherwise, I think it is ready for a vote - Matt: maybe bring. it to a vote next meeting? Does that give us enough time to go through everything one more time? It would mean a finalized claim spec around October. - Vaughn: what is the process for releasing a new spec? - Matt: the person who takes lead opens a PR with all appropriate changes, we create a branch from that, core maintainers vote on the issue, then the directors vote on it and mark it as final. For the next time, the person who volunteers who take the lead on this should create the PR for the spec. - Ralph updates on GH status - CNAB bundles work in the new GH registry (using Docker Distribution in the backend). Thanks to Chris and team for all of that work. And to Radu for all the extra work. - Porter action in GitHub marketplace works for pushing to GH registry, but pull still needs a little work. (Media type is not recognized) Same is true for CLI - Ralph will open a proof of concept repo - Ralph, Santiago and Trishank discussing rudimentary In Toto signature trees--and how to start this process. Approach GH to create a preview? - First target is to demonstrate with this that metadata can be attached to a bundle, with validation endpoint - Trishank: Theoretically, signy is ready for use in that last step - Trishank: Add in-toto links to make it work should be easy. Hard part will be making the root layout - Ralph: We can limit the scope of the signing chain using GitHub's roles, and that might make it possible to rely on the tooling for most of the main process - Radu: Does that interact with the CODEOWNERS file? Do we need an owners file for the root layout? - Trishank: Worth looking into, but not necessary - Ralph: Next issue will be getting people to use this, and GH itself might be the ones to push this. "Give professional tools to the entire world." - Requirements for having something working: Notary endpoint on registry side, and signy support for the entire workflow - Signy #80 - Radu: Trishank has done most of the work to make this happen. - Radu: Last step is to perform the verification step. It was working before; just needs to be re-enabled - Trishank/Radu: Convention for doing delegation is our first step ## September 2nd - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Simon Davies, Ralph Squillace, Matt Butcher | | Note Taker |Matt Butcher | Informal checkin on things. - Claims and Reg spec discsussion defered - Work with GitHub to add support (via MS) Meeting adjourned early ## August 19th - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Carolyn Van Slyck, Vaughn Dice, Chris Crone, Karen Chu, Jacob LeGrone, Radu Matei | | Note Taker | Radu Matei | ### Agenda * Welcome new CNABBERS! 🦀 * KubeCon CNAB Demos/Presentations! * Chris's demo: https://github.com/chris-crone/kubecon-eu-20 * https://kccnceu20.sched.com/event/Zet9/simplify-your-cloud-native-application-packaging-and-deployments-chris-crone-docker * https://kccnceu20.sched.com/event/Zemr/sharing-is-caring-push-your-cloud-application-to-an-oci-registry-silvin-lubecki-djordje-lukic-docker * https://kccnceu20.sched.com/event/Zexh/deep-dive-harbor-enterprise-cloud-native-artifact-registry-steven-zou-daniel-jiang-vmware * https://sched.co/Zeiq (Where to Put All That YAML: Secure Content Management for Cloud Native Apps - Ryan Abrams, Mirantis) * cnab-go PR bump: https://github.com/cnabio/cnab-go/pull/227 * we noticed that there is a part of the spec that `cnab-go` is not implementing yet, validating image digests prior to bundle invocation. * it seems like it has to be driver specific, but would love feedback to the current implementation, which adds digest validation to the Docker driver. * there are some inline questions that would require some feedback. * in some cases there can be more than one repo digest - is it acceptable to have exactly one match? * Radu: the security spec explicitly invalidates the signature of a bundle if the content digest of an image changed, so I don't think we should allow multiple digests. * Jacob: there should only be one digest. In the Kubernetes driver we download by digest, so we pull by that digest, and digest validation is offloaded to the kubelet. * Vaughn: action item - we should revisit https://github.com/cnabio/cnab-go/pull/166 * Jacob: do we have clarity on what the spec "content digest" refers to? * Radu: I think the wording has been changed to reflect that the "content digest" refers to the manifest digest. ## August 11th - Security and Registry Meeting | | | | -------- | -------- | | Recording | | | Attending | | | Note Taker | | ### Agenda * Welcome new CNABBERS! 🦀 * Signy PR #80 * Discuss with Jacob how to test and deploy Signy with CNAB-to-OCI instead of ORAS ## July 29th - Security and Registry Meeting | | | | -------- | -------- | | Recording | | | Attending | | | Note Taker | | ### Agenda * Welcome new CNABBERS! 🦀 * Updates on Notary v2 * TUF might be _one_ option for the implementation for Notary v2 * prototype released by Steve Lasker and Shiwei - https://github.com/notaryproject/nv2/pull/2 * Pairing on in-toto verification * (We might need to use another room for this meeting only, so we can share screen. More info in the #cnab channel in the CNCF Slack). * Ping Ralph about https://github.com/cnabio/cnab-spec/pull/362 ## July 22nd - General Meeting | | | | -------- | -------- | | Recording | | | Attending | Radu Matei, Vaughn Dice, Carolyn Van Slyck, Trishank Kuppusamy | | Note Taker | Carolyn Van Slyck | ### Agenda * Welcome new CNABBERS! 🦀 * Trishank to show Matt his new Moka pot * Vaughn: Demo Parameter Sets **Canceled due to lack of agenda and no host** ## July 15th - Security and Registry Meeting | | | | -------- | -------- | | Recording | | | Attending | | | Note Taker | | ### Agenda * [Registry PR](https://github.com/cnabio/cnab-spec/pull/362) still needs one more LGTM from a spec maintainer * Radu still working through the remaining TODOs [here](https://github.com/cnabio/signy/pull/80) (mainly verifying keys and performing the verification process). ## May 6th - Security and Registry Meeting | | | | -------- | -------- | | Recording | | | Attending | | | Note Taker | | ### Agenda * ~~Welcome new CNABBERS 🦀~~ * ~~[Discuss Registry PR](https://github.com/cnabio/cnab-spec/pull/362#pullrequestreview-404372943)~~ * ~~Known implementations [`cnab-spec`#288](https://github.com/cnabio/cnab-spec/pull/288)~~ * ~~Clarify how to verify & inspect in-toto metadata [`cnab-spec`#364](https://github.com/cnabio/cnab-spec/pull/364)~~ * ~~Option to verify in-toto metadata on OS or container [`signy`#73](https://github.com/cnabio/signy/pull/73)~~ * ~~Minimal example of a software supply chain [`signy`#74](https://github.com/cnabio/signy/pull/74)~~ * Postponed ## April 22nd - Security and Registry Meeting | | | | -------- | -------- | | Recording | | | Attending | Trishank Karthik Kuppusamy, Radu Matei | | Note Taker | | ### Agenda * Welcome new CNABBERS * Discuss where `signy` should verify in-toto metadata * OS? * Container: user-specified or bundle-developer-specified? Why not bundle-developer-specified? Discuss infinite regress of in-toto metadata verification. * Discuss [`signy#73`](https://github.com/cnabio/signy/pull/73), a PR that provides both OS or container verification * Jacob: would like to see a declarative API/DSL for well-known inspections * Radu: would like to see better UX for signy to OS/container verify * Jacob & Radu: inspections, just like invocation images, do open up some security issues about resource consumption for invocation and verification images * Discuss simple in-toto software supply chains aka root layouts * ABC example * Datadog example * Collect CNAB uses cases users would like to see * CNAB use case: two steps * Image use case: also two steps * Jacob & Radu would like to support out-of-the-box signing for both bundles and images * Trishank: need to fix `signy` to verify images referred to in bundles, will be off by default, but people like Jacob can turn it on * Radu: `signy` can check signatures on bundles copied to a different registry, even with a different GUN, unlike Notary-v1 * Please review: * [`cnab-spec#288`](https://github.com/cnabio/cnab-spec/pull/288) * [`cnab-spec#364`](https://github.com/cnabio/cnab-spec/pull/364)