changed 2 years ago  
Linked with GitHub
  • 1. # CNAB Community Meeting

link text

Meeting time: Every other Wednesday 9:00 AM - 10:00 AM US Pacific Time
Zoom Link: https://cnab.io/zoom with passcode 77777
GitHub Repo: https://github.com/deislabs/cnab-spec

Meeting Minutes and Agenda

Future Agenda Items

  • Installation state https://github.com/cnabio/cnab-spec/pull/411 (Radu)
  • non-normative section for naming dependencies installation name, e.g. myinstall-redis
  • upgrade path for dependencies
    • Installation Lockfile
      The resolved dependencies' installations
      e.g. Wordpress required MySQL, the lockfile has the installation name of mysql stored
  • unused dependencies, ownership tracking
    • Detecting unused dependencies
      • Query for any lockfiles that reference an installation

March 2, 2022 - General Meeting (CANCELED)

February 16, 2022 - General Meeting (CANCELED)
Recording
Attending Carolyn Van Slyck
Note Taker

February 2, 2022 - General Meeting
Recording
Attending Carolyn Van Slyck, Steven Gettys
Note Taker

  • 🚨 Carolyn: Next meeting the zoom link will change! I'll remind everyone and update this doc with it.

  • Updates on the "is upgrade required" controversy from last time https://github.com/cnabio/cnab-spec/blob/main/103-bundle-runtime.md

    An implementation of a CNAB runtime MUST support sending the following actions to an invocation image: install, upgrade, uninstal.
    Invocation images SHOULD implement install and uninstall. If one of these REQUIRED actions is not implemented, an invocation image MUST NOT generate an error (though it MAY generate a warning). Implementations MAY map the same underlying operations to multiple actions (example: install and upgrade MAY perform the same action)…
    And an error MUST NOT be issued if one of the three built-in actions is requested, but not present in the bundle. Errors are reserved for cases where something has gone wrong.

  • Sounds like a bundle shouldn't return an error if it doesn't define upgrade and it should always be okay for a runtime to call upgrade on an invocation image. It's super confusing that we made them optional sometimes but whatever.

  • Naming conventions for generated installations https://github.com/cnabio/cnab-spec/issues/423#issuecomment-1022363162

    • Carolyn: I think this can be non-normative. Tooling may have different way to generate the name. Different tool may not be able to extract the output of an dependency based on its name if the generated name is not normative. We shouldn't directly reference a dependency that's deep in the dependency tree bc interface is defined in the parent bundle.
    • Jacob: maybe it can be up to the tooling. Is there any place we would need to reference the name for the installation. We are always targeting the parent bundle so we don't really need to have a way to reference the dependency bundle name. design an interface to provide path to a child dependency through parant bundle name.
    • Steven: What would be the use case for having different tools managing bundle lifecycle. Have a warning in the spec to alert the impelmentation to handle this.
    • Ralph: There's not really real world usecase for the scenario that people will change tooling between bundle lifecycle.

  • dependency graph resolution https://github.com/cnabio/cnab-spec/issues/425#issuecomment-1022400402

    • Carolyn: I've provided additional questions and comments

    • What needs to be defined in the spec for resolution? Inputs/outputs?

      • Is any/all of resolution non-normative?
      • being able to speicify an existing installation as dependency
      • what bundle we should use when installing

    • Jacob: it can be up to the tool. rollout plan can be tool speicific as well. How to understand a bundle implement an interface? And how can a bundle specify that. Keeping the state of what bundles can satify an interface is tool specific.

      • kubeconfig is refreshed upon each action.

    • Steven: developing against bundle interface. Very interested in multi-layer dependencies. What if a dependency has changed during an upgrade?(table this for now).

      • Distributing kubeconfig to devs but have expiration without having to rebuild the cluster

    • Ralph:

  • Passing creds/params/outputs https://github.com/cnabio/cnab-spec/issues/424

    • Carolyn: this is a required part of the spec. I think we are ready to write something up for it based on that issue.
      • this is going to make the dependency graph resolution more complicated
      • this is going to guide the resolution.
      • do I want to consume things through dependency or pramaters? It should be up to the tool, not the spec.
      • Can only reference one level down
    • Jacob: this needs to be in the spec.

  • What's the state of the dependency spec and how will new additions be treated?

    • Steven: we don't really have the usecase to directly reference a deeply nested dependencies
    • Carolyn: add versioning to the spec. Dependency will always be an extension.

  • What should happen when a dependency fails?

  • What needs to go in the spec around upgrade, custom actions and uninstall vs. what can be non-normative covered by tooling?

    • We need to spec out things that affect "can the bundle run"

  • Next steps:

    • Porter will be our validation of the spec before we lock it down
    • We need to reimplement graph resolution to support:
      • dependencies of dependencies
      • bundle interfaces
      • version ranges
      • overriding with an existing instance or another bundle reference
      • wiring up dependency params/creds/outputs

  • Jacob: Invocation image can know if it has dependencies and how to call into it. Alias actions to bubbling up actions from child dependencies.

  • Carolyn: I don't see how invocation image has enough information to call an action from its dependencies.

  • Jacob: is non-digested image reference allowed in the spec?

  • Carolyn: it's sort of in the spec. We will make this into a new issue to discuss more

Janary 19, 2022 - General Meeting
Recording
Attending Yingrong Zhao, Carolyn Van Slyck, Jacob LeGrone
Note Taker Carolyn
  • Can we switch which zoom we use so that it's easier for me to download and post recordings?
  • Dependencies Issues:
    • Define a bundle interface https://github.com/cnabio/cnab-spec/issues/422
      • carolyn: I think that the spec only requires install and uninstall, upgrade is optional. You can't tell from the bundle.json if upgrade will work…
      • Everyone read the spec and figure out if we have a gap or if carolyn can't read
      • The interface should include custom actions that it wants to use, and the parameters/credentials that they take, and the outputs that the action generates.
    • Pass params/creds to a dependency https://github.com/cnabio/cnab-spec/issues/424
    • Naming conventions for generated installations https://github.com/cnabio/cnab-spec/issues/423
    • Dependency graph resolution https://github.com/cnabio/cnab-spec/issues/425
    • Reuse an existing installation for a dependency https://github.com/cnabio/cnab-spec/issues/426
      • jacob: scenario #2 looks like datadog's workflow functionality. The workflow handles parameter sources and passing them to each bundle in the workflow, or lookup up outputs and passing them to a later task in the workflow.
      • jacob: individual bundles don't define dependencies, that's all handled at the workflow level
      • may use dependencies int eh future just for exclusive dependencies that need to be kept in sync together.

Janary 5, 2022 - General Meeting
Recording
Attending Carolyn Van Slyck, Jacob LeGrone, Ralph Squillace
Note Taker
  • trishank: Disband the biweekly security meeting
  • Signy
    • Can do we do a signy release and blog post about where it is now?
    • carolynvs: I'd like to start work on making signy usable as a library
    • Ralph: when we last talked with Radu and Trishank, #80, was 5 separate issues. Only 1 was a must have fix, the rest addressed a UX issue that we can optionally try doing later. Possible to create a signature that had a payload too big for notary. We haven't seen anyone actually hit this limit. Workaround is to use some notary hacks to dump history. The remaining 1 fix is: https://github.com/cnabio/signy/pull/91
    • We think that we could fix this message size concern in the future, though the question of "which version of notary" complicates the answer a bit.
    • ✅ Ralph will follow up with Trishank to decide if 80 can be merged, cherry-picked or abandoned.
    • ✅ Ralph will chat with Trishank, Radu and Scott Buckel to get info on remaining 4 tasks (93-96)
  • Dependencies Reboot
    • See use cases in https://github.com/getporter/proposals/blob/9ed393e424b99f3d2bb09ee1773719dc46278b4f/pep/003-dependency-namespaces-and-labels.md#motivation
    • Let's update the dependency spec to enable these use cases
    • Polymorphic dependencies requires additional thought about the search space and if it should be arm waved by the spec and left up to the tools
    • we can use user provided resources to implement polymorphic and avoid the search space question again, the user gives the install action the name of the instance to use.
    • fun thought: a bundle can ship a default impl withthe invocation image, but the runtime could choose to swap the invcation image with another (i.e. default is what we use locally but in remote execution we use something else)
    • need to address naming conventions for generated instances for dependencies https://github.com/cnabio/cnab-spec/issues/423
    • could address unique name constraint in the spec by having a naming convention, e.g. namespace/name
    • Need to address how to pass params / creds / outputs into dependencies https://github.com/cnabio/cnab-spec/issues/424
    • Need to discuss Graph resolution, depth, cycle detection, or aborting searching for a solution. https://github.com/cnabio/cnab-spec/issues/425
      • Ralph: understand when resolution occurs and how that interacts with very long running bundles
      • Carolyn: we will resolve a static execution plan up front before executing anything.
    • Ralph: people want rollbacks, how do they come into play when a dep fails and they would like to rollback the entire graph of changes. Let's think about single bundle rollbacks first and then later evaluate how that can work with a dependency graph
    • Jacob: Sending signals to invocation images, e.g. cancel. Split into new topic for single
    • encapulate dependencies:
      • wordpress needs a db, postgres on clouda. top level bundle controls who gets which params, cred, and handles mapping them to dependencies. deps do not impact the public interface of parent bundle.
    • Need to think about how we can bubble up, compose or expose custom actions of dependencies as a custom action on the top level bundle
    • Define what comprises a bundle's public interface: param, cred, output. Should it include actions? https://github.com/cnabio/cnab-spec/issues/422

September 15, 2021 - General Meeting
Recording
Attending Carolyn Van Slyck, Hadrien Patte, Vaughn Dice, Matt Butcher, Brian DeGeeter, Ralph Squillace
Note Taker Vaughn Dice
  • Reminder: Anyone can add items to the agenda
  • Carolyn: Demo how Porter is using parameter sources and outputs to support a bundle "state bag"
    • Demo link: https://github.com/carolynvs/tabbycat-demo
    • Slated to arrive in an upcoming v1 alpha release
    • Background: Had been authoring bundles with the Terraform mixin, but anytime install failed, the .tfstate file wouldn't be persisted, causing all sorts of issues
    • Solution: combine two existing concepts: Parameter Sources and Outputs. Treat the .tfstate file as state that shouldn't/needn't be managed by the bundle author. Rather, Porter does the plumbing in the background in the form of its new 'state bag', e.g. state section of the Porter manifest
    • Anything in this state bag will be persisted regardless of the result of an action (install, etc.)
    • Additionally, entries won't show up in the Parameters (or Outputs) section when viewing/explaing a bundle (they were in the past)
    • Vastly simplifies managing state. Recall that users of the Terraform mixin can still configure/use a remote backend (using a cloud provider, etc.) but this brings more complexity/overhead. Still perhaps recommended for production, however.
    • (Tangent): Recall that a K8s Service IP value can be acquired via the Helm3 mixin. Handy!
    • How is this implemented?
      • Although this is a Porter feature, it uses concepts that are in CNAB Spec today (i.e. other implementations can do similar!). Main spec concept used is Parameter Sources
    • Question (Ralph): How does this state feature interact with bundle version?
      • Clarification: Bundle version represents the semver of the bundle itself, App version represents the semver of the bundled app (in demo, this value went under the custom section), Installation revision represents a particular instance+modifying action+result
      • Example: Say the same/unmodified bundle is 1. installed (revision 1) then 2. upgraded (revision 2). The state used on upgrade is from the previous revision (1, install).
      • Ralph: This looks great. Understand that bundles are getting more and more diverse. Does this feature sufficiently cover the growing array of use cases for tracking state?
    • Hello from Brian DeGeeter! (Terraform mixin contributor!) Q: Is more metadata necessary in the state section? For example, knowing what version of the bundle is associated with state.
      • Carolyn: Recall that we track both the previous and current version of the bundle.json within the installer image when running an action.
    • Follow-up idea: Do we really want bundle authors to always create standard state sections (say, tfstate and/or tfvars for a Terraform-based bundle)? Perhaps we can streamline it such that this isn't needed.
  • Needs attention (reviewers!):

August 18, 2021 - General Meeting
Recording
Attending Carolyn Van Slyck, Radu Matei, Matt Butcher, Vaughn Dice
Note Taker Vaughn Dice
  • Carolyn: Demo v1 changes in Porter Operator that are part of the new installation state spec
    • Carolyn has been working on functionality to store additional information around installations
    • An installation record will now be a first-class resource – vetting in Porter first, CNAB Spec next!
    • Demo in Porter…
      • Support for namespaces (for installations as well as cred and parameter sets)
      • Empty namespace = global
      • Support for labels (helps with organization, mgmt of installations)
      • Other items tracked on an installation:
        • Bundle repository
        • Bundle version
        • Bundle digest
        • Credential sets (named mapping of cred values)
        • Parameter sets
        • Parameter values
      • Setting things up so users can edit the installation record to reflect what they want it to look like; then Porter can apply those changes. (Shift towards imperative approach)
  • Discuss lessons learned about what should go in the spec, and what should stay in the tooling
    • Data access layer that existed in cnab-go was not very performant; did not support rich querying
    • Moving towards removing this logic from spec libraries like cnab-go and putting the control into the implementations
    • Porter now uses mongo for storage backend
      • The default is the 'mongo-docker' plugin which runs mongo in a Docker container with an associated/persisted Docker volume
      • Can also use the 'mongo' plugin to provide a mongo connection string to Porter to connect to
  • Questions
    • Radu: What are the items that are anticipated to land in CNAB Spec
    • Carolyn: In general, only things related to the runtime
    • Installation State Spec is first up
    • A lot of this work is being done to lead up to a more robust dependencies spec
    • Group (present attendees) consensus agrees this is a good approach

August 4, 2021 - General Meeting
Recording
Attending Matt Butcher, Radu Matei, Vaughn Dice, Aries
Note Taker
  • Deferred dependency discussions and operator demo until a time convenient for Carolyn.
  • Update on signy "It makes sense and we are happy with the current plan"
    • Verification process is nearly complete as well
  • Butcher/Vaughn: Carolyn is working on storing installation records (claims and labels) in MongoDB.
  • No major or minor spec changes between now and September

June 23, 2021 - General Meeting
Recording
Attending Carolyn Van Slyck, Jacob LeGrone, Matt Butcher, Aries, Radu Matei, Ralph Squillllace
Note Taker

Agenda (Notes inline):

  • Carolyn: Mistakes were made, taking storage out of the CNAB spec and cnab-go.
    • Here's what we have now: installation states, using namespaces and labels
    • Current interface is CRUD for one item
    • No current need to query the data
    • Would be nice to say "list all items in a namespace" or "list all things with labels X, Y, Z"
    • Started to feel like creating an ORM w/o knowing the storage layer
    • Suggestion: Claims store should not be in cnab-go. Only the format should. Storage is implementation-specific.
    • Jacob: Agree… interop between runtimes is a non-goal of CNAB
    • Butcher: I agree with Jacob, it is a non-goal to have interop between different runtimes at the claim level.
    • Carolyn: So we can remove it from cnab-go, and things will be backed out of the spec so that the spec is just about the format.
    • Jacob: Could add it back in the future. See also Temporal's storage interface.
  • Carolyn: Any new agenda items? Signy?
    • Ralph: Trishank, Radu, Ralph have been chatting about the remaining four issues to achieve 1.0
    • Main issue was to handle situation where notary can be flooded with work
    • Need to document what Notary thinks it can handle
    • Impl works, though
    • Radu: Scenario: You have been pushing in-toto md for a few versions. The actual TUF target JSON for your repo (version/tag) has been growing to lots and lots of MD. All of that data attaches to the same file as notary (shared w/ in-toto). There is an (internal) limit to how much MD Notary can handle. So somehow we should mention that the MD can be stored externally. But… (a) this was not originally a requirement, and (b) it exists regardless of in-toto. So probably we should just test and document rather than coming up with an external storage. That would be a spec bump.
    • Ralph: Agree. Document the Notary issue (what is the max size?) and declare 1.0.
    • Radu: Workaround is trivial: When you approach the limit, create a new tag
    • Jacob: Is this actually likely in practice?
    • Ralph: We don't know
    • Clarification about what sort of data we are talking about, and that there are definitely limits. Don't know how high those limits are. And this impacts all MD for Notary, not just signatures.
    • Carolyn: So there are three other issues?
    • Radu: Hackathon maybe next week to solve remaining 3
    • Jacob: Is there a way to remove old MD to reduce the size?
    • Radu: You'd have to look into Notary. Not sure.
    • Discussion about a few ways to delete or overwrite
  • Aries: introduction: Cloud Cover as a PM. Have a similar tech to CNAB, but would like to observe.
  • FIN

June 9, 2021 - General Meeting
Recording
Attending Carolyn Van Slyck, Sam Boyer, Vaughn Dice
Note Taker Vaughn Dice

Agenda (Notes inline):

  • Discuss progress with installation spec POC
    • https://github.com/cnabio/cnab-spec/pull/411
    • Seeing limitations with the default filesystem installation data store (which is only intended for dev anyways)
    • Options: drop support for local filesystem backing store, move to query-able store (SQLite?)
    • Would rather support a more production-friendly default (query/filter/etc.)
  • Data Storage Access patterns
    • Slow due to how we pull documents individually.
    • Can we ditch the filesystem store for sqlite?
    • Add claim document cache?
  • How do we handle migrations when it comes to changes in the claim documents?
    • Thinking this will/should be the responsibility of tool/impls as opposed to cnab-go.
  • Pull Requests
  • Updates from Sam around po11y + CNAB

May 26, 2021 - General Meeting
Recording
Attending Carolyn Van Slyck, Vaughn Dice
Note Taker Carolyn Van Slyck

Agenda (Notes inline):

May 19, 2021 - Security Meeting
Recording
Attending Carolyn Van Slyck, Scott Buckel, Ralph Squillace, Radu Matei, Trishank Karthik Kuppusamy, Carlos O Kieffe
Note Taker Trishank

Agenda (Notes inline):

  • Pull requests:
    • https://github.com/cnabio/signy/pull/92
      • Doesn't block on Notary, so we are good to go with signing
      • Concerns about being able to verify previously signed bundles, because the canonicalization might be different now
      • Are there security (availability) concerns? For example, can different tools verify bundles signed by other tools? What about backwards-compatibility? Decision is that we can break it for now because no production use yet, but we do need to advertise it explicitly (perhaps as a MAJOR SemVer bump).
    • https://github.com/cnabio/signy/pull/89
    • https://github.com/cnabio/signy/pull/80
      • Trishank to break up issues and put into a project
      • Scott and others will pick up issues as seen fit

May 12, 2021 - General Meeting
Recording
Attending Carolyn Van Slyck, Vaughn Dice, Sam Boyer, Matt Butcher, Jacob LeGrone, Ralph Squillace
Note Taker

Agenda (Notes inline):

  • Revisit canonical json and non-integer numbers: https://github.com/cnabio/cnab-spec/issues/413
  • Cancelled is official canceled: https://github.com/cnabio/cnab-spec/issues/412
    • Carolyn will open a PR to fix the spec
  • Dependencies meetings starting next week!
  • Is anyone blocked or waiting on comment/reviews?
  • Issue Queue Spluenking
  • Sam: Grafana, Prometheus monitoring mixins https://groups.google.com/g/mixins
    • Working on a cue base system for writing schemas, helps create migrations across versions as breaking changes are introduced.
    • https://groups.google.com/g/mixins/c/uE1A3gXQ6vs
      • some parameters are known at the beginning of the process vs template variables which are deferred to the end
      • can't even assume that you are deploying to k8s, must work on any platform/installation
      • three stage lifecyle of observability objects: produce, distribute, consume
      • produce loop: user creates and refines the object, determine parameters
      • distribute: perhaps backed by oci registries, git vcs, prometheus endpoint to distribute their mixins, TBD
      • consume: some parameters aren't known until the consume phase
        • mixin package is a tree, some parameterized, includes grapha dashboards and prometheus rules
        • expand parameters (explode), filter out some objects, transform object into format for deployment tool
        • needs to fit into anyone's toolchain
      • grafana today: use mixins to power 1 click integrations (monitor a go app, wasmy stuff, javascript, etc). No parameterization atm
    • https://grafana.com/blog/2021/01/14/how-prometheus-monitoring-mixins-can-make-effective-observability-strategies-accessible-to-all/
    • jsonnet based mixins, has standard labels that are attached to all scraped metrics. A generic mixin can assume some template variables are available, such as environment. Right now you just have to hope that the mixin has a replaceable parameter, and when you use it you have to manually wire up that parameter.
    • Need a preview, what is the impact of changing a parameter
    • would like to track the origin of the parameter (the resolution of the parameter)
  • Jacob: We could provide an extension to validate parameters with something other than jsonschema, such as cue

April 28, 2021 - General Meeting
Recording
Attending Carolyn Van Slyck, Jacob LeGrone, Vaughn Dice
Note Taker Vaughn Dice

Agenda (Notes inline):

  • Alt week meeting - Can we use it for dependencies? Sounds like the security meeting may be starting up again?
    • Showdown! We'll all just show up and sort it out :)
  • Migrating Docker App to Porter - Call for contributors and ideas!
  • Inconsistent spelling of cancelled/canceled: https://github.com/cnabio/cnab-spec/issues/412
    • cnab-go/impl uses 'canceled'; spec 'cancelled'
    • Leaning towards an errata fix in the spec (keeping 'canceled')
  • Installation State Specification
  • Hey! Check out Carolyn's KubeCon talk!!!

March 31, 2021 - General Meeting
Recording
Attending Carolyn Van Slyck, Jacob LeGrone, Vaughn Dice, Simon Davies, Matt Butcher
Note Taker Vaughn Dice

Agenda:

  • Sync our spec maintainers teams

    • (GitHub) Maintainers out of sync w/ OWNERS

  • (carolynvs): Declarative bundles and a v2 spec ideas

    • Porter Proposal for Mixins as Bundles: https://github.com/getporter/porter/discussions/1499
      • Problem: Porter packs all mixins in a bundle in the bundle's invocation image, leading to big images and other issues
      • Idea: What if we distro'd mixins as bundles themselves (as opposed to separate binaries, as they are in Porter now)
      • (Jacob): We thought of the workflow interface to be a bundle itself; decided against it. Fairly happy with this decision. They are similar in many ways, though.
      • (Jacob): They are fairly simplistic, e.g. "I want installations x,y,z installed in order a, b or c"
      • (Carolyn): In this case, none of the mixin bundle executions would be tracked as separate installations – only the one installaion for the main bundle using them.
      • (Carolyn): Efficiency gains: multiple bundles that use the same mixin/version combos (helm, terraform, etc.) can now re-use the same cached mixin bundles
      • (Carolyn): see other Pros in issue above (further provenance assurances, optimized for layer cacheing, performance boost, etc.)
      • (Simon): Q: Considered having a CNAB mixin in Porter?
      • (Carolyn): Perhaps, open to discussing this in a more Porter-centric context. Here we're trying to guage if this model might fit well in CNAB
      • (Jacob): One difference with what we use internally and this approach: We like the ability to track mixin installations separately for dashboarding/debugging. In this approach, that same visibility may not be present
      • (Carolyn): Right. It could be a matter of what makes sense to the bundle author. If sub-installations need to be tracked separately, this would be a good use case for a dependency (which does get its own installation) vs if they do not (maybe go for these mixin bundles)
      • (Carolyn): Realize the origins of this idea are/were Porter-specific, but wanted to present it here before considering a full-fledged proposal
      • (Carolyn): This would represent a shift from the CNAB runtime tool knowing how to run one bundle to knowing how to run multiple bundles and track via a graph (still possibly representing just one tracked installation)
      • (Jacob): Q: How to organize creds/params/outputs to/from these internal/mixin bundles
      • (Carolyn): Thinking this can/will be defined in the workflow section (either new section in bundle.json or maybe under well-defined custom extension)
      • (Carolyn): Do we want to setup a working group meeting?
      • (Jacob): Currently more interested in the Dependencies proposal, but open to discussing this as bandwidth allows

  • Item: (Jacob) Question around the Dependencies proposal. For the Exclusive Resource case, we need to figure out how to name the dependency according to the parent bundle name. (We utilize installation names heavily for tracking/filtering/displaying installations)

    • (Future agenda item added)

  • Needs Review:

    • Installation State Specification - https://github.com/cnabio/cnab-spec/pull/411
      • Originated from wanting to add label and namespace support to the Claims Spec (think K8s labels/namespaces)
      • Evolved into a new spec
      • Introduces a new document dedicated to the installation itself, which also features label and namespace fields prominently
      • Also tracks bundle reference and version to know if a newer version might be available, etc.
      • (Jacob) Very interested in this; Use similar approach for internal tool. Ex: dashboard showing all installations for a bundle, also filtering by other factors (k8s cluster, datacenter, etc.)
      • (Jacob) Does seem like a good idea to normalize this approach
      • Application version (as label) discussion:
        • (Jacob) we interrogate the bundle used for the installation to get this info (often stored separately in a graph db; in general if certain metadata changes, they don't want it to change/update the digest)
        • (Carolyn) there are use cases where bundle authors may want to specify a non-semver app version
        • (Carolyn) this is a good opp to look at how we might want to change how/where we query data
    • Pod Affinity for K8s Driver - https://github.com/cnabio/cnab-go/pull/245

March 17, 2021 - General Meeting
Recording
Attending Carolyn Van Slyck, Ralph Squillace, Vaughn Dice
Note Taker

Agenda:

  • (carolynvs): Declarative bundles and a v2 spec ideas

March 3, 2021 - General Meeting
Recording
Attending Carolyn Van Slyck, Vaughn Dice, Matt Butcher, Jacob LeGrone
Note Taker Vaughn Dice

Agenda:

  • carolynvs: Add apply to field for credentials was merged and is in Porter.
    • Scope creds to a particular set of actions; will be omitted for others
    • In Porter, porter explain takes an optional action arg to show what creds/etc. apply to a particular action
    • Parity with Parameters
    • Eventually will be included in the next 1.x spec release
  • Optionally save logs as an output
    • https://github.com/cnabio/cnab-go/pull/243
    • Save them via a persistent storage option
    • Consider if the bundle execution happens in a remote environment, would want the ability to reference logs after the action
    • Mech: logs are captured/persisted like other outputs
    • Jacob: What about streaming logs?
    • Carolyn: Different effort… this isn't 'real' streaming of logs
    • Jacob: Consider a long-running action and we'd like to see streaming status/progression. Maybe the runtime can read concurrently from the output file as its being written?
    • Brainstorming a strategy of communication between the driver and the invocation image…
    • If we had a well-known output file, the driver could 'watch'/tail as its being written
    • Trying to come up with a more general solution… maybe a custom extension… 'io.cnab.somethin'.streamable'
    • (As written currently, the output file logs only available after the action has run/completed)
    • Carolyn going to look at drafting up something for this streamable scenario
  • carolynvs: Support installation level data storage
  • carolyn: Walk through dependencies proposal for Porter
    • https://github.com/getporter/proposals/pull/8
    • https://github.com/cnabio/cnab-spec/pull/409
    • Carolyn: Overview: How can we utilize k8s approach to resolving instances of software (via namespaces and labels) for dependency resolution
    • See proposal for scenarios intended to be supported
    • Discussion on Polymorphic Dep section:
      • Butcher: Gentoo did something similar; this is a good problem to solve
      • Carolyn: Experience w/ K8s Service Catalog which attempted a rigid enforcement of service interfaces. We'd like to take a less-strict approach.
      • Jacob: Could the API be extended to allow for differing output names (e.g. 'root-connection-string' vs 'connection-string')?
      • Carolyn: Ah, also tried to solve in Service Cat. Really hard! Had the sys admin setup mappings… no one was really keen to do this. Much preferred to come to an agreement at the bundle level, e.g. authors agree that to impl the mysql bundle interface, the output needs to be named 'connection-string', etc.
      • Butcher: Say we needed a key/value store. Redis vs etcd, etc. Both fulfill the interface. But maybe differ on driver used, etc. How do we support this finer-grained specification?
      • Carolyn: Could utilize parameters (already in spec), or maybe labels (e.g. "supports-driverx=true", etc.) on top of using the interface
      • Jacob/Carolyn: Lockfiles. When we resolve deps per requires/interface, we should keep the records of each in a dep graph lockfile.
      • Butcher: How crucial would a lockfile be? Is the dep resolution system already repeatable/dependable?
      • Carolyn: (segues into the 'Register a Bundle' section of the proposal) e.g. 'helm-mysql' is my preferred impl of the mysql iface
      • Discussion on shared dep resources… similar concerns/exp in Helm, e.g. https://github.com/helm/helm/issues/7699
    • On to selectors: namespace, label, name, interface
      • e.g. I rely on Wordpress where label = "app=myapp"

February 17, 2021 - General Meeting
Recording
Attending Carolyn Van Slyck, Vaughn Dice, Matt Butcher, Chris Crone
Note Taker Chris Crone

Agenda:

Notes:

  • carolynvs: Our current jsonschema library doesn't support concurrency.
    • Chris: PR looks good to me
      • Reached out to Sebastiaan at Docker to see if he knows what's up as he's contributed to the library before
  • carolynvs: K8s Driver Updates
    • Need some eyes on this
    • Biggest change is on volumes
  • carolynvs: Support delegation in Signy
    • Could use more eyes on the PR
  • carolynvs: Moved all repos from master to main branch :tada:
  • carolynvs: Proposed Dependencies Change for Porter that we will want to evaluate for the CNAB Dependency spec: https://github.com/getporter/porter/discussions/1441
    • Better model for dependency management with impacts on storage layer

February 3, 2021 - General Meeting
Recording
Attending Matt Butcher, Ralph Squillace, Jacob LeGrone, Chris Crone, Carolyn Van Slyck, Vaughn Dice
Note Taker Vaughn Dice

Agenda:

  • Media Type: Just need to formalize this: https://github.com/cnabio/cnab-spec/issues/381
    • Just need to decide on the appropriate media type and update the spec
    • Two things: 1. Need clarification of wording, 2. Assemble in correct order aka "Everyone's learning the Registy spec Yay!"
    • Ralph is stoked to grab the IANA stuff
    • ? Do we already spec what goes in the config blob itself?
      • We just need the config media type
    • ? Do we want to say where annotations show up (e.g. title, version)?
      • Depends on Docker v OCI manifest spec
      • Some registries don't allow a mix
      • Suggestion: annotations could be optional for now
      • Differentiation between applying at root vs config object would be helpful (makes sense to be the latter, this is what Jacob does)
      • Jacob will file an issue to track
    • ? How is the v1 version used on the mediaType?
      • Is it the overall manifest version or bundle.json?
      • It seems like we should tie it to the bundle.json spec version
      • Leaning towards just tracking the major version - and make sure we only allow backward-compatible v1.x releases (can add fields, etc. but can't change existing schema fields)
      • Full spec version is encoded in bundle.json, so if tools need finer-grained validation, they can extract/inspect
      • We could add a semver annotation to the manifest itself to check w/o fetching the config object
  • carolynvs: I am going to rename our default branches in the cnabio org to main unless someone objects.
    • No objections
  • Pull Requests:
    • Make validating a bundle thread-safe https://github.com/cnabio/cnab-go/pull/237
      • Radu suggested that we use a different library. Carolyn is looking into it.
      • How'd we discover? In Porter, we are keen on doing things in parallel (dependencies, running tests, etc.) Porter is using the current patch.
      • Carolyn has reached out to see if maintainers would want to integrate the patch(es)… have not heard back
      • Consensus: if alternate library thread-safe, let's do that way, otherwise we're comfortable with the patched lib

January 20 2021 - General Meeting
Recording
Attending Chris Crone, Simon Davies, Matt Butcher
Note Taker Matt Butcher
  • What is missing from the registry spec?
    • Last time, we were wondering if we have sufficient info for "as a CNAB runtime, I can tell the difference between artifacts"
    • Chris gave background about what the issues where, what facilities are available today, and what OCI features might be coming in the future.
    • Chris: Some registries don't allow mixing OCI types and Docker types, so that prevents us from taking some approaches. So we took the approach of making it work with compatibility with today's registries, thus CNAB-to-OCI
    • Issue https://github.com/cnabio/cnab-spec/issues/382
    • Chris: Annotations don't exist on Docker registry indexes, so… Do we want to fix (have partial solution) as part of the spec now? And have v2 when this gets sorted?
    • Matt: I think the question is do we write a specification with today's capabilites, or do we completely defer writing a spec?
    • Simon: Enable success today, and that means writing a spec for today, accepting that we may need to do a breaking change for a v2 registry spec
    • Some discussion about switching to ORAS. Doing so would give up on a number of off-the-shelf tools that do image layer scanning, etc. None of us found a compelling reason to justify rewriting.

January 14th - Registry and Security meeting

Canceled due to the lack of response.

January 6, 2021 - General Meeting
Recording
Attending Matt Butcher, Carolyn Van Slyck, Simon Davies, Trishank Kuppusamy, Jacob LeGrone
Note Taker Matt Butcher
  • Issue Backlog Review
    • https://github.com/cnabio/cnab-spec/issues/363 (yes we would like to do this)
      • Proposed change to section 810
      • Bundle should be able to declare "needs Docker socket" or "needs privilege"
      • Carolyn: Porter does something like this with extension. We could make this official.Uses a custom extension.
      • Because this is critical for a runtime to be able to work correctly, we should standardize this
      • Simon: Does this tightly couple us to containers?
      • Carolyn: With the extension, we just mark that the bundle has Docker required, and has a custom extension.
      • Simon: There are a number of niggly edge cases with Docker being more flexible in execution backends.
      • Matt: Do we want to do this in core or in 810-well-known-extensions
      • Carolyn: We probably need two extensions in 810. One is for remotes, one is for tighter requirements for a Docker daemon
      • Simon: And a design that recognizes that this is driver-specific.
    • https://github.com/cnabio/cnab-spec/issues/366 (yes we would like to do this)
      • Use applyTo for credentials
      • Proposed change to CNAB Core
      • Hoping to do this as a minor patch
      • Simon: What is the usecase?
      • Carolyn: Install command interacts with cloud and creates a new cluster. To do that, it needs credentials for the cloud provider itself. But after that, it does not need cloud provider creds, just the new KubeConfig. This prevents us from having to give creds that are unnecessary for subsequent commands (like getting logs, etc.).
    • https://github.com/cnabio/cnab-spec/issues/385
      • Created https://github.com/cnabio/cnab-spec/issues/401 to track state concerns as a whole
      • Proposed change to 810 (parameter-sources)
      • Carolyn: We can go this way or talk about workflows
      • Jacob and Carolyn: The use cases are pretty similar. Should look at both and decide between the two.
      • Jacob: There are cases where we want params that are disjoint from bundle. This might help with that.
      • Simon and Jacob: Similar to Teraform. NB: The notetaker lost track of this conversation. :-D
      • Discussion over whether there are still gaps between the state discussion and the workflow discussion
      • Simon: May be information in a claim that is useful in subsequent action, but that is not explicit input to the further action. State storage (data that is generated but not fed as input) might be modeled as output, but right now we are restricted in that it has to be a param for a subsequent action. Workflow or task-level dependencies might help here.
      • Carolyn: Porter was starting to use parameters to store state, and that was problematic because we don't want that exposed to the user. Hid those params in the explain feature. So it's not advertised, but still doesn't feel like the right way to solve it
      • Simon: Hard for the runtime to extract claim information on behalf of the bundle.
      • Carolyn: Explicit outputs and parameters – we can condsider a new "state" concept to the bundle author and have the runtime automatically attach state per the bundle author's definition (e.g. tf state). This makes it clear that this is an internal concern of the bundle.
      • Jacob: TF state definitely is a good use case for this
      • Carolyn: Will make a new issue covering all of these three. We can then dive in on that issue.
    • https://github.com/cnabio/cnab-spec/issues/338
      • Extension for Declarative K8S Installers
      • Carolyn: This was originally for VMWare. Are we still interested?
      • Nobody seems too interested in this. Shall we close-won't-fix for now?
  • Status on Registry Spec
    • https://github.com/cnabio/cnab-spec/issues/382
      • Some discussion on whether this is already handled by OCI.
      • Jacob: This probably does not need to be part of our spec. We could make a suggestion to use the OCI annotations…
      • Tishank and Carolyn: If we have some specific labels we use, we probably should define them in the spec
    • Other things
      • Jacob: We should make sure the tools can figure out how to resolve a bundle from an OCI reference
      • Need issue if there is not one already

November 4th - Registry and Security meeting

Canceled due to the lack of agenda.

October 28th - General meeting
Recording
Attending Matt Butcher, Chris Crone, Vaughn Dice, Radu Matei, Carolyn Van Slyck
Note Taker Radu Matei

Agenda

  • final approval for the Claims specification:

    • Chris: I approved before, and there have been no changes in the meantime.
    • Matt: I also approve.
    • Matt: we need create the two tags (for the Claims spec and 1.1 core) and we should be done with the release of both Claims and 1.1

  • update on the registry specification:

    • Chris: Radu created issues on
    • Radu: we merged an initial draft of the specification, and we have issues for the remaining work items.

  • Matt: a blog post about the claims and 1.1 Core?

    • Chris: sure, we never over-communicate
    • Matt: I volunteer for a blog post.

  • Chris: Jacob's talk about CNAB: https://www.youtube.com/watch?v=H67uuwVO1tc

  • Chris: how is Porter coming along, when do we see another demo?

  • Carolyn: we can show another demo! We showed an air-gapped demo for SIG App Delivery, and we can it.

  • Chris: Any KubeCon talks we know about?

  • Carolyn: now that Porter is CNCF, we should have a talk next conference.

October 21st - Security and Registry meeting
Recording
Attending
Note Taker Radu Matei

Agenda

  • review / questions about signy#80 - document

  • "intoto" vs "in-toto" as the JSON key for the in-toto custom metadata field (see comment)

  • other Signy issues:

    • signy #81 - Use length from Notary to cap bundle download size
    • signy #70 - Disallow pushing trust collection to a repository without a tag

October 14th - General Meeting
Recording
Attending Carolyn Van Slyck, Radu Matei, Simon Davies, Ralph Squillace
Note Taker Radu Matei

Agenda

  • maintainer vote on Claims spec - cnab-spec#390
    • we tried to have unanimity in the past from spec maintainers
    • :tada: claims spec 1.0.0 has now been merged
  • review cnab-go-#232 - Fix invocation image digest validation (Carolyn)
    • when multiple repo digests were present for an image, cnab-go would previously fail; but this is guaranteed to happen for cnab-to-oci.
    • example use case for this scenario in this Porter blog post
    • we need one more person to have a look at the PR before merging, ideally one of cnab-to-oci's maintainers
    • the fix has been tested (and is currently used by Porter), but we want to make sure there are no edge cases that haven't been considered yet.

Oct 7th - Security and Registry meeting
Recording
Attending
Note Taker

Agenda

  • No meeting

September 30th - General Meeting
Recording
Attending Jacob LeGrone, Matt Butcher, Carolyn Van Slyck, Radu Matei, Vaughn Dice, Simon Davies
Note Taker Carolyn Van Slyck

Agenda

🚨 New zoom link: https://carolynvs.com/zoom

  • Welcome new CNABBERS! 🦀
  • CNAB_LAST_REVISION https://github.com/cnabio/cnab-spec/pull/386
    • This has been merged as of yesterday
    • Could be useful for stateless invocation images, e.g. doesn't have config maps but does add annotations. Maybe it does a diff action if it had the revision.
    • DD doesn't do that today because they use helm instead.
    • It's not clear which revision for the runtim to grab, due to custom actions.
    • We removed to pave the way for a final release of CNAB Core 1.1.0 and CNAB Claims 1.0.0. For now let's remove and wait for a solid use case.
  • CNAB Core 1.1.0 Release
    • https://github.com/cnabio/cnab-spec/pull/388 (approved + merged)
    • It's not approved by the working group yet. We need approval from the WG:
      • radu, butcher, jeremy, silvan, Jacob, carolyn (from OWNERS)
      • Need a majority vote from the above people before we can create a branch
      • Then the executive team can approve and cut a tag from the branch
    • We just need to cut the tag to formalize the release (process above)
  • CNAB Claims 1.0.0 Release
  • We need change logs for both releases. Vaughn volunteers to create them.
  • Let's add a checklist item to the PR template to add to the change log.
    • Vaughn will follow-up and add an issue for it.
    • Not sure if we have a PR template?
  • Demo at a later time the Data Dog Worflow and trick them into upstreaming it into the spec
    • Is it cloud native if it's not YAML? 🤔

September 23rd - Security and Registry meeting
Recording
Attending
Note Taker

Agenda

  • Welcome new CNABBERS! 🦀
  • Chatting about Signy and registries

September 15th - General Meeting
Recording
Attending Trishank Kuppusamy, Matt Butcher, Radu Matei, Chris Crone, Vaughn Dice, Ralph Squillace
Note Taker

Agenda

  • Welcome new CNABBERS! 🦀
  • Porter was accepted into the CNCF Sandbox! (link)
  • review cnab-spec#384 - clarification for contentDigest
  • vote for the claims spec?
  • signy#80 - finish the verification implementation
  • quick update on cnab in GH, in-toto, and signing work

Notes

  • review cnab-spec#384
    • this was a blocker to one of Jacob's PR, and a follow-up adding verification for the content digest on the Docker driver. One more comment to address, but mostly follow-up issues. It needs one more reviewer.
    • Matt: If we merge #384, it is a change in the spec - do we need to release a new minor release, or just an erratum?
    • defer to Chris on this PR
    • Matt: I think it requires a minor release.
    • Chris: I think it requires a minor
  • Voting for the claims spec
    • Vaughn: Carolyn has implemented it in Porter
    • Vaughn: Reading through spec to see if it is ready. Otherwise, I think it is ready for a vote
    • Matt: maybe bring. it to a vote next meeting? Does that give us enough time to go through everything one more time? It would mean a finalized claim spec around October.
    • Vaughn: what is the process for releasing a new spec?
    • Matt: the person who takes lead opens a PR with all appropriate changes, we create a branch from that, core maintainers vote on the issue, then the directors vote on it and mark it as final. For the next time, the person who volunteers who take the lead on this should create the PR for the spec.
  • Ralph updates on GH status
    • CNAB bundles work in the new GH registry (using Docker Distribution in the backend). Thanks to Chris and team for all of that work. And to Radu for all the extra work.
    • Porter action in GitHub marketplace works for pushing to GH registry, but pull still needs a little work. (Media type is not recognized) Same is true for CLI
    • Ralph will open a proof of concept repo
    • Ralph, Santiago and Trishank discussing rudimentary In Toto signature trees–and how to start this process. Approach GH to create a preview?
    • First target is to demonstrate with this that metadata can be attached to a bundle, with validation endpoint
    • Trishank: Theoretically, signy is ready for use in that last step
    • Trishank: Add in-toto links to make it work should be easy. Hard part will be making the root layout
    • Ralph: We can limit the scope of the signing chain using GitHub's roles, and that might make it possible to rely on the tooling for most of the main process
    • Radu: Does that interact with the CODEOWNERS file? Do we need an owners file for the root layout?
    • Trishank: Worth looking into, but not necessary
    • Ralph: Next issue will be getting people to use this, and GH itself might be the ones to push this. "Give professional tools to the entire world."
    • Requirements for having something working: Notary endpoint on registry side, and signy support for the entire workflow
  • Signy #80
    • Radu: Trishank has done most of the work to make this happen.
    • Radu: Last step is to perform the verification step. It was working before; just needs to be re-enabled
    • Trishank/Radu: Convention for doing delegation is our first step

September 2nd - General Meeting
Recording
Attending Simon Davies, Ralph Squillace, Matt Butcher
Note Taker Matt Butcher

Informal checkin on things.

  • Claims and Reg spec discsussion defered
  • Work with GitHub to add support (via MS)

Meeting adjourned early

August 19th - General Meeting
Recording
Attending Carolyn Van Slyck, Vaughn Dice, Chris Crone, Karen Chu, Jacob LeGrone, Radu Matei
Note Taker Radu Matei

Agenda

August 11th - Security and Registry Meeting
Recording
Attending
Note Taker

Agenda

  • Welcome new CNABBERS! 🦀
  • Signy PR #80
  • Discuss with Jacob how to test and deploy Signy with CNAB-to-OCI instead of ORAS

July 29th - Security and Registry Meeting
Recording
Attending
Note Taker

Agenda

July 22nd - General Meeting
Recording
Attending Radu Matei, Vaughn Dice, Carolyn Van Slyck, Trishank Kuppusamy
Note Taker Carolyn Van Slyck

Agenda

  • Welcome new CNABBERS! 🦀
  • Trishank to show Matt his new Moka pot
  • Vaughn: Demo Parameter Sets

Canceled due to lack of agenda and no host

July 15th - Security and Registry Meeting
Recording
Attending
Note Taker

Agenda

  • Registry PR still needs one more LGTM from a spec maintainer
  • Radu still working through the remaining TODOs here (mainly verifying keys and performing the verification process).

May 6th - Security and Registry Meeting
Recording
Attending
Note Taker

Agenda

April 22nd - Security and Registry Meeting
Recording
Attending Trishank Karthik Kuppusamy, Radu Matei
Note Taker

Agenda

  • Welcome new CNABBERS
  • Discuss where signy should verify in-toto metadata
    • OS?
    • Container: user-specified or bundle-developer-specified? Why not bundle-developer-specified? Discuss infinite regress of in-toto metadata verification.
    • Discuss signy#73, a PR that provides both OS or container verification
    • Jacob: would like to see a declarative API/DSL for well-known inspections
    • Radu: would like to see better UX for signy to OS/container verify
    • Jacob & Radu: inspections, just like invocation images, do open up some security issues about resource consumption for invocation and verification images
  • Discuss simple in-toto software supply chains aka root layouts
    • ABC example
    • Datadog example
    • Collect CNAB uses cases users would like to see
    • CNAB use case: two steps
    • Image use case: also two steps
    • Jacob & Radu would like to support out-of-the-box signing for both bundles and images
    • Trishank: need to fix signy to verify images referred to in bundles, will be off by default, but people like Jacob can turn it on
    • Radu: signy can check signatures on bundles copied to a different registry, even with a different GUN, unlike Notary-v1
  • Please review:
Select a repo