# Compose Spec Community Meeting **Meeting time:** Every other Thursday 10:00 AM - 10:30 AM US Pacific Time **Zoom Link:** https://docker.zoom.us/j/970819001 **GitHub:** https://github.com/compose-spec **Group slack channel:** #compose-specification @ cloud-native.slack.com (Get an invite from: https://slack.cncf.io) **Past meeting minutes:** https://github.com/compose-spec/community ## Meeting Agenda and Notes ## NEXT MEETING TEMPLATE | | | | -------- | -------- | | Attendees | | | Note Taker | | ### Agenda * Welcome newcomers! * Any demos? * (Your name) What you'd like to discuss ## 2021-09-23 | | | | -------- | -------- | | Attendees | Lorena Rangel, Nicolas De Loof, Rachid Zarouali, Ulysses Souza, Steph Rifai | | Note Taker | Steph Rifai | ### Agenda * Welcome newcomers! * (ndeloof) welcome @ulyssesouza as maintainer [#200](https://github.com/compose-spec/compose-spec/pull/200) * (ndeloof) introduce top-level profiles [#201](https://github.com/compose-spec/compose-spec/issues/201) ### Notes * PR to make Ulysses a maintainer is merged! https://github.com/compose-spec/compose-spec/pull/200 * Discuss top-level profiles [#201](https://github.com/compose-spec/compose-spec/issues/201) * If you run without a profile compose should be able to detect this * Propose to have profile automatically enabled based on some conditions * we could add some environment variables to be set * Valid use case: working with a client that would like to have the ability to switch between two different services at once especially two different databases at once, so this would be really useful * You would have to switch the variable to which db you will run, but will be better than having to use different compose files * Prepare a proposal PR to describe the syntax and use cases. We can have a prototype to experiment with * Compose v2 release is not yet packaged with the docker CLI on Linux * We had a proposal to create a system package for the plugin but make it a required depenedency * Feedback: good idea, will be really helpful because users will get it automatically * Having it be a plugin is useful because you can still have the ability to get it separately if you want ## 2021-06-17 | | | | -------- | -------- | | Attendees | Mike Irwin, Eric Hripko, Chris Crone | | Note Taker | Chris Crone | ### Agenda * Welcome newcomers! * Any demos? * (Your name) What you'd like to discuss * (Eric) Release process for the spec https://github.com/compose-spec/compose-spec/pull/164 * Let's define maintainership * Release approval requirements ### Notes * (Eric) Release process for the spec https://github.com/compose-spec/compose-spec/pull/164 * Chris: Docker created the spec with some maintainers * We reached out to people we thought would be good on the spec * Lots of people helped us get things going but are not very active now * It's a good time to review maintainership on the project * Jacob: How many maintainers should we have? * Balance between spread of ideas and not too many people such that it doesn't move * Would Docker keep maintainer seats as docker compose v1 and v2 are references? * Eric: We don't have a lot of active people right now * We don't want a structure where we can't get quorum * We can resolve some of this with more async work * Vote by poll for things rather than in meetings * e.g.: The build secrets issue has been blocked for ages * Chris: Would prefer less more active people than too many * We can ask for help on issues we're not comfortable with * We could move this meeting to get more engagement if we need (based on where maintainers are) * Chris: Should we define what an active maintainer is? * Would be good to define this * Ensure maintainers at least vote on a certain percentage of issues * We need a minimum commitment of work on the spec for each maintainer * Chris: We have users on the spec and tool builders. Sometimes issues get stuck as tool builders don't have features on their roadmap * Jacob: Maybe having an issue threshold would help. If you want a major feature (to be defined), show a code example of it * compose-ref was created for this but it hasn't been used much. Maybe Compose v2 or another code base can be used instead ## 2021-06-03 | | | | -------- | -------- | | Attendees | Chris Crone, Pablo Chico de Guzman, Ramiro Berrelleza, Nicolas de Loof | | Note Taker | Chris | ### Agenda * Welcome newcomers! * Okteto Stacks demo: a Kubernetes backend implementation of the Compose Spec * (Pablo Chico de Guzman) Okteto Stacks demo and feedback about uses cases for running Compose on Kubernetes. Share our learnings while implementing Okteto Stacks and go over the main incompatibilities between the Compose Spec and Kubernetes: ingress rules and DNS compatible names. * (Eric) Release process for the spec https://github.com/compose-spec/compose-spec/pull/164 * Let's define maintainership * Release approval requirements ### Notes * Demo of Okteto * Okteto are building cloud developer tools * Using the Compose spec to map to a Kubernetes backend * Compose because it's what developers are using * Developers move to Kubernetes because of local resource constraints or requirements * Could be one part of the app in a cluster or the whole app * Interested in L7 support in the Compose spec * Ran into some spec issues * Network/Port mapping in Compose * Names with underscores * Demo of Okteto hosted developer environment ## 2021-04-22 | | | | -------- | -------- | | Attendees | Chris Crone, Marina Kvitnitsky, Michael Irwin, Eric, Anuj Sharma, Nicolas de Loof, Eric Hripko | | Note Taker | Chris | ### Agenda * Welcome newcomers! * Any demos? * (Your name) What you'd like to discuss * (Michael) Implementing compose-go in other languages * (Eric) Are we able to land https://github.com/compose-spec/compose-spec/pull/147? (once conflicts are resolved) * (Chris) [Versioning](https://github.com/compose-spec/compose-spec/issues/104) ### Notes * Intros * Anuj: AWS, working on Docker AWS integrations * Want to streamline deployment of Compose to ECS * (Michael) Implementing compose-go in other languages * Michael: Would like to have a Compose AWS CDK integration and that's where this idea started * Team uses Javascript so Typescript would be ideal * Would love a compose-ts like compose-go * Anuj: Something like this would be useful for handling corner cases not currently handled by Compose ECS integration. CDK is the right place to do this. * Chris: Plus one! We could add this to the compose-spec org but should make sure projects are maintained * Spec versioning would help here * Nicolas: Don't want to put in place barrier to entry to putting this in compose-spec * Would prefer to encourage people to contribute to the spec directly * Eric: What does this mean for Python docker-compose? * Should this be converted to an SDK * Michael: Will be starting this in private namespace and then move it to the compose-spec org * Will collaborate with Anuj * (Eric) Are we able to land https://github.com/compose-spec/compose-spec/pull/147? (once conflicts are resolved) * Eric: We were in agreement last meeting but this is missing one LGTM * Chris: Found where the compose-spec.io site code is, will work on moving it to the repo * (Chris) [Versioning](https://github.com/compose-spec/compose-spec/issues/104) * Chris: Presented proposal * Eric: Having a changelog will be really useful * Nicolas: We have automation for changelogs in the docker-compose release process * Agreement that this is OK ## 2021-04-08 | | | | -------- | -------- | | Attendees | Mike Irwin, Chris Crone, Nicolas de Loof, Marina, Eric, Jacob | | Note Taker | Chris | ### Agenda * Welcome newcomers! * Any demos? * Better documentation for Compose spec, see https://github.com/compose-spec/compose-spec/issues/115 * Native system service wrapper for compose apps * (Chris) [Spec versioning for tool makers](https://github.com/compose-spec/compose-spec/issues/104) ### Notes * Eric [Better documentation for Compose spec](https://github.com/compose-spec/compose-spec/issues/115) * Based on issues from collegues and external issues * Often heated arguments * Docker docs are quite prominent but not as up to date as the Compose spec docs * Causing a lot of confusion * Potential solution is to generate docs * Chris: Would be good to mirror this on docs.docker.com and elsewhere as well * Mike: Will be some drift between spec and tooling * Eric: Would be good to have a compatibility matrix for features so that users understand * Decision: Get this merged in the repo and then work out where to host, probably compose-spec.io * Mike: Would be good to have standarized documentation as this makes moving between tools easier * Nicolas: Could use badges on features to show which tools support them * Native system service wrapper for compose apps * Punted to next time as none of the attendees raised this * (Chris) [Spec versioning for tool makers](https://github.com/compose-spec/compose-spec/issues/104) * Chris: Would like to make it easier for tool makers to specify a point in time in the spec * Should not be exposed to users * Parsing libraries like compose-go could use semver, this should not * Proposal to use a date tag * Chris will propose a lightweight process for this * Eric: [Build secrets follow-up](https://github.com/compose-spec/compose-spec/issues/81#issuecomment-782254271) * Eric: Secrets would be useful * Chris: There might be a difference between build secrets and deployment secrets * Could cause confusion * Eric: Could be up to tools to decide where to get secrets * Chris will add this to the issue * TOML support * Chris: Think other formats are fine so long as they can represent the Compose model * Nicolas: The spec should have at least one canonical text format (e.g.: YAML in UTF-8) * Tools should be able to handle this and export to it * Jacob: What about literals * nerdctl is adding Compose support :tada: * Nicolas: Would be good to get Akihiro to do a demo ## 2021-03-11 | | | | -------- | -------- | | Attendees | | | Note Taker | | ### Agenda Skipped because of Docker Community All Hands. ### Notes ## 2021-02-25 | | | | -------- | -------- | | Attendees | Marina Kvitnitsky, Mike Irwin, Eric Hripko, Chris Crone, Peter Valdez | | Note Taker | Chris | ### Agenda * Welcome newcomers! * Any demos? * Build secrets follow-up, see https://github.com/compose-spec/compose-spec/issues/81#issuecomment-782254271 ### Notes * Build secrets follow-up, see https://github.com/compose-spec/compose-spec/issues/81#issuecomment-782254271 * Chris: BuildKit's syntax can handle all of these things but isn't very user friendly * Eric: Most of these are essentially host mounts * Eric: BuildKit is actually just a subset of mounts because it's per step not for the whole build * Eric: Start with file and ssh proposed in the initial issue description * Consensus is that sounds good * Chris: Will put this on the Slack chnanel for a vote * Chris: Kubernetes mapping? * Peter: Want to setup a meeting for this next week, will announce in the channel ## 2021-02-11 | | | | -------- | -------- | | Attendees | Chris Crone, Ben Gotch, Nicolas de Loof, Jacob Howard, Peter Valdez, Marina Kvitnitsky, Eric Hripko | | Note Taker | Chris | ### Agenda * Welcome newcomers! * Any demos? * (Jacob) Compose with cloud-based Docker Engine ([slides](https://docs.google.com/presentation/d/18ebn5M2NYycKuQ24gWJ4NOoNu-RRHi4G4Q_BGWcj8b4/edit?usp=sharing)) * (Eric) [Build-time secrets](https://github.com/compose-spec/compose-spec/issues/81) * SSH * File-based ### Notes * (Jacob) Compose with cloud-based Docker Engine * Watch the video, it's a great demo! * [Slides](https://docs.google.com/presentation/d/18ebn5M2NYycKuQ24gWJ4NOoNu-RRHi4G4Q_BGWcj8b4/edit?usp=sharing) * Peter: What's the main use case? * More computing power potentially, for very large applications * M1 -> amd64 native development * Colocation of cloud components with dev containers * (Eric) Build-time secrets (see https://github.com/compose-spec/compose-spec/issues/81) * Issue is getting lots of traction, equal interest in SSH and file-based * Ensure that we can do this in at least two places: locally and somewhere else * Eric has enough direction and will draft a solution ## 2021-01-28 | | | | -------- | -------- | | Attendees | Michael Irwin, Nicolas de Loof, Marina Kvitnitsky, Peter Valdez, Chris Crone | | Note Taker | Chris Crone | ### Agenda * Welcome newcomers! * Any demos? * (Eric) Build-time secrets (see https://github.com/compose-spec/compose-spec/issues/81) * SSH * File-based * (Nicolas) align Pull_policy valus with docker CLI (see https://github.com/compose-spec/compose-spec/pull/136) * (Chris) Compose -> Kubernetes mapping ### Notes * Welcome newcomers! * Nicolas: Engineer at Docker on the Compose CLI * Michael: Virginia Tech, using Compose with Helm charts * Marina: PM at Docker * Demos! * Jacob promised to have one next time * (Eric) Build-time secrets (see https://github.com/compose-spec/compose-spec/issues/81) * Bumped to next time when Eric's here :) * (Nicolas) align Pull_policy valus with docker CLI (see https://github.com/compose-spec/compose-spec/pull/136) * Currently the Docker CLI and Compose spec have different values * Proposal is to align Compose spec with the Docker CLI * Michael: Kubernetes differs from Docker * Not sure this is important though * Nicolas: Use aliases to support values already specified in the spec * No objection * (Chris) Compose -> Kubernetes mapping * Meeting was on Tuesday * Michael gives rundown of the meeting * Listed existing tools that used Compose with Kubernetes * Canonical place for discussions will be on `KUBERNETES_MAPPING.md` * Plan is to meet monthly * Peter's notes: * Secret handling * Extension fields can fill in gaps - e.g. x-ingress * Helm charts are tough to extend * k8s-to-compose - 10x more questions * Compose CLI - docker compose up --context kube * Take a look at the WIP branch mapping (feedback on compose-spec repo) - https://github.com/compose-spec/compose-spec/blob/wip_kube_mapping/KUBERNETES_MAPPING.md * Testing is tough - don’t want 20 min cluster startup time (kind may help) * Chris wrote this a long time ago: https://github.com/docker/compose-on-kubernetes/blob/master/docs/mapping.md * (Michael) [Add a dependsOn condition to wait for successful completion](https://github.com/compose-spec/compose-spec/pull/134) * Some minor changes but looks good ## 2020-12-17 | | | | -------- | -------- | | Attendees | Chris Crone, Nicolas de Loof, Peter Valdez, Michael Irwin, Jacob Howard | | Note Taker | Chris | ### Agenda * Welcome newcomers! * (Michael, Peter) Deploying a Compose file to Kubernetes * (Peter) An approach at handling local routing using an nginx/docker-gen based container (wrt https://github.com/compose-spec/compose-spec/issues/111) * (Chris) Sebastiaan has hesitations about [adding the project name](https://github.com/compose-spec/compose-spec/issues/106) * (Chris) Platform for build in the spec ### Notes * Michael and Peter demo deploying Compose on Kubernetes * Mike's demo * Comes from building a platform at VT * [See code here](https://github.com/mikesir87/helm-charts) * Want tooling that doesn't require developers be experts in the underlying orchestrator * Using Flux v1 * Find Compose the be the right abstraction level for this * Would like source of truth to be Compose file * Consumes Compose file as values for Helm chart * Allows using Kubernetes now with option to pivot to ECS or something else later * Need some extensions to Compose for things like ingress * Peter's demo * Working on a cloud platform with a similar goals * https://shipyard.build * Creates template Compose file based on languages developers want to use * Creates ephemeral environment for pull requests * Ask [Peter](mailto:peter@shipyard.build) for an invite if you'd like access! * Nicolas linked a [Kubernetes mapping doc](https://github.com/compose-spec/compose-spec/tree/wip_kube_mapping) so that we can get more aligned on this mapping * (Peter) Routing [issue](https://github.com/compose-spec/compose-spec/issues/111) * Need some kind of path based routing * Currently using labels * https://github.com/shipyardbuild/compose-ingress * Nicolas worried about adding extensions that are too platform specific * Goal of custom extensions is a playground for things that can be pulled into the spec later * Want us to be able to run things locally and very platform specific extensions will break this * (Chris) [Project name in the spec](https://github.com/compose-spec/compose-spec/issues/106) * Lots of debate about pros and cons * Nicolas didn't have a strong opinion originally but now against this * Seems like main use case is for deploying the same app multiple times * Could solve this in tooling: flags, arguments, etc. * Nicolas will add a comment on the issue * (Chris) Platform for build in the spec * Apple's new Macs have made this particularly necessary * Would be good to align Compose better with what's in the Docker CLI for build * Only where it makes sense for others * Multiplatform is obvious ## 2020-11-05 | | | | -------- | -------- | | Attendees | Nicolas de Loof, Roman Anasal, Jacob Howard, Chris Crone, Justin Cormack, Eric Hripko | | Note Taker | Chris | ### Agenda * Welcome newcomers! * Profiles PR (https://github.com/compose-spec/compose-spec/pull/110) * (Nicolas) Introduce routing (ingress and more) (https://github.com/compose-spec/compose-spec/issues/111) * (Chris) [Versioning of spec](https://github.com/compose-spec/compose-spec/issues/104) ### Notes * [Profiles PR](https://github.com/compose-spec/compose-spec/pull/110) * Thanks to Roman for adding a clearer PR for this feature! * Nicolas doesn't see a blocker with current implementation * Justin: Would profiles be a good way to handle things like changing service port or a single setting like TLS or should we keep it simple for now? * Nicolas: Networking has lots of local vs prod concerns that we need to handle. Substition of service is a good first stab that solves lots of problems * Chris: Like that this is simple to start and that we can iterate on it * Nicolas: Profiles approach has lots of possibility as it's defined * Jacob: How does disabling and enabling services handle things like volumes? * Nicolas: We can leave this up to tool providers * Jacob: How would this interact with depends_on? * Nicolas: Profiles are like a preprocessor; they generate a valid Compose file * [Introduce routing (ingress and more)](https://github.com/compose-spec/compose-spec/issues/111) * Original issue popped up on the Compose CLI repo asking for ingress * Lots of possibilities here with things like service meshes * Good starting point might be path based routing * Intention is to get more eyes on this proposal * Chris: Path based routing would also enable things like lambdas (FaaS) * Nicolas: We need to be sure that we have local development solutions for this functionality. A lot of this functionality is provider specific currently. * [Versioning of spec](https://github.com/compose-spec/compose-spec/issues/104) * Need a way for tool makers to pick up the spec and consume it * Justin: What users care about is just if a feature is supported by their tool * Chris: Imagined just having versioned schema * Jacob: We want to avoid the C++ versioning approach where there are years between feature releases * Might want to cut releases when tools support a given feature ## 2020-10-08 | | | | -------- | -------- | | Attendees | Chris Crone, Jacob Howard, Justin Cormack, Roman Anasal | | Note Taker | Chris Crone | ### Agenda * Welcome newcomers! * Project name in Compose file (https://github.com/compose-spec/compose-spec/issues/106) ### Notes * Welcome newcomers! * Anca, maintainer on docker-compose works at Docker * Project name in Compose file (https://github.com/compose-spec/compose-spec/issues/106) * tl;dr: Should project name be added to the Compose file * Justin: Very old issue * Chris: Possible problems with this: * Cannot up the same stack multiple times on the same machine * Risk of up replacing an existing stack * Risk of down doing something unexpected * Justin: Could use (reverse) FQDN for names as a workaround * Chris: What about random names or using the name in the Compose file as a seed for the full name? * Not backward compatible * Jusin: Other suggestion was to have this in a different file * Jacob: Current docker-compose adds label with path that could be used as a check * Jacob: When merging Compose files with project name, last one wins * Roman: Don't see downsides that we don't have already to adding this * Just another place that one can specify the name * Chris: I'll summarize discussion and add to issue tomorrow ## 2020-09-24 | | | | -------- | -------- | | Attendees | Chris Crone, Nicolas de Loof, Roman Anasal, Junlin Chen, Peter Valdez, Eric Hripko | | Note Taker | Chris | ### Agenda * Welcome newcomers! * Profiles proposal: https://github.com/compose-spec/compose-spec/pull/98 * [Chris] Cutting specification releases * [Chris] [Dedeprecate runtime field](https://github.com/compose-spec/compose-spec/issues/101) ### Notes * This meeting will now only take place when we have an agenda * Check in the Slack channel to make sure there is one * [Profiles proposal](https://github.com/compose-spec/compose-spec/pull/98) * Lots of interesting use cases for this * Proposal to move forward: Reduce scope to single file that can be used for local dev and then deployed to the cloud * Move other use cases to another issue/PR * Nicolas will action this * docker-compose 1.27.4 was just released :tada: * [docker/compose-cli](https://github.com/docker/compose-cli) was just open sourced * Also a [Node SDK](https://github.com/docker/node-sdk) * Cutting specification releases * Reported by Sebastiaan that for tool builders it's good to have a versioned release * Semver vs something like the date * No strong opinion * Would also be useful to have schema hosted somewhere * Could also investigate schema.org * [Dedeprecate runtime field](https://github.com/compose-spec/compose-spec/issues/101) * Brought up by Akihiro on the spec tracker * Goal thus far has been to keep the spec backward compatible * Chris: Not clear what the long term use case for this but agree about backward compat * Something like Docker Context could be used to do this for all services * Junlin shared his containerd Compose tool: * https://github.com/mc256/containerd-compose ## 2020-09-10 | | | | -------- | -------- | | Attendees | Chris Crone, Jacob Howard | | Note Taker | Chris | ### Agenda * Welcome newcomers! ### Notes * No topics and not many attendees so cancelled ## 2020-08-27 | | | | -------- | -------- | | Attendees | Nicolas de Loof, Guillaume Lours, Roman Anasal | | Note Taker | Nicolas de Loof | ### Agenda * Welcome newcomers! * Eric: [GPU support proposal](https://github.com/compose-spec/compose-spec/issues/74#issuecomment-670851982)! * Find a way to declare GPU requirement in a generic way * Use of `device_request` seems to offer enough flexibility * Sounds comparable to swarm's `generic_resources` to express device requirements * Eric will bootstrap a proposal to synthetize proposals and move forward * Nicolas: [Profiles proposal](https://github.com/compose-spec/compose-spec/issues/97) * Profiles proposal is comparable to maven/spring profile to group optional "services" by usage * Covers various scenarios, including "test/CI" with additional mock/test-runner containers * Spec does not yet define rules about multiple yaml files being merged, we can work on this as well to support `compose.yaml + compose-{profile}.yaml + compose-overrides.yaml`, the latter being used for local changes not to be shared with teammates. * Guillaume: ran tests for docker-compose 1.27-rc2 on docker/awesome-compose. ## 2020-08-13 | | | | -------- | -------- | | Attendees | Nicolas de Loof, Guillaume Lours, Chris Crone, David Killmon | | Note Taker | Chris | ### Agenda * Welcome newcomers! * (Nicolas) Proposal to support hierachical secrets (Kubernetes/AWS style) ([Issue](https://github.com/compose-spec/compose-spec/issues/93)) * Docker Compose now versionless! See the [latest RC](https://github.com/docker/compose/releases/tag/1.27.0-rc1) ### Notes * Welcome Guillaume! * (Nicolas) Proposal to support hierachical secrets (Kubernetes/AWS style) ([Issue](https://github.com/compose-spec/compose-spec/issues/93)) * While working on support for secrets on AWS ran into limitation of Compose: does not allow secret hierarchy * Docker ran into this with Compose on Kubernetes too so know this is broader issue than just AWS * Proposal for new attribute in the Compose format for secrets * Another possible issue is that the secret path is hard coded (`/run/secret/<secret name>`) * Proposal works for AWS * Release of Docker Desktop Edge next Tuesday will include this to get feedback * Docker Compose now versionless! See the [latest RC](https://github.com/docker/compose/releases/tag/1.27.0-rc1) * Download it and try it! * Feedback will be really useful while we have this out as an RC * Also drops Python 2 support * Guillaume will test against Awesome Compose * [GPU support](https://github.com/compose-spec/compose-spec/issues/74#issuecomment-670851982) * Will discuss more next time when Eric is here * It looks like a good proposal ## 2020-07-30 | | | | -------- | -------- | | Attendees | Chris Crone, Ben de St. Paer-Gotch, Eric, Peter Valdez, Jacob Howard | | Note Taker | Ben | ### Agenda * Welcome newcomers! * (Chris) docker-compose removing requirement for spec version ([PR](https://github.com/docker/compose/pull/7588)) * (Chris) Remove merge requirement for PRs to be up to date with main branch * Specifically related to https://github.com/compose-spec/compose-spec/pull/60 ### Notes * Ben will make sure the calendar invite is updated. * Welcome newcomers, Peter Valdez. * PR 7588 - removed the version requirement from Compose, the decision was to make Compose based on the features not the version of the spec. This is merged into docker-compose. * The change to no longer use version will be released as a beta to make sure Docker has not broken anyone * The next release of dockerCompose will also have dropped Python 2 as that is EOL * Currently on the Compose spec repo you cannot merge branches unless they are up to date with the main branch * No strong opinion on whether we have this in place * Eric: With V2/V3 merge, who can we talk to about the Compose spec changes in Docker-Compose * Ping Chris Crone on this and he will review * Peter: Things that are merged into Compose spec and how that moves into the Docker-Compose CLI * Making a best effort to move these into the Compose CLI * Have released Compose support for ACI and ECS in latest Docker Desktop * So have some Compose Go started and want to release this in the next ~6 months of so * K8s mapping, not sure how we start doing this. * Ben will kick off a thread in the community slack channels to start discussion on this going forward ## 2020-07-16 | | | | -------- | -------- | | Attendees | Chris Crone, Ben de St. Paer-Gotch, Ian | | Note Taker | Chris Crone | ### Agenda * Welcome newcomers! ### Notes * Meeting cancelled as we didn't have quorum, agenda moved to next meeting ## 2020-07-02 | | | | -------- | -------- | | Attendees | Nicolas de Loof, Jacob Howard, Jeremy Kreutzbender, David Killmon, Chris Crone, Eric Hripko | | Note Taker | Chris Crone | ### Agenda * Welcome newcomers! * Reviews for [Specify the behaviour of extends field](https://github.com/compose-spec/compose-spec/pull/86) ### Notes * Nicolas: Eric's [PR](https://github.com/compose-spec/compose-spec/pull/86) * Work on readding v2 fields to v3 of spec * Specifically this PR adds `extends` fields * Aim is to include this in compose-go so that it can be used elsewhere * How should we handle services that reference other services? * Eric: Could block or allow this * Nicolas: Could allow implementers to choose * Feel we should be conservative here * Might need real use cases to make a decision on this * Eric: Agree that restricting this would be good and will update PR * Nicolas: docker-compose updates * Anca (at Docker) is working on updating docker-compose to match changes to the spec * Removing requirement on `version` field * Chris: We released ACI support in the Docker CLI (in beta) this week * It's included in Docker Desktop Edge or for Linux [here](https://github.com/docker/aci-integration-beta) * We're using the updated Compose spec and compose-go at Docker! * Run through open PRs that need review: * [Add NOTICE file](https://github.com/compose-spec/compose-spec/pull/85) * Nicolas: Next issues that need discussion or work * [Add option to specify services to be started by default](https://github.com/compose-spec/compose-spec/issues/87) * Multiple options for how to implement this * Appears that _services_ abstraction is too restrictive for some use cases * Another example is for jobs * Jobs * Eric: Are these a developer use case? * Chris: Maybe, we need to dig into the actual use cases * David: Would be useful as people drive workloads to the cloud * Connect your job to a network with long running services would be useful * Nicolas: Compose extension mechanim will allow people who write tools to propose things and then we can make a decision * Jeremy: [Compose issue](https://github.com/docker/compose/issues/7539) ## 2020-06-18 | | | | -------- | -------- | | Attendees | Nicolas De Loof, Ben De St Paer-Gotch, Jacob Howard, Till Wegmueller, Gustavo, Eric | | Note Taker | Ben | ### Agenda * Welcome newcomers! * Nicolas - overview to date * Open forum ### Notes * Nicolas: Given an overview of what has been done in the Compose Specification so far * Want to re-introduce all the Compose V2 attributes * Drop the requirement for version, so what you put in the file will be checked against the platform & your tool will warn you * Want to avoid people thinking V2 is for Local and V3 is for cloud - want the Compose file format to be universal * Nicolas: In terms of implementation * Docker extracted the Compose Go library into a dedicated repo in the Compose spec * Merged a PR to provide a model 'project' that will be the result of agregating multiple compose files * Has a layer to check compatibility * Nicolas: Interesting proposal, sent on the Docker Compose repo but the idea is generich https://github.com/docker/compose/issues/7539 * Acknowledge that a Compose file may have 'other' containers that are used as one off tasks * Run debugs * Run tests * Proposal is to allow specific syntax in the Compose file so you can define these containers in your compose file but won't be started by default when you do a Compose Up * Interesting topic to discuss * Nicolas" One of the challenges we have today is to make keep Docker Compose compliant with the Compose specification * Want as least that Compose spec compliant files will work with Docker Compose * Till - Adding tasks to Compose, there was a similar discussion for Docker images in general. * You have a very limited amount of things you can specify an image can do * Would this be something to take up with OCI people to support in the image, to support some tasks in the image/app container to do with DB * Nicolas - the issue at the image level is that this becomes an OCI runtime discussion, lack understanding of how we would move this into OCI * If we want to address at the Compose level, we would need a more declartive approach. The hook approach from K8s may make sense * Till: also taking part in OCI meetings which is why this option is something to consider * Eric: reminded the talk 'labels labels labels' from Dockercon where you could store a K8s manifest in a Docker label. Which is a gross over use but gives an example * Eric: for lifecycle it could be interesting to see what lifecycle events you might define, if you could do something before main service has started it may not be useful. Then if you are trying to check if your main service has started that would be a lot of overhead for developers * Eric: compose tools tend to be file/forget and don't have a daemon. Don't have interaction with the lifecycle. * Till: Previous trick, when NGinx starts up and needs lifecycle things, this needs to be handled in the startup script. Define specific outcomes in that startup script * Nicolas: need to dig in to understand the usecase/benefit that this could provide and make sure it is implementation agnostic * Eric: Discuss V2 story, is extends the last bit that is missing from V2 before this is closed out? * Nicolas: Extends is one we need to define clearly, from the Docker Compose code there are no complications in how this is implemented * Eric: how can we help * Nicolas: propose in the spec how we could do the extend in the Compose specification ## 2020-06-04 | | | | -------- | -------- | | Attendees | Nicolas de Loof, Chris Crone, David Killmon, Jacob Howard, Ben De St Paer-Gotch, Eric | | Note Taker | Chris Crone | ### Agenda * Welcome newcomers! * (Nicolas) Requirement of version field in the spec (https://github.com/compose-spec/compose-spec/issues/13) ### Notes * Nicolas: Requirement of having version field in the spec * https://github.com/compose-spec/compose-spec/issues/13 * Now that spec is open, we are more of a "loose" specification, we don't want to force all implementers to support all features * Feel that it would make more sense to do compatibility per feature rather than with a specified version * Proposal is to drop version field * PR for pull v2 features into spec is up and needs reviews * https://github.com/compose-spec/compose-spec/pull/78 * Dropping the version field will remove confusion between v2 and v3 * Eric: Possibly something like "use strict" for Javascript? But in YAML ;) * Eric: Agree with everything above * Think that strict mode would be interesting to add * Versions don't make sense * Chris: Also agree with this * Nicolas: Put up PR on compose-go that allows whitelisting which features your implementation supports: * https://github.com/compose-spec/compose-go/pull/45 * Chris: would like to see an auto generated version of this * David: I do wonder though - the underlying implementations from the various cloud providers might change or our implementations might have bugs that people take dependencies on. I wonder how we might do that without versions, ya know? * Nicolas: version in Compose file is of the specification * If we adopt this we would not be able to remove features * Tools will need to decide what features they support and what the feature means for their backend * Can still mark features as deprecated (leave them in the schema) and tools can reject the feature * Example of something that could be deprecated: scale in v2 has been replaced by deployment.replicas in v3 * scale should still be part of our spec but should be translated by implementations into deployment.replicas * David: would we also warn on fields we don't know? e.g.: we add a new field like "dogs" will old versions of implementations warn or err on this? * Nicolas: Can offer both * Nicolas: No strong opinion against the removal of the version field on the public issue * Primary feedback is that people are surprised that the Compose spec is looser than most others * Chris: Follow up on question last time: Where are the videos going? * Ben is currently owner of the meeting * Needs to get access to either the Docker YouTube or create a separate Compose account * Ben took an action to get this done for next time * Nicolas will update the version issue ## 2020-05-21 | | | | -------- | -------- | | Attendees | Justin Cormack, Jacob Howard | | Note Taker | Justin Cormack | ### Agenda * Welcome newcomers! * (Chris Crone) Compose spec and GPUs (https://github.com/compose-spec/compose-spec/issues/74) * Dropping version requirement (https://github.com/compose-spec/compose-spec/issues/13) ### Notes * GPUs. Declarative vs non declarative APIs. Declarative API is what pretty much everything except Docker engine supports. Look at existing practise, eg CUDA (device caps etc), OpenCL, cloud providers, Nvidia-docker. Are we using generic attributes or vendor specific? Helpful to do an overview of existing APIs. More generic than the Docker "just use this device" model. See also Microsoft post on WSL2 https://devblogs.microsoft.com/directx/directx-heart-linux/. Could use CUDA and opencl but now there are so many frameworks. Does it make sense to use these as entry points? * Compose Trademark. AFAIK not a trademark. Anyone should be able to implement and call it Compose X. * Where are recordings going? ## 2020-05-07 | | | | -------- | -------- | | Attendees | Ben De St Paer-Gotch, Nicolas de Loof, Chris Crone, David Giffin, Vanessa Sochat, Eric Hripko, Alex Hicks, Erik Landerholm, Jacob Howard, Jeremy Kreutzbender, Laszlo Fogas, Till Wegmueller | | Note Taker | Ben De St Paer-Gotch | ### Agenda * Welcome newcomers! * Scope of Compose - feedback to be added in Github about https://github.com/compose-spec/compose-spec/blob/master/VISION.md * V2 support: https://github.com/compose-spec/compose-spec/issues/12 * provider-specific extensions https://github.com/compose-spec/compose-spec/issues/17 to drive spec changes * (Chris Crone) Format of this meeting feedback: Long enough? Anything we can do better? ### Notes * Nicolas de Loof * Welcome, overview of last meeting * First item - establish scope of Compose spec. We don't want Compose file format to become another KubeYaml 'thing' with everything inside it, increasing the complexity. We want to keep Compose simple and still covering the use cases required for production deployment * Ben: Wrote initial vision doc. Based off feedback from users that often the first step for Compose is as a tool for scripting run of single container. Useful as it can then be used to string containers together and run them in production. * Vanessa Sochat - a lot of researchers who are creating simple applications locally that may never make the jump to production * Tim W * - when developing locally and pushing it into a cluster, we have a lot of low level concepts like C-groups/types. These differ between a local computer and a high power cluster - you won't be able to make that work smoothly but it is a good idea to have it in at least to define it in 2 different places * Nicolas D * - we want to cover most of the interesting use cases, so we need to cover a single host but also being able to target a cluster. Most of the current Docker options today don't work on Windows, it is the same with Docker options targetting a cluster which is why we split Compose between V2 and V3. As such the vision in this area is - we should not block on things that could be applied to a significant number of use cases. We need to consider the Compose spec as a way to express a container based application model with some options for common parterns that are portable. * Nicolas D * - the vision has already been split between core & deploy, we should probably change delpoy to be for more platforms. We should open a PR against the vision to make this clearer. We are trying to create a common language for how we represent containers, when people are using compose it is ok for some platforms to not support some features of the platform. This shouldn't be limited because 'you aren't' using version 3, but it should be based on what features your platform supports. * Till W * - are we looking to generalize some of the options? * Nicolas - * Looking at an example of linux C groups, if we have a way to express that this attribute is here and how a device can be used by your container. * Chris C * A good test would be could this feature run on multiple platforms. A good example is adding GPU, then we could schedule workloads for GPU and it could be supported on multiple platforms. * Nicolas * For legacy reason, GPU was implemented as switching the runtime. But end users doesn't want to know about runtimes and just wants the container to have access to the device. These are the sorts of changes we could make. * Eric * Where does CI fall with the vision? * Nicolas * CI is a typical use case for Compose. * Could Eric open up an issue on the Compose spec to itterate on this * One to clarify with everyone here, V2 support * Work so far - listed fields that were removed between V2/V3. Already went into discussion that some of them are still valid, a few have similar commands in deploy section. There are a few use cases that we may not want to support * The idea is to enable a new set of tools that are not specifically tied to the version set in the Compose file, as users are just running locally or deploying to 'something' They just use Compose. * We need to make sure that tools read a compose file and see an attribute they don't support, that they just warn the user rather than refuse the file. This needs to be part of Compose Go so it is not too hard for people to implement * Add support for extension attributes - you can already add some custom extensions to your compose file, it is enabled at the top level within your services or networks. Wanted to suggest that we introduce support, as a best practice, of extensions everywhere in the Compose file. And ask that people can use vendor specific extensions, so users can experiment with the spec and make some interesting use cases. * Chris * We will still need to have reasonable constraints on what is accepted into the spec as an extension * Nicolas * We could perhaps limit it to needing to be implemented/supported by at least 2 major providers and be able to mock it/use it locally * Is the meeting long enough/is the format good? * Till * Could be longer? Perhaps 1 hour? * Chris * Once the spec stabilizes we can probably extend it ## 2020-04-23 | | | | -------- | -------- | | Attendees | Chris Crone, Justin Cormack, Ben Firshman, Nicolas de Loof, Darragh, Silvin Lubecki, Alex Hicks, David Giffin, Jacob Howard, Jeremy Kreutzbender, Jochen Zehnder, Ben de St. Paer-Gotch, Till Wegmueller | | Note Taker | Chris Crone | ### Agenda * Welcome newcomers! * Introductions ### Notes * Welcome! * Introductions * Justin Cormack: Docker * Excited to finally have the specification out as an open spec * Want to find out how people are using Compose and make it no longer just Docker specific * Nicolas de Loof: Docker * Work on Compose and related things * Chris Crone: Docker * Engineering manager at Docker responsible for Compose * Ben de St Paer-Gotch: Docker * Product manager on Compose * Ben Firshman: ex-Docker * Co-creator of Compose * Here because he has historical knowledge of Compose specification * Very happy to see this happen, intention of Fig (original name) was to build something like this * David Giffin: Release * Using Compose for getting workloads to Kubernetes * Want Compose to be more production * Silvin: Docker * Work on Docker App which consumes Docker Compose * Till Wegmueller: OpenFlowLabs and illumos Community * Using Compose and Kubernetes on illumos * Implementing OCI compliant runtime and support tools Aurora OpenCloud https://git.wegmueller.it/opencloud/opencloud * Jeremey: Release * Here to learn more and see how specification progresses * Jacob Howard: Mutagen * Interested in the spec and bringing it to the cloud * Darragh: 56k Cloud * Big users of Compose for local development * Experienced in using ECS CLI Compose tools * Jochen Zehnder: 56k Cloud * Alex Hicks: PhD student * Using Docker Compose for projects * Taught computational history course and included containerisation * Djordje Lukic: Docker * Compose, Docker App * Excited about the spec and feedback on it! * JC: There are a bunch of open issues on the specification containing things that we think we should change * We'd love your feedback on them and help in evolving them * Particularly around reconciling v2 and v3 * JC: Feel that people should not need to put version number at the top of the Compose file * Darragh: Being able to pin on a version is a good thing for projects that don't evolve quickly * Backward compatibility is important * DG: Interested in extends and combining Compose files * JC: Don't remember why extends was removed * BF: Think this was with move to Stacks and Swarm * CC: Spec is rough because we wanted to share it early and genuinely engage with the community * JC: If you have issues, please add them to the repo * JC: Would like a working group on Kubernetes mappings * DG: Interested in Kubernetes mapping * JC: Can create an epic to track issues associated with this * DG: Mounts and ports are particularly tricky * JC: Networking as a whole is tricky as nothing maps that well to the Docker model * TW: Worried that Compose spec becomes too complicated if we don't set scope and goals clearly * CC: Vision document covers some of this: https://github.com/compose-spec/compose-spec/blob/master/VISION.md * CC: Future of Compose livestream link: https://www.youtube.com/watch?v=JPpyDfPyuJ0