Try   HackMD

Porter Community Meeting Minutes

Bi-weekly call with the Porter maintainers, contributors and community.

Future Topics

Got something that you want to talk about?
Add it here

  • https://cloud-native.slack.com/archives/CN8NA4F8V/p1726695535147919
  • Proposal: Write a bundle once and deploy on many clouds
    • Bundles can declare that they "provide" a resource
    • Users either configure their environment with the desired providers, or can specify it with provider azure|google|amazon|
    • Register implementations in an OCI registry for discoverability
  • Documentation
    • Currently the website is published on merge to main, the result is that the documentation can be for the "main" build instead of the current stable version
  • Discuss current WIP implementation:
    • SQL storage support, beginning with PostgeSQL #3236

Feb 20th, 2025

Attendees

  • Sarah Christoff
  • Kim
  • Steven
  • Brian

Agenda

  • (Sarah) We are getting more PRs from our friends at Microsoft, and it's really cool. Let's review them. Reviewed this PR
  • (Sarah) What's going on with the Operator - should we archive it? How can we get more traction with it?
    • (Steven) Let's ask around and see if we can get more help with it
    • (Sarah) Let's be clear in the README that this is not production ready and where we need help
  • (Sarah) SBOM support is important, we should get that done. Call to Action?
  • (Steven/Kim) How to update the Mongo plugin?
    • It's hard to tell what version is actually being used
    • We need to get this upgraded though
    • Let's provide documentation on the upgrade path
  • (All) For KubeCon EU, let's focus on:
    • Modern Mongo (Cut over)
    • SBOM support
    • Terraform or Atmos Mixing Completion
  • (Steven) Let's finish the Terraform mixin PR
    • We could write a Atmos mixin instead of a terraform mixin PR with specifying different folders
    • (Brian) What if we had something for Dockerless Porter
      • (B) I have POC'd something out

October 31, 2024

Attendees

  • Sarah Christoff
  • Joel
  • Kurt
  • Arun
  • Shivani
  • Steven

Agenda

  • (Joel) There's a PR open on cnabio, it updates things that haven't been updated in over a year. Sarah to get merge capabilities to [cnabio](Bumping github.com/docker/docker version to 27.1.3 to fix the package security vulnerability by jotoude · Pull Request #307 · cnabio/cnab-go)

October 3, 2024

Attendees

  • Kim Christensen
  • Sarah Christoff
  • Joel
  • Kurt
  • Arun
  • Shivani

Agenda

  • (Joel) has been working on adding tracing into mixins

September 19, 2024

Attendees

  • Steven Gettys
  • Kim Christensen
  • Kurt Schenk
  • Joel
  • Raj
  • Arun
  • Craig
  • Shivani
  • Amos Ehiguese

Agenda

  • Welcome Amos!
  • Need a minor release of porter to address some dependency vulnerabilities
    • Azure plugin for sure needs a new release
    • Inventory plugins and mixins to determine if we need to cut new releases for vulnerabilities
    • Get fix release out by end of next week
  • ARM mixin has some updates, PRs incoming for it
  • Follow up on 1857 with Joel for design review
  • Terraform mixin multiple plans
    • Code is working, need to do docs update then PR will be coming shortly
    • Tofu mixin needs the same update
  • Asynchronous mixins design
    • Should be caller of the bundle handle mixin status?
    • Why is the helm status part of the helm mixin? We should take a look at the helm mixin to make sure it's doing best practices
  • Follow up on where we are for Kubecon
    • Minor release of Porter
    • Operator functionality complete
    • OpenTofu mixin complete

September 5, 2024

Attendees

  • Sarah Christoff
  • Steven Gettys
  • Troy Connor
  • Joel
  • David
  • Raj
  • Shivani

Agenda

  • (Joel) Running Porter in a container in AKS
    • Some of the packages in Porter are being flagged by S360
    • Can we update the dependencies that have the CVE (https://github.com/PuerkitoBio/goquery and https://pkg.go.dev/golang.org/x/net)
    • Is there an update needed in cnab-go as well?
    • Sarah to repro security scan issue
    • go.sum causing the issue by being out of date? Try regenerating go.sum
      • Get PR up with fresh go.sum
  • Issue 1857 needs to have more description so it can be started
    • Joel sync with Kim, might already be in progress

August 8th, 2024

Attendees

  • Sarah Christoff
  • Kim Christensen
  • Brian DeGeeter
  • Ian Howell
  • Joel

Agenda

  • (Joel) Issue 1857, could we pass in the context via the mixinconfig and then have that be used as a logger during runtime?
    • Brian, Kim and Sarah to add addtl context to this issue

July 25th, 2024

Attendees
Steven Gettys
Ayan Sengupta
Joel
Kurt Schnek
Troy Connor
Shivani

Agenda

  • Tofu mixin initial commit
  • Make sure there's an issue cut for managed identities with storage plugin
    • (TODO gettys) look into implementing managed identity authentication for the mongo stroage plugin
      • Azure CosmosDB doesn't currently support managed identity authentication, is coming soon
  • Operator V1.1 is cut
  • Mixin tracing (https://github.com/getporter/porter/issues/1857)
    • Should we pass the OTEL ctx down to the mixins? Or should they get their own context
    • Follow up with Sarah on the issue
  • Mixins running inside of Microsoft using managed identities/oauth flow
    • How do long running deployments handle when tokens expire mid deployment?
    • Could provide JIT secret resolution between steps in an action and update mid-flight?
      • Need to double-check how secrets are resolved between action steps now.
      • Could add a refresh of secrets before kicking off the next step.
      • Lots of edge cases/race conditions to consider
    • How do we mirror the azure cli flow for a user installing a bundle?
      • mount the ~/.azure as a credential?

July 11th, 2024

Attendees

  • Sarah Christoff
  • David Gannon
  • Kurt Schnek
  • Ayan Sengupta
  • Steven Gettys
  • Troy Connor

Agenda

  • Updating MongoDB in MongoDB Docker Storage Plugin
    • What to do about data migration?
      • Add a storage db migrate command?
      • (Discussion) We could have pinned versions that update slowly or a feature that rolls through a migration up to Mongo v7
  • Support for managed identity https://github.com/getporter/porter/issues/939
  • Mixin documentation on porter.sh
    • How would we like to update it? Is not automatically updated at the moment.
  • Where to store OCI images for example documents (ticket)
    • Maybe just another github project containing them all, and pipelines?
      • Wherever they need to go, can someone cool create the space and I can crack on :)
    • Should we migrate the carolynvs/* images at the same time?
  • How to progress the mixin version work (ticket)
    • (Discussion) Add this to v1.2 milestone
  • Decision on OpenTofu mixin (ticket)
    • Do we simply fork existing terraform mixin? If yes, can someone cool create a space for this please. (YES)
      • (TODO gettys) make the opentofu mixin repo from the terraform
      • Mixin publishing should just "work"?
        • Kim to lookinto how to publish
    • Do we try and make the existing mixin build both? It will be much harder, but might make it easier to keep them in line? (NO)
  • Operator GRPC server discussion
    • How do we handle PorterConfig for the GRPC server?
  • Terraform mixin update
    • Check if the mixin config is part of the tmp CNAB directory

June 27th, 2024

Attendees

  • Sarah Christoff
  • Troy Connor
  • Kim Christensen
  • David Gannon
  • Kurt Schnek
  • Ayan Sengupta

Agenda

  • Porter storage plugins
    • Updating mongo version to support latest?
    • How do we add in other storage backend types?
      • (Kurt) Looking to see if making a cosmosdb plugin or waiting for cosmosdb to update itself
      • (Troy) I know you mentioned managed identity, what about tokens?
      • (Sarah) what about adding an init or a pre-step that would get the credential (since we already do that) and then authenicates into cosmos/mongo
      • (Kurt) explore certificate based authenicate maybe?
  • Signing information for embedded images
    • Do we want to include signatures of embedded images in the bundle?
      • https://github.com/getporter/porter/issues/1457
      • Right now, the artif. and the invocation image is signed, but any images around the invocation image that the user pulls will not be signed
        • one path forward is signing pulling all the images down
        • we need to do research to see what other projs do
    • How do we find signatures?
      • Using the OCI Referrers API?
      • Based on the configured signing plugin?
    • (Troy) I know v1.1.0 is going out today, but maybe Operator later?
      • Yea, ofc
    • (Troy) Kubernetes Support Matrix?
      • n-2?
      • Right now testing on kind 1.27
    • (Sarah) Adding brew support
    • (Joel) We are trying to capture the logs from mixins

June 13th, 2024

Attendees

  • Sarah Christoff
  • Steven Gettys
  • Troy Connor
  • David Gannon
  • Kurt Schenk
  • Arun
  • Joel
  • Kim Christensen

Agenda

  • Porter operator lifecycle
    • Delete policy "delete/orphan"
      • TODO: There was a note here about K8s versions that I think I blatted by mistake. Sorry!
      • Add docs for managing porter managed resources created by the operator as a porter "admin"
      • Orphaned resources will not be removed from the porter state
    • Creating deployment at startup
      • Porter server image is not published as part of porter CI
      • Integrated server management only supporte
    • Managing deployment (configmap changes, when operator restarts, etc)
    • Correct porter agent/published image/porter agent action
  • What part of the porter.yaml is sent to mixins at "action" time?
    • Porter core should add in the mixin config for execution of the mixin actions
  • Porter mixin version field questions. (PR: https://github.com/getporter/porter/pull/3148)
    • Replicate terraform provider version functionality?
    • Add a porter mixin install -f porter.yaml?
    • Support multiple versions of the same mixin installed
    • What needs to happen for mixins at runtime? Nothing!
  • Is there a way to find out when https://github.com/google/go-containerregistry will release their next version? It's a blocker on a bug for v1.1.0.
    • Sarah will follow up on this

May 30th, 2024

Attendees

  • Steven Gettys
  • Kurt Schenk
  • Arun
  • David
  • Joel
  • Sarah Christoff
  • Neerja Ginotra

Agenda

  • Mixin versioning
    • Specify the version of the mixin in the porter.yaml
    • What should the behavior be? Pull in mixin version at build time or fail build if mixin isn't at correct version?
    • Are there any security concerns?
    • Schema change for the porter.yaml
  • Porter archive fails on windows Issue 2917
    • PR is still open, needs to be merged before this can be fixed.
    • Sarah will follow up
  • Operator
    • Porter server image should be built and published alongside Porter
    • Operator Deletion Policy is started, should be in place for Porter 1.1 release
    • GRPC outputs server is being added to the installer workflow
    • Outputs update will be investigated this weekend

May 16th, 2024

Attendees

  • Sarah Christoff
  • Kurt
  • Schenk
  • Neerja Ginotra
  • Joel Otoude
  • Arun
  • David
  • Troy Connor

Agenda

  • Kubecon - Workshops, CFPs, Panels?
  • [Kurt] - intermediate status - tracing execution of mixins and bundles

May 2nd, 2024

Attendees

  • Sarah Christoff
  • Brian DeGeeter
  • Kurt Schenk
  • Kim Christensen
  • Neerja Ginotra
  • Naveen
  • Erickson Moskito
  • Deepak Khetwal

Agenda

  • Operator Update - Quickstart is broken, we should get that updated. Troy to focus on updating dependencies. Troy to draft PEP for delete
    • Brian to look at the quickstart issue
  • Porter Lint - We should document that about adding linters, and want more "louder" errors at build time then at run time.
  • When should we error versus warn? - This should be a PEP or added to our contributor guide
  • Sarah to add monthly Operator call

April 4th, 2024

Attendees

  • Sarah Christoff
  • Steven Gettys
  • Troy Connor
  • Kurt Schenk
  • Kim Christensen
  • Neerja Ginotra
  • Jens Arnfast
  • Brian DeGeeter
  • Zel Foster
  • Navig

Agenda

  • v1.1.0 Planning (Slated for June)
    • We are looking for a release manager for this release, DM your Porter Maintainer team if you'd be interested in this
    • The project board for v1.1.0
    • Need help on getting operator testing resources, Sarah to reach out to CNCF to see if we can get clusters in Azure/GCP/AWS

March 21st, 2024

*Attendees

  • Steven Gettys
  • Kurt Schenk

Agenda

  • Porter docs

    • Rework is needed throughout the whole docs to make sure things flow and make since to where they're linked
      • Exercise the docs and capture any inconsistencies or things that don't make sense
    • Ensure consistent usage of aliases
      • Audit current docs and ensure we're using sane aliases
    • Investigate a way to correlate aliases to file location using Hugo
  • Terraform Mixin supports multiple working directories (https://github.com/getporter/terraform-mixin/issues/81)

    • Add a new mixin config called "WorkingDirs" that accepts a list of directories
      • At build time override the default WorkingDir with what's defined in the list
      • Follow up with a usecase example since this is a unique terraform implementation
  • Is there a default timeout for porter commands?

    • Steven will follow up
  • Does the porter kubernetes runtime driver support running porter outside of the cluster and specify the kubeconfig

Feb 8th, 2024

Attendees

  • Sarah Christoff
  • Steven Gettys
  • Shivam

Agenda

  • Setting up Shivam's local environment for development, came across an issue with Mage Install and an issue was made
  • Kubecon EU 6 minute deep dive, what does that look like?
    • Operator would be good to highlight
    • SBOMs are very hype, we should ship that
  • Let's plan an Operator meeting to go over issues and divvy up the work, see if we can get it more stable by kubecon

Agenda

January 10th, 2024

Attendees

  • Troy Connor
  • Sarah Christoff
  • Steven Gettys
  • Kurt Schenck
  • Salman Shah

Agenda

  • Porter mixin as bundles proposal (need PEP link)
  • Porter operator delete logic
  • Porter operator upgrade logic

November 30th, 2023

Attendees

  • Sarah Christoff
  • Steven Gettys

Agenda

  • Meeting time should be moved back 1 hour (Sarah to figure out how)

  • Operator bug where kubectl delete doesn't do an uninstall it just deletes the installation resource.

    • Previous behavior would run uninstall agent action and then put the installation CR into "installed false" state
    • Need to do a design on what our expected behavior
      • Shouldn't have an undeletable namespace because of porter CRs
      • Porter CR lifecycle management should adhere to Kubernetes lifecycle management
  • The operator needs a design document that fully defines behaviors

    • Should deleting a namespace cause installations to automatically uninstall?
    • What does upgrade look like?
    • Operator compatibility matrix for cloud managed clusters
  • Reach out to Crossplane and setup a maintainers meeting

  • Get some WASM bundles built, collaborate with Fermyon or MS

  • Cancel and communicate out no meetings in December

  • Get some useful bundles and reach out to community. Tech talks, podcasts, etc.

November 16nd, 2023

Attendees

Agenda

November 2nd, 2023

Attendees

  • Sarah Christoff
  • Allan Guwatudde
  • Steven Gettys
  • Mani Bindra
  • James Sevedge

Agenda

  • [Sarah] - Discuss add mount PR

  • [Troy] - What should we show at the KubeCon demo?

    • a kubectl apply, and the generation of the resources
    • "I can do a kubectl apply of a porter resource, and a kubectl get and see the status of that resource"
  • [James] - At F5 they had issues with devops tooling drift, and used Porter to stop the drift between tooling, it simplied the tooling stack (just need Docker + Porter). Created a Go CLI wrapper backed by Porter bundles

    • Keeping database in sync across pipeline jobs would be a good feature
    • Persisting state between CI jobs

October 19th, 2023

Attendees

  • Allan Guwatudde
  • Sarah Christoff
  • Troy Connor
  • [Name] - [Org]

Agenda
[Allan G] - Open telemetry in the operator
* The operator should pass through ambient opentel env vars to the porter agent so that we can collect trace data from it?
* This is passing through existing environment variables that the sysadmin may have set on the cluster. Generate a correlation id/user sets it on the cluster. The kubernetes job environment variables can be provided by setting the porter-env secret in the namespace that the Agent Action jobs are executed. Any values set in the porter-env secret will be added to the jobs environment using EnvFrom on the Agent Action job. Base64 encoded.
* made deploy created kind cluster runs out of resources? Mac problem?
* porter list command and relation to PORTER_HOME/config . Does command consider all namespaces when one is not defined in config file?

[Sarah C] - We should make docs better

  • Operator docs: namespace in contributor.md needs to be updated for porter.config
  • We should give example specs for local
  • For Operators/Admins: we shouldn't expect them to use Mage - it should just be "run porter commands"
  • Administrator vs Devleoper persona on the website to organize documents

October 5th, 2023

Attendees

  • [Name] - [Org]
  • Phill Gibson - MSFT
  • Allan Guwatudde

Agenda
[Phill] - KubeCon NA '23 activities
* What's everyone doing?

September 21st, 2023

Attendees

  • Sarah Christoff
  • Troy Connor
  • Mohit Bisht
  • Ludvig Liljenberg
  • Allan Guwatudde

Agenda

  • [Name] - Subject
  • Mohit Bisht - What are the key resources you recommend for someone new to Porter and the cncf ecosystem? Are there specific technologies or skills I should prioritize learning as a beginner?
    • Understanding Docker + containers (and having docker installed)
    • Kubernetes is really helpful to understand the Operator
    • A lot of CNCF is written in Go, so understanding Go
    • The CNAB spec is really important, check it out!
    • The CNCF App Delivery Tag is really good to understand the space we are in
  • Sarah - Operator v1.0 next week - what is left? What needs merged? What is the release process, and who wants to run it?
    • What is left?
      • Operator Bundle Upgrade Path
      • Documentation
    • Community Call next Thursday for release
    • Sarah to look at release process
  • Sarah - What is our triage process, timeline, should we add an auto-labeler?
    • Allan is interested in looking into autolabeler
    • What can Porter as a team* handle so we don't overhwelm ourselves?
    • Create an issue for triaging to source community feedback
    • Labeling good first issue and start labeling on how hard and easy they are
  • Sarah - Hacktoberfest is happening, how should can we get involved? Should we do a hacktoberfest tagging and triage session?
    • Let's do Hacktoberfest!
    • Let's do a big focus on docs updates and getting the website help :)
  • Allan Guwatudde - Updating operator within k8s context? Some errors faced in dev environment. New approach to stale bot. Slack threads.
    • The upgrade action for the porterop bundle should use the kubernetes mixin to update the deployment spec via kubectl to update the operator image - we can update the operator bundle to accept a operator image version param
    • For the errors installing porteroperator we will run through the operator kind setup
    • The links for integrations need to be updated
  • In Porter channel we should use threads!

Septemeber 7th, 2023

Attendees

  • [Name] - [Org]
  • Phill Gibson - MSFT
  • Sumit Kumar Soni
  • Sarah Christoff

Agenda

  • [Name] - Subject
  • Sarah - Moving website to it's own repository
    • (Decision) Move it into it's own repo! :)
  • Phill - Moving to Google docs follow-up discussion
    • (todo) github issue to make managed porter state for ado

August 24, 2023

Attendees

  • Anders Lybecker
  • Phill Gibson
  • Ludvig Liljenberg
  • Troy Connor

Agenda

  • [Name] - Subject
  • Anders Lybecker - re-use existing installation to satisfy dependency
  • Phill G - Docs repo discussion
  • Phill G - Should we move our notes site to a Google doc
  • Troy Connor - Status of the WIP installation outputs PR https://github.com/getporter/operator/pull/245

Notes

  • Problem: Sharing things between bundles e.g Mysql installation Bundle A has it, Bundle B needs to use it instead of installing a new one.
    • Flag --weak-dependencies was mentioned as a minimal solution before PEP Advanced dependencies becomes a thing
    • Creating a GitHub issue around this
  • Docs dedicated website?
    • Should we create a seperate repo to just host the website? This makes it easier for just document maintainers to push updates and lessens any incident of compromising project code base.
  • Google Doc?
    • Most CNCF projects are using Google docs and tools (calendar, groups, etc). Would it be good for the community to work with more OSS project friendly tools?
  • Status of WIP InstallationOutputs PR
    • Need to create a helm install to deploy operator with grpc service
    • Need to see updates in bundle with new outputs
    • Need to add this to porter install porterops

August 10, 2023

Attendees

  • ADD YOUR NAME HERE
  • Troy Connor
  • Phill Gibson
  • Sarah Christoff
  • Steven Gettys
  • Brian DeGetter
  • Allan Guwatudde

Agenda

  • [Troy] Demonstration of the progress with grpc/installationoutputs creation
    • Integration Test Scenarios we should add:
      • Doing an upgrade and the output changes
      • needs a new bundle to test this
  • [Sarah/Phill] Docs
    • [Brian] Mixins should be renamed to Extensions
      • PEP or GH issue/PR to establish Porter Glossary to discuss naming
      • Leave plugins as plugins
    • Users of Porter "As a bundle author" and "as a bundle operator" docs need to be front of mind
    • [Phill] Can we create an issue for deliverable for Kubecon
      • Let's tag issues with a kubecon2023 label!
    • For introducing people to the porter opertaor we should keep the operator bundle

July 27, 2023

Attendees

  • ADD YOUR NAME HERE
  • Sarah Christoff
  • Ludvig Liljenberg
  • Troy Connor

Agenda

  • [Troy] Discuss progress with grpc/installation outputs feature.

July 13, 2023

Attendees

  • Troy Connor
  • Phill Gibson
  • Steven Gettys
  • Sarah Christoff
  • Brian DeGetter

Agenda

  • [Troy] Discussing the approach using gRPC for expose outputs
    • We were running an agent action that run the CLI command, and parsing out the kube log file for infomration
    • Ran into issues implementing this into a CRD
      • Fields had trouble dynamically updating in the CRD
  • [Phill] Project Board strategy for tracking
  • [Phill] KubeCon NA 2023 Sprint items

June 29, 2023

Attendees

  • Phill Gibson
  • Troy Connor
  • David Justice
  • Sarah Christoff
  • Brian DeGeeter
  • Steve Belton
  • ADD YOUR NAME HERE (getporter.org/dev-meeting)

Agenda

  • [Phill] New Docs TOC layout
    • Restructing of document in this PR
    • Should we move docs into it's own repo?
      • It's super nice when adding features for it to be in the same repo/PR, maybe not for now!
    • Let's focus on getting this PR merged but going forward should we make a PEP/Epic Issue (how do we drive community involvement to this?)
    • Adding guiding princs. into contributing guideline for documentation!
    • TODO: Add docs needing search ability in an issue
  • [Phill] Thoughts around working with Bundle ISVs/Creators on hosting "verified" bundles on a designated repo. Something similar to Docker Hub etc.
    • We need to be able to sign bundles beforehand
    • What applications are most popular and then we can put it in a registry so operators can onboard easier?
    • We should set guidelines around bundles that are community submitted but are usable, testable, badges for trusted developers
    • Creating Bundles for top used products/tools
  • [Troy] Should we ship a clientset controller-runtime client with a getporter/porter/api/v1 scheme with porter to programmatically allow people to write go code against the api surface.
    • ClientSet vs using Generic Client?
      • It's more of what users are used to, it can be easily bundled with the Operator
      • This will allow users to utilize the same client that the operator uses to interact with the custom resources available when the porter operator installed.
        • Your pipeline can include this within go code to install bundles as part of the declaritive state you wish to achieve
      • NOTE: The word(s) of clientset was being conflated with a generated set of clients (like client-go) vs a "client" that can run queries against a set of types that are included with getporter/porter.sh/api/v1.
    • What's the maintainabiility of this?
      • Where to put this because you don't want to include this as part of the process where it ends up carrying dependecies that make this not a viable solution. (dependencies meaning the whole operator. Do we multi module it? It's own repo? Release it as part of a process that includes this?)
    • Demonstration on what this means
  • [Sarah] Let's talk about Helm
    • Ralph to link Helm repo in issue and put HERE
    • Discussion to continue as the Helm charts mature and eventually put them in their own repo in the future, perhaps

June 1, 2023

Attendees

Agenda

  • Operator currently doesn't support telemetry, but as it stands the Operator doesn't support the complete Porter config
  • Currently Operator Documentation isn't working with those using minikube - it may be really great to update the documentation and ensure we're supporting users on all different tools
    • Going through all the operator documentation is a need
  • What tools should we make sure we support for "developer workflows"
    • KinD, minikube (should be able to get it work, but not a huge priority), k3s
  • Demos(?)
    • Showcasing telemetry (see issue above)
      • "It would be nice to go in and interactively troubleshoot a failed install"
    • Showcasing RBACs and how Operator A vs Operaotr B can only do certain things
  • Giving names to the steps in a bundle would be really cool and helpful for troubleshooting
  • Brian demo Operator on multi-cloud (how can we support?)

May 18, 2023

Attendees

Agenda

May 4, 2023

Attendees

Agenda

April 20, 2023

Attendees

Agenda

Office Hours - Come and meet other users, chat and ask questions

April 6, 2023

Attendees

  • Carolyn Van Slyck and Smokey the Cat
  • Steven Gettys
  • Matthew McNeilly
  • Karpagam Balan
  • David Justice
  • Erickson Moskito
  • ADD YOUR NAME HERE (getporter.org/dev-meeting)

Agenda

Office Hours - Come and meet other users, chat and ask questions

March 23, 2023

Attendees

Agenda

  • Defining a v2 dependency in porter.yaml
  • CNAB spec changes for dependencies v2
    • We will release dependencies v2 as a custom CNAB extension, org.porter.dependencies.v2
    • Will not propose changes until AFTER we ship and there is a desire at the CNAB level to have this spec.
    • https://github.com/carolynvs/cnab-spec/blob/dependencies-v2/500-CNAB-dependencies.md is one way we can propose our changes upstream when ready but this will depend on consensus at the CNAB level.
    • Changes between what we implement and what CNAB will implement are different because of lack of support for things such as namespaces, installation resources, and critically temlating in bundle.json
  • Can we rely on mysql 1 in one dependency and mysql 2
dependencies: requires: - name: mysql2 bundle: version: 1.x # we do pick compatible versions between the two, go for the highest possible match - name: mysql1 bundle: version: <=2.0.0 sharing: mode: none
  • how about bundles of bundles, "meta bundle"
    ​​​​name: my meta bundle
    ​​​​
    ​​​​parameters:
    ​​​​- name: shared service endpoint
    ​​​​
    ​​​​dependencies:
    ​​​​  requires: # .list 5 deps, but not install anything itself
    ​​​​  - name: my app
    ​​​​    parameters:
    ​​​​      endpoint: ${bundle.parameters.endpoint}
    ​​​​      
    ​​​​install:
    ​​​​ exec:
    ​​​​     command: ./install.sh
    ​​​​     
    ​​​​# custom actions
    ​​​​test-deployment:
    ​​​​  exec:
    ​​​​    command: ./mytests.sh
    

porter install my-meta-bundle
porter invoke action test-deployment

March 9, 2023

CANCELLED due to people being out

Februbary 24, 2023

CANCELLED due to lack of agenda

February 9, 2023

Attendees

Agenda

  • Project Governance Updates
    • Code Owners
      • automatically request for code reviews
      • needs to be up-to-date
      • feel free to ask to be added if you are interested in doing reviews
    • Emeritus Maintainers: https://github.com/getporter/porter/issues/2546
      • when maintianers are no longer active but we still would like to give them the credits but update their status and gracefully step down from an active mantianer
      • PR for the definition of the activity. We would like to get more eyes on this criteria
      • We are going to vote on the proposal for this once we have defaults agreed on.
  • Porter Operator v1 Planning
    • v1 milestone
    • v1 milestone is tracking a stable release of the operator with compatibility guarantees
    • must haves:
      • delete temporary resources created by the operator using TTL and config to deterime how long to keep resources before deleting
      • Retrying porter agent jobs
      • lots of docs to support someone trying out the operator and how to install/run in production
      • Accessing data from inside the cluster, outputs grpc service
    • Nice to haves
      • passthrough of open telementry env vars to the porter agent from the cluster
      • install the operator into an arbitrary namespace
      • split up the operator bundle's uninstall action for "uninstall the components of the operator" and another for "data cleanup"
    • F5 installs the operator with kustomize, not a bundle
    • multi-architecture builds of the operator (without rosetta), to support running on an arm cluster may or may not be delivered before v1 but it's in progress (MAKE AN ISSUE)
    • ISSUE: how to install the operator with just kubectl / kustomize, and support that officially just like we do with a bundle
  • Porter GRPC API config
    • two endpoints implemented: list installation and list installation outputs
      • list installation: mirrors with CLI outputs
      • list installation outpus: return the latest output of the installation
    • issue: running against the resolved porter config currently. we don't know which porter config is used for a particular installation.
      • option 1: ignore the fine grade resolved porter config. Just use the namespace level config
      • option 2: operator can be the one that creates the API server using a particular porter config
      • option 3: manually deploy the API server
      • option 4: tie porter agent config with the GRPC server
        • however, the porter config is also a top level objec that's also tied directly to an installation
      • option 5: we are not v1 yet, maybe we don't need installation level porter config.
    • issue: two identical porter config but with different names. This may cause confusion when managing installations.
    • next step: how do we integrate the GRPC service inside the operator and how we handle the output resource
      • create an issue inside the operator repo for its v1 milestone.
    • a feature flag for the GRPC service?
      • no change in the porter internals so far
    • carolyn - new issues
      • decide how we want to split up the cmd/porter cmd/porterrpc
      • operator - how to install the grpc service with the operator
      • operator - consume the outputs of an action using the grpc service and provide "A WAY" for someone in k8s to use the output (make an Output CRD)
      • track down who is using the operator and if they use instllation level config
      • Steven will make other issues to track work
    • Workflow of workflows - A lot of people are running porter in a larger pipeline/workflow. Let's understand how people are running porter in larger workflows, document best practices, and identity gaps (like scraping outputs).
      • provide best practices for this use case
    • managing porter state inside a larger pipeline/workflow has been challenging
      • be able to save/restore porter database in a new environment
      • alternative representation of porter's database using porter's driver interface. For example, a sql driver. (making an issue for this so people are aware of this option)
  • Handling duplicate configs in the operator
    • can we document/detect this misconfiguration so people can be aware of this issue?
    • make an issue to track the docs/warning

January 26, 2023

Attendees

Agenda

  • Introductions
  • Want to contribute to Porter? We have issues that are suitable for new contributors at https://getporter.org/find-issue/
  • Looking for meeting feedback! Let us know if you would like to see different content in this meeting, or maybe a particular topic.
  • Porter v1.0.5 release and its replacement v1.0.6
  • Porter plugins installation with file flag demo
    • requires the latest release, v1.0.6 or higher
    • Previously had to install plugins one at a time
    • Now you can pass a file to porter plugins install, with -f plugins.yaml and specify the plugins to install along with the version, source, mirror, etc.
    • The file doesn't use an array (it's a map) to make sure people don't assume that the order in the file is the order used when the command runs to install the plugins. We install in alphabetical order always for consistency.
    • We will use this feature in the Porter Operator, so that the porter agent can be configured easily with a single command (which fits better with the patter of how the operator runs porter commands in a job)
    • The operator PR is still in draft and is not merged yet.
    • If you had made a plugins.yaml with v1.0.5, you MUST update the file to match the fixed schema. Apologies for the inconvenience!
  • Upcoming changes to the operator
  • grpc questions: multithreading with the grpc service/client
    • we have a GRPC POC that can list installations
    • the porter client is long lived and that doesn't work well with the hashicorp go-plugin framework with concurrent requests. We have workaround that makes a new porter struct per request but we'd like to see if we an optimize this better.
    • we want to eventually have this in the operator. different namespace may have different plugins and different configurations. if we want to share a porter client for multiple connections, we need to keep these in mind.
    • we want to make sure that each porter client is initialized correctly so we are not reading the entire configuration each time.
    • right now, we will move forward with client per rpc
    • Should we have a separate porter binary for the grpc server?
      • as if right now, we don't seem to see a benefit of that
      • we have a PR incoming that will improve how we process configurations
      • one benefit of splitting the porter binary is that we can have less dependencies
  • Update on using Porter at F5
    • F5 demoed how to setup multicloud test envs using porter bundles! 🎉 Used by a couple hundred engineers on a daily basis to setup test envs and have them write their own bundles as well.
    • There are hopes to eventually have a public facing registry of test envs, big ip, where a user can quickly setup their own test env. Not planned yet just an idea so far.
    • We'd like to see more companies share porter bundles as building blocks for others to use/improve on.
    • Brian is working on a whitepaper to explain how it all works and showcases (joy of nginx bundle) how it can quickly setup environments.
  • Porter 2022 annual review

December 15, 2022

Attendees

Agenda

  • Operator plugin installation user experience
    • default value
      • default plugin when no plugin config is specified
        • Notes:
          • © let's say the porter plugin doesn't specify anything. They would still get the K8s plugin if there is no default
          • (Y) yes
          • © I kind of like it b/c folks get a working
          • (Y) I think there is another condition, then if someone was running the setup and then they delete the defaults which are preconfigured on the sys level namespace.
          • © Catch up: when we setup the bundle it will setup some config locally; they run the installer, run the customize namespace, then delete the namespace agent config. I still think this is good b/c the config is heirarchial. We have the namespace level we pop by default, then we have a lower level config. If you still have nothing, then the operator will default. We need stronger documentation when you override it and have no config at all.
          • (Y) That can run into a problem with current behavior. When we merge the agent config, the override only applies if the user sets the config. If we implement this logic here when the plugin is empty, we set a default. That value will overwrite.
          • © When we do a merge, we take the sys level config, then namespace, then explicit config. You're saying that if they make an explicit config and set plugins to empty (invalid config), the config will ignore?
          • (Y) The default in the config will overwrite the previous value in the sys or namespace level config.
          • © Is this in invalid case or any case?
          • (Y) In all cases.
          • © I think we need to pair on this as it is not what is expected.
      • should version and feedURL be required?
        • Notes:
          • © do you have concerns using the Porter CLI default behavior
          • (Y) If we don't have anything and try to generate the value
          • © I see we don't have the hash. When we install porter plugins, we create a volume with a hash. If the plugin is mutable, then when a new version comes out, it will be stale. This will lead to people being surprised that they are stuck on the old version with no clear way forward. In a prod env, folks should pin to a well-known version. In a dev env, do we want to make them do that, or is there anything we can do to make that hash accurate. When they don't specify something, we could do something weird like ask porter what the latest version is. Other option is maybe we flag the volume with a lifetime and eventually becomes stale, then after a period we rebuild it. Kind of like a TTL.
          • (Y) When we check a plugin that has a volume installed, we check the TTL.
          • © I think that for the PR, this is not needed. Someone could follow up with a TTL to prevent folks from using a stale version. WDYT?
          • (Y) Now we need to find a reasonable TTL value.
          • © Agent config is alway how we answer that question. Perhaps, a week. We don't release plugins that often. If we did it every hour it would have a perf impact. Once a week, incurring 2 mins. Also could change the TTL on the agent config. Maybe there's a simpler solution. This is a problem more for devs that want to use latest all the time.
          • (Y) I was thinking about it and needed more perspective. I can create the issue after the meeting.
    • error handling
      • Notes:
        • (Y) When a plugin is first installed, when a porter agent config is created and at the same time an install is created, it will always error out due to the plugin is not ready.
        • © Does the operator use the exponential back off?
        • (Y) Right now, it will fail. I needed to have a status to ensure other controller understood it failed.
        • © Is there a status where the controller will wait to install the plugins?
        • (Y) Not yet
        • © If we implement it later, and build reconcile backoff, does that address error handling concerns or are there other issues?
        • (Y) The reconciler alread knows to do the backoff. When they try the job, it may say it failed. From the users' perspective, it would say an error when it was not ready.
        • © I though we had not created the agent action controller failing during the polling
        • (Y) What I mean is that both the agent config and plugins are reconciling at the same time. It would look to see if the agent config is ready and won't be ready yet.
        • © We have not created a job yet, but the logs will see it's busy. You are saying it's in the job logs. By the time the agent action controller decides to create a job, shouldn't it know for sure the config exists and should be populated. I don't understand the scenario when we schedule the job before the volume is not ready. If this is a problem, our solution would be to add more resilient handling to schedule the job when we know it's ready.
    • performance
      • (Y) David had suggested previously that is easier to PATCH a map instead of an array on a CRD
      • (Y) Need to ensure that we install plugins in a consistent order regardless of the representation
      • © Maybe let's not try to match the behavior of how porter installs mixins because at the end of the year (mixins as bundles) will remove the abilitty to install mixins entirely
      • © The porter cli should decide the order and data structure, not the operator.
      • (Y) We can use the same data structure (map) in the porter plugins install file FILE command, and in the AgentConfig CRD. Then we should document that porter will install the plugins in alphabetical order (doc in both the operator AgentConfig and the porter plugin install command).
    • (Y) Still need to finish adding additional information to the Porter AgentConfig status so that we can hold off on using a volume until it is ready or identifying that the plugin installation failed (i.e. the porter plugin install command failed). Yingrong will finish that and include in the open PR that implements the porter plugin installation in the operator.

December 1, 2022

Attendees

  • Yingrong Zhao
  • Carolyn Van Slyck
  • Jeremy Rickard
  • Mohamed Chorfa
  • David Justice
  • Joshua Abednego
  • Steven Gettys
  • ADD YOUR NAME HERE (getporter.org/dev-meeting)

Agenda

  • SBOMs
    • software bill of materials, part of the supply chain security space

    • list of the things that go into the final product (like an image or bundle)

    • can also use it for vulnerability scanning, like "oops it has log4j"

    • provides juicy metadata

    • spdx (linux foundation) is one type of sbom - analysis after the fact

    • Or you can generate it out at build time, which is more accurate

    • Docker (buildkit) has a plugin, that uses syft (https://github.com/anchore/syft), to generate attestations during the docker build command https://github.com/docker/buildx/pull/1412

    • We could generate one that is published alongside or in a bundle that says what's in the invocation image, could put it into the bundle.json

    • Building in provenance about the build itself, where it was built, who built it, the inputs that went into the build (part of SLSA - supply chain level for software artifacts)

    • i.e. was it built on a trusted server or on Carolyn's laptop, what parameters were passed

    • Use cases:

      • put in the version of client tools like kubectl or helm, terraform
      • more information in bundle.json
        • the version of porter
        • the digest
      • what did a bundle do during an installation?(it's more like a inspiration of sbom but not an exact use case for it)
    • we can generate multiple sboms that reference each other

    • how to find sboms in a bundle? How to connect the information in a sbom to the correlated data in a software?

    • does signing work together with sbom ?

      • they can work in parallel
    • Jeremy is going to start a hack.md for this feature

    • Joshua: Just want to pass the info that Dan from Chainguard reached out and Dan, Carlos, and team is happy to help regarding signing and integrating sigstore implementation if it's the tool we want to go with. But as Carolyn said we might definitely want to make it as generic and not get coupled to specific ones.

    • signature plugins

  • Porter data services for outputs
  • State of PEP003 Advanced Dependencies (maybe push to a later time)
    • Discuss "execution plan" or workflow
    • How to move a bundle with dependencies, and how to do resolution in an airgapped environment
    • Can we support writing a single bundle and deploy to any cloud?
  • white paper about how F5 use porter

November 17, 2022

Attendees

  • Carolyn Van Slyck
  • Steven Gettys
  • Joshua Abednego
  • David Justice
  • Mohamed Chorfa
  • ADD YOUR NAME HERE (getporter.org/dev-meeting)

Agenda

  • (brian/steven) Porter data service
  • we have a gap in the operator, getting the outputs of a bundle run back into the operator or using it in an integration
  • another example is monitoring the status of an instalation
  • trying to scrap the invocation image output was a huge text scraping hack and didn't work well
  • Need to be mindful of secrets and secret outputs as well
    • Another way was to send links to secret stores to get an output value (if it was secret)
  • Suggestion: Run a data service alongside the operator's container in k8s
  • porter repo can build two binaries, one for the cli and another for the data service
  • Both below show you the last output
    • porter installation outputs list -i installation-name -o json
    • porter installation output show -i installation-name output-name -o json
    • run RUNID, get the output from a specific run instead of the last
    • would like a way to retrieve the path to the secret, so you can pull it yourself, maybe have json/yaml give you back both the resolved secret (it does this already) and the full secret path to the location in the secret store (formatted so that it is ready to use with the secret store cli)
  • Do we really need an output CRD in the operator?
    • could have a separate RBAC for it which is nice
  • What protocol would the api be?
    • options: rest, grpc, graphql
    • concern is that we may need to evolve the api while providing a stable/consistent interface into the data on top of that without breaking people each time we expose new stuff or alter the data format
    • containerd shims use ttrpc, slimed down grpc used for ipc on local machine.
    • the storage plugin already is written in grpc, so we can reuse a lot of the data structures potentially (depends on the shape of the service endpoint/resources)
    • exposing something other than grpc, means we need to translate that into what the plugins support to query the data
    • minimize the end users need to pay attention to endpoint versioning over time.
    • public api stable -> translation layer that maps this to the actual storage and handles proper protocol version / storage representation
    • start with the internal grpc service that handles translating, use that immediately with teh operator
    • then add on top something user friendly that talks to the internal gprc service
  • Next step is a PEP for the internal data service (user facing service would be a next step)
    • Scope: This is a data only service, it doesn't trigger porter bundle executions? Running bundles and triggering is the role of the operator.
  • Nginx bundle - https://github.com/bdegeeter/JoyOfNGINX/tree/main/porter/nginx-oss
  • central the mgmt of the protos so that they can be versioned publicly
    • can use https://buf.build/ to describe the protocol, easier than using the low level google protobuf tool
  • PEP003 - https://github.com/getporter/proposals/blob/main/pep/003-dependency-namespaces-and-labels.md
  • Workflow Resource - https://github.com/getporter/proposals/blob/d31409ed77d6fcabfc14a1de006f7541742f252c/pep/003-dependency-namespaces-and-labels.md#implementation
  • Next meeting we'll go over workflow/deps and platform idependent bundles, single installer that can deploy to any cloud.

October 20, 2022

Attendees

  • Carolyn Van Slyck
  • Joshua Abednego
  • Steven Gettys

Agenda

September 8, 2022

Attendees

Agenda

  • The 1.0.0 release candidate and how you can help test
  • Reminder of the 1.0.0 final release process
    • v1 releases for all plugins and mixins
  • v1.1.0 label is being used to flag high priority issues to work on post v1
    • https://github.com/getporter/operator/pull/108 (automatically install plugins into the porter agent)
    • Feel free to common on ones that need to be looked at or just DM carolyn
    • Need a kubernetes testing environment for testing the operator and porter, another that uses cosmos and a real mongodb (carolyn will make an issue to track)
  • Applying to move from CNCF Sandbox to CNCF Incubation stage
  • Post 1.0 roadmap and plans
    • Cleanup backlog
      • comment on any issues that you still find useful
    • Curate high priority issues for initial minor releases
    • Finish Porter Operator
    • Advanced Dependencies
    • 🙋🏽‍♀️ Let us know what else you are looking for!
    • Gettin outputs captured from the operator
      • Endpoint inside the cluster where you can query porter's database
      • Look into how we can integrate with a grpc db service, with backstage.io plus the operator
        • wouldn't have to write your own UI

July 14, 2022

Attendees

  • Yingrong Zhao
  • Carolyn Van Slyck
  • Ralph Squillace
  • Jeremy Goss
  • Steven Gettys
  • Joshua Abednego
  • Aaron Schlesinger
  • David Justice
  • Add your name (https://getporter.org/dev-meeting)

Agenda

  • Advanced Dependencies

    • Current Status
    • Updated designs and notes
    • Feature Flag plan (how to release this without it being a breaking change requiring a porter v2)
    • Incremental PR plan to avoid a mega PR of doom that is unreviewable
    • Open Questions
      • Is it possible to integrate with third-party workflow engines (Argo, Brigade, or something that runs on Docker only)?

    David: third party workflow engine is a huge dependency. Is it worth it?
    Carolyn: Bundle is a workflow. Porter does not want to reinvent the wheel. It will also allow for workflow engine to be pluggable.
    Ralph: plugin approach how it affects immutability.
    Carolyn: who kicks off the running container does not really affect immutability. Container Orchestration.
    Steven: So we have argo kicking off porter currently.. You're telling me soon I could have argo kick off porter to kick off argo?? That’s pretty meta
    argo workflow -> porter's installation
    collect secrets -> terraform enterprise
    Carolyn: we don't want to write another workflow engine so we can focus on solving problems with bundles and improving security.
    abstracting workflow from porter might be too complex
    passing output from one argo stage to another, it requires to parsing k8s logs.

    • (Ralph) deployment model vs distribution model (handing it off for a customer to install like a deb)
      • have a built-in simple solution

September 8, 2022

Attendees

Agenda

June 16, 2022

Attendees

  • Carolyn Van Slyck
  • Yingrong Zhao
  • Steven Gettys
  • Joshua Abednego
  • Tanmay Chaudhry
  • Jeremy Goss
  • Add your name (porter.sh/dev-meeting)

Agenda

  • Introductions
  • Porter Hoodie! Follow the directions here to request one https://groups.io/g/porter/topic/91602108#94
  • v1 release coordination
  • Release Stages
    • Alpha
      • Unstable database schema, no migrations supported
      • Almost done! We'll move to beta when the migration PR is merged
    • Beta
      • Data migrations from v0.38 are supported
        • New PORTER_HOME directory, and porter migrates data from your old directory into the new one.
        • We don't touch old data so that you can go back if needed.
        • porter storage migrate --old-home ~/.porterv0
      • Someone should be able to use their current data with beta onwards though our final release
      • Existing bundles, parameter sets and credential sets created with v0.38 will need minor schema changes to work with v1.0.0-beta+
      • Last chance to give feedback!
      • A few breaking changes are still planned for beta. Getting these out of the way to keep 1.0 consistent
    • Release Candidate
      • We will cut an RC when we think we have a viable 1.0.0 build. If we find bugs, we'll fix them and cut another RC
      • After the RC is vetted, it will be retagged as 1.0.0
  • v0.38 EOL 🚨
    • v0 is over a year old. Move off of it quickly especially in production. A lot of security redesigns went into the v1 release.
    • We will give v0.38 security support (when possible) for 3 months.
    • So if there are high severity vulnerabilities in our dependencies or Go, we'll rebuild with patched dependencies.
  • Moving forward with v1
    • We plan to release dependencies as a minor patch to v1. More on that at our next meeting
    • No breaking changes to database schema, or document schema without increasing the major version.
    • Whenever possible, new features will be released with flags, so that they don't affect compatiblity.
    • Roadmap will be updated again, and we'll discuss in a meeting post 1.0
  • Outputs in the operator (Steven)
    • Brainstorm on how to fetch agent action stdout
    • Things we have tried:
      • inside the installation controller, after the agent action finishes
      • modified teh agent image, support a list of commands to run
      • porter install foo
      • porter installation outputs show
      • right now trying to do pod logs this is an ugly hack
      • -o /porter-shared/outputs/myoutput didn't work well due to timing issues and needing a pod around to access the mounted volume
      • It seems like it's time to just add a grpc service and support retrieve the output
      • F5 isn't blocked, they have hacks to get the data, but we don't want to bake in those hacks into theo perator.

June 2, 2022

Attendees

  • Carolyn Van Slyck
  • Steven Gettys
  • Brian DeGeeter
  • Yingrong Zhao
  • Jeremy Goss
  • Prakash Mirji
  • David Justice
  • Add your name (porter.sh/dev-meeting)

Agenda

May 5, 2022

Attendees

Agenda

April 21, 2022

Attendees

  • Yingrong Zhao (porter.sh/dev-meeting)
  • Carolyn Van Slyck
  • Steven Gettys
  • Jeremy Goss
  • Joshua Abedengo
  • Prakash Mirji
  • Brian DeGeeter

Agenda

  • (yingrong) Status Update: Storing sensitive data in a secret vault

    • sensitive parameter values and outputs are currently persisted to the claimstore (database)
    • This is a security concern that we want to address before 1.0
    • Porter will persist sensitive values into your configured secret store, and the db stores a link to where to get it
    • By default, no secret store is configured so porter will refuse to work with bundles that generate or require sensitive data
    • We will provide a secret plugin that you can use if you are okay with storing sensitive data on the filesystem, for dev/test (or trying out porter)
    • Otherwise in production you should use Azure KeyVault or HashiCorp Vault for the secret plugin
    • Parameters and outputs are marked as sensitive in the porter.yaml
    • porter installation output show MYPASSWORD (get sensitive value)
    • porter installation show (redact)
    • Human output is redacted by default, but you can get it with json
  • (carolyn) Changes to the plugin protocol

    • Secret is adding Create function
    • All plugins will pass context.Context and support opentelemetry
    • gRPC protocol instead of net/rpc
    • all plugins will need to recompile and be updated
    • We will provide migration doc for the plugin updates
  • (carolyn) Data migration will be AFTER alpha.20 (which has the sensitive data fix)

    • don't recommand to build based on source until alpha.20

    a bug in plugin.Serve method. Fix is on the way
    Storage plugin can't work with external storage plugin. The fix is in progress.
    cred/prameter set CRD in kubernete operator plugin is in progress

April 7, 2022

Attendees

  • Carolyn Van Slyck
  • Krishna Sagiraju
  • Jeremy Goss
  • Yingrong Zhao
  • Steven Gettys
  • Srujan A
  • David Justice
  • Joshua Abednego
  • Add your name here (porter.sh/dev-meeting)

Agenda

  • Supporting data migrations from v0.38 to 1.0.0

  • Moving into beta and the release candiate process

  • Roundup of recently released features

    • Build behind a proxy
    • build-args
    • secrets and ssh
    • Krishna: non-docker buildkit
  • Techincal guidance on issue#2022

    • usecase: add assets to a bundle and be able to use porter to get the assets out of the bundle
    • put the assets into a separate layer(separate from invocation image) and have ability to identify the location of those assets in the bundle
    • pull the assets only rather than the entire image
    • current workaround: base64 encode the data so it can be part of the manifest through custom metadata
  • Discuss CredentialSet and ParameterSet CRD implementation issue#18

    look very similar to the installation controller
    rethink the list of type of values supported in kubernetes
    link to native kubernete concept(like: configMap)

  • No way to extract custom resources from a bundle

  • Multiple invocation images

    • 1 image to package all the assets
    • CNAB spec allows to define multiple invocation images
    • data image
    • mount reference images into the running container

Mar 24, 2022

Attendees

Agenda

Open Office Hours

Mar 10, 2022

Attendees

  • Carolyn Van Slyck
  • Steven Gettys
  • Joshua Abednego
  • Brian DeGeeter
  • Don Stewart

Agenda

  • Porter builds bundles to run as an unprivileged user now (v1.0.0-alpha.12)
    • home directory for the user is /home/nonroot
    • Dockerfile
      • USER 65532
  • How to deal with dependabot
    • cascade merge requests, so we could first merge cnab-to-oci, tag it, then cnab-go, tag it, then porter
  • Mixins as Bundles Proposal
  • How to install plugins on the porter agent
    • current workaround: building the agent with extra plugins installed
    • Possible ideas:
      • mount a plugin volume
      • use the k8s plugin and then connect k8s to other secret stores
        • azure keyvault service, and prepoulate secrets
        • Carolyn will ask Azure about how this works
  • It's possible to get stuck
    • kubectl delete -> deletionTimestamp (stuck)
    • uninstalled = true on the installation
metadata:
    name: foo
    finalizers:
        - porter.sh/finalizer

Feb 24, 2022

Attendees

  • Carolyn Van Slyck

Agenda

  • Reminder this meeting is recorded and will be posted at https://porter.sh/videos/
  • Introductions
  • Switching to bundles that don't run as root (nonroot invocation image)
  • Upcoming AgentAction resource in the Porter Operator
  • Let's chat about dependencies!
    • Status
      • Proposal is mostly there, need to finish and merge.
      • CNAB Spec will follow after we vet with an implementation.
      • I have a branch that vets the manifest changes, graph resolution. Still need to vet the execution plan.
    • Picking a default implementation for an environment
    • Managing an installation and its dependencies
      • Visualizing
      • Working with them as a unit
      • Lifecycle management
      • When is something not used anymore?

Feb 10, 2022

Attendees

  • Vaughn Dice
  • Steven Gettys
  • Joshua Bezaleel Abednego
  • Krishna sagiraju
  • Carolyn Van Slyck

Agenda

Jan 27, 2022

Attendees

  • Steven Gettys
  • Carolyn Van Slyck

Agenda

Jan 13, 2022

Attendees

  • Carolyn Van Slyck
  • Nathaniel Hatfield
  • Yingrong Zhao
  • Joel Baxter
  • Brian DeGeeter
  • Ralph Squillace
  • Joshua Bezaleel Abednego
  • Steven Gettys
  • Vaughn Dice
  • Krishna Sagiraju
  • Jeremy Goss
  • Add your name here

Agenda

  • Introductions

    • Yay, new people!
  • Discuss submission of distroless Porter docker images to Iron Bank

    • Summary: Certain organizations have a set of standards in order to approve software use (related to Platform One?)
    • Porter has added a few things towards this goal
    • (Defer rest of discussion)
  • Demo of the latest operator build and how to install it and use it

    • https://release-v1.porter.sh/operator/
    • https://release-v1.porter.sh/reference/file-formats/#installation
    • Background: We have the porter CLI, but we'd like to run Porter in a K8s cluster to automate bundle installation/lifecycle
    • Side note: Toggle the docs on porter.sh/docs at the top left to v1 to see these docs
    • (Demo!)
    • (Looking at installation manifest for the operator): Note that the metadata section has K8s-y things while the namespace/name under the spec is for Porter
    • Should be able to interrogate operator logs via porter CLI, shouldn't need to drop down to K8s pods, etc. (though possible; note that the operator will soon start cleaning up pods, etc. when completed)
    • Editing the installation manifest triggers an upgrade
    • What is the active field?
      • Two diff ways to manage custom resources in K8s
          1. Delete CRD directly (but when it's gone, it's gone)
          1. Keep CRD but track uninstall history
      • This is what active means for the operator: Setting active to false triggers an uninstall but keeps the record around so users can see its uninstall status via the porter CLI and the CRD continues to exist
      • Thought: Issues may arise w/ the two modes and multi-party use on a shared cluster
        • Should Porter ignore the kubectl delete CRD command?! Or at least not mapping it to 'porter uninstall'
        • Let's reference other prior art (Argo), maybe create an issue/discussion
    • 99% of the operator functionality is encapsulated by the porter installation apply CLI command
    • Starting to add Status metadata to the installation object/record
      • Caveat: There can currentlly be drift between Porter's datastore (mongo) and K8s
      • To be clear, we won't ever consider K8s the source of truth for Porter installation records
  • Uninstalling a bundle with the operator

    • Not yet released
    • Maybe within the next week or so!
    • Tangential Q: State of K8s plugin for Porter?
      • Needs to be revisited to get to work w/ Porter v1
      • Currently, Carolyn set up a port forward from the mongodb instance running in K8s to access the operator installation records/data
      • Since Porter switched to the mongodb backend for data store, the K8s plugin would be revised to only handle secrets (Think: I need to access a secret/sensitive value and I want it stored in an external provider e.g. Hashicorp Vault, Azure, K8s secrets, etc.)
      • After the uninstall work is in, we'll use file formats for Creds/Params to improve this use case
        • e.g. they will be represented by CRDs in K8s as well
        • Looking for contributions/collab here!
      • Next step: resolve mongodb connection string from a secrets plugin instead of hard-coding it into config on the host (or in K8s)
  • Mixin error handling

    • https://release-v1.porter.sh/mixins/exec/#ignore-error
    • In the latest v1 release (notes forthcoming)
    • Example scenario: Using a mixin to create a resource and it already exists. Some CLIs will error in this case (or return w/ non-zero exit code). Previously, this would trigger a failure in the Porter action being run, which would halt the action.
      • Now, we can configure such a step to ignore the error, with a lot of different options:
        • Ignore all errors
        • Ignore any error with specific exit code
        • Ignore an error with certain output (either string or regex match)
    • Currently implemented in exec mixin; want to roll out to others (like az)
    • If you use a mixin and would like this logic incorporated, we can help!
  • But wait there's more! We have a new Porter dev!

    • Meet Yingrong, who has just joined the Porter team full-time
    • Last job was in OSS as well
    • Now Carolyn has more help!!!

Nov 18th, 2021

Attendees

  • Carolyn Van Slyck
  • Joel Baxter
  • Jeremy Goss
  • Krishna Sagiraju
  • Nathaniel Hatfield
  • Add your name here

Agenda

  • Operator backlog
  • Porter v1 backlog
  • Introductions: Joel, Jeremy and Krishna

Nov 4, 2021

Attendees

  • Carolyn Van Slyck
  • Joshua Bezaleel
  • Erickson Moskito

Agenda

Oct 21, 2021

Attendees

  • Carolyn Van Slyck
  • Vaughn Dice
  • Add your name here

Agenda

Oct 7, 2021

Attendees

  • Carolyn Van Slyck
  • Mohamed Chorfa
  • Steven Gettys
  • Vaughn Dice

Agenda

  • v1 milestone progress and backlog grooming
    • https://github.com/getporter/porter/milestone/16
    • Some top issues to look at:
    • Mixin schema - could break into 2 issues
        1. Get rid of the squigglies - allow arbitrary mixin config
        1. Get full auto-complete for mixin config
    • Improve DX around using Porter as a library (out of v1)
      • Of course, CLI, Porter config, etc. will be covered with semver guarantees we don't wish to focus on the same for Porter's API
    • Params and Outputs from array to map? (out of v1)
      • Not convinced the benefits outweigh the impacts
      • Perhaps a little too late to make the change (all Porter manifests would need updating, etc.)
    • WSL DNS bug attempting to install from cdn
      • Not holding up v1
      • Any WSL pros out there that can help?!
    • Validate bundle digest https://github.com/getporter/porter/issues/1626
      • Security-wise, validating digests would be preferred
      • Q: If you have the tag and the actual expected digest, what would be the value of pulling via tag and checking digest as opposed to pulling w/ the digest directly? (No compelling answer)
      • Re: supply-chain security, arguably the more pertinent missing piece is checking the signature on the artifact
      • Carolyn shows current state of support for bundle versioning: can use version, tag or digest (digest will always be given top priority when provided)
      • Thinking this just needs clarification in the docs (in fact, a dedicated Security section would be the next move)
    • Support buildkit flags https://github.com/getporter/porter/issues/1769
      • Today, we support buildkit as a driver but we don't expose buildkit opts/flags in the porter CLI
      • Open to interested contributors!!
    • Adding lint/scan fields to Helm and K8s mixin
      • Should be able to add today, to run during build
    • Add verbosity flag
      • Initially added to support better granularity on what is logged
      • Carolyn experimented with structured logs and tracing and we think this will be the way to go. (Current impl uses OpenTracing and sends to Jaeger; looking at OpenTelemetry which looks to be the successor)
        • Enthusiastic support from the community!
    • Allow setting step outputs as non-sensitive
      • They are sensitive by default
      • Community support for this!
    • Params/creds that are JSON
      • Support accessing fields of object in manifest, e.g. '{{ bundle.parameters.object.key }}'
    • Wrapping up: Encouraged to go thru the v1 milestone and add comments/feedback. We'd like to get it into a state where only the necessary or highest-pri items go to v1. Lower-pri/non-breaking-changes are more eligible to delay. Thanks!

Sept 23, 2021

Attendees

  • Carolyn Van Slyck
  • Vaughn Dice
  • Jeremy Goss
  • Brian DeGeeter

Agenda

  • Introductions when we have new attendees
    • Welcome, Jeremy! Interested in using Porter to bundle their cloud-native apps. Already have a tool for authoring apps which produce CNABs.
    • Carolyn: Reminder - these meetings are primarily for the community, so don't hesitate to suggest working sessions, ask questions, add to the agendea, etc.
  • Show and Tell: Share how you are using bundles, show a new mixin, and other neat stuff.
  • Recent Releases
    • v1.0.0-alpha.3 is out!
    • Reminder: these are alpha releases, so no guarantees of backwards compat, data migration, etc. Only for kicking the tires.
    • Though, final v1.0.0 will have migration support from pre-v1 to v1
  • Demo of new v1 features:
    • MongoDB data backend! Way more performant, both locally and remote. Supports complex querying.
    • Namespaces and labels to support organization of installations, parameter and credential sets. Porter also has a global namespace for shared/team params/creds.
    • Labels currently allow filtering of resources. In the future, this will tie in with better/more advanced dependency support.
    • Bundle state: Porter now supports use of a 'state bag' to manage application state.Have porter persist state data for you, such as a tfstate or tfvars file so that you don't have to deal with it yourself
      • Example: bundles using terraform mixin to track tfstate/tfvars. See the tabbycats demo: https://github.com/carolynvs/tabbycat-demo
      • Q: Can users still access the 'hidden' state resources? Yes, if they look at the bundle definition, say via porter inspect. But again, we don't expect users to usually interact with them.
    • Params/Creds import (via apply). Can show params/creds and output yaml/json to then edit as needed and then re-apply.
    • reconcilation: Define the desired state of an installation, and Porter will handle comparing that desired state with the installation's current state, then run install, upgrade automatically based on if there are differences.
      • Sneak peak! (not in v1-alpha.3 but coming soon)
      • Example: show an installation, modify a parameter value, apply the updated installation Porter automatically detects a difference and runs upgrade.
      • This feature also sets things up for the Porter operator to transition from imperative to declarative. Coming within the next month or two.
      • Opens up GitOps scenarios and Flux integrations. Change a file in a git repo, Flux lets the Porter Operator know, Operator runs actions as needed.
  • Conclusion: Try v1 alpha!!! Let us know how it looks. Thank youuuu

Aug 12, 2021

Attendees

  • Carolyn Van Slyck
  • Vaughn Dice
  • Add your Name

Agenda

  • Introductions when we have new attendees
  • Is the approach for mixin schema support standardized?
    • In progress of switching to use goembed to embed schema in mixins
    • When developing a mixin, it isn't strictly necessary to embed schema, but recommended, especially with custom config, etc.
  • Show and Tell: Share how you are using bundles, show a new mixin, and other neat stuff.
  • Demo of new v1 features:
    • Namespaces
      • Very similar to namespaces in Kubernetes (but not exactly the same)
      • Can search all namespaces via porter list --all-namespaces, or specific via porter list --namespace test
      • In Porter's config, the default namespace can be set to a specific value
      • Goal is to allow multiple users to work with the same Porter install/env
      • Also supported for credential and parameter sets
      • To designate installations/cred/paramsets to the 'global' namespace, set namespace to the empty string (-n "")
        • This enables bundle installations in a certain namespace to use cred/paramsets in the global namespace
        • Porter will first check the certain namespace; if not found, will check the global
    • Labels
      • Another technique to group installations (and filter searches)
      • Also supported for credential and parameter sets
      • Q: Will bundle authors be able to add labels to the bundle itself?
      • A: Currently we have keywords; there are/will be CNAB Spec proposal(s) to transfer bundle label(s) to installation
    • MongoDB
      • In order to support the namespace and label queries, we had to overhaul Porter's underlying storage mechanism
      • New default storage plugin: mongodb-docker: MongoDB running in a local Docker container
      • No longer flatfile local filesystem storage
      • Storage plugins now speak mongodb
      • Another built-in storage option: mongodb: provide the conn URL to a MongoDB instance and Porter will connect to it
      • Check your Porter config to see if you have a hard-coded driver to the old/deprecated filesystem plugin and update
      • Note that Porter stores outputs in MongoDB, which may be an issue with large outputs. May revisit in future.
      • Handy GUI to explore Mongo data: MongoDB Compass (https://www.mongodb.com/products/compass)
        • Dev tool as well to be sure indices are being used where expected
    • Extended CNAB talk wrt storage
      • The CNAB Spec defines the data representation and runtime concerns
      • Other aspects, like param and cred sets, are non-normative, i.e. just suggestions
      • We discovered we had items defined in the Go CNAB library (cnab-go) that were non-normative/generic and not nec. strictly defined by the spec. So we've decided to remove these from cnab-go to make it clear that those concerns are entirely up to each implementation.
      • Porter ran into big performance issues when using these parts of cnab-go
      • Want to vet these changes in Porter first (using fork of cnab-go) for a bit before issuing PRs in cnab-go and cnab-spec
    • Try these changes out!
      • Next alpha v1 release should be due out soon.
      • Disclaimer! As these are alpha releases, there will be no offer of data migration between releases.
      • Repeat: not for use in production :)

July 15, 2021

Attendees

  • Carolyn Van Slyck
  • Vaughn Dice

Agenda

  • Introductions when we have new attendees
  • Show and Tell: Share how you are using bundles, show a new mixin, and other neat stuff.
  • Porter Operator - Desired State Commands
    • Carolyn working on Installation CR for configuring desired state of an istallation
    • porter installation|credentials|parameters apply
    • porter installation|credentials|parameters show
    • Store additional status information on the custom field
    • Talking about validation errors or mistyped fields (param/cred sets that don't exist, etc.)
      • Porter should return error and not persist the provided CR
    • With the kubernetes plugin, we might have the ability to use k8s secrets as store for values in Param and Cred sets (or we can/should add)
    • Have had requests to define/manage Param and Cred sets as CRs as well
      • But the weird part is currently 'path' is a way to define where a p/c value is from, which isn't applicable in K8s
      • So, could use 'configMap' or 'secret' in K8s
    • In general, working towards users of the Operator never having to use the Porter CLIjust Operator interactions in tandem with CRDs
  • Porter data storage
    • Currently use an ORM model (implemented in cnab-go); we'd like to get rid of this (very inefficient, esp. via plugins that store/retrieve from cloud providers)
    • Moving all data access from cnab-go into porter, e.g. credential store
    • Storage Plugins (mongo query api)
      • local mongo daemon to replace local filesystem
      • mongodb for remote storage
      • cosmosdb for azure
    • Redefine the storage plugin interface

July 1, 2021

We have changed the meeting time to 3pm UTC

Attendees

  • Carolyn Van Slyck
  • Vaughn Dice
  • Ritesh Yadav

Agenda

  • Introductions
  • Put questions or items on the agenda
  • Helm -> Helm2 Mixin Rename
    • Update the porter readme doc with the latest helm2/helm3 changes
  • Review v1 progress
  • Backing out storage from the CNAB spec and cnab-go
    • Moving storate providers and drivers into Porter, and optimizing for reads/writes/queries.
    • Implications for storage plugin such as changing the interface or understanding domain more (i.e. knows how to read a set of related data for an installation).
  • Porter Operator status and roadmap
    • Work on operator starts back up in July
    • New direction: desired state management of installations
    • Need management of credential/parameter sets, porter config, etc.
    • Will devote an upcoming community meeting to design walkthrough
  • Take a look at FAB, https://fab.dev/
    • Could we make a fab mixin?

June 16, 2021

Agenda

Action Items

  • make a v1 page that describes high level breaking changes and features to look forward to
  • Fix the permalink

May 19, 2021

Participants

  • Jennifer Davis
  • Vaughn Dice
  • Carolyn Van Slyck

Agenda

  • v1 milestone is set! https://github.com/getporter/porter/milestone/16
  • What has gone into v1 and how to try it
    • Only install exec by default (#1588)
    • Add support for maintainers (#1572)
    • Add new build driver for buildkit (#1567)
    • Use Go 1.16 and change to go:embed
  • What has gone into main?
    • Everything merged into main, @carolynvs has been merging into v1
    • Update the docker example (#1595)
    • Updated QuickStart (#1586)
    • Fixed blog CSS (#1593)
    • Split integration tests into separate build (#1564)
  • Porter Operator roadmap is being built out now
  • Signy Integration
    • Coming from security spec (not finalized)
    • Signing and verification experience needs to be designed
    • Implement using an experimental flag so we can start integrating now, before signy is done.
    • Want a PEP that describes how this should look when complete.
  • Documentation improvements! Overview of quickstart changes and upcoming updates.
  • Potential GitHub Action to merge main to v1 branch rather than manual process.

May 5, 2021

Participants

  • Carolyn Van Slyck
  • Thorsten Hans
  • Vaughn Dice
  • Mohamed Chorfa
  • Carlos OKieffe

Agenda

  • Reminder: We have an open agenda! So all contributions to the agenda are welcome here.
  • Speed up build times: https://github.com/getporter/porter/issues/1546
    • Pull Request CI can take quite a bit of time (20-30mins)
    • Main culprit: integration tests that run real bundles against a real Docker daemon
    • Oftentimes, PRs introduce changes that don't actually affect the runtime
    • Moving forward, we can always kick the integration tests off manually, otherwise we'll be smarter about only running them if the runtime is affected. Then, the tests will always run on the merge to main event.
  • Docker app migration
    • Docker App has been deprecated and/or no longer supported
    • Carolyn has written a blog post on how to migrate your Docker App to Porter! (Will be live soon, check porter.sh/blog)
    • Surprise! Porter will honor the DOCKER_HOST/DOCKER_CONTEXT env vars when installing a bundle, so they can use a remote daemon/host.
    • Thorsten may have cycles to try with Azure/ACI :)
    • As a follow-up, could be nice to have a doc a la Compatible Registries for compatible Docker hosts
  • Update on hosting situation
    • Previous setup came with a lot of instability
    • We've migrated everything to Netlify + GitHub releases, e.g. https://github.com/getporter/porter/releases/tag/v0.38.1
    • Mohamed concurs - he always publishes to GH as well
    • All of the mixins under the getporter org publish to GH releases as well
  • Experimental flag support
    • Porter implements the CNAB core spec (and others) but oftentimes pushes the spec along
    • There are features we on Porter would like to see in a spec and so we have a need to use feature flags to toggle major experimental features (which, when on, may not strictly comply with a given spec)
    • Thorsten: Have we considered the approach of a global 'this will break spec compliance' flag?
    • Thinking more toggle per feature
    • Thorsten: Experience here with .NET. Need to be aware of messaging to the community
    • Some experimental features may not necessarily break a spec (adding Buildkit, for instance) or they might break other versions of Porter itself
    • Mohamed: We could have two categories of features: Build vs Runtime. The latter is really where spec compliance comes in, not necessarily the former.
  • CNAB Security Spec
  • Augmented Mixins
    • Forum discussion: https://github.com/getporter/porter/discussions/1522
    • Additional logic for a given mixin, like lint/scan of Helm charts for the helm mixin at build-time
    • Configuration would be specific to a given mixin; shouldn't need changes in Porter to support
    • Another approach, when talking about the Terraform mixin specifically. If the logic needs to occur at runtime, maybe a new command would be a better fit the ability to run arbitrary terraform cli commands as they are added in subsequent versions, etc. (Like terraform validate ... which needs to occur using the underlying cloud provider at runtime)
    • Vaughn/vdice can help ^^ !

April 21, 2021

Skipped, vacation! 🌴

April 7, 2021

Participants

  • Carolyn Van Slyck
  • Jennifer Davis
  • Vaughn Dice
  • Mohamed Chorfa

Agenda

March 24, 2021

Participants

  • Carolyn Van Slyck
  • Ralph Squillace (Microsoft)
  • Vaughn Dice
  • Joshua Bezaleel
  • Jennifer Davis (Google)
  • Simon Davies

Agenda

March 10, 2021

Participants

  • Carolyn Van Slyck
  • Mohamed Chorfa
  • Vaughn Dice
  • Simon Davies

Agenda

  • Continue discussing dependencies changes
    • Prevent reinstalling on top of an existing installation
    • Simon: Porter configuration allows for specifying different plugin config by namespace?
    • Mohamed: I have a dependency on foo@v1.0.0 and I get installed. Foo is then upgraded to v2.0.0.
      • I upgrade the main bundle: we have a lockfile of dependencies, we don't re-resolve dependencies.
      • What if I upgrade a bundle and that introduces a new dependency, we do resolve that and update the lockfile and then install the dependency.
      • Do we want to enforce another installation's dependency version range when upgrading a dependency. Soft warning. (need more scenarios to understand if we should )
      • Document the dependencies and rely on the lockfile, make that visible.
      • Add to doc that ref and version need to match for existing installations
    • I had a dep on mysql v1, then I upgraded the bundle and bumped by dependency of mysql to v2. What happens? Do I upgrade mysql too?
      • We need to rely on the dependencylifecycle definition and always allow the user to override and tell porter what to do.
    • Add owner (installation) flag to resources so we know who installed it.
    • We want the other feature of tracking users to actions

February 24, 2021

Participants

  • Carolyn Van Slyck
  • Vaughn Dice
  • Mohamed Chorfa

Agenda

  • Walk through of Proposals
  • Common github settings for getporter organization
    • We now have github.com/getporter/.github where we can put common settings such as labels and branch protection rules
    • We should add issue and PR templates to this repository
    • We should use the t-shirt size github app and label so PRs are automatically sized for us

February 10, 2021

  • Cancelled

January 27, 2021

Participants

  • Carolyn Van Slyck
  • Simon Davies
  • Vaughn Dice

Agenda

  • Making this a public meeting.
  • Porter Operator
tags: Meeting