--- tags: keda --- # KEDA Community Standups ## Feb 27 Meeting ### Proposed agenda (feel free to add iems) - Provide capability to extend scalers ([#624](https://github.com/kedacore/keda/issues/624) - Tom) - Provide governance for built-in & add-on scalers ([#573](https://github.com/kedacore/keda/issues/573) - Tom) - Introduce version in the trigger spec ([#613](https://github.com/kedacore/keda/issues/613) - Tom) - Provide a changelog ([#614](https://github.com/kedacore/keda/issues/614) - Tom) - Should be part of https://github.com/kedacore/keda so that feature PRs can include updates to changelog - This does not replace GitHub releases but are complementary - Tom would be happy to pick this up - Important for handling deprecations ([demo](https://changelog.promitor.io)) ### Attendees - To be added ### Notes - To be added ## Feb 20 Meeting ### Proposed agenda (feel free to add iems) - CNCF Donation Update (Tom) - [SIG-Runtime PR](https://github.com/cncf/sig-runtime/pull/10) - Remove `deploymentName` label on every `ScaledObject` ([#545](https://github.com/kedacore/keda/issues/545)) - How much effort would this be? It's redudant information which we can avoid AFAIK - Paolo Patierno from Red Hat is happy to contribute it - CNCF presentation listed Astronomer, Bupa, Cyclomedia - Do we have contacts to ask if we can list them on keda.sh? - @Daniel Ok for Astronomer to be listed? - Tom will add those that we have approval for ### Attendees - Daniel - Lukas - Satish - Jeff - Anirudh ### Notes - KubeCon sessions - Tom K got a lightning talk accepted for Serverles practitioners summit. - Jeff H got a tangentially related session accepted for practitioners - will mention KEDA - CNCF update - Spoke this morning to SIG-Runtime, went well. Opened PR (Anirudh to copmlete) and then just need to get ToC approvers - Cluster autoscaler - One item that came up was the possibility to use KEDA to drive scale out of the cluster autoscaler. Opened [an issue](https://github.com/kedacore/keda/issues/637) for discussion - Astronomer.io going to open a docs PR to add logo to users / community - Redundant deployment-target - Agreed if possible we should clean up the scaled object so you don't have to redundantly provide the `deployment` target 2xs [RELATED ISSUE](https://github.com/kedacore/keda/issues/545) ## Feb 13 Meeting ### Proposed agenda (feel free to add iems) - Knative - Aleksander joined from IBM - He has been pushing KEDA in internal componnets - Azure Monitor - Last night support was finished. - Fixed auth issues. - Need to submit docs - Need to do Pod Identity. - Docs on adding a new Scaler - Mel will add docs on how to help to add Scaler - 1.2 Release - No issues - release as expected - Docs have been updated - Were able to use the new release tooling by - https://github.com/kedacore/keda/releases/tag/v1.2.0 - @Anirudh to add link to the releases on the Keda home page - Potential Bug in the Helm Chart (Daniel) - Trying to merge the KEDA Helm chart in their own bigger chart. Using it as a Sub-Chart does not install the CRD - Maybe did not have the latest (Sandra) - Kafka trigger scaler (Rajasa) - Samples added - Should have a way for folks to have their Samples in the KEDA repo. @Anirudh to follow up - Going to add one more sample for EventHub Scaler - Non-Azure Functions workloads in other cloud (Rajasa) - Discussion on this - Depends on the Event Source ### Attendees - Anirudh (Microsoft) - Sandra (IBM) - Aleksander (IBM) - Mel (Microsoft) - Rajasa (Microssft) - Daniel (astronomer.io) - Satish (Microsoft) ## Feb 6 Meeting ### Proposed Agenda (feel free to add items) - CNCF Sandbox donation ([#501](https://github.com/kedacore/keda/issues/501) - [Proposal](https://github.com/cncf/toc/issues/335)) - Still pending approval from SIG-Runtime, pending presentation on 20th of February - Joined today's standup and Jeff is good to go to present, hopefully there are 2 chairs so they can vote. - Provide governance for built-in & add-on scalers ([#573](https://github.com/kedacore/keda/issues/573)) - Service Mesh Interface scaler (Tom - [#615](https://github.com/kedacore/keda/issues/615)) - This will allow us to autoscale HTTP workloads for all service meshes that support SMI - SMI is looking at adding this so we should collaborate with them https://github.com/deislabs/smi-spec/issues/96 - Happy to take the lead and draft out some specs if interest - Allow nested metadata properties & Azure Monitor scaler (Tom) - Do we merge it already? If so, then we'll have breaking changes pretty soon but [#613](https://github.com/kedacore/keda/issues/613) could be helpful here - Provide a changelog (Tom - [#614](https://github.com/kedacore/keda/issues/614)) ### Attendees - Jeff (Microsoft) - Sandra (IBM) - Mel (Microsoft) - Zbynek (Red Hat) - Calvin (Cyverse) - Will (Cyverse) - Lukas (Red Hat) - Jeremy (Cyverse) ### Notes - Sandra merged in labels PR, release process to automate updating versions. `Make release` - Mel making progress on the PR for monitor, just has to write the test. Going to break pod identity into a seperate. - Looking for something to look into after this one. Something that isn't super urgent but does require Kubernetes - Zbynek proposed a PR to the operator hub on 1.2 - Next plans to look at duck typing and some other changes we may want to make it 2.x. Will be out next week. - Is a proposal on the [Knative eventing PR to help scale Knative event source](https://github.com/knative/eventing-contrib/pull/886) - Discussed scaler governance - no decision made, just covered topics https://github.com/kedacore/keda/issues/573 - Operator SDK section added to readme, may be mac only? - [ ] Check operator SDK - [ ] Check SMI spec adoption - [ ] Zbynek to add some info on the zap-info ## Jan 30 Meeting 📹 [Recording here](https://youtu.be/EUH2SBrmOec) ### Proposed Agenda (feel free to add items) - Provide governance for built-in & add-on scalers ([#573](https://github.com/kedacore/keda/issues/573)) - Introduction of Azure Durable Function add-on scaler ([GitHub](https://github.com/kedacore/keda-scaler-durable-functions)) - Can we expect [v1.2 release](https://github.com/kedacore/keda/pulls?q=is%3Aopen+is%3Apr+milestone%3Av1.2) anytime soon? - FYI: very very simple [instructions](https://github.com/zroubalik/keda/tree/deployInstr#deploying-custom-keda-locally-outside-cluster) for local deployment outside cluster, Zbynek could provide demo next time if needed ([PR](https://github.com/kedacore/keda/pull/583)) ### Attendees - Jeff (Microsoft) - Anirudh (Microsoft) - Steven (astronomer.io) - Ahmed (Microsoft) - Lukas (Red Hat / Knative) - Mel (Microsoft) - Sandra (IBM) - Scott Nichols (VMware) - Ishara (BUPA) - Rajasa (Microsoft) - Daniel (astronomer.io) ### Notes - Some contributor guide pains being reported this. - Had a few conversations this week with teams internal to MIcrosoft who are planning to start using KEDA to run services - Good progress on the SQL scalers. For Airflow, Postgres and MySQL are most popular so we have those now. MySQL was merged in 3 days ago. Anirudh interested to even pick up. Steven helping now integrate KEDA / these scalers into Airflow so it can start to surface to customers. - Steven also well versed in some of the CI/CD stuff as well - On the topic of CI/CD we tweaked the blob test so it isn't as intense and passes more now - Store secrets in GitHub and then pulls them into a GitHub Action to authenticate - Potentially we could get secrets from Google / AWS to also run tests on GKE / EKS - On the Azure Monitor PR, some questions about `authParams` and `resolvedEnv` - `authParams` are the parameters / values resolved from the `TriggerAuthentication` values - `resolvedEnv` are the parameters / values resolved from the deployment. - Anirudh did have some issues getting it to build locally. But then changing code, packaging up a container, deploying it - Anirudh got stuck. - Rajasa also pointed out that experience for Windows users has a few gaps as well - Daniel brought up we should also document what variable names for consistency. A naming convention / consistency guide. - Plan to cut the 1.2 release this week - Lukas had a good question on if we should just time-based releases in general. - Knative does every 6 weeks. - github.com/google/ko - kubectl symantecs for go-lang. TGIK - 45 min of Ko - GitHub.com/n3wscott/discovery - duck-typing labels - Ishara is working on a sample for the blob trigger and plans to have the PR open soon - Rajasa was working on the Event Hub scaler - now moving to the Kafka trigger scaler with Event Hubs. Event Hubs needed a type of authentication that the Kafka scaler didn't support from the outset. Mel helped debug in Go as well. - In Event Hubs scaler it assumes the consumer is the Event Processor Host. However Kafka has the same APIs to check the checkpoints. - Secrets - was using TriggerAuthentication or w/out the TriggerAuthentication. - The log library that is being used in the Kafka scaler was not printing in logs. - VS Code remote container build was not working. Rajasa to raise an issue. - Anirudh mentioned it didn't work either - but is fixed now https://github.com/kedacore/keda/issues/568 ### Action Items - [ ] Anirudh to cut 1.2 release this week - [ ] Community to help review the [WIP PR](https://github.com/kedacore/keda/pull/584) - [ ] Jeff to create a release schedule that includes when the last commit will be grabbed, when we will cut the release, when it will go out - [ ] Rajasa to create an issue specifically around the new line characters - [ ] Jeff to reach out to Zbynek on documentation of Zap-info ## Jan 23 Meeting 📹 [Recording here](https://www.youtube.com/watch?v=8qV6PzmqYMg&feature=youtu.be) ### Proposed Agenda (feel free to add items) - Community refresh on keda.sh (Tom - [#65](https://github.com/kedacore/keda-docs/issues/65)) - Actively seeking KEDA customers for "Customers" section on keda.sh (Tom - [#551](https://github.com/kedacore/keda/issues/551)) - PostreSQL scaler ([#546](https://github.com/kedacore/keda/issues/546)) - [.NET Core Sample](https://github.com/kedacore/sample-dotnet-worker-servicebus-queue) update (Tom) - Included API for HTTP-autoscaling demo once available - Jakob Ehn is contributing a nice UI to visualize demo - Can I get more permissions to setup PR CI? - Cannot create HPA object when Deployment has missing ConfigMap (optional: true) ([#555](https://github.com/kedacore/keda/issues/555)) - Interesting issue, but need somebody with implementation experience to determine what to do ### Attendees - Jeff (Microsoft) - Daniel Imberman (daniel@astronomer.io) - Tom (Codit) - Ahmed (Microsoft) - Lance (Red Hat / OpenShift) - Lukas (Red Hat / Knative) - Mel (Microsoft) - Sandra (IBM) - Zbynek (Red Hat) - Anirudh (Microsoft) ### Notes - Basic postgres scaler merged. Plans to work on a mySQL one this week. Even got it working end-to-end with Airflow. - Interested in step scaling - Talked about some options. The main scenario Daniel looking to unblock is a SQL metric that would be like (# of running * # of rows / current). So thought is that maybe the metric could be smart enough to do metrics to help with target. Anirudh is ok with both. - Ahmed also was thinking the scalers could be smart enough to use the metrics to help influence the hpa scaling decisions. - Ahmed brought up that we would need approval to use the k8s namespace in the CRD https://github.com/kedacore/keda/issues/552. - Anirudh documented the release process for KEDA https://github.com/kedacore/keda/wiki/Release-process - Tom brought up with 2.0 we may consider breaking the ScaledObject and ScaledDeployment - Mel working on the Azure Monitor scaler. Hoping for a PR in next few days. Ran into some issues with the docs - plans to fix those as well. - Some confusion around if metadata can be nested. For Azure Monitor there are a lot of metadata properties where nesting would be nice. - Operator Hub is ready. Zbynek is going to start the process for Operator Hub. - 2.0 potentially could include the duck-typing - Mel brought up when deploying locally, if change to "PullIfNotPresent" it fails. - Daniel brought up to do the local without having to push first you could use Kind. You could do Pull Policy of "never" and it will load - If doing in the cloud, you will need to push to a repo - Daniel interested in looking into "Telepresence" tech to try out improving the dev cycle for scalers ### Action Items - [x] Jeff to create Allow nesting of metadata - [x] Jeff to create updating the scaler doc and assigning to Mel - may be related to https://github.com/kedacore/keda/pull/556 - [ ] Zbynek to add docs and look into the operator hub release process - [x] Jeff to create an issue around duck-typing for 2.0 - [x] Jeff to give Tom permissions for CI / CD - [x] Daniel to create docs for Postres sql on KEDA docs ## Jan 16 Meeting ### Proposed Agenda (feel free to add items) - CC 1.1.0 Release ([draft](https://github.com/kedacore/keda/releases/tag/untagged-06ea7afe5503e368cc13)) - https://github.com/kedacore/keda/issues/545 - Interest in postgres scaler - [CNCF Proposal](https://github.com/cncf/toc/issues/335) ### Attendees - Daniel (Astronomer.io) - Ishara (BUPA) - Lance (Red Hat / OpenShift) - Lukas (Red Hat / Knative) - Rajasa (Microsoft) - Sandra (IBM) - Israel (Microsoft) - Tom (Codit) ### Notes - Release 1.1.0 draft release out. Action to confirm that all commits that are needed are in there - Rajasa Event Hub stuff seemed to be working now. Seemed to work fine on further testing. Some variability in how it was scaling. Some further testing coming - and a sample in the works. - Ishara to update docs. Jeff to add comment on issue. - Daniel starting work on the Postgres scaler. Will create issue, Jeff will assign. - Question on using custom queries, and not stats yet today. Daniel use case is a bit like "what are the number of items that are set to this state". - Another question on if any thought on creating a generic JDBC type scaler so not limited to just Postgres? Thinking is for getting a quick version out, a Go scaler specific to Postgres. You would just provide the SQL query e.g. "SELECT COUNT() FROM FOOTABLE WHERE STATE LIKE ACTIVE" (Jeff forgot SQL syntax). Daniel has been looking to use a generic Go database driver, so potentially this could be extended to other databases? Israel is open to test this. Daniel will keep this in mind. - Partner logos - there is [an issue](https://github.com/kedacore/keda/issues/539) you can comment if willing to add your logo. Tom brought up a great point we can make it really easy for people to open a PR to add their logo to the docs site. - Jeff needs to make sure Anirudh and Ahmed provide data for missing sections of proposal - Question about what CNCF sandbox means - more or less we can keep operating standups the way we are today, but means we give up the trademark and rights to KEDA to the community. In general though we can do things like recording meetings and sharing better :) ## Jan 09 Meeting ### Proposed Agenda (feel free to add items) - Updates - Discussion of PRs and progress on project - Release 1.1 - Schedule & status (https://github.com/kedacore/keda/milestone/5) - New items and PRs - Recap KEDA meeting yesterday ([recording processing](https://youtu.be/pWSeZ-Tn4I0)) - Update docs for contributing scalers ([issue](https://github.com/kedacore/keda/issues/531)) - Knative - [StatefulSets]( https://github.com/kedacore/keda/issues/497) - CNCF Proposal ([draft](https://github.com/tomkerkhove/toc/blob/keda-donation-proposal/proposals/keda.md)) - Attendees - Ishara - Mel - Rajasa - Sandra - IBM - Notes - Azure Blob is still pending review by Ahmed. There is a limitation (same as Azure Functions has) if container > 5000 but we can add to docs - Mel working on the Azure Monitor scaler. Expecting by next week could have a PR. Would be good to consider the pod identity (Azure Queues and Azure Service Bus both support pod identity). - Rajasa - outstanding PR merged last week. Need this PR to be merged in [here]( https://github.com/kedacore/keda/pull/537
), docs also merged in. There is a bug they are hitting an issue scaling from 1 -> n doesn't seem to be working. Also has a working sample that ## Dec 19 Meeting ### Proposed Agenda (feel free to add items) - Updates - OperatorHub.io progress (Zbynek) - https://github.com/kedacore/keda-olm-operator/pull/1 - Blob support PR (Tom/Ishara) - Keys support for Azure Functions (Anirudh) - We discussed changing the frequency of the meeting to twice a month but decided to hold off until later in 2020 (Febuary) - Zbynek will send another PR for the documentation of the operator. - Israel to take a look at the first pull request for the operator (keda-olm-operator). We need more reviewers before it is publish - Anirudh has a pull request for Keys support for Azure Functions in the functions repo and should submit the PR early next month (January 2020) - Rajasa is currently working on making the EVH scaler work with Java and .NET apps as well. Support for python is not yet available. She will submit the PR for the changes. This is the PR for Event Hubs Scaler to work with Java and .NET applications https://github.com/kedacore/keda/pull/517 - Mel will submit pending changes into a branch and will resume after the break. - We will skip the next 2 meetings and resume on January 9th. - Attendees: - snichols - Zbynek - Mel Cone - Tom Kerkhove - Anirudhg - Rajasa Savant (RJ) - Lukas Berk - Ishara Pannila - Florian - BBrowning - Israel Ekpo ## Dec 12 Meeting ### Proposed Agenda (feel free to add items) - func core tools support (Anirudh) Keys support Http request based scaling - Feedback from Scott on looking at code - Do we have issues for these ? - Scott and Zbynek will sync and take ownership. - Move community standup to bi-weekly basis (Tom) - If anybody has objections, please go to #502 - Suggestions to have 30 mins every week ? - Bursting Jobs in Azure Container Instances (Matt Pasco) - https://github.com/mattpasco-microsoft/KEDA-ACI - Matt can confirmed Can burst out Jobs to Virtual Nodes - OperatorHub.io progress (Zbynek) - Closed the PR for it - 90% done. - From our side all work would be completed. ## Dec 5 Meeting ### Proposed Agenda (feel free to add items) - Standup Calendar (Tom) - You can now add our standup calendar to your client to see if it's cancelled or not - https://calendar.google.com/calendar?cid=bjE0bjJtNWM0MHVmam1ob2ExcTgwdXVkOThAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ - Product leads can ping me to get access - Release automation (Tom) - We should focus on streamlining our releases - Introduce `experimental` tag for master builds - Introduce formal release with GitHub releases (on-demand trigger) - Docs on what we have and when to use what - See milestone: https://github.com/kedacore/keda/milestone/6 - 1.1 release features / milestone - Suggestions - kedactl (Tom) - Dashboard (Tom) - CNCF Donation (Tom) - Support for StatefulSets ### Notes - Google Calendar for the meeting calendar - Release Automation - Push to master - creates master image and updates the master tag on Docker Hub. - CI/CD on push to tag. - Need to update the Charts. - Add an issue to update Charts automatically. - Add docs. - Suggestion to just tie to a release. - Issue #439 - Currently not on Operator Hub - Self Signed Certificate issue ? - We have an open PR - Zybnek will have to done relatively soon - CI/CD Should also push to Operator Hub. - Suggestions for the next release - Final release plan should be done via https://github.com/kedacore/keda/milestone/5 - New Authentication spec for Scalers - New scalers - Azure Monitor as a fallback - Pod Identity - AWS, Google others - Fix the dashboard for KEDA - 1.0 - Sekhar - Anirudh will create a issue and assign to Sekhar - Suggestion for having its own CLI - Krew plugin. - Shekar would love to have some integration with Knative for which Scott has joined - We could produce a scalable "Duck type" (ie. PodScalable) that can be scaled upon and allow Knative & KEDA to collaborate. - We should setup a Knative & KEDA meeting to discuss this further (invite snichols@vmware.com) - Knative would love to use KEDA for queue-based scaling, KEDA could use Knative for request-based scaling (which they already have) - See more in Scott his talk from KubeCon - https://www.youtube.com/watch?v=kldVg63Utuw - Support for StatefulSets - Ahmed to review - Tom has started working on the CNCF proposal and has pinged Jeff for an early review - Tom will create a GitHub issue for it to track the work - Ishara will create a PR for a blob scaler, will scale on new blob created - Thanks! ## Nov 28 Meeting ### Cancelled for US Thanksgiving 🦃 ## Nov 21 Meeting ### Cancelled for KubeCon North America ## Nov 14 Meeting ### Proposed Agenda (feel free to add items) - KEDA 1.0 (jeffhollan) ### Notes - KEDA 1.0 main open items: - Docs (Tom and Jeff) will chomp on this - Operator SDK and Operator Hub [issue](https://github.com/kedacore/keda-docs/issues/8) - Manual release of 1.0 by Ahmed (sometime before Monday) - Operator Hub still has some work items - A special procedure that Operator Hub has for installing API service that it's expecting. Operator Hub is looking for a `kind` and since it doesn't see it, it tries to re-install. Zbynek talking to the hub team to figure out best way to solve. This is limited to only the Operator Hub tooling support. - Zbynek has an issue on the hub team: [issue](https://github.com/operator-framework/operator-lifecycle-manager/issues/1122) - - Stretch goals - Auth spec gap - KEDA Dashboard ## Nov 7 Meeting ### Proposed Agenda: - Discuss KEDA 1.0 Release - Updated Helm chart so that it works with the two containers and Operator SDK - done - Set up CI through GitHub Actions for any merged PR's to master which will currently update the docker images (0.0.4) - done - Nightly tests fix - to be done - Update the readme - to be done - Version strategy - to be decided - currently we think that we will have a 1.0 stable release - different sets of helm charts for stable and dev. - Breaking changes fixes for Rabbit MQ - to be merged - Pod identity for Service Bus - to be done by Ahmed - Zbynek - First draft of the manifest file for the Operator Hub - Will show demo - Ahmed - Helping with CI stuff, Operator SDK is merged in. Enabling the End to End tests have to be enabled. Looking at PR's as they are come in. - Have to come up with a support plan. - Operator Hub has many models for support - Partner - Red Hat - Community - Suggestion is to consider this a Community. - Goal is to have KEDA submitted as a Sandbox project in CNCF. ### Proposed Agenda (feel free to edit and add): - Twitter-based scaler: Monitors specific accounts for specific words, and scale accordingly. For real-case scenarios we might use different Twiter APIs, this can be a simple demo for the community. Work has been commencing on the following branch, intention to create a PR soon. [https://github.com/eashi/keda/tree/twitter-scaler](https://github.com/eashi/keda/tree/twitter-scaler) - Stream live development of the Twitter scaler (above) on Twitch. I have streamed before other stuff but sadly there is no recording to demonstrate. My plan is to start next week, maybe Thursday 14th. - Started documenting writing scalers (the old way, not the through External Scalers). The first step was PR [#395](https://github.com/kedacore/keda/pull/395). - Keda on OperatorHub.io ## Oct 31 Meeting ### Proposed Agenda (feel free to edit and add): - PR #407 (https://github.com/kedacore/keda/pull/407) - Fix for Azure Queue Scalar AAD Metadata (stgricci) ### Notes * CSE has a hackathon coming up and are looking to extend the jobs capabilities. (Customer technical engagement team) * Need to validate that the dashboard works with v1, and if it doesn't quickly fix. Mark as help wanted. Potential table: | Scalers | Status | Sponsor | | --------------- | --------- | ------- | | Azure Event Hub | Official | @Azure | | RabbitMQ | Official | @Azure | | NATS Streaming | Community | | | Kafka | | @Azure | The incoming AMQ scaler would be @RedHat ## Oct 24 Meeting ### Proposed Agenda (feel free to edit and add): - CNCF Working group - Operator SDK work - Kafka and Event Hub setting max scale out? ### Notes ### To do items - [ ] Anirudh to look at and sort out a contributor guide - [ ] Jeff to add info on how to chat on Slack (Kubernetes) - [ ] Satish / Anirudh to fix the helm chart - [ ] Ahmed ## Oct 17 Meeting ### Proposed Agenda (feel free to edit and add): - Operator SDK work ### Notes - Zbynek has the PR ready to go, just having a hard time opening because of GitHub PR stuff. - It's using structured logging now - Seperated the metrics adapter and the activation into seperate - For jobs, we still use the same ScaledObject. Still would like to see split in the future, but for now it is the same - Rest of big changes are related to using the Operator SDK - Once the PR is merged, Zbynek can put this on OperatorHub.io. Only thing that is needed for that. - Ahmed working on refactoring the scalers to use the new auth spec, and getting pod identity to work with Service Bus scalers ## Oct 10 Meeting Today unfrotunately due to other commitments there wasnt any particpation in the call other than Anirudh. Just adding notes for the offline discussions we had during this week on KEDA. ### Proposed Agenda (feel free to edit and add): - None ### Notes - Zbynek had earlier indicated that he was close to getting the Operator SDK done. He had proposed to split the HPA and the Activator into seperate containters within the KEDA pod which we decided to go with. Further he had to refactor his implementation to work with the implementation where-in job scaling was included with the deployment scaling which Zbynek was ok with. ## Oct 3 Meeting ### Proposed Agenda (feel free to edit and add): - None ### Notes - Base auth spec is implemented, Ahmed looking to add to the queue scalers - Ahmed working on a GitHub Actions to automate release - Ahmed also looking at the Jobs PR sometime this week (Steven pingd him) - Shekhar had a question on how we deal with changes of the auth spec CRD. Thinking is that every time we do a check we'll get the latest version of each to do the check. - Zbynek still working on Operator SDK. Hoping to have a PR in the next couple of days. Uses some different logic for looping around resoures, so lots of changes to the operator loop that will need to be reviewed. - In terms of the authentication spec - it will be in the PR Zbynek is opening (CRD registration) - Also a question on the aggregate API server (listens to metrics). While not part of the SDK, Zbynek reusing code to light that up. - Cliff has a working Artemis scaler on personal GitHub, and docs on how to use it https://github.com/cliffjansen - Ahmed going to check out the bug on Helm chart deployment ## Sept 28 Meeting Proposed Agenda (feel free to edit and add): - Experimental Jobs Functionality (Added by Tom) - Testing out AAD Pod Identity ### Notes - Ahmed worked and merged in the PR on Authentication spec. He will be looking at some of the Azure scalers to implement it end-to-end - Discussion on if it makes sense to seperate or keep the same. Especially with 1.0 coming. Steve wrote most of the code and felt the amount of code share was high enough to justify same. ```yaml apiVersion: keda.k8s.io/v1alpha1 kind: ScaledObject metadata: name: azure-queue-scaledobject-jobs namespace: default spec: pollingInterval: 40 maxReplicaCount: 4 cooldownPeriod: 10 jobTargetRef: parallelism: 1 completions: 1 activeDeadlineSeconds: 60 backoffLimit: 6 template: containers: - name: consumer-job image: sgricci/queue-consumer env: - name: AzureWebJobsStorage valueFrom: secretKeyRef: name: secrets key: AzureWebJobsStorage triggers: - type: azure-queue metadata: queueName: queue-name connection: AzureWebJobsStorage ``` - During Sean's expirementation, using Managed Identities (Pod Identity), the KEDA scaler would get a token using Pod Identity, and try to use that token to pass to storage, and spin back an error that said the format was invalid. To workaround, they kill the KEDA pod during deployment and when it recreates everything works (have to delete the pod whenever a new scaled object is created) - [issue here](https://github.com/kedacore/keda/issues/368) - Document the new tagging strategy ## Action Items - [ ] Steve to make discussed changes to PR, ping Ahmed, - [x] Sean to create issue around the "have to delete KEDA" pod identity bug he ran into - [ ] Ahmed looking at potentially doing GitHub release workflow to auto-tag. He'll look at this during the week as well ## Sept 12 Meeting Proposed Agenda (feel free to edit and add): - Merge in external scaler (if not done ahead of time) - Operator SDK - needs help - Prototype for ActiveMQ Artemis - Where to put samples ### Notes - Tagged a version of KEDA to :0.2 which is a "safe" version before we start to do some breaking changes. - Zbynek can start working on the operator SDK - but did flag that it may require some big changes in things like how the controller loop would function, so he wanted to make sure we were good with this change before taking the effort - https://github.com/operator-framework/operator-sdk - Potentially could look at the namespace feature as part of the operator SDK work. One of the "out of the box" features of Operator SDK - We may need some specific ## Action Items - [ ] Get Yarons branch - [X] Jeff to create README on samples - http://github.com/kedacore/samples ## Sept 5 Meeting ### Notes * Finalized and merged in the auth spec PR * Shekhar making last changes to the external scaler support. He has a bunch of samples with those * Shekhar going to look at implementing AAD Pod Identity basic support * Aarthi looking at her issues as well * We got a session at KubeCon San Diego ## August 29 Meeting Proposed Agenda - Finalizing the proposal for defining authentication parameters - Jobs on KEDA - Current Status, Community Input, Discussion on how to proceed with docs - Making progress on the External Scaler support for Keda - Any other issues ### Attendees Ahmed Shekhar Tom Steve Anirudh ### Notes - Finalizing the proposal for defining authentication parameters: - There are two different approaches suggested on the PR - whether authentication is part of the trigger (or) authentication is via another newly defined entity (Trigger Authentication approach). - We debated the pros and cons of both approaches and decided that we will go for the seperate entity approach. However this will mean that we will have to change all the scalers to use this. (will open issues for this) We also spoke of versioning in this context. For now we decided that we will *not* version and continue with the current version. - External Scaler support for Keda - Ahmed has reviewed it and has a minor comment which Shekhar will fix - Shekhar will also add a sample - Tom will also have a look at it. - Overall looks good to mer - Jobs functionality - Came out of the work by Microsoft CSE for a customer who is using KEDA in production. Ahmed had a look at it and did not find any issues. Use case is events come in -> GCP/Azure queue, containers run to completion (long running jobs). Shekhar was added for the review. - Steve will post an example spec for Azure Queue for this. Tom was wondering if we wanted to consider making this a different CRD. He will give some comments. No changes in the Scalers themselves. - Overall the feeling was that we have to have better governance around new issues. Also would want to see if we want to identify owners for different scalers. ### Action Items - [ ] Shekhar will submit a PR for the Authentication parameters work in the scale controller. - [ ] Tom will create different issues for each scaler for the Authentication work. - [x] Tom will update the PR for Authentication parameters work saying that we decided the seperate entity approach and ask for final comments on the approach. - [ ] For external scaler support - Shekhar to fix minor comment from Ahmed and submit for merging - , he will also add a sample. ## August 15 Meeting Proposed Agenda (feel free to edit and add): - External Scaler support walkthrough (Shekhar) - Dashboard community support [repo here](https://github.com/kedacore/dashboard) ### Notes ### Action Items - [ ] Jeff to Setup an additional meeting on extensible scalers directly - [ ] Jeff to Add known issues with help wanted flags to dashboard repo - [ ] Priya / Lee to take a look at the auth one. They'll meet with Sean and see what issues they could help with - [ ] Jeff to track down Yaron directly and understand operator SDK. ### Action Items ## August 8 Meeting [Meeting recording](https://youtu.be/_Gz3ZnXL07g) - KEDA Dashboard demo starts at 05:30 timestamp Proposed Agenda (feel free to edit and add): - KEDA Dashboard demo (alpha) - Shazmin - **Viewable in meeting recording above** ### Notes - Israel working on some more tutorials on scheduling (videos) - Shazmin demod the dashboard for KEDA. Will post recording. - Shekhar brought up we likely want to integrate with prometheus by default to pull the logs - Shehar also brought up we should look into authentication options - mirroring KubeDashboard is likely a good start (kube context or headers) ### Action Items - [x] Aarthi and Ahmed about reviewing PRs ## August 1 Meeting Proposed Agenda Items (feel free to edit and add): - Image releases - HTTP Scaling Sample - Helm Chart Notes - KEDA Job Management (issue #199) - AAD Pod Identity / Getting secrets - 1.0 release overview ### Attendees * Aarthi * Jeff * Joey - Microsoft * Lee * Priya * Sean - Microsoft * Steve - Microsoft * Tom Kerkhove * Zbynek * David ### Notes - Aarthi - Shazmin almost done with a KEDA dashboard. It still needs to be integrated with a better logging story that can scale but it looks good. Next week will demo and get feedback. Plan is to create a seperate repo for the dashboard, and extend our helm chart to also pull in dashboard. - Some discussion on how best to package up in a helm chart and depending on other charts. We do something like this today with Osiris. - CLI would likely pull from the same "APIs" that the dashboard would pull from. - [Tom held a meeting and created a PR on AAD Pod Identity](https://github.com/kedacore/keda/pull/307) that the community should review and provide feedback. - Some additional work done on the job dispatch work. Steve has been working on [Lee's concept](https://github.com/lee0c/k8s-jobs) and is looking to add the PR for supporting jobs. ### Action Items - [ ] Community to review and provide feedback on authentication concept - [x] Jeff will help break out into issues per scaler - [ ] Jeff to create issue and figure out scalable way to do docs, FAQ, and samples - [ ] Consider HTTP samples as well - [ ] Helm chart notes should be updated to point to this when available - [ ] Lee to create an issue for HPA labeling - [x] Jeff to assign jobs issue `seushermsft` - [ ] Aarthi / Ahmed / Jeff to review the gRPC PR, AAD Pod Identity, Jobs (to be created) ## July 25 Meeting Proposed Agenda Items (feel free to edit and add): - Scaler & Trigger specs - AAD Pod identity update - Pluggable scaler ### Notes * Scaler & Trigger specs are moved to [Git repo](https://github.com/kedacore/keda/tree/master/spec) * Allows non-Microsoft folks to send PRs with proposals * Owners of scalers should provide docs in the spec section * Proposal PR for authentication changes in scaler spec is [open](https://github.com/kedacore/keda/pull/307), feedback/concerns are welcome * Pluggable scaler will re-review it and ping other folks in Microsoft * You can write scalers in any language that comply with the gRPC scheme * Scalers can be shipped seperately * Allows to create a community of scalers * Might need a portal over time to group all of the scalers in the wild * Similar to hub.helm.sh who are open for community Helm charts * Progress on portal is pretty good, demo might follow next week * What is the authentication that will be supported? * To be asked next week * Will be seperate pod which you can deploy ## July 18 Meeting Proposed Agenda Items: - Status updates - Technical meeting for AAD Pod Identity is being planned, see [GitHub issue](https://github.com/kedacore/keda/issues/161#issuecomment-512339124) ### Notes * Priya has been looking at doing a PR - not for AAD Pod Identity - but allowing both allowing both env variables or secret config for connection strings. Can potentially align this with the AAD Pod Identity spec from Tom K. (#161) * May just be a workaround and additional option until AAD Pod identity or something non-connection string specific is enabled. Open question on how you could authenticate with things like AWS SQS if you don't want to put connection strings directly in a configmap or secret. * Shekhar put in a PR for AAD Pod Identity for storage queues * Aarthi mentioned we may need a meeting on the spec change from Tom * Similar approach could be used for Azure Service Bus * Lee brought up that maybe we could design in a way that wasn't specific to each scaler specifically * Aarthi was looking to understand how the Azure Functions container could successfully "fail" so that Kubernetes and KEDA could cleanly have a fail-mode to prevent scaling. * KEDA dashboard progressing from Yazmin. She's making some updates to logging so that's it's parse-able by the UX * Hoping to have a first release of the dashboard in next 2-3. * Potentially do a demo in next week or following week for demo. * :warning: folks who are contributing may need to rebase * Shekhar opened a PR and proposal for pluggable scalers #294 * Anirudh working on FAQ to release in a couple of days * Event Hubs had some minor bugs, update has been pushed out. * Some discussion about how frequently we release and update `:latest`. Maybe for now we do once a week. * To help mitigate around CI/CD with scalers connected to many sources, some may be "core" scalers, and others become "extensions" that could be versioned seperately? * Today once a day we have some extensive E2E tests that runs in a Microsoft-sponsored Kubernetes cluster. A description in the `/tests` folder of the project ### Action Items - [x] Schedule a seperate meeting specifically for Tom K. spec review. Tom to schedule - [ ] Shekhar to move some of the pub-sub samples into the KEDA org - [ ] Jeff to add Shekhar to the KEDA Organization for rights - [ ] Everyone to provide feedback on the event hubs scaler ## July 11 Meeting Proposed Agenda Items: - AAD Pod Identity Configuration ([#161](https://github.com/kedacore/keda/issues/161) - Tom Kerkhove) - Check on the process for contribution to the KEDA repo - Technical review to be scheduled for the AAD Pod Identity. Add Aarthi, Ahmed, Anirudh and Israel as optional. - KubeCon workshop interest in participation. - Question about Durable Functions supported on KEDA? - Not yet but on the roadmap. ## July 4 Meeting Cancelled - US Holiday ## June 27 Meeting Proposed Agenda Items: - Intros - Status Updates - Event Hubs Scaler Demo ### Attendees Rhuss (RedHat), Lance (RedHat), Lee, Shekhar, Israel, Shazmin ### Notes - https://github.com/kedacore/keda/issues/270 - Asavari to follow up - Clarification around where to open issues for components not specifically located in the kedacore/keda repo - we should open issues in the specific repository where the code is located but no harm crosslinking to the core repo for triage during weekly standups. - Shekhar expressed interest in helping with the dashboard implementation, along with Shazmin. - Shekhar finished the scalers for Redis and GCP Pub/Sub. Working on creating samples. ## June 20 Meeting Proposed Agenda Items: - Intros - Status Updates - Proposed topics (feel free to edit and add topics) - AAD Pod Identity ([#161](https://github.com/kedacore/keda/issues/161)) ### Attendees Tom, Asavari, William, Yaron ### Notes - Reached a 1,000 stars for the repo! - AAD Pod Identity - Not a blocker but we're interested in having it align with the 1.0 release to serve multiple customer scenarios. Help wanted on the issue. - Need to create GitHub releases for the KEDA image. Already have CircleCI but need to have nightly builds to update the master. Asavari to open an issue. - Yaron's going to provide a set of scenarios for the Prometheus samples. ## June 13 Meeting Proposed Agenda Items: - Intros - Status Updates - Proposed topics (feel free to edit and add topics) - .NET Core 3.0 Worker sample is added (#216 - Tom K.) - Status update on Helm chart improvement ### Questions - Is there a timeframe in which KEDA v1.0 is aimed to be released? (Tom K.) ### Notes - v1.0 is tracked as part of the project here - https://github.com/kedacore/keda/projects. If the current momentum continues, we're looking at H2 for the release. - (Tom K.) Opened a PR to document the helm chart configurations. Logging setup needs review. Ahmed ElSayed is looking into it. - (Tom K.) Open question - Is AAD Pod Identity in scope for v1.0 - https://github.com/kedacore/keda/issues/161. Asavari to follow up. - (Shekhar) Made a PR for SQS scaler. Will be looking into providing a sample next. - (Yaron) Operator SDK work is blocked on https://github.com/kedacore/keda/issues/54. Need help verifying, Yaron will open an issue. - Event Hub scaler work progressing well. Validation in progress. - PR already open for the Prometheus scaler. Next step is to provide sample queries for HTTP and Azure Monitor. Asavari to open an issue. ## June 6 Meeting Proposed Agenda Items: - Intros - Status Updates - Proposed topics (feel free to edit and add topics) - KEDA Dashboard (Asavari) - Long running feedback (Lee) - .NET Core 3.0 Worker sample (#216 - Tom K.) - Feel free to review so we can transition it to the KEDA org - Questions - Is there a timeframe in which KEDA v1.0 is aimed to be released? (Tom K.) - Notes - Event Hubs SDK work progressing well. - Shekhar interested on providing some samples - William brought up a good point that we could look at adding a VS Code plugin. Created a placeholder issue, but should look into see what's possible, potentially even extending the existing Kubernetes extension somehow? Assigned to Asavari and myself to investigate with the VS Code team. - Lee been working with customer on long running workloads. It is *possible* to have a very long running SIGTERM delay. It sets the status as "terminating" which can be a bit confusing for the user. Also looked at using Kubernetes jobs (run to completion), has a POC of that working. Does run to completion which is nice - so job status can be trusted. * Proposal to allow KEDA to act as a job dispatcher and cleaner based on metrics * Potentially scoped to queue-only triggers? Or at least optimized for it - Action Items - [x] Jeff to add questions on job scaler to issue - [ ] Lee to use issue #199 for a proposal of the job dispatch functionality ## May 30 Meeting Proposed Agenda Items: - Intros - Status Updates - Proposed topics (feel free to edit and add topics) - KEDA Dashboard (Asavari) - KEDA and Jobs (Lee) - 1.0 Release review - Notes * Shazmin working on Event Hubs Go SDK support for checkpointing and partition data * Aarthi working on certs and auth work items, to give update on other ones * We have the release project setup with the deliverables currently in scope for 1.0. We broke off a seperate column for items that are more focused on the "Azure Functions on K8s" feature-set which may extend past KEDA. * Lee had a meeting around jobs and scaling. Thinking was to intercept SIGTERM and delay terminiation until long running job completes. Similar suggestions in the sig-autoscaling K8s group - Action items - [ ] Jeff to email lee about long running approach ## May 23 Meeting Proposed Agenda Items: - Intros - Status Updates - Proposed topics (feel free to edit and add topics) - Demo of KEDA autoscaling Knative Kafka Event Source (Ben B.) - Status of the 1.0 release project ### Attendees Anirudh, Jeff, Aarthi, Ben, Israel, Lee, Asavari ### Notes * Ben showed the work he'd been doing with Knative. A few options on how to do this. One of them was to just integrate with Knative and KEDA with no changes and let KEDA help drive the scaling of the Kafka consumer. Also another world where Knative could also support "container pull" based deployments. * Lee had some asks around supporting long running jobs and some PoCs around [this issue](https://github.com/kedacore/keda/issues/157). For the "Kubernetes Jobs" strategy it looks like the pattern would be 1 queue message per job. TTL is still in alpha, looking at cron jobs as well. Doesn't seem to be a very good way to remove completed jobs or select on status. Ben also brought up the point where you may be able to modify the label so that the pod is no longer part of the deployment. HPA may react to that but at least the pod would continue to live and KEDA could then terminate the pod? ### Action Items - [x] Jeff to talk to Cosmos DB team about Go support for their SDK - [ ] Jeff to talk to Ahmed about reviewing PR - [x] Jeff to create issue about kubevalidator and helm - [x] Jeff to flag Aarthi on the multiple signal issue ## May 16 Meeting Proposed Agenda Items: - Intros - Status Updates - Proposed topics - `Needs Discussion` tag - Release criteria label / project - Operator SDK / hub - Support for multiple resources under same metric (#194 - Tom K.) - CloudEvents & Kubernetes Events (#165 - Tom K.) - Streamlining the Helm Chart (Tom K.) - Long running workloads ### Attendees Jeff (MSFT), Ben Browning (Red Hat), Tom Kerkhove (Codit), James Sturtevant (MSFT), Lee Cattarin (MSFT), Thiago Custodio (MNEO), Ahmed ElSayed (MSFT), Israel Ekpo (MSFT), Yaron Schneider (MSFT), Asavari Tayal (MSFT), Luke Kim (MSFT) ### Notes * Ben looking to open PR around autoscaling and Knative * Loop into the HPA "scale to zero" capabilities that are slated for 1.15 alpha * Some discussion around the "Long running workloads" work item. Issue has details * CloudEvents driving scale in Kubernetes. Regardless there is likely a CloudEvents SDK that could be leveraged, but still an open question of how the event makes it way to KEDA. Knative auto-TLS is the way that Knative has done this as well. Hybrid connections? * Operator Hub / SDK - will have an update for next week. Most of code already written but have to sort out dependency conflict. Likely land by mid-June. Ben is working to get things on operator hub right now that are doing more around operator hub. * Helm charts - adding KEDA to hub.helm.sh (Tom) * Scaling a single resource on multiple event sources (multiple queues that all scale a single deployment) * **Everyone** start to flag issues that should move to the "RELEASE" milestone for the 1.0 release ### Action Items - [x] Jeff to link to the HPA capability for scaling to zero on Prometheus issue - [x] Jeff to create an issue on aligning with the HPA scale to zero for custom metrics - [x] Jeff to create an issue and assign Tom to add KEDA to hub.helm.sh - [ ] Israel to create a PR and sample for Kafka. Potentially also including how you could scale kafka cluster ## May 9 Meeting Proposed Agenda Items: - Intros - Status Updates - Proposed topics - Azure Event Hubs trigger - Dashboard - Operator SDK - OperatorHub.io - Prometheus ### Attendees Jeff, Aarthi, Asavari, Anirudh, Yaron, Israel, Tom, Eduardo, Luke ### Notes * Yaron is going to be working on the Prometheus scaler so that you can scale based on any prometheus scaler, working on the string / int mapping, operator SDK would be later * Aarthi is working on the Event Hubs scaler, and look to validate the minikube work * Anirudh to work with Aarthi * Some discussion about Kubernetes Event scaler * Israel to work on some samples on things like Service Bus * Tom will provide a sample of using the Prometheus scaler with Azure Monitor once Yaron finishes ### Action items - [x] Jeff to create an issue for minikube - [x] Jeff to create an issue for Kubernetes Events and add help wanted - [x] Jeff to create a doc issue for documenting the HPA - [x] Jeff to create a management dashboard issue - assign Asavari - [x] Create item for FAQ (likely would be a wiki). Is KEDA Azure Functions only? How does this relate to Knative? I'll assign Anirudh - [x] Jeff to create an item on a sample on Azure Monitor -> Prometheus - [x] Jeff to create an issue to add samples for triggers we don't have samples for. Israel interested in helping with Service Bus samples (izzymsft)