--- tags: keda --- # KEDA Community Standups ## Aug 6th Meeting ### Proposed Agenda - 2.0 Release (T) - Let's not forget the config alignment - Alpha & full release planning? - Proposal - Alpha next week, full end of Aug if possible - Number of replicas remains the same after deleting ScaledObject (T - [#947](https://github.com/kedacore/keda/issues/947)) - Maintain last known state if prometheus is unavailable (Z - [#965](https://github.com/kedacore/keda/issues/965)) - Migrate to the new Kubebuilder based Operator SDK framework (Z - [#969](https://github.com/kedacore/keda/issues/969)) - Any volunteer to implement Script to do the migration from v1 to v2? (Z - [#946](https://github.com/kedacore/keda/issues/946)) - Standup authentication changes with Zoom (T - [#963](https://github.com/kedacore/keda/issues/963)) - Cluster Autoscale ### Attendees - Jeff Hollan - Zbynek Roubalik - Travis Bickford - Tsuyoshi - Patrick McQuighan - Tom Kerkhove (Codit) ### Notes - 2.0 release is almost done - Migration guide is on its way [link](https://keda.sh/docs/2.0/migration/) - Fix job related problems and ship alpha next week - Full release at end of the month, maybe sooner - Tom will start working on the blog post next week ([WIP](https://docs.google.com/document/d/1vhT-wwEPPNImFmLZHGJxdKjUvKWk6ckeE7QCLSOJzLY/edit?usp=sharing)) - We should try to get the samples up-to-date based on the alpha version and assign an owner or archive them - Number of replicas remains the same after deleting ScaledObject - People agree that it should scale back to the original number - However, Kubernetes & HPA without KEDA use the same approach - [ ] Tom will update the issue to keep current approach and give option to scale back to the original instance count - Maintain last known state if prometheus is unavailable - Prometheus metric adapter will maintain the same instance count, while KEDA will scale back to 0 if it doesnt' have the count - HPAs receiving errors from metric servers stop taking any action; this seems liek a safe aproach for us. Otherwise we can create autoscaling loops flooding whole cluster. - This is how it works today so we are good to go - We shouldn't do 1 -> 0 nor 0 -> 1 scaling if we cannot fetch metrics - Would be nice for 2.0, otherwise afterwards - Migrate to the new Kubebuilder based Operator SDK framework - We are using Operator SDK which is being merged with Kubebuilder, we should check if we can migrate - Has no functionality changes but just reshuffling of the code - Will be done for 2.0 stable release - Any volunteer to implement Script to do the migration from v1 to v2? - Travis will look at this - Aiming for full release - Standup authentication changes with Zoom - We need to migrate before Sept 21st - We'll use password for meetings - Cluster Autoscale - A few folks at Amazon are managing EKS and looking at KEDA for event-driven autoscaling - It won't be easy for cluster autoscaler as they don't provide APIs - Long-requested feature for Kubernetes upstream - Still in exploration phase - Related - https://github.com/kubernetes/autoscaler/issues/3182 - e2e tests - They seem flaky, what cluster are we running on? - Jeff will check with Ahmed - That's why we need cluster autoscaler integration ;) ### Action Items - [ ] Jeff will switch to standups with password ## Jul 23rd Meeting ### Proposed Agenda - External metric provider via HTTP ([Issue](https://github.com/kedacore/keda/issues/929) | Tom) - Will we support this? What about the [proposed OpenAPI spec](https://github.com/kedacore/keda/issues/929#issuecomment-658718989)? - What about [proposal](https://github.com/kedacore/keda/issues/929#issuecomment-658718989) concerning renaming external, external-push and external-http? - Provide support for explicitly stating workloads to scale to zero. ([Issue](https://github.com/kedacore/keda/issues/944) | Tom) - Migration from v1 to v2 (Travis) - v2 ScaledJob progress (Tsuyoshi) ### Attendees - Zbynek (Red Hat) - Travis (Shutterstock) - Ahmed (Microsoft) - Tsuyoshi (Microsoft) ### Notes - Live migration from v1 to v2 - would be hard to achieve because there could be only one Metrics Server in the cluster and live migration would require 2 running instances in the same itime - Migration guide is in the progres (Zbynek) - script to convert ScaledObjects in the cluster from v1 to v2: https://github.com/kedacore/keda/issues/946 - v2 ScaledJob - first part of the work is in the PR (refactoring step) - rest or the work will follow in subsequent PRs - adding a retention period to clean to resources - Tsuyoshi will add Getting started guide for ScaledJob - once there is a working version, we are good to release KEDA v2 beta - External metrics provider via HTTP - should not be a problem to proceed with this and add it as a proper scaler - authentication needs to be solved - Ahmed to follow on that issue - Provide support for explicitly stating workloads to scale to zero - should be easy to implement the scaled to zero, adding a separate CR would be an overhead, adding an option in the spec to disable scaling should be more convenient - "freezig" the scaling on a particular replica count (ie. disable scaling and keep metrica count on 10) is much more difficult to achieve, because we would need to fake metrics for metric server. As a workaround user could set minReplica to needed value and use the option proposed in the previous point - would be nice to implement this before v2 is released ### Action Items - [ ] to look and comment on https://github.com/kedacore/keda/issues/929 (all) - [ ] to look and comment on https://github.com/kedacore/keda/issues/944 (all) ## Jul 9th Meeting ### Proposed Agenda - KEDA v1.5 is out πŸŽ‰ - Providing a consistent experience for referencing env vars in trigger metadata ([doc](https://docs.google.com/document/d/1DWjG2GfhxO8nWrLciTLtHW2y6YDkj2CF7n3abj4-pmE/edit) | [issue](https://github.com/kedacore/keda/issues/753) | Tom) - Tracking unshipped features (Tom) - Status of `ScaledJob` (Tom) - Should we move Kafka improvement to v2? https://github.com/kedacore/keda/pull/925 - FOSSA check to be fixed the next week ### Attendees - Zbynek Roubalik (Red Hat) - Ahmed ElSayed (Microsoft) - Travis Bickford (Shutterstock) - Tsuyoshi (Microsoft) - Tom Kerkhove (Codit) ### Notes - During e2e tests we use additional containers which are now under Travis & Jeff so we should move these under KEDA. - We should create an issue to list all images to migrate - We'll pull and push the images under KEDA, but add the Dockerfiles under /tests/tools, for those that contain code we'll create a `test-tools` repo under under KEDA - Should we move Kafka improvement to v2? https://github.com/kedacore/keda/pull/925 - We will move it to v2 since it doesn't make sense to release a new minor release - Providing a consistent experience for referencing env vars in trigger metadata - This is very confusing - Easy fix is to use an envRef as suffix in the property names - See issue for proposal doc & review - Tracking unshipped features - We should create a "vnext" milestone once PRs (and related issues) are merged to help track shipped features - When released, milestone should be renamed and closed - Status of `ScaledJob` will be started by Tsuyoshi - Tsuyoshi will join Kubernetes Slack in case he has questions - https://github.com/kedacore/keda/issues/789 - 2.0 Release - We can ship a beta after next standup ### Action Items - [ ] Travis to create an issue for e2e containers under KEDA - [ ] Everybody to review proposal for configuration changes - [ ] Tom to update release process with new milestone thingy ## Jun 25th Meeting ### Proposed Agenda - Adding Standard Resource metrics to KEDA (Tom - [PR](https://github.com/kedacore/keda/pull/874)) - Support for multiple triggers (Tom) - PR to be finished https://github.com/kedacore/keda/pull/866 - (Related proposed Feature: [Move replica capping decision out of scaler](https://github.com/kedacore/keda/issues/373)) - Release 1.5 - Release 2.0 - Authentication & secret providers ### Attendees - Zbynek Roubalik - Anirudh Garg - Alexander Yermakov - Ahmed ElSayed - Tom Kerkhove - Travis Bickford ### Notes - Standard Resource metrics is more of an advanced scenario where people want to manipulate the HPA that is being created - Tom has the feeling that we should align it with other triggers to provide a consistent approach for users and carve out the mutation of the HPA - This brings a lot of additional complexity so it's not sure if it's worth it - We will start with the advanced scenario and see if people start asking for a CPU/Memory trigger - Support for multiple triggers - This will be supported when ['use unique metric for all scalers' PR](https://github.com/kedacore/keda/pull/866) will be merged - Release 1.5 - Redis Streams & CRON scalers were added - pending [ActiveMQ Artemis](https://github.com/kedacore/keda/pull/888) - nice to have is [secrets from HashiCorp Vault](https://github.com/kedacore/keda/pull/890) - Anirudh is going to start the release - Release 2.0 - Somebody from Anirudhs team is going to get the Jobs part going next week - Add issue for blog post (Tom) - Add issue for migration guide from v1.x to v2.x (Tom) - Alpha release aiming for mid/end-june - Authentication & secret providers ## Jun 11th Meeting ### Proposed Agenda - Release 1.5 - Should be shipped as soon as these are merged: - https://github.com/kedacore/keda/pull/873 - https://github.com/kedacore/keda/pull/765 - Scaler governance (Tom - [PR](https://github.com/kedacore/keda-docs/pull/120)) - @jeffhollan Can you have a look at this before standup or too busy? - Guidance on building an external scaler (Tom - [#104](https://github.com/kedacore/keda-docs/issues/104)) - If somebody can give me some pointers I'm willing to pick this one - Failing FOSSA check in PRs - [#808](https://github.com/kedacore/keda/issues/808#issuecomment-641269201) ### Notes - End to end testing container in registry - Prometheus metrics endpoint for both modes (activiation and hpa) - OpenCensus / Prometheus - Progress around the cluster autoscale issue spec - https://github.com/kedacore/keda/issues/637 - https://docs.google.com/document/d/1CgUaidS8Jz6iCDJ_qIzkElBa25_OdgPv75RZ2qFhAOI/edit?usp=sharing ## May 28th Meeting ### Proposed Agenda - https://github.com/kedacore/keda/issues/753 - Replica capping discussion (Zbynek) - https://github.com/kedacore/keda/issues/373#issuecomment-631446814 - Add HPA Standard Metrics to ScaledObject Spec (Zbynek) - https://github.com/kedacore/keda/issues/852 - ScaledJob fix & implementation (Zbynek) - https://github.com/kedacore/keda/issues/789 - Beta release of v2? - Tom - Fine by me ### Attendees - Zbynek - Jeff - Chaitanya - Anirudh ### Notes ## May 14th Meeting ### Proposed Agenda (feel free to add agenda items) - Scheduled release week (1.5) - Going to postpone this release as not enough stuff in there - Scaler configuration can be confusing (Tom) - https://github.com/kedacore/keda/issues/753 - Some are resolved, some are raw values - How can we improve this? - add some more sample of the scaler files for diffect type of triggers. Add more helping/supporting documents for KEDA or where can seek for help for the issues. - Sharing secrets - Update (Tom) - Azure Event Hubs scaler checkpointing should be language agnostic (Tom) - Discussion of Cron Scaler (ckuduvalli) https://github.com/kedacore/keda/issues/812 - Monitoring the metrics adapter - Potentially prometheus metrics - Push activator pattern - Release schedule / lead cycle - Unique metric name for each scaled object - v2 updates and latest ### Attendees - Jeff - Tom - Travis - Daniel - Zbynek - Amith - Anirudh - Sandra - Ahmed ### Notes - We're going to delay the release as only some minor non-critical bug fixes. Will circle back in 2 weeks and potentially punt for 4 weeks - Discussed ways to make it more explicit to define which properties resolve from Env, but also even some potentiall `{{ syntax }}` to let people choose to have any value pulled from TriggerAuth / Env / Secret. Using this issue to track discussion and will circle back in 2 weeks. Jeff to investigate how many properties would be impacted. https://github.com/kedacore/keda/issues/753 - Today KEDA works today by running the polling interval. So some polling interval where KEDA checks to see if `isActive`. Ahmed has a proposal to create an interface for something to notify KEDA that is should make a scaling decision immediately. Initial thinking for this is a generic gRPC scaler. - Will you be able to interact with this and provide your own scaler? There is some interface for you to have this work with whatever scaler you prefer. - Could this work with like Service Bus 'queue not empty' Event Grid notification? Yes, but unclear if a scaler could be BOTH push and pull, but potentially yes. - Trying to keep KEDA from being a server, and more of a client to connect to as many servers as it needs. So you'll need to implement some external server that KEDA could connect to via gRPC as a generic client and it could push. But scalers, and external scalers, can act as the adapter / interface to relay events to KEDA via gRPC. - Ahmed hoping to post an RFC today / tomorrow. - Maintainers now have OnePassword to share secrets and credentials. - Talked about the fact that the event hub scaler today has some language incompatibilities - it stems from the fact that Event Hubs SDK behaves slightly different with different language SDKs. Our approach has been to write some adapters in the scaler. Will likely just need to keep flagging incompatibilities and fixing one-off. - Punted discussion of the CRON scaler until creater could join - Discussion on how we could provide better metrics or status to help monitor what metrics that KEDA is pulling and publishing to HPA - especially because the HPA only shows the average, so sometimes hard to understand the specific values. - Potentially allow you to wire up Prometheus? - Potentially integrate with Cloud Events? - Likely worth looking at what telemtry info we have, and even flowing telemetry somewhere else. OpenTelemtry? - Travis to create an issue - Going to spin up a release schedule process so it doesn't always fall on one person - Decided we can have a metric name as an option for all scaled objects. The change required making the change to nearly all scaled objects. Exposing this and making it so scalers can set the name. - Work already based on the v2 branch. - Amith to create a pull request, even [WIP] by end of week to review changes and provide feedback. - v2 work progressing well. Biggest pending work items is going to be supporting ScaledJobs in v2. Hoping that same contributors who helped with v1 can move to v2, but TBD. - Was also some memory usage / leak issues, but does seem to be resolved in v2. But one reason we may want to get v2 out there sooner than later as it now has a good amount of improvements unique to v2. - Would also want to get the push version into v2, even have a beta for people to start testing out. - Plug for anyone interested in creating content / resources / use cases that we could add to our docs site https://keda.sh/resources/ ## April 30th Meeting ### Proposed Agenda (feel free to add agenda items) - Maintainers should vote for Daniel Imberman's consideration for maintainership - Versioned docs review (Tom) - v1.4 release retrospective (Tom) - We should no longer release on Friday unless somebody can review PRs - Hotfix was shipped for regression issues - Points of improvement - [x] Apply branch policy to require 1 human to review - [ ] We should automate more - [ ] We need more robust testing infrastructure - Idea: - Should we use issues to track roll-out and tag them to find related PRs? - Provide guidance how to build an external scaler (Tom - [#104](https://github.com/kedacore/keda-docs/issues/104)) - Introducing External Scaler community (Tom - https://github.com/kedacore/external-scalers) - Do we have this in docs / README for discoverability? - Do we support or deprecate Helm 2.0 support for KEDA 2.0? (Tom) - Poll: https://github.com/kedacore/charts/issues/40 - Blog (Tom) - Proposal from community to write a post about Strimzi + KEDA ([#150](https://github.com/kedacore/keda-docs/issues/150)) - Should we post a blog post per release? - v2 - volunteer to implements ScaledJob functionality? [#789](https://github.com/kedacore/keda/issues/789) - v2beta2 for HPA - Issue around scaling to zero of resetting the last active time - CI / CD ### Attendees - Jeff - Tom - Travis - Amye SP (CNCF) - Daniel - Robin - Zbynek ### Notes - Amye was able to join and flag on some of the CNCF - YouTube playlist - Key sharing - Infra for testing and such - Docs update - Tom to add example of how we version for new updates - Release process ### Action Items - [x] Jeff to share with Amye the youtube playlist - [ ] Jeff to add Travis to team for RabbitMQ issue - [ ] Jeff to reach out to CSE for ScaledJobs - [ ] Travis to create an issue on the v2beta2 stuff ## April 16th Meeting πŸ“Ή Recording: https://youtu.be/ZUTTPrhcrYw ### Proposed Agenda - KEDA 1.4 release - v2 progress / updates - ScaledJob spec - Move standup recordings to CNCF YouTube account (Tom - [#744](https://github.com/kedacore/keda/issues/744)) - Scalers (Tom) - New proposal for Redis Streams ([#746](https://github.com/kedacore/keda/issues/746)) - SMI is WIP - Meeting times moving forward - Rajasa Use Cases / issue - FTP autoscaling / incoming requests (Rahul) - HTTP Scaling ### Attendees - Jeff - Tom - Israel - Anirudh - Ahmed - Zbynek - Rajasa (RJ) - Abhishek - Amith - Rahul - Travis ### Notes - Zbynek to create an issue to flesh out the job stuff. Specifically Jeff brought up that there is some confusion on which setting a user should specify to control how many jobs are spun up concurrently (queueLenght, maxReplicatCount, or parallelism) - Also an open question about multiple scalers. So may need to create a unique HPA [733] metric name for the HPA to drive scaling on - ### Action Items ## April 2nd Meeting πŸ“Ή **[Recording](https://youtu.be/G-CW5p09M-4)** ### Proposed Agenda - KEDA Website Refresh (Tom) - Path to commiter/maintainer privileges (?) - v2.0 - Milestone introduced - https://github.com/kedacore/keda/milestone/8 - Provide consistent naming for configuration - Autoscaling v2beta1 to v2beta2 - Move away from k8s.io namespace - Duck-typing update ([#591](https://github.com/kedacore/keda/issues/591)) - Scaling any Resource with `/scale` subresource ([#703](https://github.com/kedacore/keda/issues/703)) - Introduction of `ScaledJob` & `ScaledDeployment` ([#653](https://github.com/kedacore/keda/issues/653)) - Move Standup to earlier time? - CLA bot on docs - Helm chart releases (Tom) - Job scaling issues (Sowmya) ### Attendees - Jeff Hollan - Anirudh Garg - Tom Kerkhove - Sowmya - Ahmed - Satish - Zbynek ### Notes - https://github.com/cncf/servicedesk revamped our website. Could potentially use them to provide lastpass as well. - Zbynek did a bunch of investigation around Duck typing and how we could support more things to scale. Duck typing has some limitations given how many ways things like replica count can be defined in sub resources. He has another [proposal](https://github.com/kedacore/keda/issues/703) to change the scaled spec that would be more flexible, but is an open question around how to deal with in-line secrets given changes. - Some discussion if it makes sense in v2 to have `ScaledObject` or move to an approach with a specific CRD for Jobs and Deployments. - Potentially keep `ScaledObject` as the flexible one that could do deployments, stateful sets, argo deployments, but introduce a new kind called `ScaledTask` or `ScaledJob` or something that would scale jobs. - Talked about providing a naming convention for `metadata` of scalers to help create some consistency. https://github.com/kedacore/keda/issues/712 - One downside is the individual services use different names. E.g. azure calls things "ConnectionString" where something like RabbitMQ may use the term "host" which both provide similar info (where and how to connect). Kafka and `broker` is another example. - Worthwhile discussion - plan is to review the curent ones and flag opportunities to improve. - Discussed timing for these standups as it's challenging for Europe primarily to join. Right now we meet at 10am PDT. Discussed maybe moving to 9am. That will work for now but may be difficult for some (Anirudh) when school starts again. WE would go later but would be like 12am PDT. Will continue chat on email. ### Action Items - [x] Tom to request LastPass for KEDA ([#722](https://github.com/kedacore/keda/issues/722)) - [x] Tom to request KEDA being listed on CNCF Community Calendar ([#723](https://github.com/kedacore/keda/issues/723)) - [x] Jeff to create PR for Maintainer onboarding process - [ ] Sowmya to create repro steps for the issues and Ahmed / Jeff can help look and try to understand ## March 19 Meeting ⚠️ With daylight savings time will start [an hour earlier for non-US attendees](https://www.thetimezoneconverter.com/?t=10%3A00%20am&tz=Seattle&) ⚠️ ### Proposed agenda (feel free to add iems) - CNCF Donation Update (Tom) - We have 3 TOC sponsors and are finalizing the donation πŸŽ‰πŸŽ‰πŸŽ‰ - We are adding maintainers, governance, security, passing CII Badge, logos, et al (see open actions below) - License has been changed to Apache License 2.0 - Changelog has been introduced (Tom) - This changes the way we release, see https://github.com/kedacore/keda/blob/master/RELEASE-PROCESS.MD - Do we need a nicer doc design? (Tom) - CNCF can help with this - Provide governance for built-in & add-on scalers ([#573](https://github.com/kedacore/keda/issues/573) - Tom) - Plan for next release (Tom) - We need to define milestones and use them to track what is coming so folks can get an idea - Follow-up on last meetings release topic - HTTP Scaling - big opportunity here (Jeff) - Extending redis scaler ([#647](https://github.com/kedacore/keda/issues/647) - Tomek, if we have time) - Service Mesh Interface (SMI) scaler ([#615](https://github.com/kedacore/keda/pull/615) - Tom) - Spec proposal is done by Tom, anybody up for implementing it? - Kubernetes event scaler ([#306](https://github.com/kedacore/keda/pull/306) - Tom) - Shekhar Patnaik is no longer responding - Do we abandon it or? - 2.x items (Zbynek) - 1.4 release (Anirudh / Jeff) - Security issue for e2e tests (Jeff) ### Attendees - Daniel Imberman (Astronomer) - Jeff Hollan (MSFT) - Tomek Urbaszek (Polidea) - Zbynek Roubalik (Red Hat) - Michael Droessler (CUNA Mutual Group) - Anirudh Garg (MSFT) - Tom Kerkhove (Codit) ### Notes - Astronomer wrote a nice blog post on KEDA - https://www.astronomer.io/blog/the-keda-autoscaler/ - 1.3 released last week - **We are officially a CNCF Sandbox project** - CNCF provides metrics related to our project on a [dashboard](https://keda.devstats.cncf.io/d/8/dashboards?orgId=1&refresh=15m) - Tracking action items below for CNCF transition - Had some discussion about if we should move the channel from Kubernetes slack to Kubernetes - Covered the new changelog.md process for releases - Design for KEDA.sh - We can request help for designs for logos and website via CNCF - Tom requests help via CNCF https://cncfservicedesk.atlassian.net/servicedesk/customer/portal/1/CNCFSD-382 (only for maintainers) - Talked about 2.x items - specifically duck typing. Zbynek doing some initiatl investigations that would include ScaledObjects / Jobs / Deployments / StatefulSets / any CRD. - Wouldn't have an impact on scalers - ### Action Items - [x] Jeff to transfer keda.sh domain to CNCF ([#108](https://github.com/kedacore/keda-docs/issues/108)) - [ ] Jeff to remove old Netlify account ([#109](https://github.com/kedacore/keda-docs/issues/109)) - [x] Jeff to give Tom permissions for passing CII Badge ([#667](https://github.com/kedacore/keda/issues/667)) - [ ] Jeff & Ani to review project governance proposal ([#690](https://github.com/kedacore/keda/issues/690)) - [x] Jeff to remove the Microsoft CLA GitHub app - Emailed the team to remove. If they don't respond I'll just remove access - [ ] Ahmed to review StatefulSet PR ([#498](https://github.com/kedacore/keda/pull/498)) - [ ] Ahmed to review Storage Queue PR ([#639](https://github.com/kedacore/keda/pull/639)) - [ ] Tom to add security policy ([#668](https://github.com/kedacore/keda/issues/668)) - [x] Tom to finalize CNCF mention ([#678](https://github.com/kedacore/keda/issues/678)) - [x] Folks to review open License PRs ([overview via #658](https://github.com/kedacore/keda/issues/658)) - [x] Tom to add google doc for blog (see [here](https://docs.google.com/document/d/1WbZ7Ph687oxJMR0aHLjzTu9HfnA7EkxPXiXZo8uzKl0/edit)) - [ ] Jeff to create google survey for slack - [ ] Jeff to assign the redis issue to Tomek ## March 5 Meeting πŸŽ₯ https://www.youtube.com/watch?v=O8wG1_tpAS4 ### Proposed agenda (feel free to add iems) - CNCF Donation Update - Already have 2 TOC sponsors (Michelle Noorali & Liz Rice) - Issue opened for our artwork to get it in CNCF artwork ([#650](https://github.com/kedacore/keda/issues/650)) - Change the weekly call to be 30mins only? Bi-weekly? - v2 dev branch on GitHub? - Governance of scalers and what's included ### Attendees - Jeff - Anirudh - Sandra - Zbynek ### Notes - On the release: asked Satish to start the 1.3 release. Haven't gotten to it this week yet but will work to get out soon (start this week - potentially complete this week) - Brought up last weeks conversation about clear governance around KEDA. - Something to wonder - Do we add to core by default or discuss up front? (Tom) - Tom get out of here ;) - I added an action item. I'll take a stab at documenting and open a PR. - Zbynek starting to look at the duck typing feature ask. Thinking is it would be in a v2 branch and include other tracked changes we called out - https://github.com/kedacore/keda/milestone/8 - Mentioned the CNCF work - Discussed the cadence of these standups. Some thinking that we get folks on general attending every two weeks, so maybe do that? Not as many big changes happening so bi-weekly may be good. Even a shorter bi-weekly meeting. - Discussed having a scheduled release for KEDA like Knative has. Agreement that we should shift to a regular cadence. Makes the release process a bit more open as well. - Propsal is that we shift to once every 4 weeks. Also makes it so releases fall on a standup week. - Starting with 1.4. Potentially 4 weeks from today (Apr 2) - Question on status of the Knative PR - link here https://github.com/knative/eventing-contrib/pull/886 ### Action Items - [ ] Jeff to take a stab at writing governance proposal to go onto the docs site and linked in README.md ## Feb 27 Meeting ### Proposed agenda (feel free to add iems) - CNCF Donation Update - SIG-Runtime is now officially supporting us as the recommendation has been merged and is available on - https://github.com/cncf/sig-runtime/blob/master/recommendations/sandbox/keda.md - Actively looking for TOC sponsors - Provide capability to extend scalers ([#624](https://github.com/kedacore/keda/issues/624) - Tom) - Provide governance for built-in & add-on scalers ([#573](https://github.com/kedacore/keda/issues/573) - Tom) - Introduce version in the trigger spec ([#613](https://github.com/kedacore/keda/issues/613) - Tom) - Provide a changelog ([#614](https://github.com/kedacore/keda/issues/614) - Tom) - Should be part of https://github.com/kedacore/keda so that feature PRs can include updates to changelog - This does not replace GitHub releases but are complementary - Tom would be happy to pick this up - Important for handling deprecations ([demo](https://changelog.promitor.io)) ### Attendees - Ani (MSFT) - Tom (Codit) - AndrΓ©s (Enterprise) - Sandra (IBM) ### Notes - Provide a changelog - Tom will create a CHANGELOG.md and guidance when to update it and how - Introduce version in the trigger spec - Tracked as a feature - Provide governance for built-in & add-on scalers - Will be discussed when more people join as it has a big impact - Provide CloudEvent to extend KEDA - Good idea - We should definately do this - Provide capability to extend scalers - Good idea, we should wait for the CloudEvents basis and take it from there - Service Bus Scaler - SAS authentication scoped to namespace does not work - A lot of demand for this - https://github.com/kedacore/keda/issues/215 - We tried to do a 1.3 release but got postponed to next week - This will include Azure Monitor for example ## Feb 20 Meeting ### Proposed agenda (feel free to add iems) - CNCF Donation Update (Tom) - [SIG-Runtime PR](https://github.com/cncf/sig-runtime/pull/10) - Remove `deploymentName` label on every `ScaledObject` ([#545](https://github.com/kedacore/keda/issues/545)) - How much effort would this be? It's redudant information which we can avoid AFAIK - Paolo Patierno from Red Hat is happy to contribute it - CNCF presentation listed Astronomer, Bupa, Cyclomedia - Do we have contacts to ask if we can list them on keda.sh? - @Daniel Ok for Astronomer to be listed? - Tom will add those that we have approval for ### Attendees - Daniel - Lukas - Satish - Jeff - Anirudh ### Notes - KubeCon sessions - Tom K got a lightning talk accepted for Serverles practitioners summit. - Jeff H got a tangentially related session accepted for practitioners - will mention KEDA - CNCF update - Spoke this morning to SIG-Runtime, went well. Opened PR (Anirudh to copmlete) and then just need to get ToC approvers - Cluster autoscaler - One item that came up was the possibility to use KEDA to drive scale out of the cluster autoscaler. Opened [an issue](https://github.com/kedacore/keda/issues/637) for discussion - Astronomer.io going to open a docs PR to add logo to users / community - Redundant deployment-target - Agreed if possible we should clean up the scaled object so you don't have to redundantly provide the `deployment` target 2xs [RELATED ISSUE](https://github.com/kedacore/keda/issues/545) ## Feb 13 Meeting ### Proposed agenda (feel free to add iems) - Knative - Aleksander joined from IBM - He has been pushing KEDA in internal componnets - Azure Monitor - Last night support was finished. - Fixed auth issues. - Need to submit docs - Need to do Pod Identity. - Docs on adding a new Scaler - Mel will add docs on how to help to add Scaler - 1.2 Release - No issues - release as expected - Docs have been updated - Were able to use the new release tooling by - https://github.com/kedacore/keda/releases/tag/v1.2.0 - @Anirudh to add link to the releases on the Keda home page - Potential Bug in the Helm Chart (Daniel) - Trying to merge the KEDA Helm chart in their own bigger chart. Using it as a Sub-Chart does not install the CRD - Maybe did not have the latest (Sandra) - Kafka trigger scaler (Rajasa) - Samples added - Should have a way for folks to have their Samples in the KEDA repo. @Anirudh to follow up - Going to add one more sample for EventHub Scaler - Non-Azure Functions workloads in other cloud (Rajasa) - Discussion on this - Depends on the Event Source ### Attendees - Anirudh (Microsoft) - Sandra (IBM) - Aleksander (IBM) - Mel (Microsoft) - Rajasa (Microssft) - Daniel (astronomer.io) - Satish (Microsoft) ## Feb 6 Meeting ### Proposed Agenda (feel free to add items) - CNCF Sandbox donation ([#501](https://github.com/kedacore/keda/issues/501) - [Proposal](https://github.com/cncf/toc/issues/335)) - Still pending approval from SIG-Runtime, pending presentation on 20th of February - Joined today's standup and Jeff is good to go to present, hopefully there are 2 chairs so they can vote. - Provide governance for built-in & add-on scalers ([#573](https://github.com/kedacore/keda/issues/573)) - Service Mesh Interface scaler (Tom - [#615](https://github.com/kedacore/keda/issues/615)) - This will allow us to autoscale HTTP workloads for all service meshes that support SMI - SMI is looking at adding this so we should collaborate with them https://github.com/deislabs/smi-spec/issues/96 - Happy to take the lead and draft out some specs if interest - Allow nested metadata properties & Azure Monitor scaler (Tom) - Do we merge it already? If so, then we'll have breaking changes pretty soon but [#613](https://github.com/kedacore/keda/issues/613) could be helpful here - Provide a changelog (Tom - [#614](https://github.com/kedacore/keda/issues/614)) ### Attendees - Jeff (Microsoft) - Sandra (IBM) - Mel (Microsoft) - Zbynek (Red Hat) - Calvin (Cyverse) - Will (Cyverse) - Lukas (Red Hat) - Jeremy (Cyverse) ### Notes - Sandra merged in labels PR, release process to automate updating versions. `Make release` - Mel making progress on the PR for monitor, just has to write the test. Going to break pod identity into a seperate. - Looking for something to look into after this one. Something that isn't super urgent but does require Kubernetes - Zbynek proposed a PR to the operator hub on 1.2 - Next plans to look at duck typing and some other changes we may want to make it 2.x. Will be out next week. - Is a proposal on the [Knative eventing PR to help scale Knative event source](https://github.com/knative/eventing-contrib/pull/886) - Discussed scaler governance - no decision made, just covered topics https://github.com/kedacore/keda/issues/573 - Operator SDK section added to readme, may be mac only? - [ ] Check operator SDK - [ ] Check SMI spec adoption - [ ] Zbynek to add some info on the zap-info ## Jan 30 Meeting πŸ“Ή [Recording here](https://youtu.be/EUH2SBrmOec) ### Proposed Agenda (feel free to add items) - Provide governance for built-in & add-on scalers ([#573](https://github.com/kedacore/keda/issues/573)) - Introduction of Azure Durable Function add-on scaler ([GitHub](https://github.com/kedacore/keda-scaler-durable-functions)) - Can we expect [v1.2 release](https://github.com/kedacore/keda/pulls?q=is%3Aopen+is%3Apr+milestone%3Av1.2) anytime soon? - FYI: very very simple [instructions](https://github.com/zroubalik/keda/tree/deployInstr#deploying-custom-keda-locally-outside-cluster) for local deployment outside cluster, Zbynek could provide demo next time if needed ([PR](https://github.com/kedacore/keda/pull/583)) ### Attendees - Jeff (Microsoft) - Anirudh (Microsoft) - Steven (astronomer.io) - Ahmed (Microsoft) - Lukas (Red Hat / Knative) - Mel (Microsoft) - Sandra (IBM) - Scott Nichols (VMware) - Ishara (BUPA) - Rajasa (Microsoft) - Daniel (astronomer.io) ### Notes - Some contributor guide pains being reported this. - Had a few conversations this week with teams internal to MIcrosoft who are planning to start using KEDA to run services - Good progress on the SQL scalers. For Airflow, Postgres and MySQL are most popular so we have those now. MySQL was merged in 3 days ago. Anirudh interested to even pick up. Steven helping now integrate KEDA / these scalers into Airflow so it can start to surface to customers. - Steven also well versed in some of the CI/CD stuff as well - On the topic of CI/CD we tweaked the blob test so it isn't as intense and passes more now - Store secrets in GitHub and then pulls them into a GitHub Action to authenticate - Potentially we could get secrets from Google / AWS to also run tests on GKE / EKS - On the Azure Monitor PR, some questions about `authParams` and `resolvedEnv` - `authParams` are the parameters / values resolved from the `TriggerAuthentication` values - `resolvedEnv` are the parameters / values resolved from the deployment. - Anirudh did have some issues getting it to build locally. But then changing code, packaging up a container, deploying it - Anirudh got stuck. - Rajasa also pointed out that experience for Windows users has a few gaps as well - Daniel brought up we should also document what variable names for consistency. A naming convention / consistency guide. - Plan to cut the 1.2 release this week - Lukas had a good question on if we should just time-based releases in general. - Knative does every 6 weeks. - github.com/google/ko - kubectl symantecs for go-lang. TGIK - 45 min of Ko - GitHub.com/n3wscott/discovery - duck-typing labels - Ishara is working on a sample for the blob trigger and plans to have the PR open soon - Rajasa was working on the Event Hub scaler - now moving to the Kafka trigger scaler with Event Hubs. Event Hubs needed a type of authentication that the Kafka scaler didn't support from the outset. Mel helped debug in Go as well. - In Event Hubs scaler it assumes the consumer is the Event Processor Host. However Kafka has the same APIs to check the checkpoints. - Secrets - was using TriggerAuthentication or w/out the TriggerAuthentication. - The log library that is being used in the Kafka scaler was not printing in logs. - VS Code remote container build was not working. Rajasa to raise an issue. - Anirudh mentioned it didn't work either - but is fixed now https://github.com/kedacore/keda/issues/568 ### Action Items - [ ] Anirudh to cut 1.2 release this week - [ ] Community to help review the [WIP PR](https://github.com/kedacore/keda/pull/584) - [ ] Jeff to create a release schedule that includes when the last commit will be grabbed, when we will cut the release, when it will go out - [ ] Rajasa to create an issue specifically around the new line characters - [ ] Jeff to reach out to Zbynek on documentation of Zap-info ## Jan 23 Meeting πŸ“Ή [Recording here](https://www.youtube.com/watch?v=8qV6PzmqYMg&feature=youtu.be) ### Proposed Agenda (feel free to add items) - Community refresh on keda.sh (Tom - [#65](https://github.com/kedacore/keda-docs/issues/65)) - Actively seeking KEDA customers for "Customers" section on keda.sh (Tom - [#551](https://github.com/kedacore/keda/issues/551)) - PostreSQL scaler ([#546](https://github.com/kedacore/keda/issues/546)) - [.NET Core Sample](https://github.com/kedacore/sample-dotnet-worker-servicebus-queue) update (Tom) - Included API for HTTP-autoscaling demo once available - Jakob Ehn is contributing a nice UI to visualize demo - Can I get more permissions to setup PR CI? - Cannot create HPA object when Deployment has missing ConfigMap (optional: true) ([#555](https://github.com/kedacore/keda/issues/555)) - Interesting issue, but need somebody with implementation experience to determine what to do ### Attendees - Jeff (Microsoft) - Daniel Imberman (daniel@astronomer.io) - Tom (Codit) - Ahmed (Microsoft) - Lance (Red Hat / OpenShift) - Lukas (Red Hat / Knative) - Mel (Microsoft) - Sandra (IBM) - Zbynek (Red Hat) - Anirudh (Microsoft) ### Notes - Basic postgres scaler merged. Plans to work on a mySQL one this week. Even got it working end-to-end with Airflow. - Interested in step scaling - Talked about some options. The main scenario Daniel looking to unblock is a SQL metric that would be like (# of running * # of rows / current). So thought is that maybe the metric could be smart enough to do metrics to help with target. Anirudh is ok with both. - Ahmed also was thinking the scalers could be smart enough to use the metrics to help influence the hpa scaling decisions. - Ahmed brought up that we would need approval to use the k8s namespace in the CRD https://github.com/kedacore/keda/issues/552. - Anirudh documented the release process for KEDA https://github.com/kedacore/keda/wiki/Release-process - Tom brought up with 2.0 we may consider breaking the ScaledObject and ScaledDeployment - Mel working on the Azure Monitor scaler. Hoping for a PR in next few days. Ran into some issues with the docs - plans to fix those as well. - Some confusion around if metadata can be nested. For Azure Monitor there are a lot of metadata properties where nesting would be nice. - Operator Hub is ready. Zbynek is going to start the process for Operator Hub. - 2.0 potentially could include the duck-typing - Mel brought up when deploying locally, if change to "PullIfNotPresent" it fails. - Daniel brought up to do the local without having to push first you could use Kind. You could do Pull Policy of "never" and it will load - If doing in the cloud, you will need to push to a repo - Daniel interested in looking into "Telepresence" tech to try out improving the dev cycle for scalers ### Action Items - [x] Jeff to create Allow nesting of metadata - [x] Jeff to create updating the scaler doc and assigning to Mel - may be related to https://github.com/kedacore/keda/pull/556 - [ ] Zbynek to add docs and look into the operator hub release process - [x] Jeff to create an issue around duck-typing for 2.0 - [x] Jeff to give Tom permissions for CI / CD - [x] Daniel to create docs for Postres sql on KEDA docs ## Jan 16 Meeting ### Proposed Agenda (feel free to add items) - CC 1.1.0 Release ([draft](https://github.com/kedacore/keda/releases/tag/untagged-06ea7afe5503e368cc13)) - https://github.com/kedacore/keda/issues/545 - Interest in postgres scaler - [CNCF Proposal](https://github.com/cncf/toc/issues/335) ### Attendees - Daniel (Astronomer.io) - Ishara (BUPA) - Lance (Red Hat / OpenShift) - Lukas (Red Hat / Knative) - Rajasa (Microsoft) - Sandra (IBM) - Israel (Microsoft) - Tom (Codit) ### Notes - Release 1.1.0 draft release out. Action to confirm that all commits that are needed are in there - Rajasa Event Hub stuff seemed to be working now. Seemed to work fine on further testing. Some variability in how it was scaling. Some further testing coming - and a sample in the works. - Ishara to update docs. Jeff to add comment on issue. - Daniel starting work on the Postgres scaler. Will create issue, Jeff will assign. - Question on using custom queries, and not stats yet today. Daniel use case is a bit like "what are the number of items that are set to this state". - Another question on if any thought on creating a generic JDBC type scaler so not limited to just Postgres? Thinking is for getting a quick version out, a Go scaler specific to Postgres. You would just provide the SQL query e.g. "SELECT COUNT() FROM FOOTABLE WHERE STATE LIKE ACTIVE" (Jeff forgot SQL syntax). Daniel has been looking to use a generic Go database driver, so potentially this could be extended to other databases? Israel is open to test this. Daniel will keep this in mind. - Partner logos - there is [an issue](https://github.com/kedacore/keda/issues/539) you can comment if willing to add your logo. Tom brought up a great point we can make it really easy for people to open a PR to add their logo to the docs site. - Jeff needs to make sure Anirudh and Ahmed provide data for missing sections of proposal - Question about what CNCF sandbox means - more or less we can keep operating standups the way we are today, but means we give up the trademark and rights to KEDA to the community. In general though we can do things like recording meetings and sharing better :) ## Jan 09 Meeting ### Proposed Agenda (feel free to add items) - Updates - Discussion of PRs and progress on project - Release 1.1 - Schedule & status (https://github.com/kedacore/keda/milestone/5) - New items and PRs - Recap KEDA meeting yesterday ([recording processing](https://youtu.be/pWSeZ-Tn4I0)) - Update docs for contributing scalers ([issue](https://github.com/kedacore/keda/issues/531)) - Knative - [StatefulSets]( https://github.com/kedacore/keda/issues/497) - CNCF Proposal ([draft](https://github.com/tomkerkhove/toc/blob/keda-donation-proposal/proposals/keda.md)) - Attendees - Ishara - Mel - Rajasa - Sandra - IBM - Notes - Azure Blob is still pending review by Ahmed. There is a limitation (same as Azure Functions has) if container > 5000 but we can add to docs - Mel working on the Azure Monitor scaler. Expecting by next week could have a PR. Would be good to consider the pod identity (Azure Queues and Azure Service Bus both support pod identity). - Rajasa - outstanding PR merged last week. Need this PR to be merged in [here]( https://github.com/kedacore/keda/pull/537
), docs also merged in. There is a bug they are hitting an issue scaling from 1 -> n doesn't seem to be working. Also has a working sample that ## Dec 19 Meeting ### Proposed Agenda (feel free to add items) - Updates - OperatorHub.io progress (Zbynek) - https://github.com/kedacore/keda-olm-operator/pull/1 - Blob support PR (Tom/Ishara) - Keys support for Azure Functions (Anirudh) - We discussed changing the frequency of the meeting to twice a month but decided to hold off until later in 2020 (Febuary) - Zbynek will send another PR for the documentation of the operator. - Israel to take a look at the first pull request for the operator (keda-olm-operator). We need more reviewers before it is publish - Anirudh has a pull request for Keys support for Azure Functions in the functions repo and should submit the PR early next month (January 2020) - Rajasa is currently working on making the EVH scaler work with Java and .NET apps as well. Support for python is not yet available. She will submit the PR for the changes. This is the PR for Event Hubs Scaler to work with Java and .NET applications https://github.com/kedacore/keda/pull/517 - Mel will submit pending changes into a branch and will resume after the break. - We will skip the next 2 meetings and resume on January 9th. - Attendees: - snichols - Zbynek - Mel Cone - Tom Kerkhove - Anirudhg - Rajasa Savant (RJ) - Lukas Berk - Ishara Pannila - Florian - BBrowning - Israel Ekpo ## Dec 12 Meeting ### Proposed Agenda (feel free to add items) - func core tools support (Anirudh) Keys support Http request based scaling - Feedback from Scott on looking at code - Do we have issues for these ? - Scott and Zbynek will sync and take ownership. - Move community standup to bi-weekly basis (Tom) - If anybody has objections, please go to #502 - Suggestions to have 30 mins every week ? - Bursting Jobs in Azure Container Instances (Matt Pasco) - https://github.com/mattpasco-microsoft/KEDA-ACI - Matt can confirmed Can burst out Jobs to Virtual Nodes - OperatorHub.io progress (Zbynek) - Closed the PR for it - 90% done. - From our side all work would be completed. ## Dec 5 Meeting ### Proposed Agenda (feel free to add items) - Standup Calendar (Tom) - You can now add our standup calendar to your client to see if it's cancelled or not - https://calendar.google.com/calendar?cid=bjE0bjJtNWM0MHVmam1ob2ExcTgwdXVkOThAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ - Product leads can ping me to get access - Release automation (Tom) - We should focus on streamlining our releases - Introduce `experimental` tag for master builds - Introduce formal release with GitHub releases (on-demand trigger) - Docs on what we have and when to use what - See milestone: https://github.com/kedacore/keda/milestone/6 - 1.1 release features / milestone - Suggestions - kedactl (Tom) - Dashboard (Tom) - CNCF Donation (Tom) - Support for StatefulSets ### Notes - Google Calendar for the meeting calendar - Release Automation - Push to master - creates master image and updates the master tag on Docker Hub. - CI/CD on push to tag. - Need to update the Charts. - Add an issue to update Charts automatically. - Add docs. - Suggestion to just tie to a release. - Issue #439 - Currently not on Operator Hub - Self Signed Certificate issue ? - We have an open PR - Zybnek will have to done relatively soon - CI/CD Should also push to Operator Hub. - Suggestions for the next release - Final release plan should be done via https://github.com/kedacore/keda/milestone/5 - New Authentication spec for Scalers - New scalers - Azure Monitor as a fallback - Pod Identity - AWS, Google others - Fix the dashboard for KEDA - 1.0 - Sekhar - Anirudh will create a issue and assign to Sekhar - Suggestion for having its own CLI - Krew plugin. - Shekar would love to have some integration with Knative for which Scott has joined - We could produce a scalable "Duck type" (ie. PodScalable) that can be scaled upon and allow Knative & KEDA to collaborate. - We should setup a Knative & KEDA meeting to discuss this further (invite snichols@vmware.com) - Knative would love to use KEDA for queue-based scaling, KEDA could use Knative for request-based scaling (which they already have) - See more in Scott his talk from KubeCon - https://www.youtube.com/watch?v=kldVg63Utuw - Support for StatefulSets - Ahmed to review - Tom has started working on the CNCF proposal and has pinged Jeff for an early review - Tom will create a GitHub issue for it to track the work - Ishara will create a PR for a blob scaler, will scale on new blob created - Thanks! ## Nov 28 Meeting ### Cancelled for US Thanksgiving πŸ¦ƒ ## Nov 21 Meeting ### Cancelled for KubeCon North America ## Nov 14 Meeting ### Proposed Agenda (feel free to add items) - KEDA 1.0 (jeffhollan) ### Notes - KEDA 1.0 main open items: - Docs (Tom and Jeff) will chomp on this - Operator SDK and Operator Hub [issue](https://github.com/kedacore/keda-docs/issues/8) - Manual release of 1.0 by Ahmed (sometime before Monday) - Operator Hub still has some work items - A special procedure that Operator Hub has for installing API service that it's expecting. Operator Hub is looking for a `kind` and since it doesn't see it, it tries to re-install. Zbynek talking to the hub team to figure out best way to solve. This is limited to only the Operator Hub tooling support. - Zbynek has an issue on the hub team: [issue](https://github.com/operator-framework/operator-lifecycle-manager/issues/1122) - - Stretch goals - Auth spec gap - KEDA Dashboard ## Nov 7 Meeting ### Proposed Agenda: - Discuss KEDA 1.0 Release - Updated Helm chart so that it works with the two containers and Operator SDK - done - Set up CI through GitHub Actions for any merged PR's to master which will currently update the docker images (0.0.4) - done - Nightly tests fix - to be done - Update the readme - to be done - Version strategy - to be decided - currently we think that we will have a 1.0 stable release - different sets of helm charts for stable and dev. - Breaking changes fixes for Rabbit MQ - to be merged - Pod identity for Service Bus - to be done by Ahmed - Zbynek - First draft of the manifest file for the Operator Hub - Will show demo - Ahmed - Helping with CI stuff, Operator SDK is merged in. Enabling the End to End tests have to be enabled. Looking at PR's as they are come in. - Have to come up with a support plan. - Operator Hub has many models for support - Partner - Red Hat - Community - Suggestion is to consider this a Community. - Goal is to have KEDA submitted as a Sandbox project in CNCF. ### Proposed Agenda (feel free to edit and add): - Twitter-based scaler: Monitors specific accounts for specific words, and scale accordingly. For real-case scenarios we might use different Twiter APIs, this can be a simple demo for the community. Work has been commencing on the following branch, intention to create a PR soon. [https://github.com/eashi/keda/tree/twitter-scaler](https://github.com/eashi/keda/tree/twitter-scaler) - Stream live development of the Twitter scaler (above) on Twitch. I have streamed before other stuff but sadly there is no recording to demonstrate. My plan is to start next week, maybe Thursday 14th. - Started documenting writing scalers (the old way, not the through External Scalers). The first step was PR [#395](https://github.com/kedacore/keda/pull/395). - Keda on OperatorHub.io ## Oct 31 Meeting ### Proposed Agenda (feel free to edit and add): - PR #407 (https://github.com/kedacore/keda/pull/407) - Fix for Azure Queue Scalar AAD Metadata (stgricci) ### Notes * CSE has a hackathon coming up and are looking to extend the jobs capabilities. (Customer technical engagement team) * Need to validate that the dashboard works with v1, and if it doesn't quickly fix. Mark as help wanted. Potential table: | Scalers | Status | Sponsor | | --------------- | --------- | ------- | | Azure Event Hub | Official | @Azure | | RabbitMQ | Official | @Azure | | NATS Streaming | Community | | | Kafka | | @Azure | The incoming AMQ scaler would be @RedHat ## Oct 24 Meeting ### Proposed Agenda (feel free to edit and add): - CNCF Working group - Operator SDK work - Kafka and Event Hub setting max scale out? ### Notes ### To do items - [ ] Anirudh to look at and sort out a contributor guide - [ ] Jeff to add info on how to chat on Slack (Kubernetes) - [ ] Satish / Anirudh to fix the helm chart - [ ] Ahmed ## Oct 17 Meeting ### Proposed Agenda (feel free to edit and add): - Operator SDK work ### Notes - Zbynek has the PR ready to go, just having a hard time opening because of GitHub PR stuff. - It's using structured logging now - Seperated the metrics adapter and the activation into seperate - For jobs, we still use the same ScaledObject. Still would like to see split in the future, but for now it is the same - Rest of big changes are related to using the Operator SDK - Once the PR is merged, Zbynek can put this on OperatorHub.io. Only thing that is needed for that. - Ahmed working on refactoring the scalers to use the new auth spec, and getting pod identity to work with Service Bus scalers ## Oct 10 Meeting Today unfrotunately due to other commitments there wasnt any particpation in the call other than Anirudh. Just adding notes for the offline discussions we had during this week on KEDA. ### Proposed Agenda (feel free to edit and add): - None ### Notes - Zbynek had earlier indicated that he was close to getting the Operator SDK done. He had proposed to split the HPA and the Activator into seperate containters within the KEDA pod which we decided to go with. Further he had to refactor his implementation to work with the implementation where-in job scaling was included with the deployment scaling which Zbynek was ok with. ## Oct 3 Meeting ### Proposed Agenda (feel free to edit and add): - None ### Notes - Base auth spec is implemented, Ahmed looking to add to the queue scalers - Ahmed working on a GitHub Actions to automate release - Ahmed also looking at the Jobs PR sometime this week (Steven pingd him) - Shekhar had a question on how we deal with changes of the auth spec CRD. Thinking is that every time we do a check we'll get the latest version of each to do the check. - Zbynek still working on Operator SDK. Hoping to have a PR in the next couple of days. Uses some different logic for looping around resoures, so lots of changes to the operator loop that will need to be reviewed. - In terms of the authentication spec - it will be in the PR Zbynek is opening (CRD registration) - Also a question on the aggregate API server (listens to metrics). While not part of the SDK, Zbynek reusing code to light that up. - Cliff has a working Artemis scaler on personal GitHub, and docs on how to use it https://github.com/cliffjansen - Ahmed going to check out the bug on Helm chart deployment ## Sept 28 Meeting Proposed Agenda (feel free to edit and add): - Experimental Jobs Functionality (Added by Tom) - Testing out AAD Pod Identity ### Notes - Ahmed worked and merged in the PR on Authentication spec. He will be looking at some of the Azure scalers to implement it end-to-end - Discussion on if it makes sense to seperate or keep the same. Especially with 1.0 coming. Steve wrote most of the code and felt the amount of code share was high enough to justify same. ```yaml apiVersion: keda.k8s.io/v1alpha1 kind: ScaledObject metadata: name: azure-queue-scaledobject-jobs namespace: default spec: pollingInterval: 40 maxReplicaCount: 4 cooldownPeriod: 10 jobTargetRef: parallelism: 1 completions: 1 activeDeadlineSeconds: 60 backoffLimit: 6 template: containers: - name: consumer-job image: sgricci/queue-consumer env: - name: AzureWebJobsStorage valueFrom: secretKeyRef: name: secrets key: AzureWebJobsStorage triggers: - type: azure-queue metadata: queueName: queue-name connection: AzureWebJobsStorage ``` - During Sean's expirementation, using Managed Identities (Pod Identity), the KEDA scaler would get a token using Pod Identity, and try to use that token to pass to storage, and spin back an error that said the format was invalid. To workaround, they kill the KEDA pod during deployment and when it recreates everything works (have to delete the pod whenever a new scaled object is created) - [issue here](https://github.com/kedacore/keda/issues/368) - Document the new tagging strategy ## Action Items - [ ] Steve to make discussed changes to PR, ping Ahmed, - [x] Sean to create issue around the "have to delete KEDA" pod identity bug he ran into - [ ] Ahmed looking at potentially doing GitHub release workflow to auto-tag. He'll look at this during the week as well ## Sept 12 Meeting Proposed Agenda (feel free to edit and add): - Merge in external scaler (if not done ahead of time) - Operator SDK - needs help - Prototype for ActiveMQ Artemis - Where to put samples ### Notes - Tagged a version of KEDA to :0.2 which is a "safe" version before we start to do some breaking changes. - Zbynek can start working on the operator SDK - but did flag that it may require some big changes in things like how the controller loop would function, so he wanted to make sure we were good with this change before taking the effort - https://github.com/operator-framework/operator-sdk - Potentially could look at the namespace feature as part of the operator SDK work. One of the "out of the box" features of Operator SDK - We may need some specific ## Action Items - [ ] Get Yarons branch - [X] Jeff to create README on samples - http://github.com/kedacore/samples ## Sept 5 Meeting ### Notes * Finalized and merged in the auth spec PR * Shekhar making last changes to the external scaler support. He has a bunch of samples with those * Shekhar going to look at implementing AAD Pod Identity basic support * Aarthi looking at her issues as well * We got a session at KubeCon San Diego ## August 29 Meeting Proposed Agenda - Finalizing the proposal for defining authentication parameters - Jobs on KEDA - Current Status, Community Input, Discussion on how to proceed with docs - Making progress on the External Scaler support for Keda - Any other issues ### Attendees Ahmed Shekhar Tom Steve Anirudh ### Notes - Finalizing the proposal for defining authentication parameters: - There are two different approaches suggested on the PR - whether authentication is part of the trigger (or) authentication is via another newly defined entity (Trigger Authentication approach). - We debated the pros and cons of both approaches and decided that we will go for the seperate entity approach. However this will mean that we will have to change all the scalers to use this. (will open issues for this) We also spoke of versioning in this context. For now we decided that we will *not* version and continue with the current version. - External Scaler support for Keda - Ahmed has reviewed it and has a minor comment which Shekhar will fix - Shekhar will also add a sample - Tom will also have a look at it. - Overall looks good to mer - Jobs functionality - Came out of the work by Microsoft CSE for a customer who is using KEDA in production. Ahmed had a look at it and did not find any issues. Use case is events come in -> GCP/Azure queue, containers run to completion (long running jobs). Shekhar was added for the review. - Steve will post an example spec for Azure Queue for this. Tom was wondering if we wanted to consider making this a different CRD. He will give some comments. No changes in the Scalers themselves. - Overall the feeling was that we have to have better governance around new issues. Also would want to see if we want to identify owners for different scalers. ### Action Items - [ ] Shekhar will submit a PR for the Authentication parameters work in the scale controller. - [ ] Tom will create different issues for each scaler for the Authentication work. - [x] Tom will update the PR for Authentication parameters work saying that we decided the seperate entity approach and ask for final comments on the approach. - [ ] For external scaler support - Shekhar to fix minor comment from Ahmed and submit for merging - , he will also add a sample. ## August 15 Meeting Proposed Agenda (feel free to edit and add): - External Scaler support walkthrough (Shekhar) - Dashboard community support [repo here](https://github.com/kedacore/dashboard) ### Notes ### Action Items - [ ] Jeff to Setup an additional meeting on extensible scalers directly - [ ] Jeff to Add known issues with help wanted flags to dashboard repo - [ ] Priya / Lee to take a look at the auth one. They'll meet with Sean and see what issues they could help with - [ ] Jeff to track down Yaron directly and understand operator SDK. ### Action Items ## August 8 Meeting [Meeting recording](https://youtu.be/_Gz3ZnXL07g) - KEDA Dashboard demo starts at 05:30 timestamp Proposed Agenda (feel free to edit and add): - KEDA Dashboard demo (alpha) - Shazmin - **Viewable in meeting recording above** ### Notes - Israel working on some more tutorials on scheduling (videos) - Shazmin demod the dashboard for KEDA. Will post recording. - Shekhar brought up we likely want to integrate with prometheus by default to pull the logs - Shehar also brought up we should look into authentication options - mirroring KubeDashboard is likely a good start (kube context or headers) ### Action Items - [x] Aarthi and Ahmed about reviewing PRs ## August 1 Meeting Proposed Agenda Items (feel free to edit and add): - Image releases - HTTP Scaling Sample - Helm Chart Notes - KEDA Job Management (issue #199) - AAD Pod Identity / Getting secrets - 1.0 release overview ### Attendees * Aarthi * Jeff * Joey - Microsoft * Lee * Priya * Sean - Microsoft * Steve - Microsoft * Tom Kerkhove * Zbynek * David ### Notes - Aarthi - Shazmin almost done with a KEDA dashboard. It still needs to be integrated with a better logging story that can scale but it looks good. Next week will demo and get feedback. Plan is to create a seperate repo for the dashboard, and extend our helm chart to also pull in dashboard. - Some discussion on how best to package up in a helm chart and depending on other charts. We do something like this today with Osiris. - CLI would likely pull from the same "APIs" that the dashboard would pull from. - [Tom held a meeting and created a PR on AAD Pod Identity](https://github.com/kedacore/keda/pull/307) that the community should review and provide feedback. - Some additional work done on the job dispatch work. Steve has been working on [Lee's concept](https://github.com/lee0c/k8s-jobs) and is looking to add the PR for supporting jobs. ### Action Items - [ ] Community to review and provide feedback on authentication concept - [x] Jeff will help break out into issues per scaler - [ ] Jeff to create issue and figure out scalable way to do docs, FAQ, and samples - [ ] Consider HTTP samples as well - [ ] Helm chart notes should be updated to point to this when available - [ ] Lee to create an issue for HPA labeling - [x] Jeff to assign jobs issue `seushermsft` - [ ] Aarthi / Ahmed / Jeff to review the gRPC PR, AAD Pod Identity, Jobs (to be created) ## July 25 Meeting Proposed Agenda Items (feel free to edit and add): - Scaler & Trigger specs - AAD Pod identity update - Pluggable scaler ### Notes * Scaler & Trigger specs are moved to [Git repo](https://github.com/kedacore/keda/tree/master/spec) * Allows non-Microsoft folks to send PRs with proposals * Owners of scalers should provide docs in the spec section * Proposal PR for authentication changes in scaler spec is [open](https://github.com/kedacore/keda/pull/307), feedback/concerns are welcome * Pluggable scaler will re-review it and ping other folks in Microsoft * You can write scalers in any language that comply with the gRPC scheme * Scalers can be shipped seperately * Allows to create a community of scalers * Might need a portal over time to group all of the scalers in the wild * Similar to hub.helm.sh who are open for community Helm charts * Progress on portal is pretty good, demo might follow next week * What is the authentication that will be supported? * To be asked next week * Will be seperate pod which you can deploy ## July 18 Meeting Proposed Agenda Items: - Status updates - Technical meeting for AAD Pod Identity is being planned, see [GitHub issue](https://github.com/kedacore/keda/issues/161#issuecomment-512339124) ### Notes * Priya has been looking at doing a PR - not for AAD Pod Identity - but allowing both allowing both env variables or secret config for connection strings. Can potentially align this with the AAD Pod Identity spec from Tom K. (#161) * May just be a workaround and additional option until AAD Pod identity or something non-connection string specific is enabled. Open question on how you could authenticate with things like AWS SQS if you don't want to put connection strings directly in a configmap or secret. * Shekhar put in a PR for AAD Pod Identity for storage queues * Aarthi mentioned we may need a meeting on the spec change from Tom * Similar approach could be used for Azure Service Bus * Lee brought up that maybe we could design in a way that wasn't specific to each scaler specifically * Aarthi was looking to understand how the Azure Functions container could successfully "fail" so that Kubernetes and KEDA could cleanly have a fail-mode to prevent scaling. * KEDA dashboard progressing from Yazmin. She's making some updates to logging so that's it's parse-able by the UX * Hoping to have a first release of the dashboard in next 2-3. * Potentially do a demo in next week or following week for demo. * :warning: folks who are contributing may need to rebase * Shekhar opened a PR and proposal for pluggable scalers #294 * Anirudh working on FAQ to release in a couple of days * Event Hubs had some minor bugs, update has been pushed out. * Some discussion about how frequently we release and update `:latest`. Maybe for now we do once a week. * To help mitigate around CI/CD with scalers connected to many sources, some may be "core" scalers, and others become "extensions" that could be versioned seperately? * Today once a day we have some extensive E2E tests that runs in a Microsoft-sponsored Kubernetes cluster. A description in the `/tests` folder of the project ### Action Items - [x] Schedule a seperate meeting specifically for Tom K. spec review. Tom to schedule - [ ] Shekhar to move some of the pub-sub samples into the KEDA org - [ ] Jeff to add Shekhar to the KEDA Organization for rights - [ ] Everyone to provide feedback on the event hubs scaler ## July 11 Meeting Proposed Agenda Items: - AAD Pod Identity Configuration ([#161](https://github.com/kedacore/keda/issues/161) - Tom Kerkhove) - Check on the process for contribution to the KEDA repo - Technical review to be scheduled for the AAD Pod Identity. Add Aarthi, Ahmed, Anirudh and Israel as optional. - KubeCon workshop interest in participation. - Question about Durable Functions supported on KEDA? - Not yet but on the roadmap. ## July 4 Meeting Cancelled - US Holiday ## June 27 Meeting Proposed Agenda Items: - Intros - Status Updates - Event Hubs Scaler Demo ### Attendees Rhuss (RedHat), Lance (RedHat), Lee, Shekhar, Israel, Shazmin ### Notes - https://github.com/kedacore/keda/issues/270 - Asavari to follow up - Clarification around where to open issues for components not specifically located in the kedacore/keda repo - we should open issues in the specific repository where the code is located but no harm crosslinking to the core repo for triage during weekly standups. - Shekhar expressed interest in helping with the dashboard implementation, along with Shazmin. - Shekhar finished the scalers for Redis and GCP Pub/Sub. Working on creating samples. ## June 20 Meeting Proposed Agenda Items: - Intros - Status Updates - Proposed topics (feel free to edit and add topics) - AAD Pod Identity ([#161](https://github.com/kedacore/keda/issues/161)) ### Attendees Tom, Asavari, William, Yaron ### Notes - Reached a 1,000 stars for the repo! - AAD Pod Identity - Not a blocker but we're interested in having it align with the 1.0 release to serve multiple customer scenarios. Help wanted on the issue. - Need to create GitHub releases for the KEDA image. Already have CircleCI but need to have nightly builds to update the master. Asavari to open an issue. - Yaron's going to provide a set of scenarios for the Prometheus samples. ## June 13 Meeting Proposed Agenda Items: - Intros - Status Updates - Proposed topics (feel free to edit and add topics) - .NET Core 3.0 Worker sample is added (#216 - Tom K.) - Status update on Helm chart improvement ### Questions - Is there a timeframe in which KEDA v1.0 is aimed to be released? (Tom K.) ### Notes - v1.0 is tracked as part of the project here - https://github.com/kedacore/keda/projects. If the current momentum continues, we're looking at H2 for the release. - (Tom K.) Opened a PR to document the helm chart configurations. Logging setup needs review. Ahmed ElSayed is looking into it. - (Tom K.) Open question - Is AAD Pod Identity in scope for v1.0 - https://github.com/kedacore/keda/issues/161. Asavari to follow up. - (Shekhar) Made a PR for SQS scaler. Will be looking into providing a sample next. - (Yaron) Operator SDK work is blocked on https://github.com/kedacore/keda/issues/54. Need help verifying, Yaron will open an issue. - Event Hub scaler work progressing well. Validation in progress. - PR already open for the Prometheus scaler. Next step is to provide sample queries for HTTP and Azure Monitor. Asavari to open an issue. ## June 6 Meeting Proposed Agenda Items: - Intros - Status Updates - Proposed topics (feel free to edit and add topics) - KEDA Dashboard (Asavari) - Long running feedback (Lee) - .NET Core 3.0 Worker sample (#216 - Tom K.) - Feel free to review so we can transition it to the KEDA org - Questions - Is there a timeframe in which KEDA v1.0 is aimed to be released? (Tom K.) - Notes - Event Hubs SDK work progressing well. - Shekhar interested on providing some samples - William brought up a good point that we could look at adding a VS Code plugin. Created a placeholder issue, but should look into see what's possible, potentially even extending the existing Kubernetes extension somehow? Assigned to Asavari and myself to investigate with the VS Code team. - Lee been working with customer on long running workloads. It is *possible* to have a very long running SIGTERM delay. It sets the status as "terminating" which can be a bit confusing for the user. Also looked at using Kubernetes jobs (run to completion), has a POC of that working. Does run to completion which is nice - so job status can be trusted. * Proposal to allow KEDA to act as a job dispatcher and cleaner based on metrics * Potentially scoped to queue-only triggers? Or at least optimized for it - Action Items - [x] Jeff to add questions on job scaler to issue - [ ] Lee to use issue #199 for a proposal of the job dispatch functionality ## May 30 Meeting Proposed Agenda Items: - Intros - Status Updates - Proposed topics (feel free to edit and add topics) - KEDA Dashboard (Asavari) - KEDA and Jobs (Lee) - 1.0 Release review - Notes * Shazmin working on Event Hubs Go SDK support for checkpointing and partition data * Aarthi working on certs and auth work items, to give update on other ones * We have the release project setup with the deliverables currently in scope for 1.0. We broke off a seperate column for items that are more focused on the "Azure Functions on K8s" feature-set which may extend past KEDA. * Lee had a meeting around jobs and scaling. Thinking was to intercept SIGTERM and delay terminiation until long running job completes. Similar suggestions in the sig-autoscaling K8s group - Action items - [ ] Jeff to email lee about long running approach ## May 23 Meeting Proposed Agenda Items: - Intros - Status Updates - Proposed topics (feel free to edit and add topics) - Demo of KEDA autoscaling Knative Kafka Event Source (Ben B.) - Status of the 1.0 release project ### Attendees Anirudh, Jeff, Aarthi, Ben, Israel, Lee, Asavari ### Notes * Ben showed the work he'd been doing with Knative. A few options on how to do this. One of them was to just integrate with Knative and KEDA with no changes and let KEDA help drive the scaling of the Kafka consumer. Also another world where Knative could also support "container pull" based deployments. * Lee had some asks around supporting long running jobs and some PoCs around [this issue](https://github.com/kedacore/keda/issues/157). For the "Kubernetes Jobs" strategy it looks like the pattern would be 1 queue message per job. TTL is still in alpha, looking at cron jobs as well. Doesn't seem to be a very good way to remove completed jobs or select on status. Ben also brought up the point where you may be able to modify the label so that the pod is no longer part of the deployment. HPA may react to that but at least the pod would continue to live and KEDA could then terminate the pod? ### Action Items - [x] Jeff to talk to Cosmos DB team about Go support for their SDK - [ ] Jeff to talk to Ahmed about reviewing PR - [x] Jeff to create issue about kubevalidator and helm - [x] Jeff to flag Aarthi on the multiple signal issue ## May 16 Meeting Proposed Agenda Items: - Intros - Status Updates - Proposed topics - `Needs Discussion` tag - Release criteria label / project - Operator SDK / hub - Support for multiple resources under same metric (#194 - Tom K.) - CloudEvents & Kubernetes Events (#165 - Tom K.) - Streamlining the Helm Chart (Tom K.) - Long running workloads ### Attendees Jeff (MSFT), Ben Browning (Red Hat), Tom Kerkhove (Codit), James Sturtevant (MSFT), Lee Cattarin (MSFT), Thiago Custodio (MNEO), Ahmed ElSayed (MSFT), Israel Ekpo (MSFT), Yaron Schneider (MSFT), Asavari Tayal (MSFT), Luke Kim (MSFT) ### Notes * Ben looking to open PR around autoscaling and Knative * Loop into the HPA "scale to zero" capabilities that are slated for 1.15 alpha * Some discussion around the "Long running workloads" work item. Issue has details * CloudEvents driving scale in Kubernetes. Regardless there is likely a CloudEvents SDK that could be leveraged, but still an open question of how the event makes it way to KEDA. Knative auto-TLS is the way that Knative has done this as well. Hybrid connections? * Operator Hub / SDK - will have an update for next week. Most of code already written but have to sort out dependency conflict. Likely land by mid-June. Ben is working to get things on operator hub right now that are doing more around operator hub. * Helm charts - adding KEDA to hub.helm.sh (Tom) * Scaling a single resource on multiple event sources (multiple queues that all scale a single deployment) * **Everyone** start to flag issues that should move to the "RELEASE" milestone for the 1.0 release ### Action Items - [x] Jeff to link to the HPA capability for scaling to zero on Prometheus issue - [x] Jeff to create an issue on aligning with the HPA scale to zero for custom metrics - [x] Jeff to create an issue and assign Tom to add KEDA to hub.helm.sh - [ ] Israel to create a PR and sample for Kafka. Potentially also including how you could scale kafka cluster ## May 9 Meeting Proposed Agenda Items: - Intros - Status Updates - Proposed topics - Azure Event Hubs trigger - Dashboard - Operator SDK - OperatorHub.io - Prometheus ### Attendees Jeff, Aarthi, Asavari, Anirudh, Yaron, Israel, Tom, Eduardo, Luke ### Notes * Yaron is going to be working on the Prometheus scaler so that you can scale based on any prometheus scaler, working on the string / int mapping, operator SDK would be later * Aarthi is working on the Event Hubs scaler, and look to validate the minikube work * Anirudh to work with Aarthi * Some discussion about Kubernetes Event scaler * Israel to work on some samples on things like Service Bus * Tom will provide a sample of using the Prometheus scaler with Azure Monitor once Yaron finishes ### Action items - [x] Jeff to create an issue for minikube - [x] Jeff to create an issue for Kubernetes Events and add help wanted - [x] Jeff to create a doc issue for documenting the HPA - [x] Jeff to create a management dashboard issue - assign Asavari - [x] Create item for FAQ (likely would be a wiki). Is KEDA Azure Functions only? How does this relate to Knative? I'll assign Anirudh - [x] Jeff to create an item on a sample on Azure Monitor -> Prometheus - [x] Jeff to create an issue to add samples for triggers we don't have samples for. Israel interested in helping with Service Bus samples (izzymsft)