# Akri Developer Meeting Notes 2022 ###### tags: `Meeting Notes` Akri Developer calls take place every first Tuesday of the month at 9AM to 10AM PST on [Zoom](https://us02web.zoom.us/j/86009445047?pwd=VEd1dlBkK2NGbmlFYXd2NktvbWVqdz09). Anyone is welcome to join this call and participate. Please abide by Akri's [Code of Conduct](https://github.com/project-akri/akri/blob/main/CODE_OF_CONDUCT.md). The meetings are recorded and published afterwards on the [Deis Labs YouTube Channel](https://www.youtube.com/channel/UC90VZDjT8C7ca7zcuFi6oEQ). If you would like to propose an agenda item, add a comment to the Agenda section under the specific date. For any other questions or comments, message [Edrick Wong](https://kubernetes.slack.com/archives/DL0HAKMLJ) or [Kate Goldenring](https://kubernetes.slack.com/archives/D01HN3DJL2V) in the [Kubernetes Slack Channel](https://kubernetes.slack.com/messages/akri) ___ # May 3rd ## Assignments | Assignment | | | ------------ | --- | | Moderator | [Edrick](@LcYkPeeDR-Ky_8hTgDDLDg) | | Notes | [Roaa](@Roaa) | | Issue Triage | [Kate](@Kate) | ## Announcements - [Akri Office Hours at KubeCon EU](https://sched.co/11XBk) from 12:30-13:15 CEST - Akri Booth at KubeCon EU during AM hours ## Agenda - Discuss enabling Akri to discover authenticated devices which requires passing credentials to Discovery Handlers. This enables [discovering IP cameras that require authentication](https://github.com/project-akri/akri/issues/250) and ensures that Akri is only creating Kubernetes Resources for trusted devices. - [Kate](@Kate) - Bluetooth Discovery Handler investigation - Jake Gallow - Decluttering Akri's main repository -- moving out samples and tools (OPC UA cert generator) - Add an agenda item by commenting here - Issue triage - Open Discussion ## Discussion - Announcements: See you at kubecon! - Discussions around discovery handler authentication - Currently broker is responsible for authenticating with device before using it. No way to report back that device is not trusted. - Need a way to pass credentials to discovery handlers. Is this configured via the Akri Configuration? - Add more details on how the creds can reach discovery handler - Document the proposal in akri docs while in parallel starting on a poc. - Bluetooth discovery handler investigation - Overview of bluetooth by Jake - Discussion around publiclly accessible info (like mac address) vs info that needs pairing - More drilling required to fully understand that. - Next steps, document the proposal in akri and discuss more next meetings. - Decluttering Akri main repo: - OPCUA cert generator as an example - Other things can be moved to a separate repo: discovery handler, broker sample, sample apps. - Triage ## Assignments for Next Meeting | Assignment | | | ------------ | --- | | Moderator | | | Notes | | | Issue Triage | | --- # April 5th ## Assignments | Assignment | | | ------------ | --- | | Moderator | [Roaa](@Roaa) | | Notes | [Vince](@vinguyen) | | Issue Triage | [Kate](@Kate) | ## Announcements - KubeCon booth and virtual project office hours (announced March 9th) ## Agenda - Issue triage - Open discussion - Add an agenda item by commenting here ## Discussion Issues to prioritize for next release: - https://github.com/project-akri/akri/issues/450 - https://github.com/project-akri/akri/issues/379 Note: Today we will focus on triaging issues ### KubeCon announcement KubeCon EU is in May. We get to have our own booth and virtual office hour (announced March 9th). We will have 30 minute sessions and anyone can hop in. ### Triage issues We triaged these issues: - Issue 464: https://github.com/project-akri/akri/issues/464 Move it to in progress. Fabian is tracking it. - Issue 467: https://github.com/project-akri/akri/issues/467 Clippy error: use of ok_or followed by a function call. Marked as good first issue. Moved it to in progress. It is breaking the build right now so we should prioritize it. It is probably clippy updated their criteria. Assigned to Jake. - Issue 445: https://github.com/project-akri/akri/issues/445 This is an issue with kernel module not showing up. Suspected kernel module not being set up. More information needed. Following up with Elijah Moved it to investigating. - Issue 375: https://github.com/project-akri/akri/issues/375 When we initial made the webhook, we made it from sctach. Now that kube-rs has its own webhook admission controller, we should just their admission controller. Moved it to backlog. This is an enhancement issue. - Issue 376: https://github.com/project-akri/akri/issues/376 Krustlet preferred rwlock over mutex. Rwlock has better performance for multi readers, one writer scenario. Moved it to backlog. - Issue 192: https://github.com/project-akri/akri/issues/192 Stale issue. We would like to keep this issue alive. Moved it to in progress. - Issue 342: https://github.com/project-akri/akri/issues/342 Moved it to backlog We also looked at stale issues: - Issue 416: https://github.com/project-akri/akri/issues/416 It is about adding node selector. Pinged Kelly to confirm whether this is being worked on. - Issue 400: https://github.com/project-akri/akri/pull/400 - Issue 398: https://github.com/project-akri/akri/pull/398 Roaa to follow up on chrono and security issues Validate again after dependency updated - Issue 346: https://github.com/project-akri/akri/issues/346 Added keep-alive label. Removed stale label. - Issue 283: https://github.com/project-akri/akri/pull/283 We are not spending time on this. - Issue 174: https://github.com/project-akri/akri/issues/174 For speaker and headphone. (We could use event plug in/unplug?) One of our goals is to have different protocols implemented so we decided to keep this issue alive. Jake would like to take a look. Removed stale label. Suggestion: we could create protocol label? We also looked at a few closed issues: - Issue 384: https://github.com/project-akri/akri/issues/384 - Issue 352: https://github.com/project-akri/akri/issues/352 We also looked at open issues: - Issue 447: https://github.com/project-akri/akri/issues/447 Closed - Issue 446: https://github.com/project-akri/akri/issues/446 Agent pods failing and broker restarting issue. ### Roadmap We also discussed about roadmap docs - New broker deployment strategies should be updated now that we have job broker. - Updated " Protocols we would love to be contributed" - Sort the list of protocols by priority - Zeroconf - Bluetooth ### Open discussion - We have a big feature last release: jobs. - We should think about big features for next release and priority them. - Modify agent to reduce frequency of Pods getting unexpected admission error: https://github.com/project-akri/akri/issues/450 - Camera get updated scoped to some different name, we don't update because we are using IP/MAC. We should check identity property. - Create new set of builds with different k8s versions for back-compat - Zeroconf (MDNS based devices with discovery on top of it) ## Assignments for Next Meeting | Assignment | | | ------------ | --- | | Moderator | | | Notes | | | Issue Triage | | --- # March 1st ## Assignments | Assignment | | | ------------ | --- | | Moderator | [Roaa](@Roaa) | | Notes | [Edrick](@LcYkPeeDR-Ky_8hTgDDLDg) | | Issue Triage | [Brian](@bfjelds) | ## Announcements - [CNCF Live Webinar: Discovering and Managing IoT Devices from Kubernetes with Akri](https://community.cncf.io/events/details/cncf-cncf-online-programs-presents-cncf-live-webinar-discovering-and-managing-iot-devices-from-kubernetes-with-akri/) **right after this meeting at 10 AM PST** - Akri **release v0.8.4** out! ## Agenda - Release features walk through - [Kate](@Kate) - Bug Bash recap - [Kate](@Kate) - Open discussion - Add an agenda item by commenting here ## Discussion ### CNCF Webinar * First CNCF live webinar, one of the benefits of being a sandbox project * right after this developer call * Lastest release has Kubernetes Job support, opens up more scenarios * All recordered [Link will be added here] ### Akri New Release * Breaking change * What you can deploy either Kubernetes Jobs or Pods * crictl is embedded in the agent container * device plugin interface doesn't support deallocate * ensure that ever pod that is running is still running * Roaa is tracking the deallocate problem with SIG Node #### Validating on Lastest Kubernetes * GitHub runners needs to be updated with containerd * Is expected to work on lastest Kubernetes but it hasn't been validated on vanilla Kubernetes ### [Community Bug Bash](https://hackmd.io/@akri/HkDDHo6J9) * Great way to get started with Akri * The scenario is still whole * Was testing the three new features * We got some issues and Kubernetes support * Side note, (Katacoda)[https://www.katacoda.com/] is great * Other communities do this too * somethimes they take a long time * ours were nice and concise ### Jonathan from Goliath **Summary from Johnathan** Shared my exploration for using Akri for IoT device resource sharing for test/test automation. Here's some of the links I shared: * https://validation.linaro.org/ * https://labgrid.readthedocs.io/ * https://www.segger.com/products/debug-probes/j-link/ * https://www.iot-lab.info/docs/getting-started/design/ * https://en.wikipedia.org/wiki/JTAG The two devices I held up: * https://www.espressif.com/en/products/socs/esp32-c3 * https://docs.zephyrproject.org/2.7.0/boards/arm/blackpill_f401ce/doc/index.html **Additional notes** * exploring device testing, in the world IoT, there is no common CICD * how to share devices, especially in remote working * customers want to test a device * test eng writes scripts, waterfall testing * most of them have a USB itnerface that has the same device ID * managing this is difficult * LAVA from Linaro is one tool * labgrid DSL * mostly custom scripts in pytheon/PERL * anything in the cloud native world * Akri maps the model * site manager is a NUC (i3), dedicated container that is running the shell scripts/toolchain/CI scripts * solves a lot of classic problems * provide fundamentals for automated architecture * Two devices * both of different update stack * one has debug port and one doesn't * have to write it in their DSL or bash script * Currently there's less focus on protocols * Assumption ins none of these devices have software on it * when you're testing, there's nothing running on it, need a boostrapping process to get stack on it * How do we share these devices as resources * If you can't identify based on udev, basically identity based on pluggin order * some devices have unique USB ID * or jtag/ debugger inbetween * FIT IOT-Lab * need to run lots of test on lots of devices * Shared devices * container associated with a particular device * webapp, I would like this device in X location * get the container that has all the tools associated with it * Bootstraping scenario * multiple ways to flash firmware (>3) * is there a concept of bootstrapping process? * if there is a well defined process? * USB isn't only protocol * large scale deployments, USB becomes a bottleneck * ethernet has bigger bandwidth ## Assignments for Next Meeting | Assignment | | | ------------ | --- | | Moderator | [Kate](@Kate) | | Notes | [Brian](@bfjelds) | | Issue Triage | [Roaa](@Roaa) | --- # February 1st ## Assignments | Assignment | | | ------------ | --- | | Moderator | [Edrick](@LcYkPeeDR-Ky_8hTgDDLDg) | | Notes | [Roaa](@Roaa) | | Issue Triage | [Brian](@bfjelds) | ## Announcements - Support for deploying Kubernetes Jobs as brokers in the development Helm chart! ## Agenda - Demo of deploying Jobs to discovered devices with Akri - Discuss creating a community Jobs Bug Bash - Suggest agenda items by leaving a comment here! ## Discussion - Annnouncements: Developer helm chart for brokers as jobs merged and ready for usage. - Demo: - Current documentation explains how to use main chart or a dev helm chart, set discovery handler to deploy and specify what configuration to create. Following the same steps, now akri also has jobs as an option for a broker. - Akri already supports for brokers as non terminating pods like pod continously fetching the temperature.. etc. Now Akri added support for specifying a job (terminating pod) where user want to run a job once (like an upgrading scenario or maintenance job) - Demo using mock ip camera and doing an upgrade. - Bug bash discussion: - Brainstorm scenarios, suggestion to open discussion on slack to get more scenarios including discovery protocols in progress. - Suggestion of using a K8s playground (ie: KataKoda) for bug bash for a quick start. - Documentation of using jobs is in PR, not merged yet since helm chart changes are only in the dev helm chart. We can still point to it for bug bash. - @Kate to share more details on the slack channel. - Next release: - Target mid Feb, pending the bug bash and merging the jobs as brokers PR. ## Assignments for Next Meeting | Assignment | | | ------------ | --- | | Moderator | | | Notes | | | Issue Triage | | --- # January 4th ## Assignments | Assignment | | | ------------ | --- | | Moderator | [Kate](@Kate) | | Notes | [Roaa](@Roaa) | | Issue Triage | [Brian](@bfjelds) | ## Announcements - [PR](https://github.com/project-akri/akri/pull/437) up for supporting deploying Jobs to discovered devices ## Agenda - Birds of a feather discussion - Suggest agenda items by leaving a comment here! ## Discussion - Happy new year! 🎉 - PR with using terminating pods as brokers is active - It is a breaking change, but extends functionality and allows using jobs features. More details in the PR. - Right now only supports one job per device, may change in the future. - Looking at demos on Akri's website. Currently there's a demo for a couple of supported discovery handler protocols, we are missing one for onvif. Contributions and demo ideas are welcomed! - Issues triage ## Assignments for Next Meeting | Assignment | | | ------------ | --- | | Moderator | | | Notes | | | Issue Triage | | --- # December 7th ## Assignments | Assignment | | | ------------ | --- | | Moderator | [Kate](@kgoldenring) | | Notes | [Brian](@bfjelds) | | Issue Triage | [Roaa](@Roaa) | ## Announcements - [Call for Proposals (CFP) for KubeCon EU](https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/program/cfp/) due by December 17th. Ideas on what we should present about Akri? ## Agenda 1. Instead of PR title [FLAGS], can now add `same version` and `build dependency containers` labels to PRs via commenting to (not) execute certain workloads [[PR](https://github.com/project-akri/akri/pull/426)] - Vincent 2. Review [proposal](https://github.com/project-akri/akri-docs/pull/17) for supporting the deployment of Jobs as brokers in Akri - [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) 3. Review [proposal](https://github.com/project-akri/akri-docs/pull/18) for supporting multiple deployment strategies in the Controller - [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) 4. Comment here to add an agenda item ## Discussion * Announcement regarding Kubecon and a call for ideas about Akri presentations. Have discussed the possibility of an IoT panel. Searching for questions, participants, etc. * PR Workflow improvement! Previously, text-based flags in PR/commit title were used to control action/workflow (`SAME_VERSION`, etc). Intent is to use github-native concept (labels) to control this. Labels can be added using comments in PRs. * Deployment expansion!! How do we expand Akri's scenarios beyond the microservice or "daemon set" style (essentially using nodes to gain high-availability) deployment? * Proposal one: pods that finish/terminates without being restarted * https://github.com/project-akri/akri-docs/pull/17 * Leverage Kubernetes `jobs` concept * Requires changes to configuration CRD: additional property jobspec * Does `broker` become confusing? Is it still an applicable idea for Akri? Is it too restrictive? * Demo!! * Proposal two: deploying outside the daemon-set * https://github.com/project-akri/akri-docs/pull/18 * Possible scenarios: * achieve high availability with pods rather than nodes * brokers that require more than one configuration * deploying to a configuration rather than an instance * Breaking changes: * Configuration-level resource requesting * Breaking Configuration into 2 CRDs (discovery and usage) * Potential issue: * Pending pod creation ## Assignments for Next Meeting | Assignment | | | ------------ | --- | | Moderator | [Brian](@bfjelds) | | Notes | [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg)| | Issue Triage | [Kate](@Kate) | --- # November 2nd ## Assignments | Assignment | | | ------------ | --- | | Moderator | @Roaa | | Notes | [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) | | Issue Triage | @bfjelds | ## Announcements 1. New [project-akri](https://github.com/project-akri) organization 2. Akri's 1 year anniversary was October 20, 2021. [Celebratory blog](https://deislabs.io/posts/akri-a-year-later/) 3. Add other agenda items by leaving a comment here! ## Agenda 1. Discuss Akri's [Governance Doc](https://github.com/deislabs/akri/blob/main/GOVERNANCE.md). Feedback on criteria for moving up the contribution ladder? - [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) 1. Birds of a feather discussion. ## Discussion * KubeCon * Kate and Rods presented Akri on Krustlet [link added later] * Very exciting, WASM is secure by default and size reduction is significant * WASM still new, some features are missing, such as sockets * Akri organization * Akri is CNCF Sandbox now * Everything is open source (including the landscape), so everything is a PR * We had to move to an independent org as part of the move * Paid for and supported by CNCF * The main repos are all moved there * We have acces to Snyk now, which does container scanning for vulnerability * will put in a workflow to do vulnerability and license scanning * We have a DCO, which checks every commit is signed * if you are running into this issue, click on the error and there's one liners to fix * our update bot needs to be updated * Have access to service desk and community page in the CNCF * Can have maintainer sessions at future KubeCons * Happy Birthday Akri! * link above * [Governance Doc](https://github.com/kate-goldenring/akri/blob/7909ddc7a29c94985bea4685e0cea933d774f381/GOVERNANCE.md) * General meaning of what Governance means, what legal entity owns the project and how the project is governed and ran * Publically defined roles, how to move from role to role, and what to do when you're retiring from the project * Inspired by opengovernance.dev, and Porter * Our code of conduct also has been updated ## Assignments for Next Meeting | Assignment | | | ------------ | --- | | Moderator | [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) | | Notes | @bfjelds | | Issue Triage | @Roaa | --- # October 5th ([recording](https://youtu.be/QLN-7_Vla64)) ## Assignments | Assignment | | | ------------ | --- | | Moderator | [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) | | Notes | @Roaa | | Issue Triage | [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) | ## Announcements 1. Akri was [accepted into the CNCF as a Sandbox Project](https://github.com/cncf/toc/blob/main/docs/projects.md#:~:text=Sandbox-,Akri,-TOC%20Vote)!! 2. KubeCon NA WASM Day Talk on October 12 1:45pm - 2:15pm PT called [Living on the Edge: Using IoT Devices on Kubernetes WebAssembly Applications](https://events.linuxfoundation.org/cloud-native-wasm-day-north-america/program/schedule/) 3. New Akri release `v0.6.19` 4. Will be moving to new Akri organization: https://github.com/project-akri ## Agenda 1. Changes that come with Akri as a CNCF Sandbox Project - [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) 2. Grace Hopper Celebration Open Source Day outcomes and learnings - @Roaa 3. New Akri release - [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) 4. Akri MQTT POC demo and Web UI discussion ([context](https://github.com/deislabs/akri/issues/384)) - @vinay 6. Add other agenda items by leaving a comment here! ## Discussion - Akri is now a sandbox project in CNCF. what it means for Akri is there may be upcoming changes in docs, zoom links.. etc. Those will be communicated when they happen. Expect minor changes. - Grace hopper open source day recap. Contribution from OSD in PR. - Akri release: - Onvif improvements - Tokio version update - Akri code is moving to a different organization as part of the CNCF migration. - Open discussion on docs release and akri release synchronization. - MQTT demo and discussion: - Scenario is around using akri for asset management. - Currently using a mock MQTT broker from https://github.com/eclipse/mosquitto - Discussion around using a pod watcher vs watching to akri isntances instead. - Issue triage ## Assignments for Next Meeting | Assignment | | | ------------ | --- | | Moderator | @Roaa | | Notes | [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) | | Issue Triage | @bfjelds | # September 7th --- ## Assignments | Assignment | | | ------------ | --- | | Moderator | [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) | | Notes | @bfjelds | | Issue Triage | @Roaa | ## Announcements 1. KubeCon NA WASM Day Talk on October 12 1:45pm - 2:15pm PT called [Living on the Edge: Using IoT Devices on Kubernetes WebAssembly Applications](https://events.linuxfoundation.org/cloud-native-wasm-day-north-america/program/schedule/) 2. [OSD Day](https://ghc.anitab.org/programs-and-awards/open-source-day/) at Grace Hopper Celebration 3. [Updated dependencies](https://github.com/deislabs/akri/pull/361) to latest versions. Now using Tokio version 1.0. 4. [Docs site launched!](docs.akri.sh) ## Agenda 1. Issues and PR Bot for marking `stale` and closing - [Roaa](@Roaa) 1. Add other agenda items by leaving a comment here! ## Discussion * Quick updates on: WASM, Grace Hopper, Dependency updates, docs! * Staying fresh with our new Issues and PR bot ... driven by activity (comments, etc) * Lack of activity for 45 days will lead to stale label * Lack of activity on stale item will lead to closure * This allows us to keep our issues limited to things that are actively being worked on or researched. * Triage: * Frequent changes to `Configuration` resources causing instability (may be related to recent deps update) ... fix in progress * ONVIF duplicate instances detected ... good first issue, put on backlog * `udev` version update needed ... put in backlog ## Assignments for Next Meeting | Assignment | | | ------------ | --- | | Moderator | [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) | | Notes | @Roaa | | Issue Triage | [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) | --- # August 3rd ([recording](https://youtu.be/4dcplLZ56Q8)) ## Assignments | Assignment | | | ------------ | --- | | Moderator | [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) | | Notes | [Vince Nguyen](@vinguyen) | | Issue Triage | @bfjelds | ## Announcements 1. [OSD Day](https://ghc.anitab.org/programs-and-awards/open-source-day/) at Grace Hopper Celebration ## Agenda 1. Akri on Krustlet demo: demo of optimizing an Akri Discovery Handler to run as a WebAssembly module on [krustlet](https://github.com/krustlet/krustlet). - Rods 2. ONVIF updates- [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) 3. Documentation repository (proposals location?) - [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) 4. Issues clean up - should we have a stale issue bot? - [Roaa](@Roaa) 5. [Broker deployment strategies discussion](https://github.com/deislabs/akri/blob/main/docs/proposals/broker-deployment-strategies.md): Where and how many Pods should be deployed to discovered devices? - [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) 6. Add other agenda items by leaving a comment here! ## Discussion ### Akri and Krustlet Trying to port Akri components to krustlet node in order to reduce Akri footprint even further, enable wasm modules to use devices, and join the wasm community 1. First aimed to port Akri DebugEcho (for testing) Discovery Handler to run as a WASM module 2. Wasi constrains: only supports single-threaded environment, limited networking 3. Due to contraints (no sockets support in WASI), created a proxy for the Discovery Handler to talk to the Agent Takeaways 1. Decrease in DebugEcho discovery handler file size from 12M to 172K 2. Start up time decrease from 4s to 3s 3. Can use Akri devices on krustlet Next steps 1. Port more discovery handlers (that discover real devices) to Wasi 2. Port Agent (might need to wait for Wasi improvements since multi-threaded) Amazing demo from Rodrigo! [See meeting recording] Try out the demo [here](https://github.com/rodz/akri-wasi). ### ONVIF updates 1. Optimized discovery handler so only makes calls to device endpoint for filtering once. Reduced CPU limit from 1300m to 24m k8s CPU units. 2. Optimized sample broker, reducing size from 2GB to 800 MB. Still very large but only for demo purposes. 3. Removed errors due to not being able to connect to IPv6 cameras 4. Did IPv6 support investigation. [See discussion](https://github.com/deislabs/akri/issues/298) 5. Did authenticated camera discovery investigation. [See discussion](https://github.com/deislabs/akri/issues/250) ### Doc and repository We are looking to move Akri docs to its own doc repo. It is easer to edit and format. Using GitHub repo we can track and people can add comments/feedbacks easier. Though the proposal docs are more technical. Should technical proposal docs be part of github repo or HackMD? Idea: we will have a repo with 2 folders: 1 for user docs and 1 for technical proposal docs. ### Issues clean up Some Akri issues have been around for a while. A lot of projects have bots to mark issues stale or close them out. Idea: we will add a bot to label old issues as "stale" and add comment to the issue. The bot would also bump and email people so bug filer can reactivate if they like. ### [Broker deployment strategies discussion](https://github.com/deislabs/akri/blob/main/docs/proposals/broker-deployment-strategies.md) ### Triage [Execute workloads based on labels instead of flags in PR titles](https://github.com/deislabs/akri/issues/334) - PR on the way. [Improvements to the "Implementing a new Discovery Handler" Doc](https://github.com/deislabs/akri/issues/342) - Added "documentation" tag. [Switching from MIT to Apache-2.0 License](https://github.com/deislabs/akri/issues/337) - Moved to backlog. [Cluster setup flow for microk8s points to 1.19](https://github.com/deislabs/akri/issues/352) - Investigating. [RUSTSEC-2021-0073: Conversion from prost_types::Timestamp to SystemTime can cause an overflow and panic](https://github.com/deislabs/akri/issues/349) - We are blocked on this because we have to increase tokio version and a versions for a bunch of others. Moved to backlog. [Akri architecture for IoT protocols with standalone devices](https://github.com/deislabs/akri/issues/348) - Moved to investigating. ## Assignments for Next Meeting | Assignment | | | ------------ | --- | | Moderator | | | Notes | | | Issue Triage | | # July 6 ([recording](https://youtu.be/_e3ysRcuCJQ)) --- ## Assignments | Assignment | | | ------------ | --- | | Moderator | @bfjelds | | Notes | @Roaa | | Issue Triage | [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA)| ## Announcements 1. OSD Day at GHC ## Agenda 1. Akri + WASM - Rodrigo 2. MQTT proposal discussion - [Shan Desai](@ShanDesai) 3. GitHub Workflows improvements (Docker Buildx) - [Shan Desai](@ShanDesai) 4. [CoAP ](https://en.wikipedia.org/wiki/Constrained_Application_Protocol) discovery handler development - [Jiayi Hu](@hoy2ZEBmQnS5pWvE2KTZ2w) 5. Add other agenda items by leaving a comment here! ## Discussion ### Akri + WASM * Experimental project to extend WASM support to Akri * Explain benefits of WASM and limitations. * Project Github: https://github.com/rodz/akri-wasi/ ### MQTT proposal: * MQTT comes with the assumption that there are other publishers publishing special topics and that those topics are predefined. Application assumes by subscribing to a specific topic, it will get information back. * Akri's most prevailing benefit to MQTT is the dynamic deployment and scaling based on MQTT broker resources. * Handling devices going offline - MQTT can disconnect from a broker due to network connectivity or other reasons, in those scenarios the proposal is for Akri MQTT discovery handler to leverage `last will and testament concept` to stop advertising such topics and Akri's controller can take down the applications using it. * Defining MQTT Akri resources: Is a single device a resource? A topic? - Current proposal is to include general topics as an Akri resource. Akri can leverage the MQTT wild card option to group topics under general high level topics (ie: temperature.. etc) * Interesting followups * Can apps request akri resources target special topics? or only general ones? * When requesting a wild card topic as a resource, how can Akri make sure it deploys the app to the node including the publisher for that topic? Is deploying to all nodes a good strategy? - @ShanDesai to update the proposal with points discussed on GitHub and in meeting. More discussion offline can follow. ### CoAP: - Discuss different challenges: - Ideally, the agent and discovery handler runs only in a single node or limited number of nodes. Can this be achieved by node selectors? other techniques? - How can akri better incorporate the needs for pub/sub protocols and their architecture? @Giovani to open an issue to discuss further. ## Assignments for Next Meeting | Assignment | | | ------------ | --- | | Moderator | [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) | | Notes | [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) | | Issue Triage | @bfjelds | --- # June 1 [(recording)](https://youtu.be/wZBhxmn1A_0) ## Assignments | Assignment | | | ------------ | --- | | Moderator | @Roaa | | Notes | @bfjelds | | Issue Triage | [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) | ## Announcements 1. [Akri Blog](https://medium.com/akri) -- contribute by writing a post! ## Agenda 1. KubeCon EU take aways - [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) 2. Review a few good first issues that are available - @bfjelds 4. Donating Akri to a foundation - [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) 5. Discussion of MQTT Protocol + Eclipse Sparkplug B Integration with Akri - @ShanDesai 6. Add more items by leaving a comment here ## Discussion * Akri blog introduction #### KubeCon EU take aways * Presented an intro to Akri (117 live viewers, 230 views as of a week later, 100 more views as of now) and a breakout session. * Presentation contributed to gains in slack members, github repo views, and github clones(!). * Q/A session reflected user interest and areas for improvement * questions about ARM, etc * agnostic language support for extensibility * scalability? * stress testing? * device authentication? * iot protocols: * MQTT * roadmap * CNCF sandbox * Submitted for KubeCon NorthAmerica #### Good first items * Migrate from PR title flags to labels: https://github.com/deislabs/akri/issues/334 * Include Akri in `kubectl get all`: https://github.com/deislabs/akri/issues/268 * Use ENTRYPOINT rather than CMD in containers: https://github.com/deislabs/akri/issues/239 * Create Kustomize install: https://github.com/deislabs/akri/issues/182 #### Donating Akri to a foundation - [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) * Success correlated to users, contributors, device support * Donation is the best path towards increasing Akri's success ... but where?? * CNCF seemed natural to us * Akri built with the intent of being K8s native * Steps: #1 sandboxing * Small committment (sandbox process: https://github.com/cncf/toc/tree/main/process) * Offers more visibility * Provide guidance * Other thoughts? * Other sandboxed projects? https://www.cncf.io/sandbox-projects/ * Where to donate depends on what problem you are solving ... for example, if it is edge compute, target edge compute in CNCF (look at other similarly targeted projects like [OpenYurt](https://openyurt.io/en-us/)). * Projects considered every 2 months (next in June) #### Discussion of MQTT Protocol + Eclipse Sparkplug B Integration with Akri * MQTT * publish/subscribe (topics: xxx/yyy/zzz) ... lightweight (vs Kafka, etc) ... proactive * Good for low power devices, perfect for IoT * Many implementations, payloads can use json, xml, etc * Sessions can establish connection * Last Will & Testament mechanisms allow a last gasp communication * Akri & MQTT using Sparkplug B (sparkplug.eclipse.org) * Has standardized specification/implementation for payloads and topic syntax * Start with basic implementation and add configuration as users dictate need * Interesting questions: * Where should the monitoring broker pod run? On an Akri node? Somewhere in the network? * How does discovery work? Is it needed or redundant? * MQTT v5 has new concepts (request/response), is v5 preferable? * Monitoring pod (in architecture diagram) vs MQTT broker? Monitoring pod was borrowed from OPCUA concept. MQTT broker is an MQTT concept (not an Akri broker). MQTT brokers have been stress-tested for 100k devices. * Most common devices: env sensor off-the-shelf (temp, humid, pressure) for prototyping * MQTT apps: time-series data from accelerometer on boat, industrial setting sensor analysis ## Assignments for Next Meeting | Assignment | | | ------------ | --- | | Moderator | @bfjelds | | Notes | @Roaa | | Issue Triage | [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA)| # May 4 ([recording](https://youtu.be/-7paDBcU38U)) ## Assignments | Assignment | | | ------------ | ------------------------------------------ | | Moderator | [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) | | Notes | [@roaa](@roaa) | | Issue Triage | [@bfjelds](@bfjelds) | ## Announcements 1. KubeCon Edge Day lightning talk just happened! [KubeCon EU Session on Akri](https://kccnceu2021.sched.com/event/d7e010d4e3b08507878517f44530363a) in just two days, on Thursday at 3:25 AM PST. ## Agenda 1. Big v0.6.5 release overview - [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) 2. Ideal future release cycle and what should go into the next release? - Birds of a feather moderated by [@bfjelds](@bfjelds) 3. Documentation site - [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) ## Discussion ### KubeCon Akri Talk: * Talk on Thursday. Session will cover DHs and extensibilty model. * Recording will be posted May14th on CNCF website. ### New Release * Full release notes on [Github](https://github.com/deislabs/akri/releases/tag/v0.6.5) * New gRPC interface for discovery handlers * Support for full (wtih embedded discovery handlers) and slim agents. * Discovery handles are organized into library and folder to promote code sharing. * Better documentation on discover handlers & broker development * A webhook validating Akri's configurations. * Support for monitoring using Prometheus, exposing number of instances/brokers. ### Release cycles: * Last release had lots of features. Team will be moving to more frequent and smaller releases. * Moving to a monthly cadence going forward. ### Next release: Team is looking for feedback and feature requests. Some of the discussed features: * Improving onvif: IPv6, authentication scenarios. Open questions around frequency of IP cameras with authentication? * Support more platforms: kubeedge, kind.. * Handling stateful brokers/broker caching: Akri assumes all brokers are stateless, however there are some scenarios where the broker is stateful. Discussed ideas: broker requiring certain configurable grace period if a device is offline, assigning a new device to broker if a device is offline. [Jiayi Hu](@hoy2ZEBmQnS5pWvE2KTZ2w) to create an issue to continue the discussion. * MQTT Akri team to update roadmap and document the strategy on how team decides what is picked next relase (ie: most up-voted issues) ### Website: * Website is on track to be included in next release based of gitbooks. * Decision is to include it in the current akri repo -at least in short term- since it is based of docs in the repo. ## Assignments for Next Meeting | Assignment | | | ------------ | --- | | Moderator | @Roaa | | Notes | @bfjelds | | Issue Triage | [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) | # April 6 ([recording](https://www.youtube.com/watch?v=C4nhzW5aK2k)) ## Assignments | Assignment | | | ------------ | ------------------------------------------ | | Moderator | [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) | | Notes | [@bfjelds](@bfjelds) | | Issue Triage | [@Roaa](@roaa) | ## Announcements 1. KubeCon EU and KubeCon Edge Day 2021 proposals accepted! More information about the event [here](https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/). ## Agenda 1. New Discovery Handler Model in main - [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) 2. New default Agent without Discovery Handlers - [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) 3. More descriptive Helm chart values - [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) 4. Community outreach, where we could use some hands - [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) 5. Modularizing repo structure (should we have separate repositories for Akri core components, a Discovery Handlers, brokers, proto files, etc?) - [@bfjelds](@bfjelds) 6. Documentation site - [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) ## Discussion ### New Discovery Handler * Discovery handlers can live outside of the Agent * You can now write discovery handlers in Go * If interested in writing a Discovery Handler in Rust, we have a template to get started * Slim agent is default, seperated deploy discovery ### Community outreach * Focus has been on K8s community * Include other non-K8s community memebers (devops, etc) ... call for ideas: * github actions & workflows (e2e tests good area for improvement) * Create page for help wanted (issue? dedicated page? dedicated page that points to issues? check out other projects) ### Repo Modularization * Several options * split apart core repo * breaking out swagger for our API * seperate place for our gRPC * docs? * Would having more repos make it confusing for developers who are getting started * Having a documentation site might be useful as the first step if we had several repos * Can we serve up gPRC and swagger with GH Pages like we do with helm? * Start an Issue on this * Examples * https://github.com/dapr * Feedback: * +1 on proto * a scaffold project like Akri Proto + lang* ### Documentation * Standalone docs page using gitbooks ... gitbooks will link to docs folder in repo * Once docs folder and gitbooks are sync'd, no real effort needed to keep in sync * Intent is for this to provide "single source of truth" * Especially important if repos are modularized * Could provide various flows to ease different user roles (akri dev, DH dev, etc) * Should docs be separated out into their own repo (related to Repo Modularization) ### Other * Security audit action, other learnings ... could be really cool for an Akri blog ## Assignments for Next Meeting | Assignment | | | ------------ | ------------------------------------------ | | Moderator | [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) | | Notes | [@roaa](@roaa) | | Issue Triage | [@bfjelds](@bfjelds) | --- # March 2 ([recording](https://www.youtube.com/watch?v=qwkFLTEZQdk)) ## Assignments | Assignment | | | ------------ | ------------------------------------------ | | Moderator | [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) | | Notes | [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) | | Issue Triage | [@bfjelds](@bfjelds) | ## Announcements 1. KubeCon EU 2021 proposal accepted! More information about the event [here](https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/). ## Agenda 1. Discovery Handler Design and Demo - [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) 2. Dependency Update - @Roaa 3. Webhooks - @bfjelds 5. Deallocate - @Roaa 6. CoAP protocol - Jiayi Hu 7. How to engage more with community - [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) ## Discussion ### Discovery Handler - Protocols are decoupled from the Akri Agent now - Do not need to modify Akri Agent to add new protocols - Template to expediate protocol creation ### [Dependency Update](https://github.com/marketplace/actions/dependencies-autoupdate) - Streamline dependency updates via a GitHub Action: has update command and verify command. - If succeeds, a PR is created with updates - Monthly basis - GitHub action is generic so can be used for other languages -- feel free to use it in your other projects! - Also, now performing security audit checks on dependencies ### Webhooks - For validating Configurations that are passed to K8s and then Akri - Couldn't put PodSpec in base of CRD: would tie us to specific version of K8s - Catch errors (such as improper indentation) earlier ### Deallocate - Working to get deallocate functionality added to next version of Kubernetes ### CoAP - REST off UDP instead of HTTP - Useful for tiny/constrained devices that are too small to implement HTTP - CoRE: machine to machine communication - Advantages - Stateless (connection losses fine because not required to maintain state) - Lowest overhead (compared to MQTT AMQP and HTTP) - Low latency - High interoperability - Protocol translation broker: UDP to HTTP - References for further details: - [Choice of Effective Messaging Protocols for IoT Systems: MQTT, CoAP, AMQP and HTTP (2017)](https://ieeexplore.ieee.org/abstract/document/8088251) - [CoAP: An Application Protocol for Billions of Tiny Internet Nodes (2012)](https://ieeexplore.ieee.org/abstract/document/6159216?casa_token=VQAiA8XbWBQAAAAA:CWgK0xJkoX8IwQr_2z-l95vv1ewwCWLPqOeTD9Y47gwGGFT0KIZcuu7lmx8MsQMUBCY9RYo) ### Internet of Production - largest Manufacturing systems/digitalization research group in Germany - Setting up a K8s cluster to manage (?) machine connectivity - Large variety of machinery (sewing, welding, etc) that want to connect to one K8s cluster - How to connect all these devices that "speak" different protocols? - Looking at ways to aggregate machine data (fluentd, kafka, etc) - Don't want to "mess" with the machinery and have to install anything on it - Want device discovery, ability tochange aggregation method (want to pull data every minute vs every week), and not to have to modify devices/machinery. ### Bird of a Feather - @Shan's similar scenario as IoP - Research scientist - working with robotic arms and industrial machines (not high risk setting) - @Jiayi researching "The Continuum of Computing" bringing cloud, edge and fog toghether - want edge to have same advantages as cloud: HA, etc - Same issue as Moritz: how to update code on devices without bringing them down - devices collect data that used in ML workload - update code on devices using WASM (lower mem requirements than compiler) - Start sharing ideas and designs on HackMD ## Assignments for Next Meeting | Assignment | | | ------------ | ------------------------------------------ | | Moderator | [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) | | Notes | [@bfjelds](@bfjelds) | | Issue Triage | [@roaa](@roaa) | --- # Febuary 2 ([recording](https://www.youtube.com/watch?v=D5CyDrkOIP8&t=2081s)) ## Assignments | Assignment | | | ------------ | ------------------------------------------ | | Moderator | [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) | | Notes | [@bfjelds](@bfjelds) | | Issue Triage | [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) | ## Announcements ### We have an official logo now! ![Akri logo and wordmark](https://i.imgur.com/ga9etIR.png =300x) If you'd like to see the styleguide and other artwork, check out [the GitHub folder](https://aka.ms/akri/art) ### v0.1.5 Release More information at https://github.com/deislabs/akri/releases/tag/v0.1.5 ### Discovery Handlers as Pods More information at https://github.com/deislabs/akri/issues/198 ## Agenda ## Discussion ## Notes * Introductions, setting agenda, reminder of recording * Announcement of official logo, introduction to Akri name, process of logo creation * Announcement of OPCUA (v0.1.5), description of OPCUA implementation, and work to come * Discussion of new discovery handler extension method 1. gRPC communication 2. streaming response, allows for flexibility per handler 3. concern: push-based communication 4. still requires an agent for each node, **this remains an area for improvement** * Triage of issues, introduction of roadmap page 1. Documentation of build process w/ HackMD page containing start at docs 2. Represent CRD versioning in accessible way for Helm 3. Update dependencies like kube and tokio ( ~0.2 -> ~1.* ), they are old 4. Agent crash when API server not available 1. Kubernetes guidance ... 5 minute before full on takedown of Agent 2. Document what our decision is 3. Cache? 4. How does this impact slot usage, etc? 5. Standby mode? 6. Check out Krustlet and mimic 5. Overly verbose, unstandardized logging ... can the logging level be set in the Helm chart? 1. Document a standardized approach 2. Guidelines for INFO vs TRACE, etc 7. Helm linting needs to pass args to be effective, maybe create exhaustive set of lint options * Feedback, suggestions, concerns, etc 1. What is the broader goal/vision/use case? (including from MSFT perspective) Would be clearer if this was found in the documentation, maybe HackMD. 2. Other deislabs projects, Krustlet, have some blog info that was very helpful ... akri could benefit from similar. 3. Why HackMD? Other people simply use github (single tool is easier) 1. Thinking was: this was not code, so separate from github 2. google docs vs hackmd ... hackmd didn't require an account 3. In the end, we are just trying this out, open to migrating to other platforms 4. **Finding responses is difficult on HackMD**, notification system may be insufficient * Intention for handling Community meeting, agenda 1. All notes in HackMD 2. To suggest Agenda item, anyone can add a comment for new agenda item 3. aka.ms/akri/meeting-notes, search "akri hackmd", pinned to slack, should we add a badge to github ## Assignments for Next Meeting | Assignment | | | ------------ | ------------------------------------------ | | Moderator | [Edrick Wong](@LcYkPeeDR-Ky_8hTgDDLDg) | | Notes | [Kate Goldenring](@A95X6oltR8-ir2GbT_GrZA) | | Issue Triage | [@bfjelds](@bfjelds) | ---