OpenEBS RoadMap Review Discussions Notes

OpenEBS RoadMap Review Discussions Notes === :::info - **Location:** https://zoom.us/j/93142650382 - **Date:** Every first Tuesday of the month from 10:00PM to 11:00pm (IST) - **Agenda** 1. Walk through the current [Releases](https://github.com/orgs/openebs/projects) and [ROADMAP](https://github.com/openebs/openebs/blob/master/ROADMAP.md) 2. Open Discussion *Add your request, question or suggestion to our [issue list](https://github.com/openebs/openebs/issues)* *If you have anything you’d like to put on the agenda, please do so below for the next meeting:* ::: ## Nov 03, 2020 ### Status Update #### 2.2 Released ( Refactoring / Install updates) (Oct 15th) - https://github.com/orgs/openebs/projects/22 - Mayastor (possible beta?) - Snapshot capability - Rawfile Local PV (possible beta?) - ZFS Local PV (Backup/Restore via Velero to possible beta?) - Minor enhancements and bugfixes - Helm charts - Multi-arch builds - Documentation and test updates. #### 2.3 Planned ( Refactoring / Install updates) (Nov 15th) - Mayastor (possible beta on subset of features) - Snapshot capability - Rawfile Local PV (possible beta?). Will be done around or a week after 15th. - ZFS Local PV (continue beta). Need to work on helm chart - Helm charts (ongoing) - Multi-arch builds (ongoing) ### General Announcements/Updates - Docker Pull limits - OpenEBS registered for Open source program - Pull limits will be not applicable to open source program registered repos - Monthly release updates all the repos. - We use docker for amd64 (arch specific) and multi-arch builds - We also use quay for arch specific - OpenEBS Proposal for incubation raised. - 08/28: Presented to CNCF SIG Storage. - AI: Run licensing checks and add appropriate noticies. - AI: List all the core code repositories maintained under OpenEBS Org - 09/23: Updated the CNCF SIG Storage with cStor refactoring. - AI: Follow-up with SIG Storage leads on further steps. - 10/06: Got a response from Alex that SIG Leads will review and get back on further questions. - 11/03: got the feedback from the sig-leads on repositories: - openebs/cstor => what does the ZFS community think of the uZFS fork? - openebs/ndm => seachest. Clarified that we use it as a library, use the upstream version. ### New contributor shout-outs! - To be updated by Kiran ### Open Discussion - [Kiran] Kickstart SIG Community Meetings ? - Move some of the [current IST timezone meetings](https://github.com/openebs/openebs/tree/master/community#community-meetings) to world-wide times? - OpenEBS-SIG-cStor (Ashutosh/Bi-weekly) Thursday 12:45PM IST - NDM and Local PV (Akhil/Monthly) Every 3rd Tuesday 8:30PM IST - OpenEBS-SIG-ContributeX (Prateek/Bi-weekly) Every Thursday in alternate timezones UK and PT * Discuss Open issues * Help onboarding - ZFS Local PV ( Pawan/Weekly) Every Friday at 12.45PM IST - Mayastor (TBD) - Docs (TBD) - OpenEBS-SIG-Release (TBD) - AI: post on DoK - AI: Turn into live session deploying a workload on one of the engines. - [TODO] 101 Training materials for new users/contributors - NDM (Harsh) - Demos on Youtube - dev.to blogs? OpenEBS Blog? - DO - Write for donation = OpenEBS Tutorials - CNCF Blogs - AI: Engage technical writers. - [TODO]: Documentation and Workload specific tasks - Example: How to use OpenEBS with different types of Cassandra Operators. Mentioning the type of Engine to be used. - Cassandra = caas operator - Cassandra = awesome-lists? - blogs/community pages - 9/1: (Ashutosh) Minio and Kafka with cStor - 10/6: (Kiran) Check for a way to publish articles with Databases covered in Oct CNCF Tech Radar. - [Open Items] - Encryption at rest support - minio released a new solution KES? - Mayastor and ZFS Local PV support at-rest encryption, but key should be provided via YAMLs. Looking for better key management solutions. - Other engines support via LUKS - Integrate with Vault ### Backlog Discussion/Updates - [Akhil] NDM support for discovering LVM devices - 9/1: Currently working on the design. - 10/6: Initial PR for adding sys-parser has been merged. There is a Draft PR on LVM support. - 11/3: Discovery of LVM and LUKS devices has been added (no need to create manual block devices) - [Akhil/Harsh] NDM SMART metrics on block devices - There could be performance impact depending on reading some metrics depending on hardware vendor. - Can some errors be sampled at different intervals - like daily along with status that is at frequent intervals. - Adding gRPC API to NDM that will be used by mayactl (aka OpenEBS CLI) to perform disk operations. - gRPC API to help with integration of NDM with Mayastor - 9/1: PR under review. Have to quantify the performance impact. - 10/6: PR with metrics from SeaChest on SCSI devices is merged. Need to extend NDM/SeaChest integration to support NVMe and SATA. () - 11/3: Still under development. (currently on hold) - [Kiran] RWX - Currently supported via NFS - Note: There may be other variants of this use cases like write once, read many. NFS may not be required. - Is chuabofs an option? - Is SMB an option? - Current Plan: - Phase - 1: - Setup build automation of NFS provisioners from Kubernetes-sigs - (merged) https://github.com/kubernetes-sigs/nfs-ganesha-server-and-external-provisioner/pull/3 - (merged) https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/pull/3 - Phase - 2: - Enhance with NFS stats exporter - Enhance with defaults that help with resiliency - Fix issues around etcd dependency - Fix memory leak issues found during stale mounts - Phase - 3: - Add resiliency / high availability with multiple ganesha deployments - 9/1: Initial PR for sub-dir repo is ready for merge. Build automation will be taken up next. - 10/6: K8s sigs repos are active now. Yet to work on the build automation. Also worked on a prototype to bring up Kernel NFS on Block volumes. Working on OpenEBS-RWX provisioner to launch the Kernel NFS on OpenEBS Volumes. - 11/6: Working on automtion to gcr registry for the sig projects. Working on NFS provisioner using kernel nfs. (https://github.com/kmova/bootstrap/tree/master/k8s-stg/nfs-server). A (new) provisioner will be pushed into openebs. - [Michael/Shubham]: ARM and Multi-arch builds - Done on NDM in 1.12, need to replicate to other repos - Done on Linux-utils in 2.0 - e2e for ARM64 builds - 9/1: Shubam - openebs/upgrade multi-arch PR in review. openebs/cstor under progress. - 9/1: Sai - openebs/istgt - 9/1: Kiran - Local PV provisioner (split from maya repo and enable multi-arch) - 9/1: Peeyush - ppc builds on Travis are failing. It has been reported to IBM/Travis. (Prateek) Looks to be related to linux version. Check if travis can ignore the ppc job failures. - 10/6: cStor multi-arch images are ready for dev testing. Builds files need to be refactored to avoid dependency on Travis flags as we move towards GitHub actions. - 11/6: All the containers (except Jiva) have multi-arch images being setup - [Harsh] disk clean up? DO. - provide an option via ctl to run wipefs on block devices - RBAC the above command to admin context - There is clean-up job that already does wipefs. - May need a cli - Come up with a design document that will help with perform these operations at scale. - 10/6: Still needs to close on the discussion - 11/6: Still needs to be concluded on the design. - [Harsh]: Zero Day provisioning via Terraform/Crossplane - Example: Hugepages for Mayastor - Example: iSCSI on storage nodes - 9/1: Crossplane allows to import and work on existing clusters using K8s CRs. - 10/6: Provided a demo on how Mayastor works with CrossPlane. Working on a document and also check with GitHub actions. - [Ashutosh]: Custom Scheduler for cStor Replica placement - 9/1: Working on the design document. also looking at making this a generic scheduler. - 10/6: No update - [TODO]: Improvements for running OpenEBS on-premise Kubernetes clusters - South bound provisioners when using on-prem - Fabrics are having CSI support - RAID array features bubbling up into CSI - Will K8s Pool concept get into CSI? "Storage Capacity Contraints for Pod Scheduling" - Health reporting - currently available for cstor, what about other engines? (Harsh/Kiran) - Error messages are hard to understand for non-k8s users. - Start with NDM, bubble up errors related to presence of filesystems, why devices are usable - cStor volume attachment. - K8s message on attachment comes first. - Keep the installer clean by seperating the engines. - 10/6: Added to ROADMAP.md for picking up later. - [TODO] NDM/Local PV with Partitions * Support for LVM vs direct patitions due to complexity in managing resize options as part of Day2 * LVM vs Mayastor benchmarking * LVM is simple and would be a good addition for SREs * 10/6: Check with Mehran, if this is being address in Rawfile-LocalPV. - [TODO]: NVMe over Fabric initiators - Check with Jeffry - user space vs in-kernel Fabric initiators - setup a meeting to sync-up on approaches being considered with Mayastor. - [TODO]: Multus-CNI support - 10/6: Add this and the IPv6 support related backlogs to ROADMAP.md - [TODO]: Snapshot Operator - While applications are upgrading/migrating the schema, users would love to have their volumes snapshotted. - One option is to have periodic snapshots working with a retention of n copies over a certain duration. - This can work for any of the PVs that support Snapshot capabilities. - 10/6: Ashutosh worked on an initial design document. Need help on this one for refining the design and implementation. Add this to ROADMAP.md. ## Oct 06, 2020 ### Status Update #### 2.1 Released - https://github.com/openebs/openebs/releases/tag/v2.1.0 - Refactored cStor builds to make libcstor as main repo for building binary and containers - ZFS Local PV - Backup/Restore via Velero (alpha) #### 2.2 Planned ( Refactoring / Install updates) (Oct 15th) - https://github.com/orgs/openebs/projects/22 - Mayastor (possible beta?) - Snapshot capability - Rawfile Local PV (possible beta?) - ZFS Local PV (Backup/Restore via Velero to possible beta?) - Minor enhancements and bugfixes - Helm charts - Multi-arch builds - Documentation and test updates. ### General Announcements/Updates - OpenEBS Proposal for incubation raised. - 08/28: Presented to CNCF SIG Storage. - AI: Run licensing checks and add appropriate noticies. - AI: List all the core code repositories maintained under OpenEBS Org - 09/23: Updated the CNCF SIG Storage with cStor refactoring. - AI: Follow-up with SIG Storage leads on further steps. - 10/06: Got a response from Alex that SIG Leads will review and get back on further questions. ### New contributor shout-outs! - jamesread - didier-durand - Aman1440 - Naveenkhasyap - ItsJulian - lambda2 - Hard-Coder05 - sparkingdark - harikrishnajiju - salonigoyal2309 - zlymeda - avats-dev ### Open Discussion - [Kiran] Kickstart SIG Community Meetings ? - Move some of the [current IST timezone meetings](https://github.com/openebs/openebs/tree/master/community#community-meetings) to world-wide times? - OpenEBS-SIG-cStor (Ashutosh/Bi-weekly) Thursday 12:45PM IST - NDM and Local PV (Akhil/Monthly) Every 3rd Tuesday 8:30PM IST - OpenEBS-SIG-ContributeX (Prateek/Bi-weekly) Every Thursday in alternate timezones UK and PT * Discuss Open issues * Help onboarding - ZFS Local PV ( Pawan/Weekly) Every Friday at 12.45PM IST - Mayastor (TBD) - Docs (TBD) - OpenEBS-SIG-Release (TBD) - - [TODO] 101 Training materials for new users/contributors - NDM (Harsh) - Demos on Youtube - dev.to blogs? OpenEBS Blog? - DO - Write for donation = OpenEBS Tutorials - CNCF Blogs - [TODO]: Documentation and Workload specific tasks - Example: How to use OpenEBS with different types of Cassandra Operators. Mentioning the type of Engine to be used. - Cassandra = caas operator - Cassandra = awesome-lists? - blogs/community pages - 9/1: (Ashutosh) Minio and Kafka with cStor - 10/6: (Kiran) Check for a way to publish articles with Databases covered in Oct CNCF Tech Radar. - [Ajeet Rai] Process for contributing to Hacktoberfest - Any PRs that address a problem (even without a pre-existing issue) are considered for Hacktoberfest - OpenEBS repos have been updated with Hacktoberfest topic. - [Open Items] - Encryption at rest support - minio released a new solution KES? - Mayastor and ZFS Local PV support at-rest encryption, but key should be provided via YAMLs. Looking for better key management solutions. - Other engines support via LUKS - Integrate with Vault ### Backlog Discussion/Updates - [Akhil] NDM support for discovering LVM devices - 9/1: Currently working on the design. - 10/6: Initial PR for adding sys-parser has been merged. There is a Draft PR on LVM support. - [Akhil/Harsh] NDM SMART metrics on block devices - There could be performance impact depending on reading some metrics depending on hardware vendor. - Can some errors be sampled at different intervals - like daily along with status that is at frequent intervals. - Adding gRPC API to NDM that will be used by mayactl (aka OpenEBS CLI) to perform disk operations. - gRPC API to help with integration of NDM with Mayastor - 9/1: PR under review. Have to quantify the performance impact. - 10/6: PR with metrics from SeaChest on SCSI devices is merged. Need to extend NDM/SeaChest integration to support NVMe and SATA. () - [Kiran] RWX - Currently supported via NFS - Note: There may be other variants of this use cases like write once, read many. NFS may not be required. - Is chuabofs an option? - Is SMB an option? - Current Plan: - Phase - 1: - Setup build automation of NFS provisioners from Kubernetes-sigs - (merged) https://github.com/kubernetes-sigs/nfs-ganesha-server-and-external-provisioner/pull/3 - (merged) https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/pull/3 - Phase - 2: - Enhance with NFS stats exporter - Enhance with defaults that help with resiliency - Fix issues around etcd dependency - Fix memory leak issues found during stale mounts - Phase - 3: - Add resiliency / high availability with multiple ganesha deployments - 9/1: Initial PR for sub-dir repo is ready for merge. Build automation will be taken up next. - 10/6: K8s sigs repos are active now. Yet to work on the build automation. Also worked on a prototype to bring up Kernel NFS on Block volumes. Working on OpenEBS-RWX provisioner to launch the Kernel NFS on OpenEBS Volumes. - [Michael/Shubham]: ARM and Multi-arch builds - Done on NDM in 1.12, need to replicate to other repos - Done on Linux-utils in 2.0 - e2e for ARM64 builds - 9/1: Shubam - openebs/upgrade multi-arch PR in review. openebs/cstor under progress. - 9/1: Sai - openebs/istgt - 9/1: Kiran - Local PV provisioner (split from maya repo and enable multi-arch) - 9/1: Peeyush - ppc builds on Travis are failing. It has been reported to IBM/Travis. (Prateek) Looks to be related to linux version. Check if travis can ignore the ppc job failures. - 10/6: cStor multi-arch images are ready for dev testing. Builds files need to be refactored to avoid dependency on Travis flags as we move towards GitHub actions. - [Harsh] disk clean up? DO. - provide an option via ctl to run wipefs on block devices - RBAC the above command to admin context - There is clean-up job that already does wipefs. - May need a cli - Come up with a design document that will help with perform these operations at scale. - 10/6: Still needs to close on the discussion - [Harsh]: Zero Day provisioning via Terraform/Crossplane - Example: Hugepages for Mayastor - Example: iSCSI on storage nodes - 9/1: Crossplane allows to import and work on existing clusters using K8s CRs. - 10/6: Provided a demo on how Mayastor works with CrossPlane. Working on a document and also check with GitHub actions. - [Ashutosh]: Custom Scheduler for cStor Replica placement - 9/1: Working on the design document. also looking at making this a generic scheduler. - 10/6: No update - [TODO]: Improvements for running OpenEBS on-premise Kubernetes clusters - South bound provisioners when using on-prem - Fabrics are having CSI support - RAID array features bubbling up into CSI - Will K8s Pool concept get into CSI? "Storage Capacity Contraints for Pod Scheduling" - Health reporting - currently available for cstor, what about other engines? (Harsh/Kiran) - Error messages are hard to understand for non-k8s users. - Start with NDM, bubble up errors related to presence of filesystems, why devices are usable - cStor volume attachment. - K8s message on attachment comes first. - Keep the installer clean by seperating the engines. - 10/6: Added to ROADMAP.md for picking up later. - [TODO] NDM/Local PV with Partitions * Support for LVM vs direct patitions due to complexity in managing resize options as part of Day2 * LVM vs Mayastor benchmarking * LVM is simple and would be a good addition for SREs * 10/6: Check with Mehran, if this is being address in Rawfile-LocalPV. - [TODO]: NVMe over Fabric initiators - Check with Jeffry - user space vs in-kernel Fabric initiators - setup a meeting to sync-up on approaches being considered with Mayastor. - [TODO]: Multus-CNI support - 10/6: Add this and the IPv6 support related backlogs to ROADMAP.md - [TODO]: Snapshot Operator - While applications are upgrading/migrating the schema, users would love to have their volumes snapshotted. - One option is to have periodic snapshots working with a retention of n copies over a certain duration. - This can work for any of the PVs that support Snapshot capabilities. - 10/6: Ashutosh worked on an initial design document. Need help on this one for refining the design and implementation. Add this to ROADMAP.md. ## Sept 01, 2020 ### Status Update #### 2.0 Released - https://github.com/openebs/openebs/releases/tag/v2.0.0 #### 2.1 Planned (Maintenance release) (Sept 15th) - https://github.com/orgs/openebs/projects/21 - Mayastor (possible beta?) - Rawfile Local PV (possible beta?) - ZFS Local PV (Backup/Restore via Velero) - Minor enhancements and bugfixes - Helm charts - Multi-arch builds - Documentation and test updates. ### General Announcements/Updates - OpenEBS Proposal for incubation raised. - Presented to CNCF SIG Storage. - AI: Run licensing checks and add appropriate noticies. - AI: List all the core code repositories maintained under OpenEBS Org - GitHub Discussions enabled. Backlog and some slack conversations can be moved there. - https://github.com/openebs/openebs/discussions ### New contributor shout-outs! - Ajeet Rai ### Backlogs and Open Discussions - [Kiran] Kickstart SIG Community Meetings ? - Move some of the [current IST timezone meetings](https://github.com/openebs/openebs/tree/master/community#community-meetings) to world-wide times? - OpenEBS-SIG-cStor (Ashutosh/Bi-weekly) Thursday 12:45PM IST - NDM and Local PV (Akhil/Monthly) Every 3rd Tuesday - OpenEBS-SIG-ContributeX (Prateek/Weekly) Every Thursday in alternate timezones UK and PT * Discuss Open issues * Help onboarding - ZFS Local PV - Mayastor - Docs - OpenEBS-SIG-Release - More or less? - [TODO] 101 Training materials for new users/contributors - NDM (Harsh) - [Akhil] NDM support for discovering LVM devices - 9/1: Currently working on the design. - [Akhil/Harsh] NDM SMART metrics on block devices - There could be performance impact depending on reading some metrics depending on hardware vendor. - Can some errors be sampled at different intervals - like daily along with status that is at frequent intervals. - Adding gRPC API to NDM that will be used by mayactl (aka OpenEBS CLI) to perform disk operations. - gRPC API to help with integration of NDM with Mayastor - 9/1: PR under review. Have to quantify the performance impact. - [Kiran] RWX - Currently supported via NFS - Note: There may be other variants of this use cases like write once, read many. NFS may not be required. - Is chuabofs an option? - Is SMB an option? - Current Plan: - Phase - 1: - Setup build automation of NFS provisioners from Kubernetes-sigs - (merged) https://github.com/kubernetes-sigs/nfs-ganesha-server-and-external-provisioner/pull/3 - (ready for review) https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/pull/3 - Phase - 2: - Enhance with NFS stats exporter - Enhance with defaults that help with resiliency - Fix issues around etcd dependency - Fix memory leak issues found during stale mounts - Phase - 3: - Add resiliency / high availability with multiple ganesha deployments - 9/1: Initial PR for sub-dir repo is ready for merge. Build automation will be taken up next. - [Michael]: ARM and Multi-arch builds - Done on NDM in 1.12, need to replicate to other repos - Done on Linux-utils in 2.0 - e2e for ARM64 builds - 9/1: Shubam - openebs/upgrade multi-arch PR in review. openebs/cstor under progress. - 9/1: Sai - openebs/istgt - 9/1: Kiran - Local PV provisioner (split from maya repo and enable multi-arch) - 9/1: Peeyush - ppc builds on Travis are failing. It has been reported to IBM/Travis. (Prateek) Looks to be related to linux version. Check if travis can ignore the ppc job failures. - [TODO]: Improvements for running OpenEBS on-premise Kubernetes clusters - South bound provisioners when using on-prem - Fabrics are having CSI support - RAID array features bubbling up into CSI - Will K8s Pool concept get into CSI? "Storage Capacity Contraints for Pod Scheduling" - Health reporting - currently available for cstor, what about other engines? (Harsh/Kiran) - Error messages are hard to understand for non-k8s users. - Start with NDM, bubble up errors related to presence of filesystems, why devices are usable - cStor volume attachment. - K8s message on attachment comes first. - Keep the installer clean by seperating the engines. - [Harsh] disk clean up? DO. - provide an option via ctl to run wipefs on block devices - RBAC the above command to admin context - There is clean-up job that already does wipefs. - May need a cli - Come up with a design document that will help with perform these operations at scale. - [TODO] NDM/Local PV with Partitions * Support for LVM vs direct patitions due to complexity in managing resize options as part of Day2 * LVM vs Mayastor benchmarking * LVM is simple and would be a good addition for SREs - [TODO]: NVMe over Fabric initiators - Check with Jeffry - user space vs in-kernel Fabric initiators - setup a meeting to sync-up on approaches being considered with Mayastor. - [TODO]: Multus-CNI support - [TODO]: Snapshot Operator - While applications are upgrading/migrating the schema, users would love to have their volumes snapshotted. - One option is to have periodic snapshots working with a retention of n copies over a certain duration. - This can work for any of the PVs that support Snapshot capabilities. - [Ashutosh]: Custom Scheduler for cStor Replica placement - 9/1: Working on the design document. also looking at making this a generic scheduler. - - [Harsh]: Zero Day provisioning via Terraform/Crossplane - Example: Hugepages for Mayastor - Example: iSCSI on storage nodes - 9/1: Crossplane allows to import and work on existing clusters using K8s CRs. - [TODO]: Workload specific tasks - Example: How to use OpenEBS with different types of Cassandra Operators. Mentioning the type of Engine to be used. - Cassandra = caas operator - Cassandra = awesome-lists? - blogs/community pages - 9/1: (Ashutosh) Minio and Kafka with cStor ## Aug 04, 2020 ### Status Updates - 1.13/2.0 (https://github.com/orgs/openebs/projects/20) - NDM UUID for virtual disks (beta) - NDM UUID graduated to beta in 1.13 - upgrade/migration support code under review. https://github.com/openebs/node-disk-manager/pull/442 - gRPC API initial version is merged. - working on metrics - working on a blocker w.r.t partprobe/disk cleanup. https://github.com/openebs/node-disk-manager/pull/469 - Helm v3 and other build features - CRD installs via the Helm template - Added Multi-arch builds for linux-utils = platforms supported - arm64, arm32, power, amd64 - Also planned for other components. - Have to work on integration tests via multi-arch builds for NDM. - Support for ZFS Local PV via helm (add a seperate chart) - Build instruction overview for different engines in openebs/openebs repo - - cStor CSI Driver with enhanced pool schema - Graduating to Beta in 1.13 - Additional usability and e2e tests under development - Stabilizing upgrade and migration from SPC to CSPC - Docs needs to be updated as part of Beta - Dogfooding servers for OpenEBS e2e - migrate to CSPC - Local PV Enhancements - ZFS Local PV - backup/restore via Velero plugin in progress - Local PV on raw blocks on hostpaths (a new repo). - Split from openebs/maya into its own repo for Local PV external storage provisioner for better visibility? - Mayastor - Rebuild feature development - Usability issues with AKS being looked into - Initial Benchmarking via Octane between Local PV vs Mayastor published - Working on PCIe support - Jiva - Auto deletion of internal snapshots ### Open Discussion / Backlogs - NDM SMART metrics on block devices - There could be performance impact depending on reading some metrics depending on hardware vendor. - Can some errors be sampled at different intervals - like daily along with status that is at frequent intervals. - Adding gRPC API to NDM that will be used by mayactl (aka OpenEBS CLI) to perform disk operations. - gRPC API to help with integration of NDM with Mayastor - NDM/Local PV with Partitions * Support for LVM vs direct patitions due to complexity in managing resize options as part of Day2 * LVM vs Mayastor benchmarking * LVM is simple and would be a good addition for SREs - TODO: RWX - Currently supported via NFS - Current Plan: Phase - 1: - Setup build automation of NFS provisioners from Kubernetes-sigs - Tracked under: * https://github.com/kubernetes-sigs/nfs-ganesha-server-and-external-provisioner/pull/3 * https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/pull/2 Phase - 2: - Enhance with NFS stats exporter - Enhance with defaults that help with resiliency - Fix issues around etcd dependency - Fix memory leak issues found during stale mounts Phase - 3: - Add resiliency / high availability with multiple ganesha deployments - There may be other variants of this use cases like write once, read many. NFS may not be required. - Is chubaofs an option? - Is SMB an option - TODO: Improvements for running OpenEBS on-premise Kubernetes clusters - South bound provisioners when using on-prem - Fabrics are having CSI support - RAID array features bubbling up into CSI - Will K8s Pool concept get into CSI? "Storage Capacity Contraints for Pod Scheduling" - Health reporting - currently available for cstor, what about other engines? (Harsh/Kiran) - Error messages are hard to understand for non-k8s users. - Start with NDM, bubble up errors related to presence of filesystems, why devices are usable - cStor volume attachment. - K8s message on attachment comes first. - disk clean up? DO. - provide an option via ctl to run wipefs on block devices - RBAC the above command to admin context - There is clean-up job that already does wipefs. - Keep the installer clean by seperating the engines. - TODO: NVMe over Fabric initiators - Check with Jeffry - user space vs in-kernel Fabric initiators - setup a meeting to sync-up on approaches being considered with Mayastor. - TODO: Multus-CNI support - - TODO: ARM - Done on NDM in 1.12, need to replicate to other repos - e2e for ARM64 builds - TODO: Snapshot Operator - While applications are upgrading/migrating the schema, users would love to have their volumes snapshotted. - One option is to have periodic snapshots working with a retention of n copies over a certain duration. - This can work for any of the PVs that support Snapshot capabilities. - TODO: Custom Scheduler for Replica placement - (Ashutosh) - - TODO: Zero Day provisioning via Terraform - Example: Hugepages for Mayastor - Example: iSCSI on storage nodes - TODO: Workload specific tasks - Example: How to use OpenEBS with different types of Cassandra Operators. Mentioning the type of Engine to be used. - Cassandra = caas operator - Cassandra = awesome-lists? - blogs/community pages ## July 07, 2020 Attendees: ### Status Updates - 1.12 (https://github.com/orgs/openebs/projects/19) - NDM UUID for virtual disks (beta) - NDM UUID will still be under alpha in 1.12, upgrade/migration support code under review. https://github.com/openebs/node-disk-manager/pull/442 - Fixes for handling partition and block device cleanup via wipefs. - Added support for attaching errors as K8s events to BDC. https://github.com/openebs/node-disk-manager/pull/425 - Helm v3 and other build features - CRD installs via the Helm template - Added Multi-arch builds for NDM = platforms supported - arm64, arm32, power, amd64 - Have to work on integration tests via multi-arch builds. (post 1.12) - Support for ZFS Local PV via helm (add a seperate chart) - Build instruction overview for different engines in openebs/openebs repo - cStor CSI Driver with enhanced pool schema - Additional usability and e2e tests under development - Stabilizing upgrade and migration from SPC to CSPC - Docs needs to be updated as part of Beta - Dogfooding servers for OpenEBS e2e - migrate to CSPC - Local PV Enhancements - ZFS Local PV - added btrfs support - ZFS Local PV - backup/restore via Velero plugin in progress - Local PV on raw blocks on hostpaths (a new repo). - Split from openebs/maya into its own repo for Local PV external storage provisioner for better visibility? - Mayastor - Rebuild feature development - Usability issues with AKS being looked into - Benchmarking via Optane between Local PV vs Mayastor - Jiva - Auto deletion of internal snapshots ### Open Discussion / Backlogs - NDM SMART metrics on block devices - There could be performance impact depending on reading some metrics depending on hardware vendor. - Can some errors be sampled at different intervals - like daily along with status that is at frequent intervals. - Adding gRPC API to NDM that will be used by mayactl (aka OpenEBS CLI) to perform disk operations. - gRPC API to help with integration of NDM with Mayastor - NDM/Local PV with Partitions * Support for LVM vs direct patitions due to complexity in managing resize options as part of Day2 * LVM vs Mayastor benchmarking * LVM is simple and would be a good addition for SREs - TODO: RWX - Currently supported via NFS - Current Plan: Phase - 1: - Setup build automation of NFS provisioners from Kubernetes-sigs - Tracked under: * https://github.com/kubernetes-sigs/nfs-ganesha-server-and-external-provisioner/pull/3 * https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/pull/2 Phase - 2: - Enhance with NFS stats exporter - Enhance with defaults that help with resiliency - Fix issues around etcd dependency - Fix memory leak issues found during stale mounts Phase - 3: - Add resiliency / high availability with multiple ganesha deployments - There may be other variants of this use cases like write once, read many. NFS may not be required. - Is chubaofs an option? - Is SMB an option - TODO: Improvements for running OpenEBS on-premise Kubernetes clusters - South bound provisioners when using on-prem - Fabrics are having CSI support - RAID array features bubbling up into CSI - Will K8s Pool concept get into CSI? "Storage Capacity Contraints for Pod Scheduling" - Health reporting - Error messages are hard to understand for non-k8s users. - Start with NDM, bubble up errors related to presence of filesystems, why devices are usable - cStor volume attachment. - K8s message on attachment comes first. - disk clean up? DO. - provide an option via ctl to run wipefs on block devices - RBAC the above command to admin context - There is clean-up job that already does wipefs. - Keep the installer clean by seperating the engines. - TODO: NVMe over Fabric initiators - Check with Jeffry - user space vs in-kernel Fabric initiators - setup a meeting to sync-up on approaches being considered with Mayastor. - TODO: Multus-CNI support - - TODO: ARM - Done on NDM in 1.12, need to replicate to other repos - e2e for ARM64 builds - TODO: Snapshot Operator - While applications are upgrading/migrating the schema, users would love to have their volumes snapshotted. - One option is to have periodic snapshots working with a retention of n copies over a certain duration. - This can work for any of the PVs that support Snapshot capabilities. ## Jun 02, 2020 Attendees: Brian, Kiran, Murat, Richard, Vitta, Harsh, Ranjith, Vani, Karthik ### Status Updates - 1.11 - NDM UUID for virtual disks beta - Detect if devices already have Mayastor or ZFS Local PV. (A new probe) - NDM UUID will still be under alpha in 1.11, needs upgrade/migration support. - Helm v3 and other build features - Migrated to https://github.com/openebs/charts - Multi-arch support on NDM - Automate builds for NDM, Local PV for PowerSystems - Supportability/Usability enhancements - Adding gRPC API to NDM that will be used by mayactl (aka OpenEBS CLI) to perform disk operations. - gRPC API to help with integration of NDM with Mayastor - 1.12 - NDM SMART metrics on block devices - There could be performance impact depending on reading some metrics depending on hardware vendor. - Can some errors be sampled at different intervals - like daily along with status that is at frequent intervals. - Continue on ZFS Local PV, Mayastor, cStor ### Open Discussion / Backlogs - Review https://github.com/openebs/openebs/blob/master/ROADMAP.md - TODO:Kiran: Mention the CSI spec version to be supported. - Check with Glen, Pawan, Ashutosh for Mayastor, ZFS Local PV, cStor - TODO: NDM support for Chassis Information. (Take help from community) - TODO: CI/e2e pipeline - TODO: RWX - Currently supported via NFS - There may be other variants of this use cases like write once, read many. NFS may not be required. - Is choabfs an option? - Is SMB an option - Current Plan: Phase - 1: - Setup build automation of NFS provisioners from Kubernetes-sigs - Tracked under: * https://github.com/kubernetes-sigs/nfs-ganesha-server-and-external-provisioner/pull/2 * https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/pull/2 Phase - 2: - Enhance with NFS stats exporter - Enhance with defaults that help with resiliency - Fix issues around etcd dependency - Fix memory leak issues found during stale mounts Phase - 3: - Add resiliency / high availability with multiple ganesha deployments - TODO: Improvements for running OpenEBS on-premise Kubernetes clusters - South bound provisioners when using on-prem - Fabrics are having CSI support - RAID array features bubbling up into CSI - Will K8s Pool concept get into CSI? "Storage Capacity Contraints for Pod Scheduling" - Health reporting - Error messages are hard to understand for non-k8s users. - Start with NDM, bubble up errors related to presence of filesystems, why devices are usable - cStor volume attachment. - K8s message on attachment comes first. - disk clean up? DO. - provide an option via ctl to run wipefs on block devices - RBAC the above command to admin context - There is clean-up job that already does wipefs. - Keep the installer clean by seperating the engines. - TODO: NVMe over Fabric initiators - Check with Jeffry - user space vs in-kernel Fabric initiators - setup a meeting to sync-up on approaches being considered with Mayastor. - TODO: Multus-CNI support - - TODO: ARM - e2e for ARM64 builds - ARM 32 bit is not supported - ## May 05, 2020 Attendees: Brian, Kiran, Murat, Richard ### Status Updates - 1.10 - ZFS Local PV going beta - MayaStor alpha - cStor new schema (CSPC) alpha - NDM UUID for virtual disks alpha - 1.11 - Local PV metrics - NDM UUID for virtual disks beta - Helm v3 - NDM SMART metrics on block devices - There could be performance impact depending on reading some metrics depending on hardware vendor. - Can some errors be sampled at different intervals - like daily along with status that is at frequent intervals. - Continue on ZFS Local PV, Mayastor, cStor - Cloud Native Slack vs Kubernetes Slack? - Documentation is updated to remove links to OpenEBS slack - Website and Github needs to be updated - Any strong opinions on K8s slack or CNCF slack? ### Discussion Topics - Review https://github.com/openebs/openebs/blob/master/ROADMAP.md - Mention the CSI spec version to be supported. - NDM support for Chassis Information. (Take help from community) - Multi arch image - CI/e2e pipeline - Mayastor - Integrate with NDM to get unclaimed disks - NDM should be able to detect that mayastor is setup on the devices - RWX - Currently supported via NFS - There may be other variants of this use cases like write once, read many. NFS may not be required. - Is choabfs an option? - Is SMB an option - Cloud Native Slack - Migration will happen in May - Improvements for running OpenEBS on-premise Kubernetes clusters - South bound provisioners when using on-prem - Error messages are hard to understand for non-k8s users. - Start with NDM, bubble up errors related to presence of filesystems, why devices are usable - cStor volume attachment. - K8s message on attachment comes first. - disk clean up? DO. - provide an option via ctl to run wipefs on block devices - RBAC the above command to admin context - Keep the installer clean by seperating the engines.