## March 31, 2022 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Patrick Titzler| IBM | @ptitzler | | Alan Chin | IBM | @akchinSTC | | David C Martin | Age@Home | @dcmartin | | Alex Bozarth | IBM | @ajbozarth | | Kiersten Stokes | IBM | @kiersten-stokes | | Kevin Bates | IBM | @kevin-bates| | Martha Cryan | IBM | @marthacryan | * **Release status**: * Latest official release: 3.7.0 * [Summary of changes](https://github.com/elyra-ai/elyra/releases/tag/v3.7.0) * **conda-forge**: (https://anaconda.org/conda-forge/elyra) * Elyra is available again on conda-forge - Starting with v3.6.0 * Elyra 3.7.0 package updates in progress, should be available before noon PST today * **Open Data Hub**: * v3.6.0 based image is now available [Link to Image](https://quay.io/repository/thoth-station/s2i-lab-elyra?tab=tags) * ODH manifest updates in progress, [PR](https://github.com/opendatahub-io/odh-manifests/pull/546) * v3.7.0 image updates in progress * Next planned release: [3.8.0](https://github.com/elyra-ai/elyra/milestone/52) * Planning and curation in progress ### Agenda + Minutes - Discussion - Black formatter tooling to be added to dev process [#2542](https://github.com/elyra-ai/elyra/issues/2542) - line length etc. staying at 120 characters - enforcing of single vs double quoting - how this affects our jinja templates ? - use hooks for black during commit - option to auto migrate to f strings? mypy? - flynt - moves to f strings - - security/encryption of sensitive information ## March 24, 2022 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Patrick Titzler| IBM | @ptitzler | | Alan Chin | IBM | @akchinSTC | | Alex Bozarth | IBM | @ajbozarth | | David C Martin | Age@Home | @dcmartin | | Karla Spuldaro | IBM | @karlaspuldaro | | Kiersten Stokes | IBM | @kiersten-stokes | | Kevin Bates | IBM | @kevin-bates| * **Release status**: * Latest official release: 3.6.0 * [Summary of changes](https://github.com/elyra-ai/elyra/releases/tag/v3.6.0) * **conda-forge**: (https://anaconda.org/conda-forge/elyra) * Elyra is available again on conda-forge - Starting with v3.6.0 * Elyra-server and extension packages have all been published * **Open Data Hub**: * v3.6.0 CI tests passed, currently being deployed/published * Next planned release: [3.7.0](https://github.com/elyra-ai/elyra/milestone/52) * Release Candidate - Target date: March 25th ### Agenda + Minutes - Alan - CodeQL checks will be enabled - [PR](https://github.com/elyra-ai/elyra/pull/2570) - improved security, addresses CWEs - Kiersten - [#2522](https://github.com/elyra-ai/elyra/pull/2522) - Make component cache event-driven to allow out-of-proc updates - Ready for review - Includes the API(s) necessary for Alex's component catalog refresh ([#2532](https://github.com/elyra-ai/elyra/pull/2532)) - [#2564](https://github.com/elyra-ai/elyra/pull/2564) - Add API to fetch the component definition content - Alex adding the frontend piece directly into this PR - ptitzler - `elyra-pipeline submit ... --monitor` monitoring [2584](https://github.com/elyra-ai/elyra/pull/2584) - new breaking version of jinja (3.1.0)? ``` I 16:04:54.166] 'kubeflow_pipelines-0324090415':'hello' - notebook execution completed (2.621 secs) [E 16:04:54.212] Unexpected error: <class 'ImportError'> Traceback (most recent call last): File "bootstrapper.py", line 377, in execute NotebookFileOp.convert_notebook_to_html(notebook_output, notebook_html) File "bootstrapper.py", line 398, in convert_notebook_to_html import nbconvert File "/usr/local/lib/python3.8/site-packages/nbconvert/__init__.py", line 4, in <module> from .exporters import * File "/usr/local/lib/python3.8/site-packages/nbconvert/exporters/__init__.py", line 3, in <module> from .html import HTMLExporter File "/usr/local/lib/python3.8/site-packages/nbconvert/exporters/html.py", line 12, in <module> from jinja2 import contextfilter ImportError: cannot import name 'contextfilter' from 'jinja2' (/usr/local/lib/python3.8/site-packages/jinja2/__init__.py) ``` - [performance issue](https://github.com/elyra-ai/elyra/issues/2585) with updated [pandas rti](https://github.com/elyra-ai/elyra/blob/master/etc/config/metadata/runtime-images/pandas.json) ## March 17th, 2022 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Alan Chin | IBM | @akchinSTC | | Alex Bozarth | IBM | @ajbozarth | | Joshua Alter | IBM | @JoshuaAlter | | Karla Spuldaro | IBM | @karlaspuldaro | | Patrick Titzler| IBM | @ptitzler | | Kiersten Stokes | IBM | @kiersten-stokes | | Kevin Bates | IBM | @kevin-bates| | David C Martin | Age@Home | @dcmartin | * **Release status**: * Latest official release: 3.6.0 * [Summary of changes](https://github.com/elyra-ai/elyra/releases/tag/v3.6.0) * **conda-forge**: (https://anaconda.org/conda-forge/elyra) * Elyra is available again on conda-forge - Starting with v3.6.0 * Elyra-server and extension packages have all been published * Next planned release: [3.7.0](https://github.com/elyra-ai/elyra/milestone/52) * Target date: March 25th ### Agenda + Minutes Happy St. Patricks Day ! 🍀 <iframe src="https://giphy.com/embed/OOn7N4CfPuUTmNqFD7" width="180" height="180" frameBorder="0" class="giphy-embed" allowFullScreen></iframe><p><a href="https://giphy.com/gifs/SportsManias-sports-emoji-sportsmanias-OOn7N4CfPuUTmNqFD7"></a></p> - Alan - Elyra in ODH - [PR](https://github.com/opendatahub-io/s2i-lab-elyra/pull/38) - Updated image, testing in progress openshift, manifest changes to follow - Updating Elyra to fix vulnerable packages (CVEs) both node and python - Alex - Created the Code Viewer extension [#2553](https://github.com/elyra-ai/elyra/pull/2553) (demo) - Working on a git bugfix [jupyterlab-git#1090](https://github.com/jupyterlab/jupyterlab-git/pull/1090) - ptitzler - CLI work: - deprecate `elyra-metadata install` (replaced by 'create' and 'update'; 'install' removed in 4.0 (https://github.com/elyra-ai/elyra/issues/2563) - new `elyra-pipeline export` command () - Karla - [#2576](https://github.com/elyra-ai/elyra/pull/2576) - Add unit tests for script editor package - Working on debugger integration with script editor - Kiersten - [#2522](https://github.com/elyra-ai/elyra/pull/2522) - Make component cache event-driven to allow out-of-proc updates - APIs added - Fine-tuning the tests and some performance issues - [#2564](https://github.com/elyra-ai/elyra/pull/2564) - Add API to fetch the component definition content - [#2561](https://github.com/elyra-ai/elyra/pull/2561) - Bug fix to ensure that KFP component parsing does not raise errors ## March 10th, 2022 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Alex Bozarth | IBM | @ajbozarth | | Patrick Titzler| IBM | @ptitzler | | Alan Chin | IBM | @akchinSTC | | Karla Spuldaro | IBM | @karlaspuldaro | | Joshua Alter | IBM | @JoshuaAlter | | Martha Cryan | IBM | @marthacryan | | Kiersten Stokes | IBM | @kiersten-stokes | | David C Martin | Age@Home | @dcmartin | | Kevin Bates | IBM | @kevin-bates| * **Release status**: * Latest official release: 3.6.0 * [Summary of changes](https://github.com/elyra-ai/elyra/releases/tag/v3.6.0) * **conda-forge**: (https://anaconda.org/conda-forge/elyra) * Elyra is available again on conda-forge - Starting with v3.6.0 * Elyra-server and extension packages have all been published * TODO: Updates to ODH are now unblocked * Next planned release: [3.7.0](https://github.com/elyra-ai/elyra/milestone/52) * JupyterLab 3.3 has been merged * Target date: March 25th ### Agenda + Minutes - ptitzler - [maintenance] https://github.com/elyra-ai/kfp-notebook - purged PRs/issues (repo hasn't been updated in 8 months) - disabled issues - ok to archive? - TODO: archive both repository ok'd - [maintenance] https://github.com/elyra-ai/ issue purger - TODO: bot auto-close nogo, possible middle ground to mark and alert - Alan - Picking up PR [2259](https://github.com/elyra-ai/elyra/pull/2259) again - Elyra sub package installation - Martha - Pipeline node double-click behavior is now configurable in the settings [PR](https://github.com/elyra-ai/elyra/pull/2517) - Kiersten - Make component cache event-driven to allow out-of-proc updates [#2522](https://github.com/elyra-ai/elyra/pull/2522) (resolves: [#2515](https://github.com/elyra-ai/elyra/issues/2515)) - Work continuing on this - Will address new backend APIs - Karla - Jumpstart Open Source contributions summary - Joshua Alter - New contribution: [#2538](https://github.com/elyra-ai/elyra/pull/2538) - Metadata tag validation ## March 3rd, 2022 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Alan Chin | IBM | @akchinSTC | | Alex Bozarth | IBM | @ajbozarth | | Joshua Alter | IBM | @JoshuaAlter | | Sam Ali-mirsalari | IBM | @samali0996 | | Luciano Resende | Apple | @lresende | | Kevin Bates | IBM | @kevin-bates | | Guillaume Moutier | Red Hat | @guimou | | Patrick Titzler| IBM | @ptitzler | * **Release status**: * Latest official release: 3.6.0 * [Summary of changes](https://github.com/elyra-ai/elyra/releases/tag/v3.6.0) * **conda-forge**: [In-Progress](https://anaconda.org/conda-forge/elyra) * Elyra is available again on conda-forge - Starting with 3.6.0 * In-progress - Elyra-server and extension packages * Next planned release: [3.7.0](https://github.com/elyra-ai/elyra/milestone/52) * Upstream dependency: JupyterLab 3.3 * 3.3.0 PR merge in progress - take a look at the npm warnings in the console * Target date: March 11th ### Agenda + Minutes - Presentation - Open Data Hub with Easybuild and Lmod - https://github.com/guimou/odh-highlander - Guest Speaker is Guillaume Moutier from RedHat - Presentation (Tentative) - Elyra test suite improvements - Sam Ali-Mirsalari - Alex - The new build system has been merged to master and is ready for the 3.7 release. (No more `jupyter lab build`) - Jupyterlab 3.3 has been released and will be included in the Elyra 3.7 release - Kevin - Load component cache on-demand to address out of process updates [#2522](https://github.com/elyra-ai/elyra/pull/2522) (resolves: [#2515](https://github.com/elyra-ai/elyra/issues/2515)) ## February 24, 2022 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Alan Chin | IBM | @akchinSTC | | Patrick Titzler| IBM | @ptitzler | | Alex Bozarth | IBM | @ajbozarth | | David C Martin | Age@Home | @dcmartin | | Joshua Alter | IBM | @JoshuaAlter | | Sandhya Nayak | IBM | @sandhya-nayak | | Martha Cryan | IBM | @marthacryan | | Kevin Bates | IBM | @kevin-bates| * **Release status**: * Latest official release: 3.6.0 * [Summary of changes](https://github.com/elyra-ai/elyra/releases/tag/v3.6.0) * **conda-forge**: [In-Progress](https://anaconda.org/conda-forge/elyra) * Waiting on KFP dependency (kfp-pipeline-spec, in progress) **waiting for kfp 1.8.0 release** * 1.8.0 was cut days ago but no new spec pkg * https://github.com/conda-forge/staged-recipes/pull/16307 * Next planned release: [3.7.0](https://github.com/elyra-ai/elyra/milestone/52) * Upstream dependency: JupyterLab 3.3 * 3.3.0RC0 PR and merge in progress * Target date: JL 3.3 + 1 week ### Agenda + Minutes - ptitzler - deprecation announced in 3.7: [Airflow examples connector](https://github.com/elyra-ai/examples/tree/master/component-catalog-connectors/airflow-example-components-connector) - continued to be supported (but not enhanced) until Elyra 4.0 - use [Airflow package connector](https://github.com/elyra-ai/elyra/tree/master/elyra/pipeline/airflow/package_catalog_connector) or [Airflow Provider Package connector](https://github.com/elyra-ai/elyra/tree/master/elyra/pipeline/airflow/provider_package_catalog_connector) - not supported by Elyra < 3.7 - 3.7 includes breaking catalog connector API changes - compatibility support removed in Elyra 4.0 - all community maintained connectors (ex-Airflow Examples) will be updated in 3.7 to use the new API - Alan - JL 3.3.0 rc0 integration - https://github.com/elyra-ai/elyra/pull/2495 - Lint tests are still being fixed, everything else is passing - new end to end testing system in progress - automate the manual testing process for releases - will test against actual deployed clusters of aa and kfp - plan on running twice a day - Alex - Update to prebuilt extensiosn is **finally** complete and ready for through review: [PR #2447](https://github.com/elyra-ai/elyra/pull/2447) - Kiersten - [#2497](https://github.com/elyra-ai/elyra/pull/2497) - Support has been officially removed for the `component-registries` schemaspace, the functionality for which has been provided by the `component-catalogs` schemaspace since Elyra 3.3 - [#2492](https://github.com/elyra-ai/elyra/pull/2492) includes the breaking changes to the catalog connector API mentioned above - Built-in connectors to be updated as well (url, filesystem and directory-based catalog types) - Sandhya - [#2404](https://github.com/elyra-ai/elyra/issues/2404) - CLI: add support for metadata export - completed / PR [#2448](https://github.com/elyra-ai/elyra/pull/2448) ready to merge? - Next task - [#2500](https://github.com/elyra-ai/elyra/issues/2500) - CLI: Add support for metadata import? ## February 17, 2022 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Alan Chin | IBM | @akchinSTC | | Patrick Titzler| IBM | @ptitzler | | Alex Bozarth | IBM | @ajbozarth | | Kiersten Stokes | IBM | @kiersten-stokes | | Karla Spuldaro | IBM | @karlaspuldaro | | Joshua Alter | IBM | @JoshuaAlter | | Sandhya Nayak | IBM | @sandhya-nayak | | David C Martin | Age@Home | @dcmartin | | Kevin Bates | IBM | @kevin-bates | | Luciano Resende | Apple | @lresende | * **Release status**: * Latest official release: 3.6.0 * [Summary of changes](https://github.com/elyra-ai/elyra/releases/tag/v3.6.0) * **conda-forge**: [In-Progress](https://anaconda.org/conda-forge/elyra) * Waiting on KFP dependency (kfp-pipeline-spec, in progress) **waiting for kfp 1.8.0 release** * 1.8.0 was cut days ago but no new spec pkg * https://github.com/conda-forge/staged-recipes/pull/16307 * Next planned release: [3.7.0](https://github.com/elyra-ai/elyra/milestone/52) * Upstream dependency: JupyterLab 3.3 * JL 3.3 - Mon, 2/28, 2/21 RC, 2/14 beta * Target date: JL 3.3 + 1 week ### Agenda + Minutes - JupyterLab 3.3 updates - waiting for rc - Kiersten - [New blog post](https://medium.com/ibm-data-ai/getting-started-with-apache-airflow-operators-in-elyra-aae882f80c4a?source=friends_link&sk=5849ae43f537abdf820e01e3cda2a8d2) published - "Getting Started with Apache Airflow Operators in Elyra" - Alan - Deprecating support for python 3.6 - Sandhya - [#2404](https://github.com/elyra-ai/elyra/issues/2404) - CLI: add support for metadata export - Patrick - Improving Airflow connectors in collaboration with Kiersten (see blog post for background) ## February 10, 2022 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Patrick Titzler| IBM | @ptitzler | | Alex Bozarth | IBM | @ajbozarth | | Alan Chin | IBM | @akchinSTC | | Joshua Alter | IBM | @JoshuaAlter | | Kiersten Stokes | IBM | @kiersten-stokes | | Martha Cryan | IBM | @marthacryan | | Karla Spuldaro | IBM | @karlaspuldaro | | Kevin Bates | IBM | @kevin-bates| * **Release status**: * Latest official release: 3.6.0 * [Summary of changes](https://github.com/elyra-ai/elyra/releases/tag/v3.6.0) * **conda-forge**: [In-Progress](https://anaconda.org/conda-forge/elyra) * Waiting on KFP dependency (kfp-pipeline-spec, in progress) **waiting for kfp 1.8.0 release** * https://github.com/conda-forge/staged-recipes/pull/16307 * Next planned release: [3.7.0](https://github.com/elyra-ai/elyra/milestone/52) * Upstream dependency: JupyterLab 3.3 * JL 3.3 - Mon, 2/28, 2/21 RC, 2/14 beta * Target date: JL 3.3 + 1 week ### Agenda + Minutes - Alex - Updated build (pre built extensions) is working, but still needs to be cleaned up and get watch and release builds working. (Very) WIP PR: https://github.com/elyra-ai/elyra/pull/2447 - ptitzler - "What's new" summary (including demos) is now attached to the [GitHub release](https://github.com/elyra-ai/elyra/releases/tag/v3.6.0) - Alan - [aa parent node references](https://github.com/elyra-ai/elyra/pull/2468) - Looking for input on difficulty in implementing frontend check for duplicate labels in VPE - - Martha - Settings editor demo - WIP PR to update metadata editor https://github.com/elyra-ai/elyra/pull/2464 - JupyterLab news (3.3/4.0 updates) ## February 3rd, 2022 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Romeo Kienzler | IBM | @romeokienzler | | Patrick Titzler| IBM | @ptitzler | | Alan Chin | IBM | @akchinSTC | | David C Martin | Age@Home | @dcmartin | | Karla Spuldaro | IBM | @karlaspuldaro | | Kiersten Stokes | IBM | @kiersten-stokes | | Alex Bozarth | IBM | @ajbozarth | | Luciano Resende | Apple | @lresende | | Kevin Bates | IBM | @kevin-bates| * **Release status**: * Latest official release [3.5.2] * [3.5.2 changelog](https://elyra.readthedocs.io/en/latest/getting_started/changelog.html) * Try it out! * `pip install --upgrade "elyra[all]==3.5.2" --use-deprecated legacy-resolver` * **conda-forge**: [In-Progress](https://anaconda.org/conda-forge/elyra) * Waiting on KFP dependency (kfp-pipeline-spec, in progress) ~~**blocked**~~ **waiting for kfp 1.8.0 release** * https://github.com/conda-forge/staged-recipes/pull/16307 * ~~https://github.com/kubeflow/pipelines/pull/7165~~ * Next planned releases: 3.5.3, 3.6.0 * 3.6.0 (Targeting mid feb but subject to change) * Sprint planning in progress * [Deliverables](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.6.0) * Airflow pipeline processing improvements * GUI support for metadata cloning * Catalog connectors for Airflow packages and provider packages * ... ### Agenda + Minutes - romeo - Issue [2439](https://github.com/elyra-ai/elyra/issues/2439) Can't add runtimes - RESOLVED (known usability issue) - ptitzler - Created ["What's new" discussion threads](https://github.com/elyra-ai/elyra/discussions/categories/show-and-tell) - These complement the [changelog](https://elyra.readthedocs.io/en/latest/getting_started/changelog.html), which is too low level, and the [blog posts](https://elyra.readthedocs.io/en/latest/getting_started/published-resources.html), which are too infrequent - Show and Tell: [GUI support for metadata duplication](https://github.com/elyra-ai/elyra/pull/2436) - Decision needed: https://github.com/elyra-ai/elyra/pull/2438 and https://github.com/elyra-ai/elyra/pull/2437: to be delivered as separate installables or not -> the connectors will be part of the Elyra installation - Alex - Work on switching to pre-built extensions hit a few road blocks, builds are working but frontend extensions aren't being loaded. Will be reaching out to Lab devs today for help - Alan - Decision needed - [Dependency capping](https://github.com/elyra-ai/elyra/issues/2435) - prior to release of 3.5.3 - JupyterLab news - 3.3 Alpha with enhancements that Elyra is waiting for should be available soon - Martha will show a JL 3.3 demo/overview in the next meeting - Luciano - Should we switch to a new documentation template? - Gateway Style --> https://jupyter-enterprise-gateway.readthedocs.io/en/latest/ - Jupyter Server style --> https://jupyter-server.readthedocs.io/en/latest/?badge=latest --> create a small pilot to assess if there are issues / what they are ## January 27th, 2022 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Alan Chin | IBM | @akchinSTC | | Luciano Resende | Apple| @lresende | | Patrick Titzler| IBM | @ptitzler | | Kevin Bates | IBM | @kevin-bates | | Alex Bozarth | IBM | @ajbozarth | | Sam Ali-mirsalari | IBM | @samali0996 | | Kiersten Stokes | IBM | @kiersten-stokes | | Sandhya Nayak | IBM | @sandhya-nayak | | Karla Spuldaro | IBM | @karlaspuldaro | | Joshua Alter | IBM | @JoshuaAlter | Romeo Kienzler | IBM | @romeokienzler | * **Release status**: * Latest official release [3.5.2] * [3.5.2 changelog](https://elyra.readthedocs.io/en/latest/getting_started/changelog.html) * Try it out! * `pip install --upgrade "elyra[all]==3.5.2" --use-deprecated legacy-resolver` * **conda-forge**: [In-Progress](https://anaconda.org/conda-forge/elyra) * Waiting on KFP dependency (kfp-pipeline-spec, in progress) **blocked** * https://github.com/conda-forge/staged-recipes/pull/16307 * ~~https://github.com/kubeflow/pipelines/pull/7165~~ * Next planned release: 3.6.0 * 3.6.0 (Targeting mid feb but subject to change) * Sprint planning in progress * [Deliverables](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.6.0) * Airflow pipeline processing improvements ### Agenda + Minutes - Alan - Pipeline/runtime behavior validation work continues - Updating test suites - Releases, releases, fix for maint branch not updating readthedocs - New "Good first issues" - [Updating the existing elyra main webpage content](https://elyra-ai.github.io/website/) - Alex - Working on switching build to pre-built extensions by following how [Gator](https://github.com/mamba-org/gator/) and [JupyterLab LSP](https://github.com/jupyter-lsp/jupyterlab-lsp) did it. Should know if it will work for us by next week (most likely sooner). - Kiersten - [#2418](https://github.com/elyra-ai/elyra/pull/2418) - as promised; moves to use AST.parse in the Airflow component parser - Related issue: [#2414](https://github.com/elyra-ai/elyra/issues/2414) - Sandhya - [#2404](https://github.com/elyra-ai/elyra/issues/2404) - CLI: add support for metadata export - working on it - have a couple questions. - Josh - hi/intro - Sam - hi/into - Karla - [#2383](https://github.com/elyra-ai/elyra/pull/2383) - working on changes requested on PR to Validate numeric fields in submit file dialog - IBM jumpstart OS updates ## January 20th, 2022 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Patrick Titzler| IBM | @ptitzler | | Alan Chin | IBM | @akchinSTC | | Alex Bozarth | IBM | @ajbozarth | | Kiersten Stokes | IBM | @kiersten-stokes | | Luciano Resende | Apple | @lresende | | Karla Spuldaro | IBM | @karlaspuldaro | | David C Martin | Age@Home | @dcmartin | | Kevin Bates | IBM | @kevin-bates| | Romeo Kienzler | IBM | @romeokienzler | * **Release status**: * Latest official release [3.4.1] * [3.4.1 changelog](https://elyra.readthedocs.io/en/latest/getting_started/changelog.html#release-3-4-1-01-07-2022) * Try it out! * `pip install --upgrade "elyra[all]==3.4.1" --use-deprecated legacy-resolver` * **conda-forge**: [In-Progress](https://anaconda.org/conda-forge/elyra) * Waiting on KFP dependency (kfp-pipeline-spec, in progress) **blocked** * https://github.com/conda-forge/staged-recipes/pull/16307 * Next planned release: 3.5.0 * 3.5.0 (Targeting Jan 21 but subject to change ) * Sprint planning in progress * [Deliverables](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.5.0) ### Agenda + Minutes - ptitzler - Show and tell: support for Airflow 2.x provider packages as catalog types - [Prototype](https://github.com/elyra-ai/examples/pull/87) - [Provider list](https://airflow.apache.org/docs/apache-airflow-providers/) - [Potential enhancement request for Sandhya Nayak](https://github.com/elyra-ai/elyra/issues/2404) - Alan - Pushing for merge of the kfp-pipeline-spec package changes, conda-forge PR teed up. [KFP #7165](https://github.com/kubeflow/pipelines/pull/7165) - Kiersten - [#2399](https://github.com/elyra-ai/elyra/pull/2399) Fix Airflow component parser naming bug - Karla - Presenting Elyra at IBM Jumpstart OS next week - Kevin - Should we remove the deprecated `component-registries` schemaspace for 3.5? Was originally deprecated in 3.3 where users can use `elyra-metadata migrate component-registries` to transition them to the new schemaspace: `component-catalogs` - Yes, but removals come in major releases, right? So this would be for Elyra 4.0 ([kb] - ok, that answers that.) ## January 13th, 2022 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Patrick Titzler| IBM | @ptitzler | | Alan Chin | IBM | @akchinSTC | | David C Martin | Age@Home | @dcmartin | | Karla Spuldaro | IBM | @karlaspuldaro | | Kiersten Stokes | IBM | @kiersten-stokes | | Martha Cryan | IBM | @marthacryan | | Sandhya Nayak | IBM | @sandhya-nayak | | Kevin Bates | IBM | @kevin-bates | | Romeo Kienzler | IBM | @romeokienzler | | Luciano Resende | Apple | @lresende| | Alex Bozarth | IBM | @ajbozarth | * **Release status**: * Latest official release [3.4.1] * [3.4.1 changelog](https://elyra.readthedocs.io/en/latest/getting_started/changelog.html#release-3-4-1-01-07-2022) * Try it out! * `pip install --upgrade "elyra[all]==3.4.1" --use-deprecated legacy-resolver` * **conda-forge**: [In-Progress](https://anaconda.org/conda-forge/elyra) * Waiting on KFP dependency (kfp-pipeline-spec, in progress) **blocked** * https://github.com/conda-forge/staged-recipes/pull/16307 * Next planned release: 3.5.0 * 3.5.0 (Targeting late Jan) * Sprint planning in progress * [Deliverables](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.5.0) (subject to change) - 3.6.0 (STC - Subject To Change, Q1 2022) - JupyterLab 3.3 - 4.0 (STC, ~ Q2 2022) - JupyterLab 4.0 ### Agenda + Minutes - [Apache Airflow 1.10 reached EOL in June 2021 and only limited support is now provided.](https://github.com/elyra-ai/elyra/issues/2392) - need to add support for Airflow 2.0 - will drive BYO runtime work - ptitzler - Airflow Pipelines: [Add support for GitLab as DAG repository](https://github.com/elyra-ai/elyra/issues/1524) - Next: [Investigate support for provider packages as catalog types](https://github.com/elyra-ai/elyra/issues/2393) - alan - Chugging along with figuring out the kfp builds and our conda forge pkgs - Moving validation to base processor class for BYO validation - Karla - [PR #2383](https://github.com/elyra-ai/elyra/pull/2383) - Validate numeric fields in submit file dialog - [PR #2388](https://github.com/elyra-ai/elyra/pull/2388) - Add Script Editor language-specific subclasses - [PR #2390](https://github.com/elyra-ai/elyra/pull/2390) - Update Copyright headers - Debugger integration planned for 4.0 - Kiersten - [#2382](https://github.com/elyra-ai/elyra/pull/2382) - Removed support for exporting to KFP DSL - Kevin - [#2378](https://github.com/elyra-ai/elyra/pull/2378) Add support for CLI metadata and OVPs using JSON or file contents - Should be ready for review by end of day - [#2387](https://github.com/elyra-ai/elyra/issues/2387) Duration time showed wrongly for Python notebook after execution in pipeline - Opened [issue](https://github.com/deshaw/jupyterlab-execute-time/issues/60) with extension ## January 6th, 2022 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Alan Chin | IBM | @akchinSTC | | Luciano Resende | Apple | @lresende | | Karla Spuldaro | IBM | @karlaspuldaro | | Kiersten Stokes | IBM | @kiersten-stokes | | Martha Cryan | IBM | @marthacryan | | Kevin Bates | IBM | @kevin-bates | | Sandhya Nayak | IBM | @sandhya-nayak | | David C Martin | Age@Home | @dcmartin | * **Release status**: * Latest official release [3.4.0] * [3.4.0 changelog](https://elyra.readthedocs.io/en/latest/getting_started/changelog.html#release-3-4-0-12-17-2021) * Try it out! * `pip install --upgrade "elyra[all]==3.4.0" --use-deprecated legacy-resolver` * **conda-forge**: [In-Progress](https://anaconda.org/conda-forge/elyra) * Waiting on KFP dependency (kfp-pipeline-spec, in progress) **blocked** * https://github.com/conda-forge/staged-recipes/pull/16307 * Next planned release: 3.4.1 (Security Fixes)(Targeting later this week) * 3.5.0 (Targeting late Jan) * Sprint planning in progress * [Deliverables](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.5.0) (subject to change) ### Agenda + Minutes Happy New Year and welcome back! - Team is currently planning and prioritizing issues and milestones for 2022. If there is something you would like to see please feel free to add it to the agenda. - Alan - KFP pipeline spec package work - Validation service refactoring - Luciano - Planning to update Elyra video, in case any one want to help :) - Kevin - Metadata CLI support for `OneOf`, `AnyOf`, `AllOf` * Should be able to implement minimal support - Will add `--file` and `--json` options to bypass schema-to-options processing for files and strings (respectively) - (lresende) We have a pr to rewrite metadata cli to be using the same framework from pipeline cli, which would remove a lot of plumbing code. Any plans to continue that? - Community feedback on direction: https://github.com/elyra-ai/elyra/issues/2376