Elyra Weekly Dev Meeting Archive: June - Aug 2021

# Elyra Weekly Dev Meeting Archive: June - Aug 2021 :::info **When:** Thursdays at 9:00AM Pacific (your [timezone](https://arewemeetingyet.com/Los%20Angeles/2019-10-24/09:00/w/Elyra%20Dev%20Meeting)) **Where:** [Webex](https://ibm.webex.com/meet/akchin) ::: ## Latest Meeting Minutes https://hackmd.io/SgvSqrWWR2248mCw2BZ5gg ## August 26th, 2021 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Alan Chin | IBM | @akchinSTC | | Martha Cryan | IBM | @marthacryan | | Karla Spuldaro | IBM | @karlaspuldaro | | Kiersten Stokes | IBM | @kiersten-stokes | | Kevin Bates | IBM | @kevin-bates | | David C Martin | Age@Home | @dcmartin | | Luciano Resende | IBM | @lresende | | Alex Bozarth | IBM | @ajbozarth | Please see our current release status: * **Release status**: * [3.0.1](https://elyra.readthedocs.io/en/stable/getting_started/changelog.html#release-3-0-1-08-18-2021) (latest stable release) * **Documentation**: [OK](https://elyra.readthedocs.io/en/stable/) * **PyPI**: [OK](https://pypi.org/project/elyra/) * **Docker**: [elyra (OK)](https://hub.docker.com/r/elyra/elyra/tags?page=1&ordering=last_updated) [kf-notebook (OK)](https://hub.docker.com/r/elyra/kf-notebook/tags?page=1&ordering=last_updated) * **Binder**: [OK](https://mybinder.org/v2/gh/elyra-ai/elyra/v3.0.0?urlpath=lab/tree/binder-demo) * **conda-forge**: [In-Progress](https://anaconda.org/conda-forge/elyra) * Waiting on KFP and Tekton Packages * **Open Data Hub** [In-Progress](https://github.com/opendatahub-io/odh-manifests/tree/v1.0.11) * 3.1.0 (in progress) * https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.1.0 * Support data exchange between KFP custom components * Move component registry to metadata ### Agenda + Minutes - Devin - Website updates - Alan - [Data Exchange updates - Design in progress](https://github.com/elyra-ai/elyra/issues/1761) - Behavior will not differ from current limitation on data passing - Updating the component registry handling for types - Karla - [#975](https://github.com/elyra-ai/elyra/issues/975) - Working on debugger integration with script editor - [#2075](https://github.com/elyra-ai/elyra/issues/2075) - Bring Your Own Schemas and Schemaspaces - Kiersten - Working on component registry migration into metadata service - [Issue](https://github.com/elyra-ai/elyra/issues/2031) has been updated to reflect the latest design direction and minor implementation details - Martha - Settings UI work - more refactoring - Looking into [#2072](https://github.com/elyra-ai/elyra/issues/2072) - Kevin - Bring Your Own Schemas and Schemaspaces [#2075](https://github.com/elyra-ai/elyra/issues/2075) - David - Check out https://github.com/quantumblacklabs/kedro ## August 19th, 2021 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Romeo Kienzler | IBM | @romeokienzler | | Alan Chin | IBM | @akchinSTC | | David C Martin | Age@Home | @dcmartin | | Martha Cryan | IBM | @marthacryan | | Alex Bozarth | IBM | @ajbozarth | | Karla Spuldaro | IBM | @karlaspuldaro | | Kevin Bates | IBM | @kevin-bates | Please see our current release status: * **Release status**: * [3.0.1](https://elyra.readthedocs.io/en/stable/getting_started/changelog.html#release-3-0-1-08-18-2021) (latest stable release) * **Documentation**: [OK](https://elyra.readthedocs.io/en/stable/) * **PyPI**: [OK](https://pypi.org/project/elyra/) * **Docker**: [elyra (OK)](https://hub.docker.com/r/elyra/elyra/tags?page=1&ordering=last_updated) [kf-notebook (OK)](https://hub.docker.com/r/elyra/kf-notebook/tags?page=1&ordering=last_updated) * **Binder**: [OK](https://mybinder.org/v2/gh/elyra-ai/elyra/v3.0.0?urlpath=lab/tree/binder-demo) * **conda-forge**: [In-Progress](https://anaconda.org/conda-forge/elyra) * Waiting on KFP and Tekton Packages * **Open Data Hub** [In-Progress](https://github.com/opendatahub-io/odh-manifests/tree/v1.0.11) * 3.1.0 (planning in progress) * https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.1.0 ### Agenda + Minutes - Romeo - *friendly reminder* / status inquiry [#1843](https://github.com/elyra-ai/elyra/issues/1843) - lresende - no progress afaik, I will assign this to 3.1/3.2 - romeo - okidoki - friendly reminder* / status inquiry [#2021](https://github.com/elyra-ai/elyra/issues/2021) - C3 compiled 1st running component from notebook to KFP component yaml - working - Devin - Updates on Elyra website efforts - Alan - Custom component data exchange design - in order of priority kfp, aa - creating sub tasks for backend implmentation and front end discussion points - Martha - Settings UI demo - For more info on design plans: https://github.com/jupyterlab/jupyterlab/issues/8148 - Alex - Working in parallel with Martha, cleaning up the metadata library code were donating that she's leveraging - Kiersten - Migrating component registry design recap [#2031](https://github.com/elyra-ai/elyra/issues/2031) - Karla - Debugger integration with Script Editor [#975](https://github.com/elyra-ai/elyra/issues/975) - progress demo ## August 12th, 2021 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Alan Chin | IBM | @akchinSTC | | Alex Bozarth | IBM | @ajbozarth | | Patrick Titzler | IBM | @ptitzler| | Karla Spuldaro | IBM | @karlaspuldaro | | Kiersten Stokes | IBM | @kiersten-stokes | | Luciano Resende | IBM | @lresende | | Kevin Bates | IBM | @kevin-bates | | Martha Cryan | IBM | @marthacryan | | David C Martin | Age@Home | @dcmartin | | Romeo Kienzler | IBM | @romeokienzler | Please see our current release status: * **Release status**: * 3.0.0 (latest stable release) * **Documentation**: [OK](https://elyra.readthedocs.io/en/stable/) * **PyPI**: [OK](https://pypi.org/project/elyra/) * **Docker**: [elyra (OK)](https://hub.docker.com/r/elyra/elyra/tags?page=1&ordering=last_updated) [kf-notebook (OK)](https://hub.docker.com/r/elyra/kf-notebook/tags?page=1&ordering=last_updated) * **Binder**: [OK](https://mybinder.org/v2/gh/elyra-ai/elyra/v3.0.0?urlpath=lab/tree/binder-demo) * **conda-forge**: [In-Progress](https://anaconda.org/conda-forge/elyra) * Waiting on KFP and Tekton Packages * **Open Data Hub** [In-Progress](https://github.com/opendatahub-io/odh-manifests/tree/v1.0.11) * 3.0.1 (3.0.0 patches)(in progress) * https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.0.1 * Fixes to operator examples * Node label validation will warn instead of error now * Fix typing issues during pipeline compilation * 3.1.0 (planning in progress) * https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.1.0 ### Agenda + Minutes - Alex - Working on donating the metadata service and editor to core JupyterLab [#1997](https://github.com/elyra-ai/elyra/issues/1997) - Where can others follow this? Pr? Branch? (lresende) - Martha - ^Same as Alex but for settings UI discussion refer to [#8148](https://github.com/jupyterlab/jupyterlab/issues/8148) (nothing new from us there yet because we're still getting it set up) - Alan - Updating/fixing failing examples - ODH testing - component data sharing planning for 3.1 - Kiersten - [Issue #2020](https://github.com/elyra-ai/elyra/issues/2020) - Updating the processing logic for list- and dictionary-type properties for runtime-specific components (3.0.1) - [Issue #2031](https://github.com/elyra-ai/elyra/issues/2031) - Looking into moving component registry to metadata service (3.1.0) - Karla - [Issue #975](https://github.com/elyra-ai/elyra/issues/975) - Debugger integration with Script Editor - Luciano - Planning 3.0.1 and 3.1 release - [3.0.1](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.0.1) - [3.1.0](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.1.0) - Romeo - [2021](https://github.com/elyra-ai/elyra/issues/2021) Turn Elyra into a no-code/low code tool which blows your mind (far better than KNIME) ## August 5th, 2021 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Romeo Kienzler | IBM | @romeokienzler | | Alan Chin | IBM | @akchinSTC | | Kiersten Stokes | IBM | @kiersten-stokes | | Patrick Titzler | IBM | @ptitzler| | Martha Cryan | IBM | @marthacryan | | Alex Bozarth | IBM | @ajbozarth | | Karla Spuldaro | IBM | @karlaspuldaro | | David C Martin | Age@Home | @dcmartin | | Kevin Bates | IBM | @kevin-bates | Please see our current release status: * **Release status**: * 3.0.0 (latest stable release) * **Documentation**: [OK](https://elyra.readthedocs.io/en/stable/) * **PyPI**: [OK](https://pypi.org/project/elyra/) * **Docker**: [elyra (OK)](https://hub.docker.com/r/elyra/elyra/tags?page=1&ordering=last_updated) [kf-notebook (OK)](https://hub.docker.com/r/elyra/kf-notebook/tags?page=1&ordering=last_updated) * **Binder**: [OK](https://mybinder.org/v2/gh/elyra-ai/elyra/v3.0.0?urlpath=lab/tree/binder-demo) * **conda-forge**: [In-Progress](https://anaconda.org/conda-forge/elyra) * Waiting on KFP and Tekton Packages * **Open Data Hub** [In-Progress](https://github.com/opendatahub-io/odh-manifests/tree/v1.0.11) * 3.0.1 (3.0.0 patches) * Nothing specific lined up yet ### Agenda + Minutes - Romeo - to present Elyra and Kubeflow at IBM Call4Code Spot Challenge 17th of August - searching for QA Volunteers - Count me in (lresende) - Issue [2017](https://github.com/elyra-ai/elyra/issues/2017) Double-click on a node should expand properties view (not opening the notebook) - Needs discussion (lresende) - Is common-properties for dynamic UI rendering? - Can a notebook cell be executed and it's output rendered into the properties view? - Devin - Website - Alex - Martha, Karla and I have started the process of donating the metadata service and extension to core jupyterlab/server. - How should licenses be handled during donation sicne we use Apache and jupyter uses BSD? - https://github.com/ajbozarth/jupyterlab/pull/1 - Martha - Have been working on settings editor for JL - [#1986](https://github.com/elyra-ai/elyra/pull/1986) relative paths PR re-opened - Alan - Updating airflow examples - Testing Elyra 3.0 on ODH ## July 29th, 2021 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Martha Cryan | IBM | @marthacryan | | Karla Spuldaro | IBM | @karlaspuldaro | | Luciano Resende | IBM | @lresende | | Alan Chin | IBM | @akchinSTC | | Kiersten Stokes | IBM | @kiersten-stokes | | Alex Bozarth | IBM | @ajbozarth | | David C Martin | Age@Home | @dcmartin | | Patrick Titzler | IBM | @ptitzler| | Kevin Bates | IBM | @kevin-bates | Please see our current release status: * **Release status**: * 2.2.4 (latest stable release) * **Documentation**: [OK](https://elyra.readthedocs.io/en/stable/) * **PyPI**: [OK](https://pypi.org/project/elyra/) * **Docker**: [elyra (OK)](https://hub.docker.com/r/elyra/elyra/tags?page=1&ordering=last_updated) [kf-notebook (OK)](https://hub.docker.com/r/elyra/kf-notebook/tags?page=1&ordering=last_updated) * **Binder**: [OK](https://mybinder.org/v2/gh/elyra-ai/elyra/v2.2.4?urlpath=lab/tree/binder-demo) * **conda-forge**: [OK](https://anaconda.org/conda-forge/elyra) * **Open Data Hub** [OK](https://github.com/opendatahub-io/odh-manifests/tree/v1.0.11) * **3.0.0rc2 release status** * [3.0.0rc2](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.0.0) - is out and available on pypi - pip install elyra --upgrade --pre * **3.0.0 release status** * [3.0.0](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.0.0) * Support for pipeline specific components - in 3.0.0rc0 * Pipeline parser refactoring - in 3.0.0rc0 * Pipeline Editor - Additional palette location option - in 3.0.0rc0 * Update to latest kfp and kfp-tekton dependencies - in 3.0.0rc1 * Documentation - [In **latest** only](https://elyra.readthedocs.io/en/latest/) * [Custom components limitations status](https://github.com/elyra-ai/elyra/discussions/1890) * Pipeline Validation service - in 3.0.0rc2 * Auto-Migration of existing example pipelines - in 3.0.0rc2 * Remaining Issues * [Error executing notebook using "run notebook using papermill" kfp component ](https://github.com/elyra-ai/elyra/issues/1967) * Verification of kfp and aa components * CLI/validation - [**3.1.0 milestones**](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.1.x) ### Agenda + Minutes - Karla - Elyra talk at PyData Sao Paulo (co-host: Luciano) - Alan - Update PR for merging of airflow-notebook package back into core - ptitzler - custom component examples documentation should be ready next week - https://github.com/elyra-ai/examples/tree/master/pipelines/kubeflow_pipelines_component_examples - https://github.com/elyra-ai/examples/tree/master/pipelines/airflow_component_examples - lresende - Getting rc out - kfp-notebook has been merged into elyra package (as of 3.0.0rc2) - updating pipeline cli to use validation services - updating release scripts to properly deploy single-package for kfp-notebook ## July 22nd, 2021 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Martha Cryan | IBM | @marthacryan | | Karla Spuldaro | IBM | @karlaspuldaro | | Luciano Resende | IBM | @lresende | | Alan Chin | IBM | @akchinSTC | | Kiersten Stokes | IBM | @kiersten-stokes | | Patrick Titzler | IBM | @ptitzler| | Alex Bozarth | IBM | @ajbozarth | | David C Martin | Age@Home | @dcmartin | | Patrick Titzler | IBM | @ptitzler| | Kevin Bates | IBM | @kevin-bates | | Romeo Kienzler | IBM | @romeokienzler | Please see our current release status: * **Release status**: * 2.2.4 (latest stable release) * **Documentation**: [OK](https://elyra.readthedocs.io/en/stable/) * **PyPI**: [OK](https://pypi.org/project/elyra/) * **Docker**: [elyra (OK)](https://hub.docker.com/r/elyra/elyra/tags?page=1&ordering=last_updated) [kf-notebook (OK)](https://hub.docker.com/r/elyra/kf-notebook/tags?page=1&ordering=last_updated) * **Binder**: [OK](https://mybinder.org/v2/gh/elyra-ai/elyra/v2.2.4?urlpath=lab/tree/binder-demo) * **conda-forge**: [OK](https://anaconda.org/conda-forge/elyra) * **Open Data Hub** [OK](https://github.com/opendatahub-io/odh-manifests/tree/v1.0.11) * **3.0.0rc1 release status** * [3.0.0rc1](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.0.0) - is out and available on pypi - pip install elyra --upgrade --pre * **3.0.0 release status** * [3.0.0](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.0.0) * Support for pipeline specific components - in 3.0.0rc0 * Pipeline parser refactoring - in 3.0.0rc0 * Pipeline Editor - Additional palette location option - in 3.0.0rc0 * Update to latest kfp and kfp-tekton dependencies - in 3.0.0rc1 * Documentation - [In **latest** only](https://elyra.readthedocs.io/en/latest/) * [Custom components limitations status](https://github.com/elyra-ai/elyra/discussions/1890) * Remaining issues * Pipeline Validation service - Rebase in prog. * Migrate existing example pipelines - [**3.1.0 milestones**](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.1.x) ### Agenda + Minutes - Alan - [1592](https://github.com/elyra-ai/elyra/pull/1664)- Rebasing and testing with migrated example pipelines - ptitzler - Tutorials awaiting reviews - [Create generic pipelines and run them in JupyterLab](https://github.com/elyra-ai/examples/pull/54) - [Run generic pipelines on Kubeflow Pipelines](https://github.com/elyra-ai/examples/pull/55) - [Run generic pipelines on Apache Airflow](https://github.com/elyra-ai/examples/pull/56) - Karla - [#1937](https://github.com/elyra-ai/elyra/pull/1937) - Fix Open File command for pipeline nodes (merged) - [#141](https://github.com/elyra-ai/pipeline-editor/pull/141) - Enhance node properties error message style (merged) - [#1939](https://github.com/elyra-ai/elyra/pull/1939) - follow up PR on Elyra regarding above - [#1935](https://github.com/elyra-ai/elyra/pull/1935) - Update pipeline editor dialog message for unsupported files ## July 15th, 2021 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Romeo Kienzler | IBM | @romeokienzler | | Martha Cryan | IBM | @marthacryan | | Karla Spuldaro | IBM | @karlaspuldaro | | Luciano Resende | IBM | @lresende | | Alan Chin | IBM | @akchinSTC | | Kiersten Stokes | IBM | @kiersten-stokes | | Patrick Titzler | IBM | @ptitzler| | Alex Bozarth | IBM | @ajbozarth | | David C Martin | Age@Home | @dcmartin | | Kevin Bates | IBM | @kevin-bates | Please see our current release status: * **Release status**: * 2.2.4 (latest stable release) * **Documentation**: [OK](https://elyra.readthedocs.io/en/stable/) * **PyPI**: [OK](https://pypi.org/project/elyra/) * **Docker**: [elyra (OK)](https://hub.docker.com/r/elyra/elyra/tags?page=1&ordering=last_updated) [kf-notebook (OK)](https://hub.docker.com/r/elyra/kf-notebook/tags?page=1&ordering=last_updated) * **Binder**: [OK](https://mybinder.org/v2/gh/elyra-ai/elyra/v2.2.4?urlpath=lab/tree/binder-demo) * **conda-forge**: [OK](https://anaconda.org/conda-forge/elyra) * **Open Data Hub** [OK](https://github.com/opendatahub-io/odh-manifests/tree/v1.0.11) * **3.0.0rc1 release status** * [3.0.0rc1](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.0.0) - is out and available on pypi - pip install elyra --upgrade --pre * **3.0.0 release status** * [3.0.0](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.0.0) * Support for pipeline specific components - in 3.0.0rc0 * Pipeline parser refactoring - in 3.0.0rc0 * Pipeline Editor - Additional palette location option - in 3.0.0rc0 * Update to latest kfp and kfp-tekton dependencies - in 3.0.0rc1 * Documentation - [In **latest** only](https://elyra.readthedocs.io/en/latest/) * [Custom components limitations status](https://github.com/elyra-ai/elyra/discussions/1890) * Remaining issues * Pipeline Validation service * Migration updates including Pipeline version bump * Pipeline component properties collision fix * Migrate existing example pipelines ### Agenda + Minutes * Romeo * Low Code/No Code UX of Pipeline Editor [1910](https://github.com/elyra-ai/elyra/issues/1910) Render component configuration UI * Martha * [#1913](https://github.com/elyra-ai/elyra/pull/1913) Added descriptions for palette nodes * [#143](https://github.com/elyra-ai/pipeline-editor/pull/143) Added component types as a heading to the properties editor * Karla * [#141](https://github.com/elyra-ai/pipeline-editor/pull/141) Enhance validation error style in pipeline property * [Issue #1888](https://github.com/elyra-ai/elyra/issues/1888) Language server warnings (upstream) * Alan * Airflow summit 2021 summary [link to cast](https://www.crowdcast.io/e/airflowsummit2021/16) * Airflow notebook follow up refactoring after [1918](https://github.com/elyra-ai/elyra/pull/1918/files) * Luciano * Merging kfp-notebook into elyra repo * Adding support for passing parameters to scripts/notebooks * Helping with 3.0.0 overall tasks * Discussion * Creating generic Elyra components * ptitzler * Documentation updated (available only in _latest_) * [Pipelines](https://elyra.readthedocs.io/en/latest/user_guide/pipelines.html) * [Pipeline components](https://elyra.readthedocs.io/en/latest/user_guide/pipeline-components.html) * [Best practices for file-based nodes](https://elyra.readthedocs.io/en/latest/user_guide/best-practices-file-based-nodes.html) * Updating tutorials and examples (for 3.0) * Introduction to generic pipelines (JupyterLab) https://github.com/elyra-ai/examples/pull/54 * Run generic pipelines on KFP * Run generic pipelines on AA * Deferred to > 3.0 * Introduction to custom components * Best practices for custom components * Created blog posts * [Created custom components feature status discussion thread](https://github.com/elyra-ai/elyra/discussions/1890) ## July 8th, 2021 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Martha Cryan | IBM | @marthacryan | | Luciano Resende | IBM | @lresende | | Karla Spuldaro | IBM | @karlaspuldaro | | David C Martin | Age@Home | @dcmartin | | Kiersten Stokes | IBM | @kiersten-stokes | | Alan Chin | IBM | @akchinSTC | | Patrick Titzler | IBM | @ptitzler| | Eduardo Blancas | Ploomber | @edublancas | Please see our current release status: * **Release status**: * 2.2.4 (latest stable release) * **Documentation**: [OK](https://elyra.readthedocs.io/en/stable/) * **PyPI**: [OK](https://pypi.org/project/elyra/) * **Docker**: [elyra (OK)](https://hub.docker.com/r/elyra/elyra/tags?page=1&ordering=last_updated) [kf-notebook (OK)](https://hub.docker.com/r/elyra/kf-notebook/tags?page=1&ordering=last_updated) * **Binder**: [OK](https://mybinder.org/v2/gh/elyra-ai/elyra/v2.2.4?urlpath=lab/tree/binder-demo) * **conda-forge**: [OK](https://anaconda.org/conda-forge/elyra) * **Open Data Hub** [OK](https://github.com/opendatahub-io/odh-manifests/tree/v1.0.11) * **3.0.0rc0 release status** * [3.0.0rc0](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.0.0) - is out and available on pypi - pip install elyra --upgrade --pre * **3.0.0 release status** * [3.0.0](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.0.0) * Support for pipeline specific components - in 3.0.0rc0 * Pipeline parser refactoring - in 3.0.0rc0 * Pipeline Editor - Additional palette location option - in 3.0.0rc0 * Remaining issues * Pipeline Validation service * Migration updates including Pipeline version bump * Pipeline component properties colision fix ### Agenda + Minutes * DEMOs * Ploomber - Welcome! * Short Elyra Demo - Luciano * Martha * [#1858](https://github.com/elyra-ai/elyra/pull/1858) Adds a runtime images button to the pipeline editor toolbar * [#1844](https://github.com/elyra-ai/elyra/pull/1844) Merged - moving palette to the left and adding categories to the palette - Alan - [1854](https://github.com/elyra-ai/elyra/pull/1854) - Rebase and remove redundant operators in airflow - Update Airflow component examples repo - PR is WIPed - Ploomber demo - https://github.com/ploomber/ploomber - "Excellent! HMU: dcmartin@age-at-home.com" - @dcmartin ## July 1st, 2021 Weekly Meeting This weekly dev meeting has been cancelled due to release work. ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | David C Martin | Age@Home | @dcmartin | Please see our current release status: * **Release status**: * 2.2.4 (latest stable release) * **Documentation**: [OK](https://elyra.readthedocs.io/en/stable/) * **PyPI**: [OK](https://pypi.org/project/elyra/) * **Docker**: [elyra (OK)](https://hub.docker.com/r/elyra/elyra/tags?page=1&ordering=last_updated) [kf-notebook (OK)](https://hub.docker.com/r/elyra/kf-notebook/tags?page=1&ordering=last_updated) * **Binder**: [OK](https://mybinder.org/v2/gh/elyra-ai/elyra/v2.2.4?urlpath=lab/tree/binder-demo) * **conda-forge**: [OK](https://anaconda.org/conda-forge/elyra) * **Open Data Hub** [OK](https://github.com/opendatahub-io/odh-manifests/tree/v1.0.11) * **2.3-beta2 release status** * [2.3-beta2](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A2.3.0.beta2) - is out and available on pypi - pip install elyra --pre * **2.3.0 release status** * [2.3.0](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A2.3.0) - Milestone STILL in progress, realllly hope to have this out in the next few days * Support for pipeline specific components * Pipeline parser refactoring * Pipeline Validation service * Pipeline Editor - Additional palette location option - Requires pipeline migration ## June 24th, 2021 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Alan Chin | IBM | @akchinSTC | | Kiersten Stokes | IBM | @kiersten-stokes | | Alex Bozarth | IBM | @ajbozarth | | Luciano Resende | IBM | @lresende | | Kiersten Stokes | IBM | @kiersten-stokes | | Karla Spuldaro | IBM | @karlaspuldaro | | Patrick Titzler | IBM | @ptitzler| | Kevin Bates | IBM | @kevin-bates | | Martha Cryan | IBM | @marthacryan | | David C Martin | Age@Home | @dcmartin | * **Release status**: * 2.2.4 (latest stable release) * **Documentation**: [OK](https://elyra.readthedocs.io/en/stable/) * **PyPI**: [OK](https://pypi.org/project/elyra/) * **Docker**: [elyra (OK)](https://hub.docker.com/r/elyra/elyra/tags?page=1&ordering=last_updated) [kf-notebook (OK)](https://hub.docker.com/r/elyra/kf-notebook/tags?page=1&ordering=last_updated) * **Binder**: [OK](https://mybinder.org/v2/gh/elyra-ai/elyra/v2.2.4?urlpath=lab/tree/binder-demo) * **conda-forge**: [OK](https://anaconda.org/conda-forge/elyra) * **Open Data Hub** [OK](https://github.com/opendatahub-io/odh-manifests/tree/v1.0.11) * **2.3-beta2 release status** * [2.3-beta2](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A2.3.0.beta2) - is out and available on pypi - pip install elyra --pre * **2.3.0 release status** * [2.3.0](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A2.3.0) - Milestone STILL in progress, realllly hope to have this out in the next few days * Support for pipeline specific components * Pipeline specific validation service - moved back to 2.3.0 * Pipeline Editor - new release cut - 0.7.0 * KFP Notebook - release TBD - for RUN_ID - Requires pipeline migration ### Upcoming Events: - Scipy 2021 - Elyra poster sessions - July 14-16 - Apache Airflow Summit 2021 - July 12 @ 9 PDT - Pydata - July 27 @ 10AM PDT - PyData Brazil - early July (date TBD) ### Agenda + Minutes - Martha - [#1819](https://github.com/elyra-ai/elyra/pull/1819) CI artifacts - [#135](https://github.com/elyra-ai/pipeline-editor/pull/135) Upgrade canvas - [#1800](https://github.com/elyra-ai/elyra/pull/1800) Small CSS update to properties editor - [#1796](https://github.com/elyra-ai/elyra/pull/1796) Clear button bug fix - [#129](https://github.com/elyra-ai/pipeline-editor/pull/129) Fix bug with custom components without required properties - [#137](https://github.com/elyra-ai/pipeline-editor/pull/137) Move node palette to the left side panel of pipeline editor - Here is the issue with a suggested visual of the editor on the left side [#1795](https://github.com/elyra-ai/elyra/issues/1795) - Alan - Reviewing PRs [1803](https://github.com/elyra-ai/elyra/pull/1803) and [1830](https://github.com/elyra-ai/elyra/pull/1830) - Testing kf-serving component - Updating validation service with better human readable and actionable node info - Finishing up SciPy materials - Alex - Will be continuing work on https://github.com/elyra-ai/elyra/issues/1064 for 2.4 after talking to the mentors for the JupyterLab interns - Luciano - CLI prs under review [#1830](https://github.com/elyra-ai/elyra/pull/1830) and [#1805](https://github.com/elyra-ai/elyra/pull/1805) - Working with Kiersten to refactor runtime component specific code [#1801](https://github.com/elyra-ai/elyra/pull/1801) - Release Name -> 2.3 vs 3.0 ## June 17th, 2021 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Alan Chin | IBM | @akchinSTC | | Kiersten Stokes | IBM | @kiersten-stokes | | Alex Bozarth | IBM | @ajbozarth | | Luciano Resende | IBM | @lresende | | Martha Cryan | IBM | @marthacryan | | Kiersten Stokes | IBM | @kiersten-stokes | | Alona Varshal | IBM | @mudspringhiker | | Karla Spuldaro | IBM | @karlaspuldaro | | Patrick Titzler | IBM | @ptitzler| | Kevin Bates | IBM | @kevin-bates | * **Release status**: * 2.2.4 (latest stable release) * **Documentation**: [OK](https://elyra.readthedocs.io/en/stable/) * **PyPI**: [OK](https://pypi.org/project/elyra/) * **Docker**: [elyra (OK)](https://hub.docker.com/r/elyra/elyra/tags?page=1&ordering=last_updated) [kf-notebook (OK)](https://hub.docker.com/r/elyra/kf-notebook/tags?page=1&ordering=last_updated) * **Binder**: [OK](https://mybinder.org/v2/gh/elyra-ai/elyra/v2.2.4?urlpath=lab/tree/binder-demo) * **conda-forge**: [OK](https://anaconda.org/conda-forge/elyra) * **Open Data Hub** [OK](https://github.com/opendatahub-io/odh-manifests/tree/v1.0.11) * **2.3-beta2 release status** * [2.3-beta2](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A2.3.0.beta2) - is out and available on pypi - pip install elyra --pre * **2.3.0 release status** * [2.3.0](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A2.3.0) - Milestone STILL in progress, realllly hope to have this out in the next few days * Support for pipeline specific components * Pipeline specific validation service - moved back to 2.3.0 * Pipeline Editor - new release cut - 0.5.2 * KFP Notebook - release TBD * Airflow notebook - release required if RUN_NAME PR ### Agenda + Minutes Upcoming Events: Scipy 2021 - Elyra post sessions - July 14-16 Apache Airflow Summit 2021 - July 12 @ 9 PDT Pydata - July 27 @ 10AM PDT - Alan - Airflow pipeline components - operators vetted, need push updates to airflow template for contributor operators changes - Moving validation service back, make updates - Testing PRs - Martha - [#1708](https://github.com/elyra-ai/elyra/pull/1708) Merged pipeline properties - [#1769](https://github.com/elyra-ai/elyra/issues/1769) Adding a bug fix for this in pipeline-editor - [#1776](https://github.com/elyra-ai/elyra/pull/1776) Working on merging with master - Karla - IBM Jumpstart Open Source Program - Q2 highlights - Luciano - Testing, Testing, Testing - Working on backward compatibility tests - Working on corner cases of runtime specific components ## June 10th, 2021 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Alan Chin | IBM | @akchinSTC | | David C Martin | Age@Home | @dcmartin | | Layne Sadler | AIQC | aiqc | Kiersten Stokes | IBM | @kiersten-stokes | | Alex Bozarth | IBM | @ajbozarth | | Kevin Bates | IBM | @kevin-bates | | Luciano Resende | IBM | @lresende | | Patrick Titzler | IBM | @ptitzler | * **Release status**: * 2.2.4 (latest stable release) * **Documentation**: [OK](https://elyra.readthedocs.io/en/stable/) * **PyPI**: [OK](https://pypi.org/project/elyra/) * **Docker**: [elyra (OK)](https://hub.docker.com/r/elyra/elyra/tags?page=1&ordering=last_updated) [kf-notebook (OK)](https://hub.docker.com/r/elyra/kf-notebook/tags?page=1&ordering=last_updated) * **Binder**: [OK](https://mybinder.org/v2/gh/elyra-ai/elyra/v2.2.4?urlpath=lab/tree/binder-demo) * **conda-forge**: [OK](https://anaconda.org/conda-forge/elyra) * **Open Data Hub** [OK](https://github.com/opendatahub-io/odh-manifests/tree/v1.0.11) * **2.3-beta2 release status** * [2.3-beta2](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A2.3.0.beta2) - is out and available on pypi - pip install elyra --pre * **2.3.0 release status** * [2.3.0](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A2.3.0) - Milestone in progress * Support for pipeline specific components * Pipeline specific validation service * Pipeline Editor - new release cut - 0.5.2 * KFP Notebook - release TBD ### Agenda + Minutes - Alan - [1664](https://github.com/elyra-ai/elyra/pull/1664) - Updating back end for small changes to expected error list format - Testing components in pr #1620 - need to install kfserving - Working on Scipy 2021 binder proposal - Kiersten - [#1620](https://github.com/elyra-ai/elyra/pull/1620) - Support runtime-specific components - **demo** - Execution supported for KFP & Airflow - Preconfigured components are still being tested & finalized - Alex - Have been continuing work on [1064](https://github.com/elyra-ai/elyra/issues/1064) while doing reviews. Found that my original plan of attack would not work well and create a lot of spagetti code. Reevaluating my plan and figuring out a new path forward to replace non-interactive dialogs with toast notifications. - Luciano - Helping wit [#1620](https://github.com/elyra-ai/elyra/pull/1620) - Based on discussion on JupyterLab dev meeting from 06/09 - Moving metadata service to upstream (jupyter-server)? - Moving metadata-editor to jupyterlab-core / lumino? - Discuss hub concept - Possible integration with MLX - Real time collaboration - ptitzler - New runtime environment variable `ELYRA_RUN_ID` - PRs ready for review https://github.com/elyra-ai/elyra/pull/1732 - Requires new `kfp-notebook` and `airflow-notebook` releases - dcmartin - check out https://mermaid-js.github.io/mermaid a markdown for drawing graphs, flows, etc.. could be a way to document pipelines, flows, .. ## June 3rd, 2021 Weekly Meeting ### Attendees Add your name to the table below: | Name | Affiliation | Git User-id | | ----------- | ---------------- | ----------- | | Alan Chin | IBM | @akchinSTC | | Martha Cryan | IBM | @marthacryan | | Luciano Resende | IBM | @lresende | | Karla Spuldaro | IBM | @karlaspuldaro | | Kiersten Stokes | IBM | @kiersten-stokes | | Alex Bozarth | IBM | @ajbozarth | | David C Martin | Age@Home | @dcmartin | | Kevin Bates | IBM | @kevin-bates | | Patrick Titzler| IBM| @ptitzler | * **Release status**: * 2.2.4 (latest stable release) * **Documentation**: [OK](https://elyra.readthedocs.io/en/stable/) * **PyPI**: [OK](https://pypi.org/project/elyra/) * **Docker**: [elyra (OK)](https://hub.docker.com/r/elyra/elyra/tags?page=1&ordering=last_updated) [kf-notebook (OK)](https://hub.docker.com/r/elyra/kf-notebook/tags?page=1&ordering=last_updated) * **Binder**: [OK](https://mybinder.org/v2/gh/elyra-ai/elyra/v2.2.4?urlpath=lab/tree/binder-demo) * **conda-forge**: [OK](https://anaconda.org/conda-forge/elyra) * **Open Data Hub** [OK](https://github.com/opendatahub-io/odh-manifests/tree/v1.0.11) * **2.3-beta2 release status** * [2.3-beta2](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A2.3.0.beta2) - is out and available on pypi - pip install elyra --pre * **2.3.0 release status** * [2.3.0](https://github.com/elyra-ai/elyra/issues?q=is%3Aopen+is%3Aissue+milestone%3A2.3.0) - Milestone in progress * Support for pipeline specific components * Pipeline speciifc validation service * PREP - Pipeline Editor, KFP Notebook release ### Agenda + Minutes - Alan - Integrating validation checks with runtime components PR - Review docs for KF 1.3 integration - Kiersten - [#1620](https://github.com/elyra-ai/elyra/pull/1620) - Support runtime-specific components - KFP component execution functional - Working on Airflow execution - Martha - [#122](https://github.com/elyra-ai/pipeline-editor/pull/122) & [#1708](https://github.com/elyra-ai/elyra/pull/1708) Pipeline properties editor - Karla - [#1714](https://github.com/elyra-ai/elyra/pull/1714) - Pipeline editor package refactor (React components) - Alex - Working on [#1064](https://github.com/elyra-ai/elyra/issues/1064) to switch some dialogs to toast notifications - Luciano - Presented Elyra to Flyte community - Opening discussions around implementing Flyte as a supported runtime - Discussion points - [#1721](https://github.com/elyra-ai/elyra/issues/1721) - Support for parameters - How to support env_vars on distributed systems - What should we add to Elyra data efforts (next priority?) - Better support for cos integration - SQL Editor - ## Archives of Previous Meetings ### March - May 2021 https://hackmd.io/Lw1A-PdRS2WbTFXRkVpfxw?view ### Nov 2020 - Feb 2021 https://hackmd.io/k5p_LCI_TYenIsWkeIpz2g?view ### May - Oct 2020 https://hackmd.io/VMs51wxCRjqTXHhCFCazoQ?view

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.