# napari hub automation

We'd like to introduce some automated quality checks or review process in napari hub to make the plugin ecosystem more secure, reliable, and trusted. It should also have the ability to remove plugin entries from the hub.

## Motivation

In the process of migrating napari plugins from PyPI to conda-forge, plugin authors are inadvertenly introducing some packaging issues, including:

- **Outdated dependency metadata** for both PyPI and conda. Either packages that are no longer used but still listed, or packages that are now required but not listed.
- **Strict pinning** of napari versions or other dependencies which provide less compatibility among plugins within the napari ecosystem
- **Operating system** restricted compatibility due to developers/users only testing on Windows, macOS, or Linux.
- **Python** compatibility with different versions. Some authors proactively restrict the range of compatible Python versions, while in reality it does work. Other issues might include the opposite, using new Python features (e.g. `:=`) without reflecting that requirement in the metadata.
- **Development state**: plugins may not be ready to be shared with the wider audience but are published to napari-hub nevertheless.
- **Framework identifier**: used for plugins that authors do not want to be shared with a wider audience.
- **Best practices not followed** like direct listing of PyQt5 (pyqt6/pyqt) as a dependency instead of the Qt wrapper (`qtpy`) recommended [best practices](https://napari.org/stable/plugins/best_practices.html#don-t-include-pyside2-or-pyqt5-in-your-plugin-s-dependencies)

On the other hand, napari-hub might still list plugins that are:

- **Abandoned plugins**: Authors that do not reply on Github or email addresses after several months (years?)
- **Incomplete metadata**: No emails, repository information, etc...

All of the above just focuses on metadata problems, but we are also concerned about the code quality. Without a review process, plugin authors could introduce attack vectors, performance problems and other issues. Some packaging errors might result in the package accidentally vendoring licensed libraries, clobbering paths, etc.

> Example: A contributor uploads a package to PyPI named `napari-neuron-detector`, that promises automated neuron detection. However, at import time, the author forgot to delete a debugging line or actually included one on purpose, e.g. `os.system("rm -rf /")` or `time.sleep(10000)`).

## Current plugin stats

As the napari ecosystem grows, the process of creation and curation of conda recipes does not scale and will become unsustainable unless we introduce a set of minimal quality checks in the napari-hub and the tooling around plugin creation and discovery. Current statistics for plugins already migrated to conda packages are

- Current plugins on napari-hub: 257
- Current plugins available on conda-forge: 186
- Current plugins with PR on conda-forge: 58
- Missing PR on conda-forge: 3

And new plugins are created every week!

## Proposals

### napari-hub plugins quality checks

Currently napari-hub serve as an aggregator for napari plugins providing a streamlined interface for napari users. However there is currently no quality checks or curation to ensure that plugins listed on the hub comply with a minimum set of requirements including:
- Installability
- Compatibily
- Correct metadata
- Correct dependencies
- etc...

Without some sort of review process, we fear that a plugin's presence in the napari-hub catalog (and subsequently, in napari's Plugin Manager UI) might be understood as some sort of quality endorsement. Packaging issues can (and have!) irreversibly affect the installation of napari (e.g. by overwriting key files).

This process should also describe policies to remove plugin releases from napari-hub in case they are proven problematic, unmaintained or incompatible with the latest versions of napari.

### Creation of a pre-commit tool to check for package dependencies (check pip and conda dependencies)

Despite having a set of [best practices](https://napari.org/stable/plugins/best_practices.html#don-t-include-pyside2-or-pyqt5-in-your-plugin-s-dependencies), we have increasingly found the same issues over and over on different plugins. As part of the napari plugin cookiecutter template we could include additional checks and tooling to help authors keep comply more strictly with the best practices. This way we can not only list them but actually enforce them. A new pre-commit tool could be developed to check for correct listing of dependencies which could get us a long way in automating a lot of the process of recipe and package creation.

### Automatic creation of conda forge recipes

A napari-bot could automatically create conda recipes using grayskull on conda-forge. But in order to maximize the chance of success on automatic creation we need to have tools to check that dependencies are in sync. (See the previous point).

### Running compatibility checks of plugins with different versions of napari

An initial proof of concept has been started on the [`napari/packaging` repo](https://github.com/napari/packaging/pull/29).

This CI workflow lists solver errors due to irreconciliable metadata (see Motivation section above). Some examples:

Installation attempt for python=3.10.*=*cpython napari=0.4.16=*pyside* conda-forge::hesperos==0.1.33 has errors!
Reasons:
 -  package hesperos-0.1.33-pyhd8ed1ab_0 requires napari <0.4.15, but none of the providers can be installed
--------------------
Installation attempt for python=3.10.*=*cpython napari=0.4.16=*pyside* conda-forge::napari-hdf5-labels-io==0.3.dev16 has errors!
Reasons:
 -  package napari-hdf5-labels-io-0.3.dev16-pyhd8ed1ab_0 requires python >=3.6,<3.9, but none of the providers can be installed

We also check for correct solutions that incorrectly pull `pyqt` despite the bundled installation requiring `pyside` only for licensing reasons. This can reveal author plugins are not following the recommended best practices:

Installation attempt for python=3.10.*=*cpython napari=0.4.16=*pyside* conda-forge::beetlesafari==0.3.2 has errors!
Reasons:
 -  solution has pyqt5-sip==4.19.18
 -  solution has pyqt-impl==5.12.3
 -  solution has pyqtwebengine==5.12.1
 -  solution has pyqtchart==5.12
 -  solution has pyqt==5.12.3
--------------------
Installation attempt for python=3.10.*=*cpython napari=0.4.16=*pyside* conda-forge::devbio-napari==0.8.0 has errors!
Reasons:
 -  solution has pyqt5-sip==4.19.18
 -  solution has pyqt-impl==5.12.3
 -  solution has pyqtwebengine==5.12.1
 -  solution has pyqtchart==5.12
 -  solution has pyqt==5.12.3

# Decision points

* [ ] Some kind of quality check is needed before a plugin is listed on napari-hub
* [ ] The QCs will be mostly automated, but approved