# [npe2][] Design

The aim of this document is to

1. be a central place to gather thoughts/todo items for the overall design.
2. start gathering what will eventually be user facing docs around [npe2][].

> [name=Nathan Clack] I've been trying to **bold** words referencing new concepts, and *italicizing* pre-existing concepts. Some concepts may need more explicit definition or inclusion in a glossary.

## Introduction

[Napari][] loves plugins. Plugins allow people to add new readers and writers for accessing new kinds of data, customize napari's appearance, or add new widgets for interacting with data.

Plugins are managed with a *plugin engine* that

1. exposes plugin functionality and metadata for use by napari.
2. discovers plugins.
3. manages the life-cycle of a plugin within napari.

### Original plugin system.

There are two main components:
* [napari-plugin-engine][]. Based on [pluggy][].
* discovery via naming convention as well as entry_points
* support for reordering of hook calls after registration.
* enhanced API for retrieving plugin package metadata.
* modified plugin call and registration exception handling
* modified HookResult object and hook call loop, with ability to retrieve the HookImplementation responsible for the result.
* lazy plugin discovery
* a couple napari-specific convenience imports
* The napari [plugin manager][NapariPluginManager]
* evented enable/disable register/unregister
* plugin ordering
* layer between hooks and napari.

Napari declares functions that plugins can override by defining *hook specifications*. Plugins define matching *hook implementations*.

The [Napari plugin engine][napari-plugin-engine] manages discovery. One way plugins are discovered is by searching through all the installed python packages in the local environment where napari is running. Those that define an appropriate *entry point group* are imported and registered so that their *hook implementations* are available.

A significant limitation of this design is that plugins must be imported as python modules in order to query their functionality or metadata. Plugins are required to import quickly. If they block, napari will freeze.

The supported set of hook specifications are described in the [reference][hook-spec-ref]. Briefly, they are:

* `napari_provide_sample_data`
* `napari_get_reader`
* `napari_get_writer`
* Single-layer writers
* `napari_write_image`
* `napari_write_labels`
* `napari_write_points`
* `napari_write_shapes`
* `napari_write_surface`
* `napari_write_vectors`
* `napari_experimental_provide_function`
* `napari_experimental_provide_theme`
* `napari_experimental_provide_dock_widget`

### How napari interacts with plugins

#### napari's plugin manager

* Evented
* registered/unregistered
* enabled/disabled

#### Plugin preferences dialog

* ordering

#### Plugin management dialog

* enable/disable
* install/uninstall

#### File manipulation

* opening a file via a file open dialog
* saving layer data via a file save dialog

#### Menu items

* sample data
* dock widgets/analysis plugins

## Design

The "napari plugin engine 2" (npe2) introduces a **manifest**. The manifest is a specially formatted text file distributed with a plugin package that describes the plugin's functionality.

The manifest describes functionality that the plugin contributes. Most contributions comes in the form of **commands**. Each command is associated with a python [Callable][]. Commands can be associated with key bindings, menu items, and identified with specific functionality. For example, a command might be used to read or write layer data.

### Plugin life cycle

stateDiagram-v2
direction LR
[*] --> Installed: pip install myplugin
Installed --> Registered: plugin_manager.discover()
Registered --> Enabled: import myplugin
Enabled --> Disabled: deactivation
Disabled --> Enabled: activation

> [name=Nathan Clack] what happens if a plugin is uninstalled out from under napari. What if it happens between discovery and activation.

> [name=Nathan Clack] Q1: is activation required before being a command can be invoked? A1: command exec calls activate. Q2: Does exec command check for plugin enable/disable? A2: Doesn't look like disable is really a thing. Removing a command from command registry may be sufficient.

### Backward and forwards compatibility

## Contributions

### Readers

### Writers

Given a set of layers, we need to be able to select one or more compatible writers. Ideally, these are used to present the user with a list of choices in the "Save As" file dialog, enabling the user to pick their favorite. In the case where the user doesn't indicate a preference, a predictable writer choice should be made.

Current (before this PR) Napari behavior is to invoke installed writer plugins one-by-one until the first succeeds. A set of built-in plugins is tried last. The intent is that these builtins provide reasonable fallbacks so all layer type combinations may be written.

#### Compatibility

Currently, the pluggy-style hook specifications define two kinds of writers: single-layer writers and a multi-layer writer.

The `WriterContributions` assumes the command being bound corresponds to a multi-layer `writer()` function such as the one returned by a `napari_get_writer()` implementation.

Alternatively, the command could correspond to one of the single-layer writers, e.g . `napari_write_image`. This is indicated in the `WriterContribution` by setting the `use_single_layer_api` flag to true. For example:

contributions:
  commands:
  - id: napari_builtins.write_points
    python_name: napari.plugins._builtins:napari_write_points
    title: napari built-in points writer
    short_title: napari points
  writers:
  - command: napari_builtins.write_points
    filename_extensions: ['.csv']
    layer_types: ['points']
    uses_single_layer_api: true

There is no need to bind a function like `napari_get_writer()` in npe2 because writers use the manifest to declare which writer npe2 should select. That is, npe2 has the responsibility for mapping a list of layer types to a writer.

#### Layer type constraints

A writer plugin can declare that it will write between *m* and *n* layers of a specific type where *0* <= *m* <= *n*.

For example:

    image      Write exactly 1 image layer.
    image?     Write 0 or 1 image layers.
    image+     Write 1 or more image layers.
    image*     Write 0 or more image layers.
    image{k}   Write exactly k image layers.
    image{m,n} Write between m and n layers (inclusive range). Must have m<=n.

> [name=Nathan Clack] The `image {k}` and `image{m,n}` syntax is probably not that useful and, for that reason, probably confusing. The underlying implentation can handle these, but they may not be worth documenting/exposing to the user.

When a type is not present in the list of constraints, that corresponds to a writer that is not compatible with that type. For example, a writer declaring:

    layer_types=["image+", "points*"]

would not be selected when trying to write an `image` and a `vector` layer because the above only works for cases with 0 `vector` layers.

Note that just because a writer declares compatibility with a layer type does not mean it actually writes that type. In the example above, the writer might accept a set of layers containing `image`s and `point`s, but the write command might just ignore the `point` layers. The writer must return `None` for unwritten layers.

#### Writer selection

Most of the writer selection logic is on the napari side.

Given a list of layers of different types, the number of each type is counted and compared against the constraint ranges specified for each writer. This yields a set of compatible writers.

These writers can be used to populate the "Save As" file dialog with a list of compatible writer commands and their file extensions. Ultimately the user selects a specific writer command to invoke via this dialog.

When a user doesn't pick a specific writer (e.g when saving from the command line), the situation is more complicated. This PR does the following:

1. When the path has a file extension, find a compatible writer that has that same extension.

2. When there is no extension and only a single layer, take the first compatible writer and append the first listed file extension.

3. Otherwise, find a compatible no-extension writer and write to that. No-extension writers typically write to a folder -- at least that's what the built-in does.

### Widgets

### Themes

### Settings

### Translations

## Napari integration

### Plugin preferences dialog

### Plugin management dialog

### Writer considerations

[Napari]: https://napari.org/
[napari-plugin-engine]: https://github.com/napari/napari-plugin-engine
[NapariPluginManager]: https://github.com/napari/napari/blob/main/napari/plugins/_plugin_manager.py
[npe2]: https://github.com/tlambert03/npe2
[Callable]: https://docs.python.org/3/library/typing.html#callable
[hook-spec-ref]: https://napari.org/plugins/stable/hook_specifications.html
[pluggy]: https://github.com/pytest-dev/pluggy