# Plugin commands (`register` replacement) proposal

The `register` command has a few problems:

- It tries to execute arbitrary executable files at parse time, which means that binaries can be run just by typing something at the REPL
- It's overloaded with lots of potentially separate functions:
- Runs a plugin to get signatures and then puts those in the engine state
- Also writes all plugin commands in engine state out to the `plugin.nu` cache file
- Also (undocumented) use in `plugin.nu` to actually load the cached signatures, with an extra JSON parameter
- It isn't a `plugin` subcommand, which now exists with `plugin list` and `plugin stop`

My proposal (@devyn) is to separate these into two new commands, along with changing the plugin cache file into something declarative (current idea is `plugin.msgpack`) so that it can be edited incrementally:

## Commands

### `plugin add`

Synopsis:

plugin add /path/to/nu_plugin_foo
plugin add --shell python /path/to/nu_plugin_bar.py

This has basically the same user interface as `register`, but it **is not a parser keyword**. Instead, it runs the binary when the `plugin add` command gets evaluated, and only modifies the plugin cache file to replace the signatures.

This **does not** add the plugin commands loaded from the plugin into the engine state immediately (e.g. at the REPL); you must run `plugin use` afterward to actually get those commands into engine state, because that can only happen at parse time.

### `plugin use`

Synopsis:

plugin use foo

This loads the signatures for the plugin named `foo` from the plugin cache file into the engine state. It is a parser keyword, but because it only loads data that was previously prepared at runtime, it does not have the issue of potentially executing arbitrary code at parse time.

If the signatures for `foo` are not present in the cache file or are invalid for some reason, this produces an error, helpfully telling the user to `plugin add` the plugin and then try again.

I am currently thinking that `plugin use` will not be required to make plugin commands available; we may reasonably just keep the current behavior of still loading the entire plugin cache at startup.

The reason for the name being included here is so that if a script needs a plugin installed, it can still produce an error if that plugin is not present, just like it could if it were to include `register`. As a bonus, it wouldn't have to touch the user's plugin cache file.

## Changes to `nu`

`nu --no-config-file` currently doesn't allow loading plugins at all, not even with an explicit `register`. I would like to support this, but splitting `plugin add` and `plugin use` will make that impossible from a script context.

I'm thinking instead that `nu` can take a `--plugin` option to load a plugin at startup, without adding it to the plugin cache file:

nu --no-config-file --plugin /path/to/nu_plugin_gstat --commands 'gstat'

## The `plugin.msgpack` file

There are a few issues with the current `plugin.nu` file:

- It's not easy to load, modify, and then write it again, neither for Nushell, nor for a user. Editing it to remove a plugin pretty much requires a text editor. We currently regenerate it from scratch every time `register` is called.
- As we've learned from `nu_plugin_polars`, if there are a lot of commands, it can get very large. Mine is currently 497.6 KiB (uncompressed). Even uncompressed, `msgpack` could bring that down to 138.7 KiB.
- Parsing it is slow. The JSON signatures are parsed by both `nu-parser` and `serde_json`, and it can include a lot of strings for examples and usage and such. Since this needs to be loaded on every start, performance should be more of a priority.
- It doesn't support metadata at the plugin level. I would like to be able to store things such as whether the plugin supports local sockets, to be able to skip stdio if supported.

If we use a declarative format, not only can `plugin add` just load it and make the modifications and save it again, but we could also provide a `plugin rm`, or it would even allow users to do something like:

open $nu.plugin-path | where name !~ '^foo' | save -f $nu.plugin-path

If we use `msgpack`, we'll want to add `from msgpack` and `to msgpack` to core.

For filesize comparison (roughly) using `nu_plugin_msgpack`:

open $nu.plugin-path | str length | into filesize
open $nu.plugin-path |
  parse -r '(?ms)register (?<path>[^\s]+) (?<json>(?:(?!\n\n).)+)' |
  update json {from json} |
  rename path command |
  group-by --to-table path |
  rename path commands |
  update commands { each { get command } } |
  to msgpack |
  bytes length |
  into filesize

We could compress it using brotli, in which case we might want to use an extension that makes that clear, like `plugin.msgpackz`. This additionally cuts the filesize by 8-12x, so it's potentially worth it.