Nushell core team meeting 2024-07-31

Attendees

ctrlc -> plugin, when to reset plugin side Signals? https://github.com/nushell/nushell/issues/13501
stretch, pie in the sky, assigning pipelines to variables, without collecting?
Add nu_plugin_plist functionality to nu_plugin_formats?
Adding content type metadata for other types.. should we pull in a lib to use with open or manually assign.
If an underlying Pipeline reader is capable of Seek, should we expose a method to grab to expose Seek.. this would be perf optimization when using polars open as part of a pipeline.. The readers require Seek.
ANSI contamination in the pipeline, harmful strip -> PipelineMetadata
main stuff

currently doesn't reset after Ctrl-C, quits on the next call
for internal stuff the repl will reset after one whole loop
- Andy: should also broadcast to the plugins?
- Devyn: for symmetry with internal commands (if multiple commands are spawned e.g. par-each to ensure safe shutdown)
- Andy: sounds the easiest to do the same, plan a PR (Thanks!)
- some responsibility on the plugin author (if you want to respond to it)

lazy
scoping challenge, can only be consumed once and can't outlive its closure etc.
with the IR VM it would behave like binding a register
style of utilities like the rust iterator adapters imaginable
- e.g. take one entry, then pass the stream to further processing

Devyn: at some point invert the default

when opening a file with open should we set the mime-type based on the extension into the PipelineMetadata
database source, are we accepting the breaking data changes in a crate vs manually setting in the from ... converter commands
when performing transformations with to need to update
useful when piping to http family commands (motivated by jack's endeavour into a S3 plugin)

composability of cloud/storage provider plugins with polars would be nice
expects Seek (includes .rewind etc.) not quite compatible with a stream
dumping into a temp file may be wasteful for some usecases
can we dispatch on kind of input
? sidestep stream and instead use the datasource?
- our current datasource in PipelineMetadata doesn't get updated should you do transformations in between
- would need a separate trusted channel/metadata field

we shouldn't be stripping ANSI escapes implicitly ever
change find for now so that whatever is needed for the highlighting is in the PipelineMetadata
try to follow this pattern for internal commands
discussion about what to do about extensibility:
- ways to make it possible to define custom behavior on PipelineMetadata for styling?
- how can plugins add to this?
- is this losing pipeline composability? there are filters that would potentially know how they have to modify PipelineMetadata to preserve the right styling, and they'd probably just end up throwing it out not knowing what to do
- possible alternatives
  1. styled strings (like emacs) within the Value variants?
    - would be kind of nice because all commands could use it
    - support multiple spans within the string with different styles
    - not too heavy, easy to remove the styles and concat all of the strings
    - do we need syntax for this or no? possible to just construct it with commands
    - prior art (linked by @rose): https://www.gnu.org/software/emacs/manual/html_node/elisp/Text-Props-and-Strings.html
  2. Value::WithMetadata type wrapper that wraps a Box<Value> with some kind of metadata, when needed?
    - it could get complicated to know how to handle this correctly at different places
    - a bit inefficient perhaps, though it does preserve best case performance when this isn't needed
    - probably not something we can hope to represent in NUON