Nushell core team meeting 2024-04-03

Attendees

Release - patch release needed?
- 12383 hook fix
- 12341 drop notification fix
Nushell 1.0: Bringing crates to soft-1.0 in stages
1. Language (nu-parser, nu-protocol, nu-cmd-lang, nu-engine)
2. Commands (nu-command)
3. Standard library (nu-std)
4. Plugins (nu-plugin)
5. etc.
Safety and Security in/around/with Nushell
- "supply-chain"
- Safety: mistakes/bugs that can cause serious damage
- Security: which threats apply to users using Nushell?
- policies we can actually implement?
Derive macros & traits for converting structured Rust data to/from Values (Devyn)
- FromValue
- ToValue
- UpdateFromValue (for error-tolerant config pattern)
- compare this vs. trying to make serde work
Performance engineering
- benchmarks/visibility
- priorities

Hook fix by Ian
- regression due to changes to stack/engine state (root stack)
- annoying as it breaks virtualenv activation logic (and potentially other similar env or overlay related)
- looking for willing testers
- Sentiment: worth a patch release
drop notification behavior of plugins
- devyn started on a PR to address non-compliant behavior before the release
- stefan head some questions but not reviewed/signed off before release
- test of the wrong behavior that should now be fixed
default_config.nu breaks on termux
- clipboard feature doesn't work on all platform but the file currently included a binding using the system clipboard
(antoine's nu_plugin_explore)
- broke with the plugin protocol changes
- (Devyn: surprised that it worked even before)
- STDIN is not available to the plugin now
Decision: patch release as soon as the relevant commits are ready

Jakub: gradual stabilization strategy
- what can we leave out of the initial 1.0 date
- the volume of commands is overwhelming to get into a "we can guarantee stability" state
Ed?: are there change planned with new-nu-parser
- different approach
- we don't expect too many deliberate language changes
- edge cases may change
- ripping out problematic ambiguity (may lead to changes)
- may affect our type checking logic
Devyn: level of stabilization for internal crates
- need to maintain semver for the cargo build system
- but we could still keep the internal crates evolving
The main promise
- any script written for 1.0 should work forever
challenge compared to bash etc.
- we also contain a ton of commands
- thanks to the new plugin frontier we are able to implement more of the commands in plugins
plugins
- strategy to ensure semver logic for plugin/engine compatibility check
Jack:
- deprecation strategy
- what to do if we have to do a breaking change for good cause (do a major version bump)?
- plugin installation strategy
  - this is currently hard to distribute if you don't live in the cargo ecosystem
  - nupm -> become part of the 1.0 roadmap
- dataframe deprecation for 1 release than remove as the polars plugin
Summary of things we want on the 1.0 roadmap
- nupm for plugin distribution
- new-nu-parser switch if possible

Distinction: core of nushell and the surface of all the commands
Examples for Security
- query db hardening against sql injection
- dependencies / supply-chain
  - moving to plugins reduces our surface
  - reducing
Examples for Safety
- mv affected a user on discord
  - using the uutils mv
  - operation not fully atomic to roll back
  - Devyn: should we make trash the default
    - in the past we had it for a while
    - pushback due to slowness on windows
    - probably not good for scripts, so interactive mode only
procedure/system
- responsible disclosure
- security email

serde does not get us at the span while parsing
can we have an error tolerant version for config
- here the management of the manual code is hard
Sophia: serde bit us in the past for performance reasons
Darren: we were/are pretty strict on our "let's not write macros" rule
- exceptions exist: record!
- what is intuitive can be subjective
- maintainenance cost (fewer maintainer/contributors are macro proficient)

benchmarking
- divan vs criterion
  - divan doesn't support machine output
  - simpler
  - faster local runs
  - criterion is not moving as much
  - overconfident
  - more stuff to configure
benchmark ideas
- more complex functions/scripts
- testing different conditions (e.g. operation, operation as a function, operation as a closure)
- overall diversity (math, fileops, load+mutate+save etc.)