Nushell core team meeting 2023-02-08

Attendees

• Darren
• JT
• Jakub
• Michael
• Reilly
• Stefan

Agenda

• (New contributors!)
  • Any unmet questions/directions?
• JT's object/record oriented module proposal
• JT's notes about invisible dataframes
• Lightning topics:
  • Stefan: codecoverage tracking
    • Currently failing to merge coverage on some runners
    • Some restructuring of the tests still needed.
• New category of debug commands is growing!
  • debug, our old loyal friend
  • inspect, thinking about renaming to explain. PR 7957
  • view source, same command just renamed without the dash. PR 7989
  • view span, give you the ability to view spans. PR 7989
  • view files, shows what's in the engine_state files vec. PR 7989
  • profile, times each line of a script and shows you. PR 7854
  • nu_plugin_dbg, looking for a better name, shows description and values. Darren's repo
  • benchmark, times scripts, no changes

Discussed Topics

New contributors

• dunno where they came from

nu-lang domain names

Not sure if we'll do anything with this for now, but JT has a feeling that we might eventually; the "nushell" name probably deters a lot of people who don't realize it's a programming language

Code coverage

Stefan: got pretty far but there's some issue with the tooling, where it can't do a final merge of the coverage data on macOS
Maybe we just do code coverate on Ubuntu (and Windows?)

Compiling Nu scripts

JT was able to make a small app that runs a single script

replace create_default_context and use only those commands needed and include_str the script

Tree-shaking of dependency only happens at the last step/linking (so still >400 crates to compile)

Binary streaming

open foo.exe does not display nicely, need to do into binary and it does not stream
TODO: should into binary stream?

Debug commands

Naming: should we avoid view in the names? dbg?
JT:
-benchmark name is too long to type.
- What if we allowed benchmark at the end of a pipeline instead of using a closure? But that won't capture the time to calculate parameters

"Invisible Dataframes"

JT looked into Polars/Arrow/Datafusion more, nothing seems like it's a great fit for streaming data

df open "foo.csv" | where name = bar

idea: Liskov Substitution Principle as applied to Nu command overloading

• base/any command is the authoritative one
• specialized overloading commands should not change their behaviour

Nushell is not trying to be a perfect data science/processing platform for everyone; little more focus on the programming+shell sides

engine-q afterthought: separating the containers/streams (i.e. transport types) from the primitive types would be better than the PipelineData/Value split we have now (we have to repeat a bunch of container broadcasting code. typechecking in multiple places.)
Adding dataframe/sql etc. would be primarily on the transport/container layer

Reilly: how do we ensure that overloaded commands are functionally the same as the base version? i.e. differences in flags

Stefan: are commands the right abstraction?

• we do unpacking/typechecking/input validation/operation/restructuring all in Command::run
• ?interfaces/structural type style

JT's object/record oriented module proposal

JT's brain, late at night: "objects, modules, records are all the same thing"

Darrens dataframe defragmentation/renaming

JT thinks we should do the renaming now and work toward a better overloading solution later

PR's

WindSoilder's addition of examples to plugins

Vote: Land and iterate with more plugins!