Nushell core team meeting 2023-04-12

Attendees

Agenda

• Overarching topic: 1.0 goals
• Reverting the breaking syntax changes
  • What can we learn from the tradeoffs and ultimate principle driving the reversal
• Performance of the parser (cost of std)
  • nu-parser2 fast 2 furious
• (760 issues)
  • duplicates
  • issues that are potentially resolved
  • issues that are a mishmash -> break into PR level pieces.
• (Review process)
  • PR approval
  • https://chelseatroy.com/2019/12/18/reviewing-pull-requests/
• 1.0
  • Vision and goals
  • No more language breaking changes!
  • What breaking changes do we want/need to land?

Discussed Topics

Maturing nushell - Road to 1.0

We are moving well forward towards the 0.80 goals we set our selves (Jakub's notion kanban)

Next milestone to spitball: 1.0

What is the scope of 1.0. What do we want to achieve.
(not fully coverable in this meeting)

Everyone: collect the 1.0 wishlist. What do you want to see stabilized or which breaking change you really want to make soon?

Recurring guest: dataframe

currently an ugly duckling

What is the part of the 1.0 package?

• core language is fixed
• but maybe later
  • dataframe (the underlying polars is not yet 1.0 either)
  • ? plugins

Rollback of the parser changes

• goal of the changes originally: make the parsing rules easier to explain

  • 1numberlike:
    • would open the door to arbitrary unit system
    • surprisingly inconsistent behavior (e.g. git commit hashes starting with number)
  • || requirement for closure
    • strong impact for the community

• observation

  • JT: nushell is a shell for the large part of it is usage
  • the same design philosophy applies for the interactive programming language
  • especially painful in the altering existing code
  • is more shell like maybe more than just more bash like
  • what are the anti-patterns for a shell
    • long stuff to type
  • Darren: we don't want to turn into sigil soup (no APL/perl replacement)
  • Jakub: you should be able to read the meaning (strong suit over bash)
    • || closure is coming from a specific niche of programming languages
  • "lightweight" and "intuitive"
    • easy coming from other languages
  • partially more strict parsing for help messages?
    • we can get this power with more type annotations by the (library writers/)users
  • can we not care about blocks and closures?
    • JT: blocks
    • yo it just mutates its environment
    • only scope if you require it to not make changes to the env
  • the closure capture story is weird in context with the mutability
    • we don't have a particular reason to split the mutation from closure capture as it is by value.
    • capture to mutate would have to be slightly different.
      • is it just mutation vs. capture by value.
    • def vs def-env
    • could we trace the mutation so we can give good error messages and automatically make that part of the closure/block signature
    • Problem: $env.foo = inside a block:
      • Is it scoped or is it mutation.
      • maybe make scoping the explicit part to say you don't want to modify the env
      • is def the escape hatch
  • how to make the if $nu.os-info.name == windows { do something } story good.
    • source
    • parse time evaluation (e.g. marked by const) (we already have some logic living in the parser for that)
      • hard if you could reach infinite loops etc.
      • snapshot the environment appropriately

Parser performance

With the std and big config we parse a lot of code on startup:
in debug it is slow. in release mode is not that bad.

nu_parser2 at the moment is a lot faster but it is in no way feature complete yet.

Data oriented design: nodes and spans live separate from the AST structure. (how to get them cache-local is it is own challenge)

Type checking could become its own stage/pass

Darren: slippery slope ahead! growth in the std library will still skyrocket and this can impact performance.

JT: jetbrains pioneered ultra fast parsing similar to lite parse. only materialize on call. penalty will be delayed or spread out.
Jakub: julia's lazy compiling is a warning sign that doing lazy compilation is non trivial. (it aint no hotspot vm)

Curse of success

We need more benchmarks/ tracking of that to catch slow regressions
Performance work needs to be data driven

can we crowd source the benchmark and dashboard creation.

LSP

time to dogfood!

There:

• goto-def
• error
• hovers
• type hints
• completions

wip

• multi-file support

only barrier: building the latest VScode plugin
Darren has been building VSIX regularly.

but should be easy to get started with a nodejs install.

removing let-env

JT has been trying to deprecate it in favor of just $env

But we support filling in the variable name to some extent (mind blown members of the core teams didn't know this one neat trick, doctors are shocked)

let-env $foo = "bar"
let $foo = "BAR"

keep in mind we have the cool load-env

PR's