owned this note   changed 2 years ago  
Published Linked with GitHub

Nushell core team meeting 2023-04-19

Attendees

  • Antoine
  • Darren
  • Jt
  • Michael
  • Jakub
  • WindSoilder
  • Stefan

Agenda

Discussed Topics

Scope of the 0.79 release

  • need to make a call on:
    • standard library
      • default
        • more people to play with it
      • opt in?
        • startup penalty
        • complexity
      • config shouldn't be too hard
      • we could make it config point on by default.
    • defer removing the rust impls for now.
  • Type annotations in https://github.com/nushell/nushell/pull/8914
    • list<int> vs list(int)
      • what is the meaning of list(string) in bare form if we were to introduce classical call syntax foo()
      • also def foo (a: record(b: string)) {} is challenging to read if not formatted well
      • we rejected {a: string}/[int] for readability reasons a few meetings back
        • has to be constrained to type position (i.e. behind colon)
    • wait out the syntax change
  • Land the new IDE extension!

Standard library perf

  • windows Darren > 1sec (with std) else 700 or 30 ms with nu -n --no-std-lib
  • std help is still significantly slower than help
    • you can see std help ansi printing
      • buffered terminal write through a string?
      • we probably do blocking I/O through the whole stack
    • $nu is even visible in the flamegraph of startup (it clones state and stack via the LazyRecord logic for access)
      • caching is fine as long as we don't grow the scope with new definitions

Announcements

Include an anouncement for the stdlib and what it provides yet.
We have a new issue label specifically for std lib work

We get the cool new VS code extension!!!

Flag passing

No changes to discuss yet:
doc in [Passing Flags and Named Parameters Around](/A62iXv89RpqceJ2R0kiUvw

update in [Discord message](https://discord.com/channels/601130461678272522/683070703716925568/1096163164204314694

JT update on the parser upgrade

perf improvements by incremental work on the existing.

  • no big wins, lots of smaller improvements

Observation:

  • we are in a corner with our current design
  • lot's of clones
  • some of the slowdown is not necessarily directly parsing
    • name resolution
      • overlays etc. require relatively expensive work right now
      • algorithms and data structures ftw (hopefully)

nu-parser2

  • fast and loose
  • tons of syntax changes currently included
    • e.g. 1+2
  • JT wants to sneak in foo() call
  • Jakub: I think the nu_parser2 would be easy to modify to support current Nushell syntax, it's quite easy to work with.
  • We had to archive as people hoped to work on it but we weren't ready

nu-parser3 would be a drop in replacement as much as possible

span experiment
currently start and end on every AST node
drop spans to a separate Datastructure
would require changes to other parts

Also win for the nuon codepath

Update tree-sitter

1kinoti has been pushing hard over the last couple weeks
hopefully we can land this with darren's repo and move it into the org for official work

Data layout: what is Value

row-vs-columnar data

should table be more than list of records

dataframes/arrow is really about columnar data
great to type a bunch of data

2+2 kinds of performance optimizations

  • internal

  • code facing

  • get the relevant default user experience

  • high performance data-crunching performance

Sentiment:

  • get our core competencies right
  • 1.0 ship without, can wait on dataframe to settle in the wider ecosystem.

1.0 story

JT: general python-performance ballpark but we can improve later (compiled backend, dfs)

cranelift backend

post 1.0 sky is the limit

JT was comparing simple loopy benchmarks with different VM like languages
beating node is possible (take that V8)
Julia with or without compile pauses

for naive code should nu >1.0 should strive to be a great experience out of the box

PR's

Select a repo