Nushell core team meeting 2024-05-22

Attendees

  • Darren
  • Devyn
  • Antoine
  • Ian
  • Sophia
  • Cassaundra
  • Michael
  • Andy
  • Stefan
  • Jakub

Agenda

  • Checking in on dataframe removal PR - any issues so far?
  • Checking in on ByteStream PR - any issues so far?
  • Cassaundra has some contributions to discuss from her work on quake-build
  • Podcast interview invitation
  • Range semantics inclusive vs. exclusive

Discussed Topics

Dataframe removal? Any gotchas?

  • polars plugin
  • Maxim-uvarov is doing a lot of testing
    • opening csv
      • weird perf outliers on mac only
      • another polars-based tool also observed something similar -> report upstream
      • (potentially related to macOS 14.5 update)
        • side-channel mitigation?

ByteStream changes, ripple effects?

  • Devyn: buffering stuttering with htttp get/save downloads?
    • more detailed triangulation necessary
  • Raw "benchmark" open -r file1 | save file2

Cassaundra's project:

  • https://github.com/quake-build/quake
  • build system using nu component crates.
    • DSL to create tasks
    • registering some existing commands
  • own def-task command (with complexities)
    • runs into the keyword-y behavior of our def and friends -> own parsing step to handle the blocks?
  • (thread on ideas https://discord.com/channels/601130461678272522/615329862395101194/1242194131371954296)
    • parser refactor
    • librarification
    • error handling
      • parse error
    • POSIX concerns
  • upstream has been overtaking some of the local patches
  • Sophia: goal keep the integration as close to "vanilla" Nushell to ensure most users have the same and improved experience
  • Darren: reference to couchbase shell (by couchbasedb as a repl)
    • ship their own commands
  • crates that needed hacking for the quake requirements
    • primarily nu-parser to integrate a new keyword-like command

new nu parser open forum

  • is it requirement for 1.0 is recurring question
  • perf as a motivation
    • stdlib config load times
    • how many passes make sense to keep things in the happy cache land of the flat-AST design
    • Sophia's work on truffle with a VM with op-codes with tuned opcodes
  • span/node ids
    • jakub has been hacking on the existing parser/protocol system
      • replace Span with SpanId (only the parser knows about Span): monster pr with constant breakage
      • flattening is a challenge as the Span are gone into SpanId at the point we would flatten currently: move to different pass
      • how to move incrementally to new SpanIds?
      • would permit integration of new-nu-parser in a faster as we have IDs beyond the parser/AST
      • testing
        • ? have tests that snapshot/verify the stuff marked by the spans
        • some of the tests for completions break
  • loss less parsing for documentation
    • Darren: introducing annotation attributes
    • Devyn: nuon edit?
    • nufmt: wouldn't need the following passes but could benefit from a parser pass (with minimal validation verifying whitespace rules)

h3

Select a repo