Nushell core team meeting 2022-11-02

Attendees

  • Jakub
  • JT
  • Michael
  • Reilly
  • Darren
  • Stefan
  • Fernando
  • Dan
  • The public listening on discord

Agenda

  • Public meeting schedule?

  • 15 min's on the clock for PRs

  • How to proceed with the indexing commands? (tables from dandavision and webbedspace)
    –> JT will take a look

  • Overloading:

    • Related: input/output type draft from Dan: https://github.com/nushell/nushell/pull/6796
      • Introduces a mapping: depending on input different output
      • Overloading in terms of vectorisation/selection
      • But a runtime challenge (some stuff will appear as Any atm)
    • Document from Jakub: https://hackmd.io/@nucore/Skt0PGjVj/edit
      • allowing input output type annotations for custom commands with new syntax
      • Callsite overloading (dynamic but add a manual type enforcement at call site like Rust turbofish)
      • Question of dynamic dispatch
      • input $in is PipelineData instead of Value and supports lazy iteration and POSIX streams.
      • Action item for all: read through the document
      • good starting point to experiment with syntax into the new parser

  • Plugins:

    • We want to trim the # of commands shipped with Nushell. Need better plugin functionality to move things out of core. Ex: dataframes
    • Better core compiletimes/smaller binary
    • Thinking about a install dataframe command
      • Good example how simple julia or R handle plugin/package installation
      • how to persist safely. (see abi stability below and package/folder management)
    • Update on libloading/abi_stable experiment
      • Open question: ABI compatibility/versioning
        • #[repr(C)] would be the stable but hard path to go
        • reprC would open up other languages for core plugins with internal data performance (currently value is the abstraction instead of PipelineData -> not lazy)
      • A lot of additional macro magic (in cglue or safer_ffi)
      • Get's us dll/.so loading
    • Feedback from Dan Gohman (creator of cranelift) on using wasm
      • Also language agnostic
      • Going back and forth across process boundaries works if you serialize… or if you're working across a C ABI… not really an improvement over libloading :)
      • Feels experimental. Not an obvious win

  • New parser / typechecker

    • Making progress, needs more help
    • Goal also get match
    • Checkout the checklist
    • Sideshow: error handling (minimal implementation from JT vs miette as in current nushell)
    • Action item: jump in and ask JT if you get stuck

  • RFC (or similar) procedure to stay more focused with PRs

    • Our RFC repo got dormant 2 years ago
    • We might had to reject some cool PRs that didn't fit in the current road map.
    • Could we get a light weight alternative
      • for small breaking changes make sure to get buy in from a set of frequent/experienced devs
    • Discord,
    • also announce what is hot on the inofficial roadmap
      • Status update channel
    • project board is stale

  • Question from Adam: is there a plan to simplify/consolidate commands that have grown organically?

    • webbedspace has been looking at commands
    • Important problem as we aim for 1.0
    • Dan: Don't want to burn out focusing on many small changes if what we need is a larger/holistic review. Nu 1.0 will be judged on consistency
    • Get a consensus on guidance how nushell should feel and behave should drive how we update the commands in a idiomatic way

-> Action item: it is our current goal:
Standardize our commands/semantics/ get the input output types right. Polish the organically grown stuff.

Discussed Topics

Discussed PR's

  • 6796 Suggestion: input and output syntax shapes (@dandavison)
    • Input output shapes
    • We are moving away from type directed parsing (in the parser the types are known as SyntaxShape)
    • WIP in progress but are getting there
    • Has to affect all commands
    • Additional assertions in the tests based on the Input Output Types
    • We need to have the ability to subtype: Int/Float: Numeric
    • General consensus: get this moving in (even though we haven't moved yet from SyntaxShape)
    • Moving from Draft as soon as tests pass
    • Maybe worth plan time in the next meeting to resolve any open issues
    • Follow up: help text format
  • 6798 Skip parsing named flags when not specified (@Yesterday17)
    • Dangerous because if you mistype a flag you get no help, and it will be treated as a positional arg
    • Conclusion: reject
    • Maybe accept unknown flags even though the custom completion definition doesn't know about them
    • Rest flags
    • Maybe revisit with the general parser rewrite
  • 6799 fix: fixcd (@Decodetalkers)
    • Getting close
    • Unix checks the x bit
    • windows
  • 6806 Protocol: debug_assert!() Span to reflect a valid slice (@dbuch)
    • Waiting on virtualenv tests passing
  • 6809 Add column completion support (@nibon7)
  • 6893 make take behave like first (@WindSoilder)
    • Dan and webbedspace have put together summaries
    • JT to follow up
  • 6898 prevent error when lines reads bad bytes (@merelymyself)
    • Stefan thinks it should operate on bytes instead of converting to strings in a lossy fashion
    • JT: if lines fails to decode, should give an error telling the user to manually decode
    • Better error message would be the nushell way

PR list

gh pr list --json url,number,author,title | from json | each {|i| $"- [($i.number)]\(($i.url)\) ($i.title) \(@($i.author.login)\)"} | reverse | to text

Not yet discussed (will either be resolved on them or scheduled next week)

  • 6885 [WIP] table: Change Record view in expand-mode (@zhiburt)
  • 6896 Fix ignore-errors for select (@nibon7)
  • 6910 Make seq return a ListStream where possible. (@merelymyself)
  • 6915 New "display_output" hook. (@PerBothner)
  • 6938 removes unused features. (@FilipAndersson245)
  • 6958 Add –ignore-program-errors option to do command (@SUPERCILEX)
  • 6966 Refactor ansi stripping into nu-util functions (@sholderbach)
  • 6975 Improve do command docs (@SUPERCILEX)
  • 6976 Fixed $in in where blocks (@webbedspace)
  • 6983 Grouped config commands better (closes #6911) (@webbedspace)
Select a repo