Nushell core team meeting 2022-06-01

Attendees

• Jakub
• Michael
• Fernando
• Andres
• Reilly
• sholderbach (note-taking)
• jt

Open Topics

• Hook and overlays getting everything to work
  • Dataframe overlay

Discussed topics

Arrow and dataframes

• Arrow provides efficient operations and memory representation
• core arrow: memory format spec
• Where to get our query planning from?
  • arrow rust is currently fragmented into two implementations.
  • polars with arrow2
  • other projects use arrow
• Ideas
  • Everything in memory vs chunked.
  • Backing storage if likely going out of memory (e.g. parquet files)
• Good examples of using Arrow and Dataframes
  • Iox
  • Ballista

Detour: Asynchronous execution model

• Q: Is it a fit for nushell after engine-q?
• JT reports from/about tokio
• Obvious: UI interactiveness
• But: traditional shell use not many tasks like a webserver
• Shell has some significant (file) I/O
• For direct use in the nu language we would need language components (select the first succeeding, collect unordered, collect in sequence)

Thought to sleep on: Who is the client of nushell?

• Jakt's design was so fast due to the explicit rails of the SerenityOS project. Nushell is more than one thing but what do we want to address
• Nushell as a toolbox (current developments: overlays, "overloading")

Nu is more than just shell

• Do we want to drop shell from nushell in official communication as we can more than just traditional shell work with the language?
• Reaching out to get github .nu support
  • Resolving the name/syntax highlighting conflict with the current but abandoned nu

Permeable/Transparent/Environment touching blocks

• Idea JT: split closure / block

• Special sigil for immediate evalution on the block to create environment effects.

• would not be available for functions

• Corner case: overlays already provide defs etc. and are special scopes

• maybe requires two modes: basic environment affecting and overlay definition affecting

• Kubouch: define at the callsite (like rust unsafe block?)

  • but do we really want those annotations to become viral (annotation on every function that might be affected by use of @env)?

 def-env foo [x: int] {
    let-env FOO = $x + 10
}

foo 10 

# Very common use case users expect
@env @overlay if 4 > 2 { 
    overlay add spam
} else {
	overlay add eggs
}

​def foo [x: Closure] {
​    @overlay do $x 
​    
​    # ???? who knows what happened
​}