# Nushell core team meeting 2022-11-16

## Attendees

- Darren
- JT
- Jakub
- Reilly
- Michael
- Stefan
- Dan
- WindSoilder
- FilipAnderson

## Agenda

- Timing for GitHub Nushell support
- Plans for parser changes
- Trimming down commands / messaging our intent as we do
- More thoughts on plugins
- Input/output signature syntax
- Final <15min PRs

# Discussed Topics

## Observations before the scheduled agenda

`table --expand` is going great (and with the `display_hook` you can expand it based on terminal size)

This release packaging was quick

## GitHub nu(shell) integration

nu extension still refers to the objective-C lisp `.nu`
We are starting to reach critical mass to reach out to github/linguist for `.nu` as nushell

Needs 200 repos
https://github.com/github/linguist/blob/master/CONTRIBUTING.md#adding-a-language

## Plans for parser changes

JT's thinking of landing small incremental changes instead of a big bang
- will support bashisms like >, &&, ||. some changes to the pipeline data structure needed (can no longer be a simple `Vec<Command>``)

a | b
Pipeline {items: &[Command]}

a > file.dat
a || b
a && b
// Needs a more flexible structure
enum Pipeline {
  Redirect(...)
  Or(..)
  And(..)
  ..
}

[Experiment now archived](https://github.com/jntrnr/nu_parser2)

## Trimming down commands

- Areas to trim:
- build-string
- trivially replaced > gone
- shells command (`n`, `p`)
- hogging valuable chars
- dataframes commands (big chonker)
- Balance value (biggish data runtime performance, conference wowfactor) and cost (build times, docs complication, dev expertise needed)
- currently no direct replacement available
- indexing commands

Darren's command spreadsheet: https://docs.google.com/spreadsheets/d/1w8FWy7FXhkiIQ7EJN0wF8SdDWImGwVwxywEg85LJtpU/edit#gid=0

Please put in your hottakes: commands that can be useably replaced, renaming to consistency (also inspect flags)

**Dan:** we can still make breaking changes and can really reflect on the state of the standard library
**Stefan:** should strike the right balance between combining a million tiny commands and using 1 big bloated command with 100 flags
**JT:** latent fear that we loose momentum but we should feel comfortable to make the breaking changes! we are in a reflective period to inspect naming/organization. No we have momentum for the editorial process and reviewing -> let's do it
Help get users down the happy paths -> help with common patterns
**Darren:** removing commands is hard when we lose functionality, tough decisions to make. is it OK to lose functionality like shells?

**JT:**
`from ...` :`parse` `read` `convert` `load(s)`
`to ...` :`serialize` `write` `format` `dump(s)`

`read`/`write` file vs pipe
**Dan:**
`to` and `into` are confusing
The pair for file format parsing is core to nushell functionality + casting/interconversion

**JT:**
Steady stream of breaking changes vs. ripping the bandaid in a batch
*Consensus:* At least make the topical changes together

Maybe rename the `from`/`to` pair to something and make `to` the casting command (most languages seem to have `To...` methods/functions)

## Plugin update

**JT:** been thinking about options for plugins: new plugin system (WASM? DLLs?) vs augmenting the existing plugin system incrementally. Leaning toward incremental
**Jakub:** agrees w/ incremental
**Stefan:** agrees w/incremental from risk and community perspectives
**Darren:** Gonna need a big jump in functionality to do things like dataframes in plugins
**Michael:** Need to get Fernando's input and consult him on the future of dataframes
**JT:** we should remove complicated parts that don't have enough maintenance energy behind them. maybe we put out a survey asking how many people are actually using dataframes

**-> Action Item:** community survey about dataframes (Michael, JT)

## Input output type signatures

[Jakub's exploration](https://hackmd.io/@nucore/Skt0PGjVj)

def foo(in: int) -> string { 
   ls | get $in | get name 
}

def ls(path?: Path) -> Stream<{ name: Path, size: Filesize, .. }>
   ls 
}

JT's notes:

def ls(path?: Path) -> Stream<{ name: Path, size: Filesize, .. }> {
   ls 
}

ls | get Foo

ls | get n<tab> (gives you `name`)

# Stream => Stream<Any>
def ls(path?: Path) -> Stream {
    # ...
}

type LSEntry = { name: Path, size: Filesize, .. }

# Stream => Stream<Any>
def ls(path?: Path) -> Stream<LSEntry> {
    # ...
}

# Scripts to Programs

gradual progression in the type stringency (we currently have opaque types, moving to wards structural types, + named parts in the structure. + type aliases/ "structs")

(one-liner to) Scripts to programs

**Q: WindSoilder:** How stringent should we be for the input type.

Can we infer based on the implementation or do we assume `Any` if not given.

`$in` is kind of overused as a name
(additional reserved name `input` as a potential solution to clarify)

# PR's

## Open for discussion
Triage if necessary, fill in small details if you are sheparding the PR
- [6983](https://github.com/nushell/nushell/pull/6983) Grouped config commands better (closes #6911) (@webbedspace)
- breaking-change (darren - i have reservations. it's probably not a breaking change because old configs still work, but the new "structured" config format works too. which means the config rust code is implemented twice in order to support old and new formats. the implication is we'll eventually remove the non-structured format which then will be a massive breaking change since it changes the format of the config.nu file.)
- Will cause future user churn/ apparently still supports old configs
- [6994](https://github.com/nushell/nushell/pull/6994) Allow iteration blocks to have an optional extra index parameter (alternative to `-n` flags) (@webbedspace)
- we touched on this one in the last meeting but didn't get to a decision due to merge conflict
- pre-meeting vote - land (darren)
- [7028](https://github.com/nushell/nushell/pull/7028) Move the config of complete rules to other place (@Decodetalkers) - breaking-change
- Currently marked as draft
- [7066](https://github.com/nushell/nushell/pull/7066) Support interpolation in backtick strings (@webbedspace)
- pre-meeting vote - land (darren) i don't really see any harm in having it
- [7079](https://github.com/nushell/nushell/pull/7079) add signature information when get help on one command (@WindSoilder)
- pre-meeting vote - land (darren, stefan) - we can continue to iterate if we want
- [7082](https://github.com/nushell/nushell/pull/7082) feat: Use Raw text to save if pipeline data is ExternalStream or String (@Decodetalkers)
- breaking-change
- pre-meeting vote - land (darren)
- [7087](https://github.com/nushell/nushell/pull/7087) Check for WRAP_PROMPT_XXX in environment (@PerBothner)
- I'm on it (Jakub)
- [7107](https://github.com/nushell/nushell/pull/7107) Use try_exist to handle the permission in cd (@Decodetalkers)
- try_exist is a +nightly feature so they wrote their own nu_try_exist until it is stable
- [7119](https://github.com/nushell/nushell/pull/7119) Fix for escaping backslashes in interpolated strings (fixes #6737) (@gavinfoley)
- We had to revert the predecessor, they figured out a fix, just wondering if we have enough tests.
- pre meeting Stefan & Darren: sentiment landing
- [7122](https://github.com/nushell/nushell/pull/7122) Restore original do -i behavior and add flags to break down shell vs program errors (@SUPERCILEX)
- [7124](https://github.com/nushell/nushell/pull/7124) new command `url parse` (#6854) and `url` subcommands tests (@raccmonteiro)
- review: Dan
- pre-meeting vote: land (darren)
- [7136](https://github.com/nushell/nushell/pull/7136) Make `first` and `last` always return 1 element, let `take` return from end of list (@rgwood)
- breaking-change
- Currently marked as draft
- pre-meeting vote: land (darren)

## Looking for reviewers
- [6806](https://github.com/nushell/nushell/pull/6806) Protocol: debug_assert!() Span to reflect a valid slice (@dbuch)
- CI green
- Several areas are affected: we should make sure the fixes are not hiding larger problems
- [7132](https://github.com/nushell/nushell/pull/7132) chore: chrono_update (@Decodetalkers)
- Currently marked as draft
- Would like to get this one in before the next release (darren)
## In process, no need for the full team
Pull out of here if you have concerns
- [6898](https://github.com/nushell/nushell/pull/6898) prevent error when `lines` reads bad bytes (@merelymyself)
- We discussed that this approach was slightly going in the wrong direction, merelymyself recently had less time to work on nushell.
- [6910](https://github.com/nushell/nushell/pull/6910) Make `seq` return a `ListStream` where possible. (@merelymyself)
- pretty close, (Stefan: could pick it up if necessary)
- [6984](https://github.com/nushell/nushell/pull/6984) [MVP][WIP] `less` like pager (@zhiburt)
- cool, most bikeshedding seems to have succeeded, polishing, (crossterm 0.25)
- [6990](https://github.com/nushell/nushell/pull/6990) make ls work like dir on windows (@richardmarklund)
- breaking-change
- CI currently not green
- discuss if we don't want to go that route
- [7010](https://github.com/nushell/nushell/pull/7010) Make json require string and pass around metadata (@Kixunil)
- breaking-change
- Good fix of weird coercions to strings that give wrong parses after `from ...` commands
- They expanded to more than just `from json`
- Currently merge conflict that needs resolution (+code review)
- [7056](https://github.com/nushell/nushell/pull/7056) Fix needs_quotes() in `to nuon` (closes #6989) (@webbedspace)
- Generally approved, Blocked by Stefan's review comments (Will try to get around to addressing them)
## Cold:
Only discuss if closing vs waiting is contentious
- [6798](https://github.com/nushell/nushell/pull/6798) Skip parsing named flags when not specified (@Yesterday17)
- Currently marked as draft
- [7002](https://github.com/nushell/nushell/pull/7002) return Error if get meet nothing and without "i" (@Decodetalkers) - breaking-change
- unclear status and PR description
- [7051](https://github.com/nushell/nushell/pull/7051) allow tables in ++ operator (@dmatos2012)
- Currently marked as draft
- Actual behavior dependent on the `append` implementation. weird conversion when mixing tables with other lists