# Nushell core team meeting 2022-08-03 ## Attendees - Darren - Reilly - JT - Michael - Stefan ## Agenda - Celebrations event - Module proposal - nupac development - overloading commands by input type # Discussed Topics ## Celebration stream Nushell blasted through the 20k stars!!! (past fish) Discord has a 25 people limit for video chat (not useful for a big celebration) (could be used for small office hours later on) JT has streamyard to link to youtube or twitch, take in comments, and small group in the stream **Friday August 5, 20:00 UTC** https://everytimezone.com/s/553cdb44 ## Pipeline refactor (jt) Goal avoid copyint `EngineState` `Command::run` now would take the `engine_state: &'a` and `PipelineData<'a>` constrained to the same lifetime External has to run in a separate thread (We can not share `'static` references to the engine state) Also `par-each` for rayon parallelism should still work. (Efforts by the community to get more stuff parallel in general) Possible workaround: scoped threads We don't want to `Arc<Mutex<EngineState>>` our way to a global interpreter lock (GIL) to have the same challenges as Python with multithreading/ This design challenge is probably a blocker for the current parallelization PR (https://github.com/nushell/nushell/pull/6213) ## nu core crate Trying this idea was hard due to it affecting all commands Can we separate out more stuff to do quicker testing ## nupac A package manager for nu written in nu https://github.com/skelly37/nupac skelly and friends are working on it Current gripe: package metadata as a comment in the scripts (PR on nu_scripts would introduce it to every script right now) Potential alternative: nuon file like the `package.json`/`Cargo.toml`s of the world Could also map to plugins written in different languages than nu We should make sure we get a solution that can be picked up by the community, we don't want to end up in the situation that we deem it unsuitable for the community after they did all the work. Package should be an opt-in thing and not necessary to use/get started with nu(shell) But official sources would be very helpful to serve the often mentioned needs (completions, prompt integrations) Backing data should probably be an easy to work with github repo Side note: probably among the biggest open source project in nu on github (good for regression tests and performance analysis, feedback about language constructs) ## Jakubs module proposal https://hackmd.io/@nucore/r1zilyw6q ### Problems to solve - How to import environment variables from parser definitions in modules in a clean way - Potential namespacing issues (resolve collisions in a predictable way) - Currently impossible: - `overlay add` in a module is forbidden (as it dynamically changes environment and definitions) ### Changes Separate environment from parser definitions (-> makes parsing more consistent?) module syntax would be a subset of a script (replaces two somewhat different parsing modes we currently need) `environment {}` block defines part to run when sourcing This can not interfere directly with the ## Field report: IOX (Michael) https://github.com/influxdata/influxdb_iox - `iox` currently not yet cratified - nushell as a querying/viewing tool Primarily a time series analytics/db environment Techstack: Arrow, parquet as persistance file format (allows chunking) Michael tries to combine core nushell commands with the iox commands to Binary nushell build embedded in own build of iox https://github.com/stormasm/influxdb_iox/tree/stormasm/nushell/iox_nu Would be another avenue to use nu as a domain specific language Q: instead of further cratification of our commands could we use targeted feature flags to create the core subset of `nu-command` ## Required cleanup: Command overloading based on pipeline output type Our documentation is now a superfund site... Help messages have to be polished. Website is borked Surface the types to the user in an understandable way The common behavior should be predictably same, and the options documented Seems to only work at parse time?