fundamental problem which we are trying to solve: generate code, finish execution, then evaluate newly-generated code less powerful than true eval, but more powerful than const-eval problem dubbed "metahacks" bootstrap problem: how many files have to be created manually or generated? using zoxide as an example: - currently, the user adds a line in `env.nu` to generate a file with `zoxide init nushell`, and then adds a line to `config.nu` to source that file - if we want this to be set up completely automatically, for something like the 2-file autoload construction there would need to be a command, say, `zoxide init nushell-gen`, that users could run to create `00-generate-zoxide.nu`, which itself would contain some invocation of the original `zoxide init nushell` and saves that to `01-run-zoxide.nu`. **2-file autoload generation** uses two files in autoloads dir: generate and run - pros - generates inspectable artifact - if a single restart is acceptable to finish bootstraping, then already possible - cons - 2 files have to be bootstrapped (generate and run) - if run file doesn't exist already, does not work until shell restart - can be sidestepped by creating an empty run file - can be sidestepped by checking for new autoloads after running each autoload - is autoloads-generating-autoloads a good idea? - vendor autoload dirs might be immutable - can be sidestepped by adding user autoload dirs - order between different autoload directories is unclear - can be sidestepped by pooling together all files in all vendor load directories and then executing them in order **3-file autoload generation** uses three files: generate + source in autoloads dir, and run in cache (or temp?) directory - pros - avoids autoloads-generating-autoloads - does not suffer from first generation problem - unaffected by vendor autoloads being immutable - generates inspectable artifact - already possible - cons - 3 files have to be bootstrapped (generate, source, and run) - the number of files makes it rather unwieldy - order between different autoload directories is unclear - can be sidestepped by pooling together all files in all vendor load directories and then executing them in order **self updating autoload** autoload can overwrite itself as part of an update process - pros - only requires bootstrapping one file - cons - less powerful than a true metahack, since it requires a shell restart **generators folder** introduce a new folder, called "generators" folder. each file in the generators folder is run, and its output is evaluated - pros - only requires bootstrapping one file (..technically two if you count the output of the generator?) - no file i/o required - cons - requires entirely new feature - no inspectable artifact **multiple arguments to `source`** each file is parsed then evaluated one at a time. for example, `source generate-foo.nu run-foo.nu` doesn't parse and evaluate `run-foo.nu` until *after* `generate-foo.nu` - pros - doesn't rely on autoloads at all - generates inspectable artifact - cons - requires new feature **`env.nu` / `preconfig.nu`** - pros - literally already implemented - easy to explain in documentation - generates inspectable artifact - cons - runs before config is set up (you can't make use of `ENV_CONVERSIONS` for example) - all code generation has to happen in `env.nu` / `preconfig.nu`, no separation between metahacks - need to manually modify `env.nu` / `preconfig.nu` and `config.nu` to bootstrap (can't be completely automatic) **just don't** - pros - assumes every code generation use-case is already solvable via without metahacks - we don't have to do anything (other than remove `env.nu` / `preconfig.nu`) - for edge cases, some metahack constructions are already possible, just might be somewhat clunky - this might be acceptable if we can't identify any actual use cases we want to support - cons - if there's a use-case we want to specifically support where metahacks are required, then this is a non-starter # existing integrations Send PRs updating install instructions to use autoload-dirs - [ ] [atuin](https://github.com/ellie/atuin): 🐢 Magical shell history. - [ ] [broot](https://github.com/Canop/broot): A new way to see and navigate directory trees. - [ ] [Direnv](https://github.com/direnv/direnv/blob/master/docs/hook.md#nushell): unclutter your .profile. - [ ] [Dorothy](https://github.com/bevry/dorothy): 🧙‍♀️ Bring your dotfile commands and configuration to any shell, including Nu, Fish, Zsh, Bash. - [ ] [mise](https://mise.jdx.dev/installing-mise.html#nushell): A development environment setup tool (dev tools, env vars, task runner) that integrates with Nushell. - [ ] [oh-my-posh](https://ohmyposh.dev/docs/installation/prompt): A prompt theme engine for any shell. - [ ] [starship](https://starship.rs/#nushell): The minimal, blazing-fast, and infinitely customizable prompt for any shell. - [ ] [zoxide](https://github.com/ajeetdsouza/zoxide): A smarter cd command, inspired by z and autojump. # status update for 2024-12-11 meeting - Context - Since `const NU_LIB_DIRS` and `const NU_PLUGIN_DIRS` were introduced, `env.nu` is no longer needed for its original purpose of setting up an environment to populate `$env.config` - exception: `env.nu` is still required to use `ENV_CONVERSIONS` in `config.nu`, but this should not be the case for much longer - However, some use cases require some code to be generated in `env.nu` which is then `source`d in `config.nu` - These have been dubbed "metahacks" - Last meeting, general conclusion was to rename `env.nu` to `preconfig.nu` to continue supporting metahacks, but de-emphasize its usage in favor of vendor autoloads - After last meeting, Douglas put together a draft PR to rename `env.nu` to `preconfig.nu`, but there were two major concerns raised: 1. Third party integrators still rely on `env.nu`, and we don't want to break any integrations with the rename. 2. Before going forward with the rename from `env.nu` to `preconfig.nu`, we should make sure there isn't a better and cleaner solution which provides equivalent or better functionality compared to `env.nu`. - Various ideas for how metahacks might work without `env.nu` were discussed in the [Discord thread](https://discord.com/channels/601130461678272522/1314725454850293780) (including Douglas, Darren, Bahex, and Rose). A few takeaways: - It would be preferable for metahacks to run _after_ `config.nu` rather than before, since then you also have access to `NU_LIB_DIRS`, `ENV_CONVERSIONS`, etc. - There isn't any advantage to running code _before_ `config.nu`. All configuration values, including `const`s, can be set after `config.nu` is loaded. - There are already some working patterns for metahacks using autoloads, but none are seamless. Requires more investigation. - Our general recommendation is that integrations should instruct the user to generate and save their configuration once, rather than using a metahack to regenerate every time (better for startup time) - No conclusions yet on what the path forward should be (ie., `preconfig.nu` or some other solution), but here are the next steps before `env.nu` is renamed or removed: - Move `ENV_CONVERSIONS` out of `env.nu` (Douglas and Bahex are working on this) - Make PRs to third-party integrators switching from `env.nu` to vendor autoloads (Bahex created a list to go through) - There are no plans to remove or stop generating `env.nu` in 0.101. Currently (as of 0.100), both `config.nu` and `env.nu` only contain comments. - Darren noted he doesn't want `env.nu` removed/renamed until we are certain we've determined the final solution for metahacks