fundamental problem which we are trying to solve: generate code, finish execution, then evaluate newly-generated code
less powerful than true eval, but more powerful than const-eval
problem dubbed "metahacks"
bootstrap problem: how many files have to be created manually or generated? using zoxide as an example:
- currently, the user adds a line in `env.nu` to generate a file with `zoxide init nushell`, and then adds a line to `config.nu` to source that file
- if we want this to be set up completely automatically, for something like the 2-file autoload construction there would need to be a command, say, `zoxide init nushell-gen`, that users could run to create `00-generate-zoxide.nu`, which itself would contain some invocation of the original `zoxide init nushell` and saves that to `01-run-zoxide.nu`.
**2-file autoload generation**
uses two files in autoloads dir: generate and run
- pros
- generates inspectable artifact
- if a single restart is acceptable to finish bootstraping, then already possible
- cons
- 2 files have to be bootstrapped (generate and run)
- if run file doesn't exist already, does not work until shell restart
- can be sidestepped by creating an empty run file
- can be sidestepped by checking for new autoloads after running each autoload
- is autoloads-generating-autoloads a good idea?
- vendor autoload dirs might be immutable
- can be sidestepped by adding user autoload dirs
- order between different autoload directories is unclear
- can be sidestepped by pooling together all files in all vendor load directories and then executing them in order
**3-file autoload generation**
uses three files: generate + source in autoloads dir, and run in cache (or temp?) directory
- pros
- avoids autoloads-generating-autoloads
- does not suffer from first generation problem
- unaffected by vendor autoloads being immutable
- generates inspectable artifact
- already possible
- cons
- 3 files have to be bootstrapped (generate, source, and run)
- the number of files makes it rather unwieldy
- order between different autoload directories is unclear
- can be sidestepped by pooling together all files in all vendor load directories and then executing them in order
**self updating autoload**
autoload can overwrite itself as part of an update process
- pros
- only requires bootstrapping one file
- cons
- less powerful than a true metahack, since it requires a shell restart
**generators folder**
introduce a new folder, called "generators" folder. each file in the generators folder is run, and its output is evaluated
- pros
- only requires bootstrapping one file (..technically two if you count the output of the generator?)
- no file i/o required
- cons
- requires entirely new feature
- no inspectable artifact
**multiple arguments to `source`**
each file is parsed then evaluated one at a time. for example, `source generate-foo.nu run-foo.nu` doesn't parse and evaluate `run-foo.nu` until *after* `generate-foo.nu`
- pros
- doesn't rely on autoloads at all
- generates inspectable artifact
- cons
- requires new feature
**`env.nu` / `preconfig.nu`**
- pros
- literally already implemented
- easy to explain in documentation
- generates inspectable artifact
- cons
- runs before config is set up (you can't make use of `ENV_CONVERSIONS` for example)
- all code generation has to happen in `env.nu` / `preconfig.nu`, no separation between metahacks
- need to manually modify `env.nu` / `preconfig.nu` and `config.nu` to bootstrap (can't be completely automatic)
**just don't**
- pros
- assumes every code generation use-case is already solvable via without metahacks
- we don't have to do anything (other than remove `env.nu` / `preconfig.nu`)
- for edge cases, some metahack constructions are already possible, just might be somewhat clunky
- this might be acceptable if we can't identify any actual use cases we want to support
- cons
- if there's a use-case we want to specifically support where metahacks are required, then this is a non-starter
# existing integrations
Send PRs updating install instructions to use autoload-dirs
- [ ] [atuin](https://github.com/ellie/atuin): 🐢 Magical shell history.
- [ ] [broot](https://github.com/Canop/broot): A new way to see and navigate directory trees.
- [ ] [Direnv](https://github.com/direnv/direnv/blob/master/docs/hook.md#nushell): unclutter your .profile.
- [ ] [Dorothy](https://github.com/bevry/dorothy): 🧙♀️ Bring your dotfile commands and configuration to any shell, including Nu, Fish, Zsh, Bash.
- [ ] [mise](https://mise.jdx.dev/installing-mise.html#nushell): A development environment setup tool (dev tools, env vars, task runner) that integrates with Nushell.
- [ ] [oh-my-posh](https://ohmyposh.dev/docs/installation/prompt): A prompt theme engine for any shell.
- [ ] [starship](https://starship.rs/#nushell): The minimal, blazing-fast, and infinitely customizable prompt for any shell.
- [ ] [zoxide](https://github.com/ajeetdsouza/zoxide): A smarter cd command, inspired by z and autojump.
# status update for 2024-12-11 meeting
- Context
- Since `const NU_LIB_DIRS` and `const NU_PLUGIN_DIRS` were introduced, `env.nu` is no longer needed for its original purpose of setting up an environment to populate `$env.config`
- exception: `env.nu` is still required to use `ENV_CONVERSIONS` in `config.nu`, but this should not be the case for much longer
- However, some use cases require some code to be generated in `env.nu` which is then `source`d in `config.nu`
- These have been dubbed "metahacks"
- Last meeting, general conclusion was to rename `env.nu` to `preconfig.nu` to continue supporting metahacks, but de-emphasize its usage in favor of vendor autoloads
- After last meeting, Douglas put together a draft PR to rename `env.nu` to `preconfig.nu`, but there were two major concerns raised:
1. Third party integrators still rely on `env.nu`, and we don't want to break any integrations with the rename.
2. Before going forward with the rename from `env.nu` to `preconfig.nu`, we should make sure there isn't a better and cleaner solution which provides equivalent or better functionality compared to `env.nu`.
- Various ideas for how metahacks might work without `env.nu` were discussed in the [Discord thread](https://discord.com/channels/601130461678272522/1314725454850293780) (including Douglas, Darren, Bahex, and Rose). A few takeaways:
- It would be preferable for metahacks to run _after_ `config.nu` rather than before, since then you also have access to `NU_LIB_DIRS`, `ENV_CONVERSIONS`, etc.
- There isn't any advantage to running code _before_ `config.nu`. All configuration values, including `const`s, can be set after `config.nu` is loaded.
- There are already some working patterns for metahacks using autoloads, but none are seamless. Requires more investigation.
- Our general recommendation is that integrations should instruct the user to generate and save their configuration once, rather than using a metahack to regenerate every time (better for startup time)
- No conclusions yet on what the path forward should be (ie., `preconfig.nu` or some other solution), but here are the next steps before `env.nu` is renamed or removed:
- Move `ENV_CONVERSIONS` out of `env.nu` (Douglas and Bahex are working on this)
- Make PRs to third-party integrators switching from `env.nu` to vendor autoloads (Bahex created a list to go through)
- There are no plans to remove or stop generating `env.nu` in 0.101. Currently (as of 0.100), both `config.nu` and `env.nu` only contain comments.
- Darren noted he doesn't want `env.nu` removed/renamed until we are certain we've determined the final solution for metahacks