fundamental problem which we are trying to solve: generate code, finish execution, then evaluate newly-generated code
less powerful than true eval, but more powerful than const-eval
problem dubbed "metahacks"

bootstrap problem: how many files have to be created manually or generated? using zoxide as an example:

• currently, the user adds a line in env.nu to generate a file with zoxide init nushell, and then adds a line to config.nu to source that file
• if we want this to be set up completely automatically, for something like the 2-file autoload construction there would need to be a command, say, zoxide init nushell-gen, that users could run to create 00-generate-zoxide.nu, which itself would contain some invocation of the original zoxide init nushell and saves that to 01-run-zoxide.nu.

2-file autoload generation
uses two files in autoloads dir: generate and run

• pros
  • generates inspectable artifact
  • if a single restart is acceptable to finish bootstraping, then already possible
• cons
  • 2 files have to be bootstrapped (generate and run)
  • if run file doesn't exist already, does not work until shell restart
    • can be sidestepped by creating an empty run file
    • can be sidestepped by checking for new autoloads after running each autoload
      • is autoloads-generating-autoloads a good idea?
  • vendor autoload dirs might be immutable
    • can be sidestepped by adding user autoload dirs
  • order between different autoload directories is unclear
    • can be sidestepped by pooling together all files in all vendor load directories and then executing them in order

3-file autoload generation
uses three files: generate + source in autoloads dir, and run in cache (or temp?) directory

• pros
  • avoids autoloads-generating-autoloads
  • does not suffer from first generation problem
  • unaffected by vendor autoloads being immutable
  • generates inspectable artifact
  • already possible
• cons
  • 3 files have to be bootstrapped (generate, source, and run)
  • the number of files makes it rather unwieldy
  • order between different autoload directories is unclear
    • can be sidestepped by pooling together all files in all vendor load directories and then executing them in order

self updating autoload
autoload can overwrite itself as part of an update process

• pros
  • only requires bootstrapping one file
• cons
  • less powerful than a true metahack, since it requires a shell restart

generators folder
introduce a new folder, called "generators" folder. each file in the generators folder is run, and its output is evaluated

• pros
  • only requires bootstrapping one file (..technically two if you count the output of the generator?)
  • no file i/o required
• cons
  • requires entirely new feature
  • no inspectable artifact

multiple arguments to source
each file is parsed then evaluated one at a time. for example, source generate-foo.nu run-foo.nu doesn't parse and evaluate run-foo.nu until after generate-foo.nu

• pros
  • doesn't rely on autoloads at all
  • generates inspectable artifact
• cons
  • requires new feature

env.nu / preconfig.nu

• pros
  • literally already implemented
  • easy to explain in documentation
  • generates inspectable artifact
• cons
  • runs before config is set up (you can't make use of ENV_CONVERSIONS for example)
  • all code generation has to happen in env.nu / preconfig.nu, no separation between metahacks
  • need to manually modify env.nu / preconfig.nu and config.nu to bootstrap (can't be completely automatic)

just don't

• pros
  • assumes every code generation use-case is already solvable via without metahacks
  • we don't have to do anything (other than remove env.nu / preconfig.nu)
  • for edge cases, some metahack constructions are already possible, just might be somewhat clunky
    • this might be acceptable if we can't identify any actual use cases we want to support
• cons
  • if there's a use-case we want to specifically support where metahacks are required, then this is a non-starter

existing integrations

Send PRs updating install instructions to use autoload-dirs

• atuin: 🐢 Magical shell history.
• broot: A new way to see and navigate directory trees.
• Direnv: unclutter your .profile.
• Dorothy: 🧙‍♀️ Bring your dotfile commands and configuration to any shell, including Nu, Fish, Zsh, Bash.
• mise: A development environment setup tool (dev tools, env vars, task runner) that integrates with Nushell.
• oh-my-posh: A prompt theme engine for any shell.
• starship: The minimal, blazing-fast, and infinitely customizable prompt for any shell.
• zoxide: A smarter cd command, inspired by z and autojump.

status update for 2024-12-11 meeting

• Context
  • Since const NU_LIB_DIRS and const NU_PLUGIN_DIRS were introduced, env.nu is no longer needed for its original purpose of setting up an environment to populate $env.config
    • exception: env.nu is still required to use ENV_CONVERSIONS in config.nu, but this should not be the case for much longer
  • However, some use cases require some code to be generated in env.nu which is then sourced in config.nu
    • These have been dubbed "metahacks"
  • Last meeting, general conclusion was to rename env.nu to preconfig.nu to continue supporting metahacks, but de-emphasize its usage in favor of vendor autoloads
• After last meeting, Douglas put together a draft PR to rename env.nu to preconfig.nu, but there were two major concerns raised:
  1. Third party integrators still rely on env.nu, and we don't want to break any integrations with the rename.
  2. Before going forward with the rename from env.nu to preconfig.nu, we should make sure there isn't a better and cleaner solution which provides equivalent or better functionality compared to env.nu.
• Various ideas for how metahacks might work without env.nu were discussed in the Discord thread (including Douglas, Darren, Bahex, and Rose). A few takeaways:
  • It would be preferable for metahacks to run after config.nu rather than before, since then you also have access to NU_LIB_DIRS, ENV_CONVERSIONS, etc.
  • There isn't any advantage to running code before config.nu. All configuration values, including consts, can be set after config.nu is loaded.
  • There are already some working patterns for metahacks using autoloads, but none are seamless. Requires more investigation.
  • Our general recommendation is that integrations should instruct the user to generate and save their configuration once, rather than using a metahack to regenerate every time (better for startup time)
• No conclusions yet on what the path forward should be (ie., preconfig.nu or some other solution), but here are the next steps before env.nu is renamed or removed:
  • Move ENV_CONVERSIONS out of env.nu (Douglas and Bahex are working on this)
  • Make PRs to third-party integrators switching from env.nu to vendor autoloads (Bahex created a list to go through)
  • There are no plans to remove or stop generating env.nu in 0.101. Currently (as of 0.100), both config.nu and env.nu only contain comments.
    • Darren noted he doesn't want env.nu removed/renamed until we are certain we've determined the final solution for metahacks