Every Nix user has Nix installed, either through using NixOS or by having installed it on their preferred OS. ((Cole: preceding sentence seems weird IMO; doesn't really add anything -- "of course nix users have nix installed, otherwise they wouldn't be a nix user")) Today, users install Nix through a set of shell scripts which work across a variety of MacOS versions and Linux distributions. ((Cole: it sounds to me like the above paragraph should be something like "If you are interested in using Nix, but don't want to make the jump to a full-blown Nix-managed operating system like NixOS, you will have to install it somehow. The most common way of doing that today is through a set of shell scripts................" or something)) The maintenance of these scripts can be frustrating and error-prone. Differences in support and behavior between these disparate systems can mean even small changes can [require considerable effort](https://github.com/NixOS/nix/pull/6916#discussion_r948269863), ((Cole: preceding `,` -> `;`)) worse, it can mean some features simply never happen. A few months ago, several members of the community began discussing if it would be possible to improve both developer and user experience by replacing the current *Polymorphic Bash* scripts with something new. [Graham](https://github.com/grahamc/), a Determinate Systems founder ((Cole: I can't really explain how or how to improve it, but the "a founder" just feels weird to me)) who played a big part in the creation of the current scripts, suggested we experiment with writing an installer in Rust. We're happy to say this experiment has been quite successful and we're pleased with the results so far. At [Determinate Systems](https://determinate.systems/) ((Cole: `,` after the link)) we're already using it in our internal infrastructure. We already ((Cole: already said "already" in the previous sentence, maybe just `s/already//`)) teased it in [our Nix on the Steam Deck](/posts/nix-on-the-steam-deck) ((Cole: nit, move "our" outside of the link)) post, but today we're going to talk about it in depth. # Stability notes ((Cole: maybe a cute little "before we get started, let's talk about stability" wouldn't go amiss here? but this is a very weak suggestion)) `nix-installer` is a unstable ((Cole: "a unstable" -> "unstable"? or maybe "an unstable tool"?)) at version 0.1.0 and you can expect it to feel like it. If you encounter issues, try running the uninstall ((Cole: "try using the novel/new/brand-new/brand-spanking-new uninstall functionality"; "try running the uninstall" sounds awkward to me)), and please [send us an issue](https://github.com/DeterminateSystems/nix-installer/issues/new) ((Cole: nit, maybe "send us an issue" -> "create an issue on our repo"?)). **On a Mac?** Be aware the uninstallation process currently does [not delete users](https://github.com/DeterminateSystems/nix-installer/issues/33). You'll need to do this yourself, see the compatability note [here](https://github.com/DeterminateSystems/nix-installer#status). We expect a future version with [`auto-uid-allocation`](https://github.com/NixOS/nix/pull/3600) to remove the need for user creation entirely. ((Cole: we might want to run this by Eelco to make sure that aligns with what he's seen while testing)) # Getting Started If you're feeling curious and want to try out [`nix-installer`](https://github.com/DeterminateSystems/nix-installer), you can run this ((Cole: "this" -> "the following"?)) on a Mac or Linux (with Systemd) machine which ((Cole: nit, "which" -> "that"?)) doesn't currently have Nix installed: ```bash curl -sL https://install.determinate.systems/nix | sh -s -- install ``` This will fetch a small bootstrapping script (based off [`rustup-init.sh`](https://github.com/rust-lang/rustup/blob/master/rustup-init.sh)) ((Cole: I'd love to see a hardcoded revision here, in case they ever (re)move the script or something to otherwise break this link)) which will then invoke the correct `nix-installer` binary for your platform. This binary will then ((Cole: recommend `s/then//`)) create a plan of how to install Nix on your system. Once a plan is created, it will ask you if you want to execute this plan: ```bash ana@ubuntu:~/Downloads$ curl -sL https://install.determinate.systems/nix | sh -s -- install info: downloading installer https://install.determinate.systems/nix/nix-installer-x86_64-linux `nix-installer` needs to run as `root`, attempting to escalate now via `sudo`... Nix install plan (v0.1.0) Planner: linux-multi Planner settings: * nix_build_group_id: 3000 * nix_package_url: "https://releases.nixos.org/nix/nix-2.12.0/nix-2.12.0-x86_64-linux.tar.xz" * force: false * modify_profile: true * extra_conf: [] * nix_build_user_prefix: "nixbld" * daemon_user_count: 32 * channels: ["nixpkgs=https://nixos.org/channels/nixpkgs-unstable"] * nix_build_user_id_base: 3000 * nix_build_group_name: "nixbld" The following actions will be taken (`--explain` for more context): * Create directory `/nix` * Fetch `https://releases.nixos.org/nix/nix-2.12.0/nix-2.12.0-x86_64-linux.tar.xz` to `/nix/temp-install-dir` * Create build users (UID 3000-3032) and group (GID 3000) * Create a directory tree in `/nix` * Move the downloaded Nix into `/nix` * Setup the default Nix profile * Configure Nix daemon related settings with systemd * Place the Nix configuration in `/etc/nix/nix.conf` * Place channel configuration at `/root/.nix-channels` * Configure the shell profiles Proceed? (y/N): ``` `nix-installer` has several builtin planners (`linux-multi`, `darwin-multi`, and `steam-deck`) and we plan to create more (such as `linux-single`) in the future. You can pass `--help` anywhere to explore the options and planners more. Generally `nix-installer` can choose the planner with heuristics, but you're free to override it. Once you'd ((Cole: "you'd" -> "you've")) approved, the installer proceeds: ```bash Proceed? (y/N): y INFO Step: Create directory `/nix` INFO Step: Provision Nix INFO Step: Configure Nix ana@ubuntu:~/Downloads$ ``` If the installer fails for any reason, including you CTRL+C'ing it, it will prompt you about attempting to safely uninstall. Later, if you decide to reinstall Nix, use a different planner, or decide that you'd like Nix gone for some other reason, you can call `/nix/nix-installer uninstall`, ((Cole: `,` -> `--` (em dash) or even end the sentence with a `.`)) this should revert the install by reading from `/nix/receipt.json` which contains ((Cole: add "information about")) all the actions taken during install. Uninstalling looks like this: ```bash ana@ubuntu:~/Downloads$ /nix/nix-installer uninstall `nix-installer` needs to run as `root`, attempting to escalate now via `sudo`... Nix uninstall plan Planner: linux-multi Planner settings: * modify_profile: true * nix_build_group_id: 3000 * daemon_user_count: 32 * nix_package_url: "https://releases.nixos.org/nix/nix-2.12.0/nix-2.12.0-x86_64-linux.tar.xz" * nix_build_group_name: "nixbld" * extra_conf: [] * nix_build_user_id_base: 3000 * force: false * channels: ["nixpkgs=https://nixos.org/channels/nixpkgs-unstable"] * nix_build_user_prefix: "nixbld" The following actions will be taken (`--explain` for more context): * Unconfigure the shell profiles * Remove channel configuration at `/root/.nix-channels` * Remove the Nix configuration in `/etc/nix/nix.conf` * Unconfigure Nix daemon related settings with systemd * Unset the default Nix profile * Remove the directory tree in `/nix` * Remove Nix users and group * Remove the directory `/nix` Proceed? (y/N): y INFO Revert: Configure Nix INFO Revert: Provision Nix INFO Revert: Create directory `/nix` Nix was uninstalled successfully! ana@ubuntu:~/Downloads$ ``` > We've been working with other installer working group contributors like (alphabetical) [Cole](https://github.com/colemickens), [Michael](https://github.com/mkenigs), [Solène](https://dataswamp.org/~solene/index.html), [Théophane](https://github.com/thufschmitt), [Travis](https://t-ravis.com/), and others to build `nix-installer` and better understand what a next-generation Nix installer would look like, ((Cole: `s/, thank/. Thank`)) thank you so much for all your help, hard work, and advice. ((Cole: nit, reads easier if it goes "hard work, help, and advice" IMHO)) # Installation Differences While the Nix installed by [`nix-installer`](https://docs.rs/nix-installer/latest/nix_installer/) behaves the same as one installed by the current install script then configured with some custom settings ((Cole: "then configured..."??? that hurts my brain, recommend removing that phrase)), there are some subtle differences in the default settings. * We set the following in the `nix.conf`: ```ini experimental-features = nix-command flakes auto-optimise-store = true ``` * We slightly more aggressively set the ZSH shell profile ((Cole: recommend expanding on this and drop the "slightly" qualifier -- specifically, call out that we write the environment setup to the beginning of the file, as opposed to the official installer writing to the end)) In the future, we plan to adopt options like [`auto-uid-allocation`](https://github.com/NixOS/nix/pull/3600) as they become available. # Advantages & Disadvantages `nix-installer` is a Rust crate instead of a bash script. This brings a number of advantages: * [`tokio`](https://docs.rs/tokio/latest/tokio/index.html), a multi-threaded async executor, allows us to parallelize tasks wherever possible * [`rustls`](https://docs.rs/rustls/latest/rustls/) means we don't depend on [`openssl`](https://www.openssl.org/) * We can create process groups and handle signals in a more structured way * The [`reqwest`](https://docs.rs/reqwest/latest/reqwest/), [`tar` (the crate)](https://docs.rs/tar/latest/tar/) ((Cole: recommend removing "the crate", since you say "crates" after listing them all. you could disambiguate earlier by saying "Crates like ... allow us to...")), and [`xz2`](https://docs.rs/xz2/latest/xz2/) crates allow us to avoid external dependencies on things like `tar` (the program) or `curl` * [`tracing`](https://docs.rs/tracing/latest/tracing/index.html) offers us fine grained, tunable structured logging * [`anyhow`](https://docs.rs/anyhow/latest/anyhow/) and [`eyre`](https://docs.rs/eyre/latest/eyre/) offers us robust ((Cole: and visually satisfying tbh, but not necessary to mention if you don't want to)) error handling * Rust code can call or extend [`nix-installer`](https://docs.rs/nix-installer/latest/nix_installer/) for other applications * Rust's built in testing allows us to have more fine grained tests which are easier for contributors to use and write Unfortunately, it also creates some disadvantages: * A build toolchain is required to create the `nix-installer` binaries * We need different binaries for each platform * Inspecting the installer without the source can be a challenge Thankfully, Nix helps us overcome these disadvantages! It lets us define a reproducible build environment, produce binaries that are provably derived from our (open) source tree, and do all of this across all platforms we support. # Architecture ((Cole: I don't remember if this exists yet, but maybe this could be the basis / inspiration for an actual ARCHITECTURE.md doc down the road)) `nix-installer` consists of a Rust library and a Rust binary that uses it. The Rust library contains three major components: * [`Action`](https://docs.rs/nix-installer/latest/nix_installer/action/index.html)s which are either an executable unit (such as 'make a directory') or a group of sub-`Action`s which the parent orcestrates (possibly in parallel). Actions have `plan()`, `execute()` and `revert()` functions. * [`InstallPlan`](https://docs.rs/nix-installer/latest/nix_installer/struct.InstallPlan.html)s which define a series of `Action`s which are run sequentially. This plan is stored on `/nix/receipt.json` after install and is used during uninstall. * [`Planner`](https://docs.rs/nix-installer/latest/nix_installer/planner/index.html)s which take some settings to create a set of `Action`s then executes the `plan()` function of each. This produces an `InstallPlan` which can then be run. Programmatically, it can look like this: ```rust use std::error::Error; use nix_installer::{ InstallPlan, planner::{Planner, linux::SteamDeck} }; let planner = SteamDeck::default().await?; let mut plan = InstallPlan::plan(planner).await?; match plan.install(None).await { Ok(()) => tracing::info!("Done"), Err(e) => { match e.source() { Some(source) => tracing::error!("{e}: {}", source), None => tracing::error!("{e}"), }; plan.uninstall(None).await?; }, }; ``` The library also has an optional `cli` feature flag to expose CLI related features for [`clap`](https://docs.rs/clap/latest/clap/index.html). The Rust binary uses that feature and sets up the things like tracing and error handling a command line program is expected to have ((Cole: swap "like tracing and error handling" and "a command line program is expected to have")). The binary also is ((Cole: `s/also is/is also/`)) responsible for making a copy of itself in ((Cole: `s/in/at/`)) `/nix/nix-installer` so users can uninstall later. ## Testing `nix-installer` has both Rust tests as well as VM tests using Nix. This is because we can't exactly test installing Nix on the user's local development machine, so we need to rely on some sort of container or virtual machine. The Rust tests include things like `Action` level tests answering questions like "Does the `CreateDirectory` action actually create a directory?" as well as more broad tests like "Does the `InstallPlan` still respect semantic versioning compatability?" There is still a large amount of things which we can and should have Rust tests for, and we plan on expanding the suite as `nix-installer` matures. Rust tests are run like so: ```bash $ cargo test Compiling nix-installer v0.0.0-unreleased (/home/ana/git/determinatesystems/nix-installer) Finished test [unoptimized + debuginfo] target(s) in 12.87s Running unittests src/lib.rs (target/debug/deps/nix_installer-fab52ed9c41e49a2) running 2 tests test plan::test::ensure_version_allows_compatible ... ok test plan::test::ensure_version_denies_incompatible ... ok test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ... ``` For the VM tests, [Eelco](https://github.com/edolstra), also a Determinate Systems founder, ((Cole: is it valuable to call him a detsys founder as opposed to the inventor of Nix...? idk, seems weird to me just like the earlier mention of Graham as a founder)) originally [wrote](https://github.com/NixOS/nix/pull/7043) ((Cole: suggest extending the link to be "originally wrote")) the VM tests for the current installer, and they were quite easy to port to `nix-installer`. Contributors can test the installer against a suite of supported Linux distributions with: ((Cole: maybe remove the "For the VM tests" start and go from there? otherwise, you're repeating "the VM tests" twice in the same sentence, which sounds off to me)) ```bash $ nix build github:determinatesystems/nix-installer#hydraJobs.vm-test.all.x86_64-linux.install-default -L nix-installer> unpacking sources nix-installer> unpacking source archive /nix/store/pa5fwi8m0jrq44m8hyqjv25anckw73zj-nix-installer-source ... ((Cole: recommend making it more obvious stuff is missing here -- even `[...snip...]` would be an improvement IMHO. maybe even the fun ascii scissors git uses)) installer-test-rhel-v9-install-default> Done! installer-test-rhel-v9-install-default> qemu-kvm: terminating on signal 15 from pid 1 () ``` # In Github Actions We include an `action.yml` in the repo meaning you can use it directly in Github Actions: ```yaml on: pull_request: push: branches: [main] jobs: lints: name: Build runs-on: ubuntu-22.04 steps: - uses: actions/checkout@v3 - name: Install Nix uses: DeterminateSystems/nix-installer@v0.1.0 with: # Allow the installed Nix to make authenticated Github requests. # If you skip this, you will likely get rate limited. github-token: ${{ secrets.GITHUB_TOKEN }} - name: Run `nix build` run: nix build . ``` # Future work There's still a bunch of fun work to be done on `nix-installer`! Our next priorities include: * Fix bugs that arise from users (hopefully including you!) * Container friendly planners like `linux-single` * Resolve existing known issues with Mac uninstallation * Adopt new features like [`auto-uid-allocation`](https://github.com/NixOS/nix/pull/3600) * Explore further improvements we could make to the install process * Add more tests We'd also like to explore transforming our `InstallPlan` to use a directed acyclic graph (DAG) of `Action`s and remove the idea of "parent `Action`s" which simply run other `Action`s. We believe a structure like this would allow more safety during the planning process as well as automatic parallelization during the execution process.