Replace eval cache on path flakes

Welcome. Sit down and take a moment to listen.

I think I identified something that will help make the flakes user experience much nicer. It takes a minute to setup the context.

Here are some usability issues that I see, and they are have as a root cause the eval cache:

1. Users often forget to git add a file and get confused when the flake cannot find the file. Even me, after years of usage, I still get tripped by this.
2. Evaluation error messages point to somewhere in the /nix/store instead of the current repo location.
3. Because the cache granularity is on the whole flake, I believe that the cache gets invalidated on each change in the repo? When actively developing, this makes the cache not very useful.
4. If the repo is a 2GB monorepo, each flake invocation will make a new copy into the /nix/store, creating a lot of duplicates.
5. When using sub-flakes, you have to actively bust the cache before the flake can use the new version of the sub-flake. That makes it not very practical to work on multiple parts of a large monorepo, with each their own sub-flakes.

Before proposing a replacement, I think it’s also worth reminding why such mechanism was introduced. On top of the obvious “make it faster”, there is also a goal of making flakes more reproducible. By forcing the code to be added to the /nix/store it also confers some benefits such as:

a. Relative paths such as toString ./. are going to be reproducible between machines as they will resolve to the same /nix/store location on both systems (instead of pointing to the path of the repo location).

b. It protects users from paths reaching outside of the flake root for similar reasons (we don’t want flakes to leak the user’s ~/.ssh/id_rsa ).

c. Because flake is using git archive under the hood, the store path will also be the same content between machines, avoiding issues where temporary or .gitignored files in the repo can be injected into the build.

So this is the big picture. Flake eval cache plays a dual role, but also has a lot of issues. I don’t see a really good path where all of those issues can be addressed without changing the fundamentals. Point (4) is being addressed with the lazy-tree branch that adds even more complexity to the language, and breaks backwards-compatibility. Instead of going down that rabbit hole I propose the following changes:

1. Extend the language to introduce an optional pathRoot argument to EvalState.
2. Extend the path resolution to also check that the resolved path is below the pathRoot if it’s set. This means that ~/.ssh/id_rsa for example would get expanded, and then immediately throw an exception.
3. Set the pathRoot to the flake root when using flakes.
4. Optionally this could be exposed as a nix config as well.
5. Extend the string context to also store paths that are outside of the /nix/store.
6. For example: builtins.getContext "${toString ./.}” would return { “/home/myuser/myproject” ={ path = true }} . Happy to discuss the exact schema shape here.
7. Change the derivation creation mechanism to hard fail on those non-store paths.
8. Make that a nix config option as well, so it can be turned on, and use it in flakes.
9. Remove the eval cache for path flakes.
10. This also means releasing a new revision of the flake lock format where no hash is being added to path flakes.
11. Bonus point: I really want a FLAKE_ROOT env var that points to the flake root, that is being set before invoking sub-commands like nix run. That would unlock all sorts of nice scenarios for working in work trees.

With all of these changes, I believe we fix all of the observed pain points while still being relatively well sandboxed, keeping good backward-compatibility, and making the nix language stronger instead of pushing more features into flakes. The only point that we conceded is point ©, but I think there are viable workarounds such as the nix-filter library that make this ok.

Let me know what you think!