---
title: Nix sub-flakes design
date: 2022-05-12
tags: design-notes, flakes
breaks: false
---

# Design notes

## Linked issues

(Mostly the result of searching for `subflake` in the github issues)

- [flakes: allow composing from multiple files](https://github.com/NixOS/nix/issues/4218)
- [Allow folder ignored by git to be used as a subflake](https://github.com/NixOS/nix/issues/5305)
- [Development only flake inputs or different category of inputs](https://github.com/NixOS/nix/issues/6124)
- [More intelligent subflake locking](https://github.com/NixOS/nix/issues/6352)
- [Allow getFlake of in-store paths in pure mode](https://github.com/NixOS/nix/issues/6353)
- [Allow flakes to refer to other flakes by relative path](https://github.com/NixOS/nix/issues/3978)

## Use-cases

### Monorepo

(bazel-style)

#### Requirements

- One global lockfile (probably)

- Several components

- Components should be lightweight to declare (or discovered)

#### How do ppl do that?

- Some tools that allow/force something like that (bazel, cabal/go
workspaces) have two different kind of files (`WORKSPACE.bazel` vs
`BUILD.bazel`).

- Others (cargo/npm) have only one, with a `workspaces` field to
declare sub-elements.

One thing with Nix that makes this notion of "workspace" more flexible
than most other tools is that there's anyways only one global store. So
workspaces don't impact that (while they generally do for all the other
solutions).

#### Are subflakes useful for that?

We could have a scheme where there's only one flake at the root, and
everything below is just normal Nix flakes.That way there's no need for
subflakes.

What flakes bring is

- The common schema (with everything it entails, like
`nix flake metadata` and friends)

- (maybe, eventually) a fine-grained eval cache

### Nixpkgs

Similar in size to the bazel-style use-case, but different because
(nearly) no code is checked-in, everything is fetched from external
sources.

- If this uses flake inputs (could be cool for managing the updates),
it probably means that a single lockfile is probably not realistic
(or is it? It's hard to know what size it would have. Maybe also we
could have a sqlite lockfile, except for the Git conflicts).

- There shouldn't be a need for independently locking every subflake −
meaning that if I change `pkgs/blah/bli/flake.nix`, I can run
`nix build .#blah.bli` and get the up-to-date version without having
to run something like `nix flake lock --update-input blah/bli`

- Subflakes should be able to depend of each other one way or another

Not sure how to compose them: Should this be implicit? Just be handled
by a clever `flake-utils` function?Eventually Nix modules would probably
help a lot with that, but we're not there yet.

### Private subflakes

[Allow folder ignored by git to be used as a
subflake](https://github.com/NixOS/nix/issues/5305)

### Ad-hoc composition

- Just bring several packages at the same place. But might not have to
share the same lockfile nor anything (maybe?)

- Each package needs to be usable and extractable independently
(modulo the dependency to other packages ofc)

## Current state of things

- We can have subflakes right now by having an input pointing to a
relative path

    {
      inputs.sub.url = "path:./subdir";
    }

- These will have their own lockfile (if they declare any input).
Having them reuse the inputs from the toplevel one requires an
explicit `.follows`. Conversely, the toplevel flake can reuse input
from its subflakes by `.follows`-ing them

- It isn't really possible to refer to the toplevel flake from a
subflake

- That can (maybe) somewhat be done by having a dummy `parent`
input in the subflake, and
`inputs.sub.inputs.parent.follows = "/"`

# Discussion
Workspaces - used to share artifacts in workspace

Why not use Nixpkgs standard "proto-derivation" format? - want the input handling

[john]: Interacts with flake inputs and composition method for package sets?

Similar issues in bazel, but far more external inputs. This is a "Nix specialty". As inputs managing them is better, but can be too many. How to trim this down?

ad-hoc composition - want to re-use lockfiles for coherency, can do override/follows, but is easy to mess up and run into problems.

"callPackage design pattern"

[john]: needs are legitimate, details are tricky, nixpkgs interaction is not well defined. What is the migration story? What would good be? Do we want a version of subflakes that won't work with Nixpkgs at the moment, or use this as the mechanism for Nixpkgs change?
- proposal: nixpkgs and subflakes are either separate, or need to go into full details
- [theophane]: nixpkgs is an example, but perhaps not the distinct goal
- at small scale composition:
- side-step problem with overlays, foot-gun

- [tom] Desired things:
- pass down
- eg: "inputs.thing.register = true;"
- pass up
- eg: "src = <github:org/repo>;"
- split
- non-locking path:./sub-dir (can have own lock)
- eg: "inputs.thing.unlocked = true;" ?
- nixConfig.unlocked = true;
- siblings? (or just define the tree)
- eg: "path:../sister-folder"

"Too many lockfiles?" - who/where is it decided which behavior is used?

...

- [eelco]: Why would we want to have an army of `flake.nix` in nixpkgs rather than just one?
- @garbas mostly for the locality of the inputs definition (define it near to where it’s used)
- [eelco]: Nothing requires creating a lockfile. But flakes aren’t really designed for this kind of use-case
- [tomberek]: If the child doesn’t have a lockfile, the parent will lock it, which is what we want
- @garbas: Could we enforce that a subflake won’t have its own lockfile? Because that’s a simple footgun
- @garbas: Maybe that shouldn’t even be called a `flake.nix`
- snowflake.nix, flake-unlocked.nix, subflake.nix, blarg.nix
- having a different thing might also solve the `--update-input` problem
- [tomberek]: That solves the “passing up” issue, but not the “passing down” one
- [eelco]: That’s not specific to subflakes, so we could have a more general solution
- [tomberek]: We could register an “in-memory” registry on-the-fly for inputs that we want to transitively `.follow`
- see overrideRegistry in the '--inputs-from' implementation
- @garbas: We’re running out of time. Would be great to write a tutorial on that to get a comprehensive overview
- [tomberek]: I can do all the manual stuff myself, but I couldn’t really teach it. So yes, good test
- [john]: Really like the sound of that. Links to the public vs private deps proposed e.g. for cargo
- cargo, public vs private deps... [link](https://rust-lang.github.io/rfcs/1977-public-private-dependencies.html)
- keep an eye on the possible complexity

[yannik]: feature exploration [link](https://github.com/ysndr/flake-inherit-input)

## TODO
- [tom]: create tutorial or simple example, describe the problem