---
feature: nix-language-version
start-date: 2022-12-12
author: @fricklerhandwerk
co-authors: @thufschmitt @Ericson2314 @yorickvp @infinisil
shepherd-team:
shepherd-leader:
related-issues: https://github.com/NixOS/nix/issues/7255
---

# RFC 137 – Nix language versioning

# Agenda

- Materials
- RFC 137 draft
- Yorick's RFC draft
- Comments on RFC 137
- Procedure
- [x] merge all contents by topic, mechanically
- [x] go into each section figuring out common patterns and key points
- [x] motivation
- [x] alternatives
- sort out and weigh all arguments
- [x] detailed design (will probably emerge as one of the alternatives)
- [ ] address all comments per section
- [ ] pick one preferred solution
- [ ] summary

2023-04-20:
- [x] focus on just the motivation section and maybe finish it

2023-04-28:
- [x] Pivot to version-in-filename

2023-05-11:
- [x] Determine how and when to do version bumps

Next:

- incorporate questions & answers into statements in the design
- [ ]
- write summary

# Summary
[summary]: #summary

Introduce a convention to determine which version of the Nix language grammar to use for parsing a Nix file.

Add parameters to the Nix language evaluator, controlling the behavior of deprecation warnings and errors.

# Motivation
[motivation]: #motivation

The stability of Nix language has been praised on multiple occasions, e.g. [Nix and legacy enterprise software development: an unlikely match made in heaven](https://talks.nixcon.org/nixcon-2022/talk/QQPBFW/).
Yet, as with any software system, in order to accommodate new insights, we want to allow the Nix language to evolve.
This sometimes involves backward-incompatible ("breaking") changes that currently cannot be made without significant downstream disruption.

Therefore we propose a mechanism to introduce changes to the Nix language in a controlled and deliberate manner.
It aims to avoid breaking existing setups, and to minimise maintenance burden for implementors and users.
The goal is for new versions of the Nix language evaluator to stay backward compatible with existing Nix expressions, but not necessarily for new Nix expressions to stay forward compatible with existing evaluators.

Regardless, changes to the language, especially breaking changes, should remain a rare exception.

## Motivating examples

Incompatible changes from the past:

- [A changelog of Nix (language) versions](https://code.tvl.fyi/about/tvix/docs/lang-version.md), as reflected in `builtins.langVersion`
- There have been other, sometimes breaking changes to the language that have not resulted in an increment of the language version (e.g. the recent `fetchGit` changes).
- The `builtins.toJSON 1.000001` output [changed in Nix 2.12](https://github.com/NixOS/nix/issues/8259).

Possbile future changes that are in discussion:

- Remove URL literals (currently implemented via experimental-features)
- Remove the old `let { body = ... }` syntax
- Disallow leading zeroes in integer literals (such as `umask = 0022`)
- Disallow `a.x or y` if `a` is not an attribute set
- Simplifying semantics of `builtins.toString` and string interpolation
- Remove the `let { body }` syntax
- `__functor` and `__toString`, probably
- Remove `__overrides`
- [Make `builtins` more consistent](https://github.com/NixOS/nix/issues/7290), e.g. not exposing `map`, `removeAttrs`, `fetchGit` and and others in the global scope
- Fix [imprecision in string representation of floating point numbers](https://github.com/NixOS/nix/pull/6238)
- Make the `@`-pattern consistent with destructuring
- A syntax to index into lists, e.g. `[ 1 2 3 ].0 == 1`
- Use `,` to delimit elements of a list expression, like `[ 1, 2, 3 ]`
- Do something about `?` meaning two different things depending on where it occurs (`{ x ? "" }:` vs `x ? ""`)
- Better support for static analysis
- [Syntax for hexadecimal numbers](https://github.com/NixOS/nix/pull/7695)

Other discussions around language changes:

- [Make path semantics less surprsing](https://github.com/NixOS/nix/issues/7338)
- [RFC 110](https://github.com/NixOS/rfcs/pull/110) proposing a alternatives to the `with` expression
- [Nix 2 – a hypothetical syntax](https://md.darmstadt.ccc.de/s/nix2)
- [Nix language changes](https://gist.github.com/edolstra/29ce9d8ea399b703a7023073b0dbc00d)

## Alternatives

- Keep the language as implemented by Nix compatible, but socially restrict the usage of undesirable features.
- (+) Roughly matches the current practice, no technical change needed.
- (+) Maintains usability of old Nixpkgs versions (up to availability of fixed-output artifacts)
- (+) Does not break third-party codebases before making a decision, keeping Nix a dependable upstream
- (-) This proposal does not allow for breakages unless there is some eventual phase-out of support
- (-) Strict enforcement requires extra tooling that this proposal would obviate
- (-) The implementation of the features that are no longer desirable still incur complexity and maintenance cost
- (-) It's still not really possible to make changes to the language

- Introduce changes to the language with language extensions or feature flags
- (-) Combinatorial explosion
- See [Haskell language extensions] for real-world experience.
- (-) Even more maintenance overhead
- (+) Allows gradual adoption of features
- (-) We already have experimental feature flags as an orthogonal mechanism, with the added benefit that they don't incur support costs and can be dropped without loss

- Never make breaking changes to the language
- (+) No additional maintenance effort required
- (-) Blocks improvements
- (-) Requires additions to be made very carefully
- (-) Even incremental changes are really expensive that way
- (-) Makes solving some well-known problems impossible

- Continue current practice
- (-) There is no process for breaking changes
- (-) Breaking changes are not always announced
- (-) There are no means of determining compatibility between expressions and evaluator versions

# Detailed Design

## Language versioning

1. The language version is a natural number.

<details><summary>Arguments</summary>

- (+) It formally decouples the Nix language version from the Nix version.
- (+) Nix language is supposed to change much less often than the rest of Nix.
- (-) There are two version numbers to keep track of.
- (-) It makes more evident that the Nix language is a distinct architectural component of the Nix ecosystem.
- (+) It's currently handled that way, no change needed apart from documentation
- See [`builtins.langVersion`] (currently undocumented)
- (+) Simple and unambiguous
- (+) Concise, even in the long term, since the language is supposed to change very rarely

[`builtins.langVersion`]: https://github.com/NixOS/nix/blob/26c7602c390f8c511f326785b570918b2f468892/src/libexpr/primops.cc#L3952-L3957

</details>

<details><summary>Alternatives</summary>

- [Calendar Versioning](https://calver.org/)
- (+) provides information on when changes happened
- (-) this is not needed because only compatibility information is needed
- (-) requires a minimum amount of characters
- this may be relevant depending on where it has to be encoded
- (+) restricting to only the year would force language changes to be rare
- (+) this would allow obvious synchronisation points with Nixpkgs releases
- (-) it may be too much policy encoded in a mechanism
- [Semantic Versioning](https://semver.org/)
- (+) can distinguish additions from other changes
- (-) this is not needed for our use case, since any addition to an expression will break for older evaluators even if the major version matches
- (-) requires more characters to account for the added expressiveness
- this may be relevant depending on where it has to be encoded
- Use version numbers of Nix stable releases for specifying the version of the Nix language
- (+) More obvious to see for users what the current Nix version is rather than `builtins.langVersion`.
- (-) Would tie alternative Nix language evaluators to the rest of Nix.
- (-) One can add a command line option such that it is not more effort than `nix --version`.
- (+) That requires adding another built-in to the public API.
- (-) Using a language feature requires an additional steps from users to determine the current version.
- (-) Requires adding another command line option to the public API.
- (+) The Nix language version is decoupled Nix version numbering.
- (+) It changes less often than the Nix version.
- (-) That was probably due to making changes being so hard.
- (+) The language changing slowly is a desirable property for wider adoption.
- (-) There are two version numbers to keep track of.

</details>

1. The language version for Nix files is denoted in the file extension.

<details><summary>Arguments</summary>

- (+) sidesteps misinterpretation keeping metadata out of the actual data
- (-) will introduce a proliferation of syntax highlighters and other tooling
- (+) this is correct though, because a syntax highligher has to know the syntax if it changes with the language version
- (+) makes accidental mixing of versions impossible
- have to specify the file extension when importing a file
- (-) have to rename all files in a project to change the version
- (-) makes filenames longer, introduces visual noise
- this is the cost of being explicit
- (-) `default.nix` resolving needs specification:
- if for `import ./foo`, all of `./foo/default.nix{6,7,8}` exist, pick the one matching the version used by the evaluator, otherwise fail
- then you'd have to specify a file using a different version explicitly

</details>

<details><summary>Alternatives</summary>

- use a magic comment at the beginning of the file
- (+) Makes explicit what can be expected to work.
- (+) Enables communicating language changes systematically.
- (+) Backwards-compatible
- (+) Allows for gradual adoption: opt-in until semantics is implemented in Nix *and* the first backwards-incompatible change to the language is introduced.
- (+) Visually unintrusive
- (+) Self-describing and human-readable
- (+) Follows a well-known convention of using [magic numbers in files](https://en.m.wikipedia.org/wiki/Magic_number_(programming)#In_files)
- (-) May make the appearance that changing the language is harmless.
- (+) The convention itself is harmless and independent of the development culture around the language.
- (-) The syntax of the magic comment is arbitrary.
- (-) There is a chance of abusing the magic comment for more metadata in the future. Let's avoid that.
- (-) At least one form of comment is forever bound to begin with `#` to maintain compatibility.
- (-) Editor support is made harder, since it requires switching the language based on file contents
- (-) It requires significant additional effort to implement and maintain an appropriate system to make use of the version information.
- (-) Raises the question if the new syntax should be a breaking change to the language
- Use a magic string that is incompatible with evaluators prior to the feature, e.g. `%? Nix <version>`.
- (+) Makes clear that the file is not intended to be used without explicit handling of compatibility.
- (-) Cannot be introduced gradually.
- (+) Such a breaking change could also be reserved for later iterations of the Nix language.

- Nix 'editions'
- (-) requires metadata in the actual data, does not solve the problem
- it was [removed from the flakes schema](https://github.com/NixOS/nix/commit/e5ea01c1a8bbd328dcc576928bf3e4271cb55399) for that reason

</details>

1. The file extension consists of the language version followed by `.nix`.
Example:

   default.7.nix

<details><summary>Arguments</summary>

- (+) consistent with the convention of file extensions carrying soft meaning as metadata
– (-) a conflicting convention is for denoting nested file types, such as `tar.gz`
- (+) visually least intrusive

</details>

<details><summary>Alternatives</summary>

- no separator
– (-) hard to discern visually
- `^`
– (-) overlaps with derivation output syntax
- `-`
- (+) visually not intrusive
- `_`
– (-) visually more intrusive
- all of the following characters will interfere with some tooling
- `!` - shells
- `"` - shells
- `#` - URLs
- `$` - shells
- `%` - URLs
- `&` - shells, URLs
- `+` - URLs
- `,` - natural language
- `/` - paths, URLs
- `:` - URLs
- `;` - shells
- `=` - URLs, Nix language
- `?` - URLs
- `@` - URLs, Nix language
- `\` - Windows paths

</details>

1. The language version for bare Nix expressions is specified with a parameter to the evaluator.

<details><summary>Arguments</summary>

- (+) It needs to be specified somehow.

</details>

1. If no language version is specified in the file name, assume version 6.
This is implemented in the stable release of Nix at the time of writing this RFC.

<details><summary>Arguments</summary>

- (+) Backwards compatible with existing evaluators
- (+) Does not require changing any existing code

</details>

<details><summary>Alternatives</summary>

- Assume the evaluator's current version
- (-) When the evaluator advances in language version, evaluation may fail on existing code
- (-) Defeats the purpose of explicit versioning: Which evaluator to use for a given file is left unspecified
- (-) Following the latest evaluator version may inadvertently break the code for older evaluators
- (+) Don't have to look up the latest version of the Nix language when writing code
- (+) Does not clutter the file names for what is supposed to be the latest version of the code

</details>

1. If no version is specified when invoking the evaluator on a bare Nix expression, assume the most recent language version supported by the evaluator.

1. The Nix language evaluator provides a command to output the latest Nix language version.

<details><summary>Arguments</summary>

- (+) This is for convenience to determine which features are available.

</details>

1. `builtins.langVersion` returns the language version used for evaluating the given expression.

<details><summary>Arguments</summary>

- (+) Could make use of it for generating Nix expressions programmatically and annotating them with the correct version.
- (+) `builtins.langVersion` is already part of the stable interface, this way we can make it pure.
- (-) Requires maintaining more API surface without a clear use case.

</details>

<details><summary>Alternatives</summary>

- Return the latest language version instead.

- (-) Doesn't help to determine what is used for evaluating the current expression.
- (+) Might provide opportunities for forward-compatible Nix code.
- (-) Brittle and defeats the purpose of this RFC.
- (-) Is impure, the value depends on the environment.

- Don't expose `builtins.langVersion` at all.

- (+) No need to have this if the Nix files are versioned.
- (+) `builtins.langVersion` is not documented and not widely used.
- (-) Requires a Nix language version bump to implement this RFC.

</details>

## Deprecation warnings and errors

- Default:
- Issue warnings on deprecated language constructs, excluding optional ones
- All warnings:
- Issue all deprecation warnings, including optional ones
- Don't warn (selection):
- Do not issue warnings for selected language constructs
- Errors instead of warnings (selection):
- Throw an error instead of a warning for selected language constructs
- All errors:
- Throw an error for all deprecated language constructs.

Then, during evaluation:
- In non-verbose mode, warn once for each deprecated construct
- In verbose mode, warn for each occurrence

A the end of the evaluation, print statistics and explanations.
The specifics of displaying warnings and errors is up to implementation, but should include the symbolic name of the langauge construct in question.

Depending on the command, this can be exposed as the following flags:

--expr-warn-all
--expr-no-warn="url-literal let-body int-zeroes non-attr-select"
--expr-error="url-literal let-body int-zeroes non-attr-select"
--expr-error-all

The naming may need some bikeshedding. For example, one could use the same syntax as with C-compilers (probably not though):
- `-Wall`
- `-Wno-`
- `-Werror=`

## Drawbacks to the error semantics

- (-) Introducing breaking changes to the Nix language will cause warnings and errors
- (+) It will actually notify users about what's going on instead of just breaking
- maintenance
burden for everyone using these old constructs, or evaluating old nixpkgs.

As an example, if we were to deprecate leading zeroes, this would show
a warning on nixpkgs prior to 22.11.

## Alternatives for handling incompatible versions

- No warnings, just errors
- (-) this doesn't offer a transition window to users
- (+) easier to implement
- Opt-in warnings
- (-) can't really make breaking changes as people won't be warned ahead of time
# Examples and Interactions
[examples-and-interactions]: #examples-and-interactions

## Deprecated syntax/features

nix eval --json ./test.nix
warning: URL literals are deprecated (url-literal)
         please replace this with a string: "https://nixos.org"

       at test.nix:1:1:

            1| https://nixos.org
             | ^

"https://nixos.org"

warning: The following deprecated features were used:
  - url-literal (nix.dev/c/dXJs), 1 time

  Use `--no-warn-deprecated=<feature>` to disable this warning, and `--verbose` to show all occurrences.

## Show the current Nix language version

nix --language-version
7

# Open questions

- Q: how do Nix versions and stability guarantees interact with Nix language versions?
- if Nix language introduces a breaking change, does that still mean a minor version bump for Nix?
- possible answer: yes, if Nix keeps evaluating prior language versions. Nix major version bump would be mandated if support for older language versions is dropped.

- Q: Should it be an exact version like 2.12.0 or should bounds like >= 2.12.0 or 2.12.* also be allowed?
- New versions of Nix will support multiple grammar versions. The one used will be based on the comment at the top of the file.
- Files with different grammar versions are fully interoperable.
- Old grammar versions will continue be supported for a long time.
- This allows evolving the grammar by allowing users to opt-in to new versions in an incremental basis. The ecosystem can gradually move one file at a time.
- otherwise those older evaluators would error out on the unknown feature in an ugly way
- Example: I'm thinking of Rust "editions" as a model. If you specify Nix 2.12 any new breaking feature are not enabled when parsing the module. When you feel like it you update to the new syntax and update your tag line to Nix 2.14.
- Specification per Nix file means that during the same evaluation different files can have different grammar versions, right?
- This would allow incremental evolution.
- this makes it for a very restrictive semantics, balancing the two ideas that:
- Nix should stay backwards-compatible and an expression that is valid Nix today will still be valid Nix in ten years;
- We also want the language to evolve, if only to remove the cruft and all the in-retrospect-not-really-happy bits that it has accumulated.
- Nix should stay backwards compatible, so if a Nix version is able to evaluate the version N of the language, it should also be able to evaluate all the versions m for any m<N. This might mean adding conditionals in the parser, or even keeping different parsers in Nix – which makes it a strong incentive for being strict about 2;
- Moreover, it should be possible to freely mix files written in different versions of the syntax in the same expressions.
- TODO: However you'll have to deal with the semantics of values passed across file boundaries, this is particularly relevant for builtins
- Some of the language changes you mention cannot be handled on a per-file basis (i.e. using the tags described in this RFC). In particular, changes to builtins cannot be per-file since builtins can be passed around as an argument.
- Could different versions not get different builtin objects when required?
- TODO: do we really want to keep this deep compatibility?
- This is technically breaking if you try to compare the builtins from two different files with different versions but I don't think that is a major issue.
- And in that case you can avoid updating your declared version until this issue is resolved so it won't break existing code (but does have a slight chance of making updating your eval version a breaking change)
- implementation idea: if I pass builtins around, we already can augment them before passing
- TODO: is that worth the effort; this is a policy and economics issue
- we may postpone dealing with that particular issue until it arises, but the general setup should allow handling that case
- TODO: how do web browsers do it?
- Example: A user might expect a `stdenv.mkDerivation` in a file for an older Nix version to still support option `foo`, but it doesn't because the file where `mkDerivation` calls `derivation` is written for a newer Nix version where `foo` doesn't exist anymore. So third-party `mkDerivation` calls that use an up-to-date Nixpkgs will at some point just get an error, there's nothing warning them of the deprecation of such options before it happens.
- this is already the case in practice, and Nixpkgs is versioned already
- consideration: conditional import across language versions?
- consider: import higher version from lower version
- just no

- IMO if it's declared per file then it's also meant per file. The alternative would be to have some special file in a project's root or something (like Rust and many other programming language projects do), but given Nix's rather YOLO attitude on imports I don't think that's feasible.
- inherit the language version across imports
- that's best effort and can fail, but if all you have is metadata and that's missing, you just have to assume
- if you had a notion of a project, you could assume the language version for that entire project, but you'd still have to encode that metadata outside of the language somehow

- Q: what should be the policy for changing the version number?
- technically: whenever the language is changed *at all*
- Q: note: this would hint at that using dated versions (YYYY-MM-DD) may be sensible
- from that you can tell how much time has passed between versions
- still monotonic, sortable, essentially integers
- These numbers should be totally independent of the Nix version number (probably match builtins.languageVersion, although I didn't know it existed until now);
- Social: Changing them should always be exceptional, maybe batching the incompatible changes to prevent changing the language version too often
- scary idea: hard limit on number of changes, e.g. one or two per year??
- actually that may make sense to synchronise (to some degree) with Nixpkgs/NixOS releases
- Q: what if you add a builtin?
- Q: should we remove support for historic versions?
- could have a compatibility wrapper at the expression level, but this would be on the user to take care of
- still, bump the language version

- Q: This also raises the question if it would apply retroactively, of course.
- For example, the flakes branch has introduced breaking changes in fetchers that made old Nix code using builtins.fetchGit not evaluate in newer versions. Should that retroactively be declared a lang-version bump, and should it be rewritten in a compatible way? Or is the breakage a bug that should be fixed? Or are interfaces of builtins not part of the language at all?
- open question, could be either, should decide case by case
- e.g. is it possible to convert the old to the new `fetchGit`

- Q: What should the default assumed value be? There's some options:
- Hardcode the default to the current Nix language version 6
- this way we always know what the field is if not set, and if you use newer Nix language features you need to opt-in to use them.
- But this then begs the question, how do we migrate to newer defaults? This would lead to requiring the field to be mandatory at some point.
- open question
- The Nix version currently evaluating the file
- but this has the problem that if a user writes a file with newer Nix version and somebody with an older Nix version tries to evaluate it, it fails without any good error message

- Q: why is it a comment and not special syntax?
- because if were, you break older evaluators immediately without providing a transition period for free until the experssion actually changes incompatibly
- it allows users weaseling their way around upgrades until they really have to
- this has some downsides of course, because you may introduce syntax that is supported only superficially but makes old evaluators break in subtle ways, but this is of lesser importance and also somewhat of a social problem

- Alternative: this should *really* use a construct with semantics to communicate the intent instead, that way formatters can't break it and evaluators that don't know it can reject it
1. statically: reuse `assert` and add something to builtins instead, making the header e.g. `assert builtins.langVersion == 6;`.
- (-) it's much more clumsy than the evaluator being able to tell you what exactly is wrong (which allows for gradual transition)
- as opposed to the magic comment it will fail hard instead of softly
- (-) this turns around the problem: instead of declaring an experssion being written in a particular version of the language, it asks the evaluator which version it is in, and fails hard. the declarative approach leaves much more options
- recognising this as a declarative construct is a recipe for confusion
- also it puts the burden of deciding what to do on the user in a way that is non-obvious
- because, if you have a n unannotated expression that fails evaluating, you just do whatever you usually do.
2. dynamically: add an entirely new construct, e.g. `use v2022;` (that should only be allowed at the top of each parse unit).
- (-) but that just breaks old evaluators instantly
- Alternative: Or should the field be mandatory with a warning (and one of the above defaults), or an error? If so, Nix could then also analyze the file to figure out which versions it supports and suggest using the oldest one of those. This then probably only works well with syntactic changes though, not semantic ones.
- No: It's very complex to implement, doesn't really allow deprecating things at all.

- claim: we think that magic comments are a *horrible* idea. they only work if *all* the tooling understands them, otherwise they lose their semantics. code formatters that don't understand them are free to remove or reformat them, but most importantly evaluators that don't know them have *no* way of acting upon them.
- answer: yes, that's the point, old evaluators should still run the code if it's sufficiently close to what they understand

# Prior art

- [Rust `edition` field]

Rust has an easier problem to solve.
Cargo files are written in TOML, so the `edition` information does not have to be part of Rust itself.

[Rust `edition` field]: https://doc.rust-lang.org/cargo/reference/manifest.html#the-edition-field

- [Haskell language extensions]

Haskell allows enabling separate language features per file.

[Haskell language extensions]: https://downloads.haskell.org/ghc/latest/docs/users_guide/exts/intro.html

- JaveScript modules
- .cjs and .mjs extensions for commonjs/es-modules syntax variants
- `function() { "use strict"; return 10 }`

- [Flakes `edition` field]

There had been an attempt to include an `edition` field into the Flakes schema.
It did not solve the problem of having to evaluate the Nix expression using *some* version of the grammar.

[Flakes `edition` field]: https://discourse.nixos.org/t/nix-2-8-0-released/18714/6

# Future work
[future]: #future-work

- Define rules deciding when a change to the language is appropriate to avoid proliferation and limit complexity of implementation.