owned this note
owned this note
Published
Linked with GitHub
---
# pnkfelix is deliberately using breaks:false so that each sentence can be put onto its own line in the markdown source, which will hopefully ease discussion of this document.
breaks: false # False means NOT to render line breaks as hard line breaks.
---
# Our Vision for the Rust Specification
## Purpose
The purpose of the Rust specification is to provide an authoritative resource for determining what source texts are valid Rust programs, and how such programs behave.
An ideal specification both (1.) defines *prescriptive bounds* on the semantics of a given Rust program for current and future Rust versions, and (2.) provides *descriptive details* of the semantics that are specific to the Rust version coupled with that instance of the specification.
The provision of the version-specific details can be provided directly in the specification, or can be indirectly provided via delegation to other documents owned by Rust teams that are the authority for that subtopic.
### Explanation of the Purpose
The above purpose statement chose some words carefully; it is worth elaborating on those words and the overall phrasing.
#### "defines"
The utility of a specifcation comes from 1. forcing authors to define things and 2. its value of those definitions to the readers of the specification.
#### "semantics"
Rust has a static and dynamic semantics.
The static semantics of Rust dictates which programs are accepted in the language, while the dynamic semantics determines which of those accepted programs are well-defined, as well as their respective meanings.
The word "semantics" in the purpose statement refers to both the static and dynamic semantics of Rust collectively.
#### "current", "future"
The Rust language has been evolving since its inception, and we expect it to continue to evolve going forward.
These evolutionary steps represent a traversal over the language design landscape.
For every Rust release, we expect the current implementation to be standing at one point in that landscape, while the Rust community's idealized goal awaits at some higher point up the mountain.
#### "prescriptive", "descriptive"
A descriptive dictionary is one that attempts to describe how a word *is* used, while a prescriptive dictionary is one that prescribes how a word *should be* used.
We take inspiration from that distinction to tease apart two important audience types.
Rust provides a stability promise: "You should never have to fear upgrading to a new version of stable Rust."
That raises a natural question: Why does the purpose statement distinguish version-crossing prescriptive definitions from version-specifc descriptive defintions?
Our answer:
That stability promise left a bit of wiggle-room for itself, in terms of what the project considers "fear" vs "reasonable labor associated with a Rust upgrade."
When defining semantics, one must be more explicit about any such wiggle-room.
Some Rust users *need* a description of the expected semantics as it stands for the Rust release that sits in their hands; they are the audience for version-specific details.
But other Rust users, such as some library developers, have a more forward-looking perspective.
The forward-looking developers may require an assurance that one specific code snippet A will always be accepted, and will also always have a particular meaning.
They may require an assurance that a different snippet B will *never* be accepted.
Or they may require an assurance that a third snippet C leveraging Unsafe Rust will always have undefined behavior (e.g. to jusify a local transformation by arguing that no *new* undefined behavior is injected by that transformation.)
These are all cases that call for a prescriptive definition of the semantics; it does not matter in these cases what the compiler currently does -- what matters is what it will do in the future, which is inherently prescriptive.
It would be premature to fix firm definitions in those areas, e.g. categorizing for each input program whether it is accepted or rejected by the type inference system, and then forcing all futures versions of Rust to follow that same categorization.
Another similar example: If we chose a fixed grammar, and then said all future versions of Rust must strictly categorize all source inputs as accepted or rejected according to that one grammar, then that would restrict our ability to add future backward-compatible language extensions to the grammar.
Therefore, these kinds of guarantees (especially with respect to details of the type inference rules, or details of what unsafe code is well-defined) are where the prescriptive *bounds* arise.
Such bounds allow for a middle ground of programs, where we do not commit all future versions of Rust to always make the same decision that the current version makes.
For example, one can then say, prescriptively, that a given grammar provides a lower bound on the set of programs that must be accepted by all future versions of Rust, while still allowing the language to evolve in a backward compatible fashion.
One can also say, descriptively, that the current version of Rust rejects source inputs that do not conform to the grammar.
The descriptive definitions tell the reader how a construct will behave with respect to that Rust version; the prescriptive bounds tell the reader what they can and cannot expect to hold true in the future of Rust.
Thus, we conclude that an ideal specification will need to address both the static and dynamic semantics, for both the prescriptive bounds and descriptive details.
#### "delegates"
There are broad areas where the questions of what semantics we want, and how they should be specified, are open research topics.
Examples of such areas include: macros 2.0, the type inference rules, the trait matching rules, and the operational semantics of unsafe code.
It is not reasonable for the T-spec team to claim authority on such topics.
Instead, other teams will be invited to contribute their own detailed descriptions, which can be published as their own documents that the specification can reference.
Each such document is, like the specification itself, coupled to a specific Rust version.
Furthermore, each such document is analogous to the detailed descriptions: the scope of each document produced via delegation is intended to be restricted to a specific Rust version.
If a contributing team has input about broader prescriptive rules that should hold beyond the current Rust version, then that should be be part of the Rust specification document itself.
Such prescriptive rules should always be the responsiblity of the T-spec team to incorporate into the document.
All such prescriptive rules are then subject to the specification approval process.
## Goals
It is ambitious to provide both prescriptive bounds for current and future Rust versions and descriptive details of the current Rust version.
We will maximize the value of our efforts by working iteratively and incrementally.
The specification can have gaps where the prescribed bounds are broader than necessary.
Subsequent releases of the specification can tighten those prescribed bounds.
We expect early versions of the spec to focus heavily on delivering the detailed description of the current Rust version.
Such a specification could be derived heavily from an existing work product, such the Ferrocene specification, since that explicitly focuses on providing a detailed description of a specific Rust version.
Feedback on those version-specific descriptions will help us learn how best to craft the prescriptive bounds in the specification.
## Scope
The specification should cover the following areas of Rust's syntax and semantics. Some parts may be inherently coupled to specific backends or target implementation techniques (e.g. inline asm).
* The grammar of Rust, specified via Backus-Naur Form (BMF) or some reasonable extension of BNF.
* Macro expansion
* Macro-by-example transcription; Hygiene
* cfg attributes
* Procedural macros; attributes and derive
* Path and identifier resolution
* Modules
* Static semantics
* Type definitions; type expressions; layout
* Type inference and type-checking; subtyping
* Lifetimes and borrow-checking
* Generics; Associated item resolution and Trait solving
* Operational semantics of safe Rust
* binding forms; match expressions; drop glue
* moving and copying of values; borrows
* field projection; method dispatch
* operator overloading
* Operational semantics of unsafe Rust
* memory model
* inline assembly
* Const evaluation
* Crates and crate linkage
:::info
**TODO:** Questions that the specification should answer
:::
:::info
**TODO:** Expected areas of fuzziness
:::
## Presentation
* The specification text will only use technical terms that it has itself defined within the specification, or that have definitions in a freely available online dictionary.
(The T-spec team is entrusted to select what dictionary to use for this purpose.)
* Individual items in the specification can be named and referenced, not just in hyperlinks but also in human text, i.e. "item [syntax-patterns.5]". When possible, these names/references to items should persist across versions of the specification.
* Iterations of the specification should include renderings that highlight the differences between versions. (See e.g. Ada Reference Manual.)
* The Rust specification should be authored in a format that encourages volunteer contribution, even if T-spec expects to have to reauthor each contribution in order to maintain a consistent voice for the specifcation.
The T-spec team is entrusted to select a document editing format that can be meet the needs above.
:::info
**TODO:** something about accessibility/readability of the document.
:::
:::info
**TODO:** note how we might include (or link) less formal content and design reasoning and motivation.
:::
## Development Process
The authority of the specification is derived from its approval by each of the appropriate teams.
The initial scope of the specification is the Rust language and its associated core library.
### Internal and External Approval
T-lang has the initial authority for approving the specification.
T-lang is expected to engage other teams or subteams to provide their approval when necessary (i.e. due to a change in scope or a need for specific expertise).
The T-spec team needs to ability to produce content and iterate on it wthout being blocked by approval processes.
All changes and additions are authored by the T-spec team and committed to its work repository.
Each item in the spec will have the ability to be tagged with metadata tags, such as "in-progress", "pending-t-lang-approval", and "of-concern-to-stakeholders".
Ideally the metadata tags will be able to embed external links; e.g. "of-concern-to-stakeholders" should ideally link to a github issue or zulip thread.
We expect there are two primary kinds of content changes: minor and major.
Minor changes are items that appear uncontroversial or trivial to the T-spec team.
For example, changes that were already approved by the T-lang team via a separate FCP might be considered minor, at the discretion of the T-spec team.
Major changes are those that are questionable, important, or controversial.
Any member of T-spec, T-lang, or a specification stakeholder can flag a change as major.
When major changes are committed to the specification and is no longer "in-progress", it is tagged as "pending-t-lang-approval" until the T-lang team can approve it.
After a suitable process has been completed with an approval, the "pending-t-lang-approval" markers are removed from content resulted from major changes.
The T-spec and T-lang teams should strive to have at least one member of T-spec who is also a member of T-lang, acting as a liason to help ensure that T-spec's calibration of minor vs major changes remains roughly in sync with T-lang.
### Release Cadence
The Rust release process proceeds independently of the specification approval process.
If a specification has not been approved for a given release, then the release goes out without an associated specification.
The specification team may choose, at their option, to deliver a specification associated with a given release at a later time; it need not be coupled to the release schedule for Rust itself.
This is by design; the specification effort must not add new hurdles for the project to overcome in order to meet its existing obligations, such as the 6-week release cadence.
Our aspiration is that we will *eventually* reach a point where delivering an updated specification is automatic and can be done according to the project's 6-week release cadence.
But we want, for the short and medium term, to have the freedom to lag behind that 6-week release cadence.
The ability to lag behind the Rust release schedule may be especially useful when T-spec team is incrementally adding new content for previously unaddressed areas, or significantly narrowing the prescriptive bounds in the current version of the specification.
:::info
**Unresolved Question: Allow Delivery Gaps?**
We identified two potential paths to follow when deciding on a delivery cadence that follows the rules above.
The T-spec team might choose to follow a delivery schedule where it produces specification documents for a *contiguous* series of Rust releases that may lag the current Rust release.
E.g. it may produce specifications for Rust versions [1.90, 1.91, 1.92, 1.93], while the current Rust version is 1.105.0.
Or the T-spec team might instead choose to follow a deliver schedule where there are *gaps* of one or more releases that do not have an associated specification. E.g. it may produce specifications for Rust versions [1.90, 1.93, 1.96, 1.99], while the current Rust version is 1.105.0.
Obviously any contiguous schedule that lags the current release is just a specific instance of one kind of gap; its just that the single gap is the lag from the final specification to the current release.
The main motivation for allowing gaps is to provide a kind of "catch up" mechanism, where T-spec is empowered to skip intermediate releases in order to focus on content that is more likely to be relevant to the most current Rust releases.
So the main question for our stakeholders is: Is there significant value to our stakeholders in trying to enforce delivery of a contiguous series of specifications, even if it implies that the lag from the latest specification and the latest Rust release may be longer than if we allowed for (arbitrary) gaps, e.g. [1.90, 1.93, 1.96, 1.99]?
:::
While the specification development process will not block releases, changes to *language features* should be coupled with relevant updates to the specification.
Once we have begun publishing a specification coupled to specific releases, then changes to the language features that are documented in the current specification cannot be stabilized without a corresponding Pull Request approved by the T-spec team to the current draft specification.
Changes to language features that are not documented in the specication can be stabilized without an update to the specification, but require a T-spec team member's acknowledgement that the corresponding feature is undocumented.
By enforcing this rule that new features must be part of the specification before they are stabilized, we will hopefully eliminate the main source of potential lag between the specification and the Rust release.
### T-spec composition
The membership of T-spec is made up, at minimum, the T-spec team (co-)lead(s) and a non T-spec team lead in a dedicated editor role. The T-spec team can choose to add new members based on what gaps it needs to be filled.
### Stakeholders
There are stakeholders whom have an interest in the delivery of a Rust specification, either as consumers of its content, or as subject matter experts to review the content. One could think of these as the "Very Important Persona (VIP) Customers." Our list of stakeholders is also follows (with motivation for that stakeholder given in parentheses)
* [ ] Entirety of T-lang design team (Rust Language Designers)
* [ ] One or more representatives from T-types team
* [ ] One or more representatives from T-opsem team
* [ ] One or more representatives from Ferrocene (High Assurance/Availabilty, e.g. Automotive Industry.)
* [ ] One or more representatives from Formal Methods Research and Development
* [ ] Operating System Development (Rust for Linux; Microsoft)
:::info
**TODO:** do we need more in the list above? less?
:::
The members of the stakeholders list are provided the ability to flag concerns in the specification text.
Once some content is no longer tagged as "in-progress", it is considered to be open to review by the stakeholders, where we invite them to provide feedback and file concerns via github issues.
The T-spec team will periodically triage the incoming issues.
For each stakeholder-filed issue, it may try to resolve the concern directly (by either making a change or discussing a point of confusion with the stakeholder).
If there is no immediate path to resolution of the concern, then the T-spec team will add the "of-concern-to-stakeholders" tag to the relevant items in the specification text.
### Incremental Development and Delivery
As noted above, we expect early versions of the spec to focus heavily on delivering the detailed description of the current Rust version.
The prescriptive bounds can start with useful high-level guarantees, such as "safe Rust cannot cause undefined behavior" and then future versions of the specification can incrementally add more details to the prescriptive bounds, such as "all undefined behavior must have its root cause in either 1. unsafe Rust, 2. a misbehaving foreign component, or 3. a bug in the Rust implementation."
Similarly, the prescriptive bounds for a component such as the type inference system can start off with properties that any suitable type inference system would need to meet, and describe class
## Thoughts / Notes / ToDo queue
* We look forward not backward
* a given spec revision is only obligated to talk about the current Rust version and future ones. It may choose to identify ways that previous Rust versions deviated from this spec, but is not required to do so.
* only the versions of the spec at or after the last stable Rust release are maintained by the project. (If previously unknown deviations between the spec and implementation are discovered, the project is not obligated to issue point releases to versions of the spec older than the most recent stable release.)
* Why a detailed description of the current version? (i.e. Why not just let the implementation details speak for themselves via the implmentation?)
* We expect the specifications to be more succinct to read and understand than the implementation.
* Any deviation between the implementation versus the specified details for the current version implies the presence of bug (either in the specification or in the implementation). Resolving such bugs will help inform how the prescriptive bounds of the specification should evolve.
* Maybe note that this is less important for limitations that rustc 'knows' and will error about. accepting code that should not be accepted (and differences in behaviour) are critical to document. The reverse, not accepting code that should be accepted, isn't as important to put in the specification.