Refined trait implementations

• Feature Name: (fill me in with a unique ident, refined_impls)
• Start Date: (fill me in with today's date, YYYY-MM-DD)
• RFC PR: rust-lang/rfcs#0000
• Rust Issue: rust-lang/rust#0000

Summary

This RFC generalizes the safe_unsafe_trait_methods RFC, allowing implementations of traits to add type information about the API of their methods and constants which then become part of the API for that type. Specifically, lifetimes and where clauses are allowed to extend beyond what the trait provides.

Motivation

RFC 2316 introduced the notion of safe implementations of unsafe trait methods. This allows code that knows it is calling a safe implementation of an unsafe trait method to do so without using an unsafe block. In other words, this works today:

trait Foo {
    unsafe fn foo(&self);
}

struct Bar;
impl Foo for Bar {
    fn foo(&self) {
        println!("No unsafe in this impl!")
    }
}

fn main() {
    // Call Bar::foo without using an unsafe block.
    let bar = Bar;
    bar.foo();
}

Unsafe is not the only area where we allow impl signatures to be "more specific" than the trait they're implementing. Unfortunately, we do not handle these cases consistently today:

Associated types

Associated types are a case where an impl is required to be "more specific" by specifying a concrete type.

struct OnlyZero;

impl Iterator for OnlyZero {
    type Item = usize;
    fn next(&mut self) -> Option<Self::Item> {
        Some(0)
    }
}

This concrete type is fully transparent to any code that can use the impl. Calling code is allowed to rely on the fact that <OnlyZero as Iterator>::Item = usize.

let mut iter = OnlyZero;
assert_eq!(0usize, iter.next().unwrap());

Types in method signatures

We also allow method signatures to be more specific than the trait they implement.

trait Log {
    fn log_all(iter: impl ExactSizeIterator);
}

struct OrderedLogger;

impl Log for OrderedLogger {
    // Don't need the exact size here; any iterator will do.
    fn log_all(iter: impl Iterator) { ... }
}

Unlike with unsafe and associated types, however, calling code cannot rely on the relaxed requirements on the log_all method implementation.

fn main() {
    let odds = (1..50).filter(|n| *n % 2 == 1);
    OrderedLogger::log_all(odds)
    // ERROR:              ^^^^ the trait `ExactSizeIterator` is not implemented
}

This is a papercut: In order to make this API available to users the OrderedLogger type would have to bypass the Log trait entirely and provide an inherent method instead. Simply changing impl Log for OrderedLogger to impl OrderedLogger in the example above is enough to make this code compile, but it would no longer implement the trait.

The purpose of this RFC is to fix the inconsistency in the language and add flexibility by removing this papercut. Finally, it establishes a policy to prevent such inconsistencies in the future.

Guide-level explanation

When implementing a trait, you can use function signatures that refine those in the trait by being more specific. For example,

trait Error {
    fn description(&self) -> &str;
}

impl Error for MyError {
    fn description(&self) -> &'static str {
        "My Error Message"
    }
}

Here, the error description for MyError does not depend on the value of MyError. The impl includes this information by adding a 'static lifetime to the return type.

Code that knows it is dealing with a MyError can then make use of this information. For example,

fn attempt_with_status() -> &'static str {
    match do_something() {
        Ok(_) => "Success!",
        Err(e @ MyError) => e.description(),
    }
}

This can be useful when using impl Trait in argument or return position.^[1]

trait Iterable {
    fn iter(&self) -> impl Iterator;
}

impl<T> Iterable for MyVec<T> {
    fn iter(&self) -> impl Iterator + ExactSizeIterator { ... }
}

Note that when using impl Trait in argument position, the function signature is considered to be "more specific" as bounds are removed, meaning this specific impl can accept a wider range of inputs than the general case. Where clauses work the same way.

trait Sink {
    fn consume(&mut self, input: impl Iterator + ExactSizeIterator);
}

impl Sink for SimpleSink {
    fn consume(&mut self, input: impl Iterator) { ... }
}

Finally, methods marked unsafe in traits can be implemented as safe APIs, allowing code to call them without using unsafe blocks.

Reference-level explanation

This is the technical portion of the RFC. Explain the design in sufficient detail that:

• Its interaction with other features is clear.
• It is reasonably clear how the feature would be implemented.
• Corner cases are dissected by example.

The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work.

Trait implementations

The following text should be added after this paragraph from the Rust reference:

A trait implementation must define all non-default associated items declared by the implemented trait, may redefine default associated items defined by the implemented trait, and cannot define any other items.

Each associated item defined in the implementation meet the following conditions.

Associated consts

• Must be a subtype of the type in the corresponding trait item.

Associated types

• Associated type values must satisfy all bounds on the trait item.
• Each where clause must be implied by the where clauses on the trait itself and/or the associated type in the trait definition, where "implied" is limited to supertrait and outlives relations. This would be expanded to all implied bounds when that feature is enabled.

Associated functions

• Must return any subtype of the return type in the trait definition.
• Each argument must accept any supertype of the corresponding argument type in the trait definition.
• Each where clause must be implied by the where clauses on the trait itself and/or the associated function in the trait definition, where "implied" is limited to supertrait and outlives relations. This would be expanded to all implied bounds when that feature is enabled.
• Must not be marked unsafe unless the trait definition is also marked unsafe.

When an item in an impl meets these conditions, we say it is a valid refinement of the trait item.

Using refined implementations

Refined APIs are available anywhere knowledge of the impl being used is available. If the compiler can deduce a particular impl is being used, its API is available for use by the caller. This includes UFCS calls like <MyType as Trait>::foo().

Older editions

For historical reasons, not all kinds of refinement are automatically supported in older editions.

You can opt in to the new behavior in older editions with a #[refine] attribute on the associated item.

impl Error for MyError {
    #[refine]
    fn description(&self) -> &'static str {
        "My Error Message"
    }
}

This enables refining all features in the table above.

Transitioning away from the current behavior

Because we allow writing impls that look refined, but are not usable as such, we need a strategy for transitioning off of this behavior. This can be done in two parts.

Lint against unmarked refined impls

After this RFC is merged, we should warn when a user writes an impl that looks refined and suggest that they copy the exact API of the trait they are implementing. Once this feature stabilizes, we can also suggest using the #[refine] attribute.

Automatic migration for the next edition

Because refinement will be the default behavior for the next edition, we should rewrite users' code to preserve its semantics over edition migrations. That means we will replace trait implementations that look refined with the original API of the trait items being implemented.

Preventing future ambiguity

This RFC establishes a policy that anytime the signature of an associated item in a trait implementation is allowed to differ from the signature in the trait, the information in that signature should be usable by code that uses the implementation.

This RFC specifically does not specify that new language features involving traits should allow refined impls wherever possible. The language could choose not to accept more specific implementation signatures for that feature. This should be decided on a case-by-case basis for each feature.

Interaction with other features

Implied bounds

When implied bounds is stabilized, the rules for valid refinements will be modified according to the italicized text above.

Specialization

Specialization allows trait impls to overlap. Whenever two trait impls overlap, one must take precedence according to the rules laid out in the specialization RFC. Each item in the impl taking precedence must be a valid refinement of the corresponding item in the overlapping impl.

Drawbacks

Why should we not do this?

Accidental stabilization

For library authors, it is possible for this feature to create situations where a more refined API is accidentally stabilized. Before stabilizing, we will need to gain some experience with the feature to determine if it is a good idea to allow refined impls without annotations.

Complexity

This RFC proposes several things that can be considered added complexity to the language:

• Allows different types to have different APIs for the same trait

Adding text to the Rust reference

Part of the reason that text is being added to the reference is that the reference doesn't specify what makes an item in a trait implementation valid. The current behavior of allowing certain kinds of divergence and "ignoring" some of them is not specified anywhere, and would probably be just as verbose to describe.

Types are allowed to have different APIs for the same trait

It is possible for a user to form an impression of a trait API by seeing its use in one type, then be surprised to find that that usage does not generalize to all implementations of the trait.

It's rarely obvious, however, that a trait API is being used at a call site as opposed to an inherent API (which can be completely different from one type to the next). The one place it is obvious is in generic functions, which will typically only have access to the original trait API.

Rationale and alternatives

• Why is this design the best in the space of possible designs?
• What other designs have been considered and what is the rationale for not choosing them?
• What is the impact of not doing this?

This RFC attempts to be minimal in terms of its scope while accomplishing its stated goal to improve the consistency of Rust. It aims to do so in a way that makes Rust easier to learn and easier to use.

Do nothing

Doing nothing preserves the status quo, which as shown in the Motivation section, is confusing and inconsistent. Allowing users to write function signatures that aren't actually visible to calling code violates the principle of least surprise. It would be better to begin a transition out of this state sooner than later to make future edition migrations less disruptive.

Require implementations to use exactly the same API as the trait

We could reduce the potential for confusion by disallowing "dormant refinements" with a warning in the current edition, as this RFC proposes, and an error in future editions. This approach is more conservative than the one in this RFC. However, it leaves Rust in a state of allowing some kinds of refinement (like safe impls of unsafe methods) but not others, without a clear reason for doing so.

While we could postpone the question of whether to allow this indefinitely, we argue that allowing such refinements will make Rust easier to learn and easier to use.

Allow #[refine] at levels other than impl items

We could allow #[refine] on individual aspects of a function signature like the return type, where clauses, or argument types. This would allow users to scope refinement more narrowly and make sure that they aren't refining other aspects of that function signature. However, it seems unlikely that API refinement would be such a footgun that such narrowly scoping is needed.

Going in the other direction, we could allow #[refine] on the impl itself. This would remove repetition in cases where an impl refines many items at once. It seems unlikely that this would be desired frequently enough to justify it.

Prior art

Discuss prior art, both the good and the bad, in relation to this proposal. A few examples of what this can include are:

• For language, library, cargo, tools, and compiler proposals: Does this feature exist in other programming languages and what experience have their community had?
• For community proposals: Is this done by some other community and what were their experiences with it?
• For other teams: What lessons can we learn from what other communities have done here?
• Papers: Are there any published papers or great posts that discuss this? If you have some relevant papers to refer to, this can serve as a more detailed theoretical background.

This section is intended to encourage you as an author to think about the lessons from other languages, provide readers of your RFC with a fuller picture. If there is no prior art, that is fine - your ideas are interesting to us whether they are brand new or if it is an adaptation from other languages.

Note that while precedent set by other languages is some motivation, it does not on its own motivate an RFC. Please also take into consideration that rust sometimes intentionally diverges from common language features.

Java covariant return types

If you override a method in Java, the return type can be any subtype of the original type. When invoking the method on that type, you see the subtype.

Auto traits

One piece of related prior art here is the leakage of auto traits for return position impl Trait. Today it is possible for library authors to stabilize the auto traits of their return types without realizing it. Unlike in this proposal, there is no syntax corresponding to the stabilized API surface.

Unresolved questions

• What parts of the design do you expect to resolve through the RFC process before this gets merged?
• What parts of the design do you expect to resolve through the implementation of this feature before stabilization?
• What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC?

Should #[refine] be required in future editions?

As discussed in Drawbacks, this feature could lead to library authors accidentally publishing refined APIs that they did not mean to stabilize. We could prevent that by requiring the #[refine] attribute on any refined item inside an implementation.

It would help to do an analysis of how frequently "dormant refinements" occur on crates.io today, and of a sample of those, how many look accidental versus an extended API that a crate author might have meant to expose.

If we decide to require the #[refine] annotation in future editions for all refinements, the only edition change would be that the lint in earlier editions becomes a hard error in future editions.

Alternatively, we may even want to require annotations for more subtle features, like lifetimes, while not requiring them for "louder" things like impl Trait in return position.

Future possibilities

Think about what the natural extension and evolution of your proposal would be and how it would affect the language and project as a whole in a holistic way. Try to use this section as a tool to more fully consider all possible interactions with the project and language in your proposal. Also consider how this all fits into the roadmap for the project and of the relevant sub-team.

This is also a good place to "dump ideas", if they are out of scope for the RFC you are writing but otherwise related.

If you have tried and cannot think of any future possibilities, you may simply state that you cannot think of anything.

Note that having something written down in the future-possibilities section is not a reason to accept the current or a future RFC; such notes should be in the section on motivation or rationale in this or subsequent RFCs. The section merely provides additional information.

Return position impl Trait in traits

One motivating use case for refined impls is return position impl trait in traits, which is not yet an accepted Rust feature. You can find more details about this feature in an earlier RFC. Its use is demonstrated in an example at the beginning of this RFC.

This RFC is intended to stand alone, but it also works well with that proposal.

Equivalence to associated types

One of the appealing aspects of this feature is that it can be desugared to a function returning an associated type.

trait Foo {
    fn get_state(&self) -> impl Debug;
}

// Desugars to something like this:
trait Foo {
    type Foo = impl Debug;
    fn get_state(&self) -> Self::Foo;
}

If a trait used associated types, implementers would be able to specify concrete values for those types and let their users depend on it.

impl Foo for () {
    type Foo = String;
    fn get_state(&self) -> Self::Foo { "empty state".to_string() }
}

let _: String = ().foo();

With refinement impls, we can say that this desugaring is equivalent because return position impl trait would give the exact same flexibility as associated types.

Replacing concrete types with argument-position impl trait

Sometimes it is desirable to generalize a concrete argument type to impl Trait or a new generic parameter. Adding generic parameters to a trait function is not allowed today, but in theory it could work as long as the parameter is defaulted. Implementing this may introduce complexity to the compiler, however.

Return position impl Trait in traits can easily be constrained to a concrete type in an impl. If and when we allow RPIT in traits, users may naturally expect that they can do the inverse for argument types.

We leave the question of whether to allow this for future RFCs.

Naming impl Trait generics

One related question is whether we will allow users to name the implicit generic parameter created by impl Trait in argument position inside of <>. You cannot do so today, but it remains undecided whether we will allow doing so in the future.

Thankfully, this extension does not hinge on that question. If we allow specifying APIT generics inside of <>, replacing a concrete argument type with impl Trait would be the equivalent of adding a defaulted new generic parameter.