# 2021-12-16

## Pattern for this time

Broad / deep

* 10 minutes reading
* 5 minutes **take notes**
* 20 minutes of chat

## Note-taking

While you read, take notes however you prefer. You can use this hackmd but you can also use paper or your brain.

At the end, we will take 5 minutes to leave some notes (in a section tagged with your handle) **in this hackmd** for others to persue/read. Note that while I don't plan to publish these notes, the hackmd is public and reachable by anyone who knows the url, so be polite.

We'll read them over and then I'll highlight particular people to expand on what they wrote.

## Focus for this time

**Early name resolution**

## Broad section

Prompts you might answer:

> * Identify 3 interesting things that caught your attention.
> * What is early vs late name resolution?
> * What other pieces of code besides the resolver did you navigate to?
> * What other sources of information (besides the code) did you find useful?
> * What would be code you'd like to read more deeply?

[Starting point for exploration](https://github.com/rust-lang/rust/blob/7734cb80782e65875994ee4709b7752623ea9ed2/compiler/rustc_resolve/src/macros.rs#L637-L645)

#### nikomatsakis

* What other pieces of code besides the resolver did you navigate to?
* I spent a lot of time spelunking in the macro resolver. I like to have some idea of the "context" in which a function is used before I dive into what it does.
* macro expansion appears to *start* at [MacroExpander::expand_crate](https://github.com/rust-lang/rust/blob/5381442abe7d0ee7f2006867ae0dbf52ec9c1934/compiler/rustc_expand/src/expand.rs#L365)
* which eventually calls `resolve_macro_invocation`
* which eventually calls `resolve_macro_path`
* which calls `early_resolve_ident_in_lexical_scope`
* What would be code you'd like to read more deeply?
* I'd probably like to go deep on the fn we have and the data structures next, to understand how `early_resolve_ident_in_lexical_scope` works

---

* `early_resolve_ident_in_lexical_scope` is called from a number of places
* scope set parameter is interesting:
* callers
* [resolve_ident_in_lexical_scope](https://github.com/rust-lang/rust/blob/5381442abe7d0ee7f2006867ae0dbf52ec9c1934/compiler/rustc_resolve/src/lib.rs#L1910)--
* walks up the "ribs" and then uses `early_resolve_ident_in_lexical_scope`
* from this I infer that this is always a "module-level path"
* and that this function is perhaps used for those
* [resolve_path_with_ribs](https://github.com/rust-lang/rust/blob/5381442abe7d0ee7f2006867ae0dbf52ec9c1934/compiler/rustc_resolve/src/lib.rs#L2200)
* kind of the same
* [resolve_macro_path](https://github.com/rust-lang/rust/blob/5381442abe7d0ee7f2006867ae0dbf52ec9c1934/compiler/rustc_resolve/src/macros.rs#L553)
* ah ha!
* resolve_derives
* smart_resolve_macro_path
* resolve_macro_invocation
* vscode can't find callers
* in ResolverExpand trait
* invoked by fully_expand_fragment
* what is an ast fragment?
* macro argument `$x:expr` I think
* [MacroExpander::expand_crate](https://github.com/rust-lang/rust/blob/5381442abe7d0ee7f2006867ae0dbf52ec9c1934/compiler/rustc_expand/src/expand.rs#L365)

#### estebank

* what *are* the differences with `resolve_ident_in_lexical_scope`? Why haven't they been unified yet?
* We don't memoize ident resolution within a scope? (This is likely not a material perf issue.)
* `fn report_errors`: we delay privacy errors for reporting after all resolutions have completed

(I gotta run) :cry:

#### jyn514

* "Late" seems strange given the comment "enum is not used for late resolution yet" - what's going on there?
* why are macros special-cased?
* AbsolutePath is giving me nightmares
* it confuses me every time that "Module" is not actually a module and just a scope - maybe it should be renamed?
* see `ModuleKind` in resolve/lib.rs
* why is there also `ModuleOrUniformRoot` aaaaaaahhhh
* is ModuleData interned?? comparing pointers in `same_def` looks sketchy
* `record_used` is related to speculative resolution I think
* doesn't `Ident` contain a span? why is there a separate `path_span` argument?
* seems to only be used for `resolve_ident_in_module`?
* what is the difference between `record_used` and `force`?
* why should `self` and `super` error?
* there are a *lot* more macro scopes than I expected (`enum Scope` in lib.rs)
* `PROC_MACRO_DERIVE_RESOLUTION_FALLBACK` is related to backwards compatibility hacks I think?
* oh, that's what "lexical" means - this is order-dependent
* it looks like `early_resolve_ident` and `resolve_ident_in_module` are mutually recursive :eyes:
* maybe "resolve_ident_in_global_lexical_scope" is a better name? not always called in early pass

---

`resolve_ident_in_lexical_scope`

* why is the value ns treated differently for purposes of normalizing span?
* wish we had Alexis King on the call
* interesting that the compiler knows and disallows some usages (e.g. outer variables in a nested `fn` item)
* oh oof I didn't even think about constant names for `const {}` blocks
* looks like most of this is just a lookup to `self.resolutions`? that's constructed in `build_reduced_graph_external` - where does it happen for local items?
* on-demand maybe? `NameResolution` is complicated
* `try_define` looks promising
* called from `update_resolution`, `resolve_glob_import`, `define`
* `define` is called from `resolve_import`, `visit_variant`, ... a whole bunch of stuff in BuildReducedGraphVisitor actually
* `build_reduced_graph_for_item` is where most non-macro calls go through
* why is `resolve_visibility` so complicated?

"ContainingScope" might be better than "Module"

#### AngelOnFira

:christmas_tree:

(Cloning the Rust repo)

- Looking at visit_scopes
- Matches scope type
- Scope includes macros and modules
- Match on the result
- Ambiguity errors?
- Pipeline
- Figure out where it is
- Figure out if it's ambiguous
- Resolve

`resolve_ident_in_lexical_scope`

- `normalize_to_macros_2_0`?
- At the end
- Why map to an enum varient?

#### metajack

- what is difference between ScopeSet and ParentScope, does the former contain the latter?
- NameBindingKind::Res?, NameBinding ambiguity
- on error case, seems to add some error reporting but returns the inner binding; seems like maybe the "weakness" of bindings should be controlled more centrally have helpers
- visit pattern is nice here, since the code doesn't have to worry about how to walk the tree, just evaluate specific scopes. visit_scopes takes care of the ordering for us
- adjusting scopes? L788

#### spastorino

`early_resolve_ident_in_lexical_scope` -> resolve identifier in lexical scope.
Is used for resolving initial segments of macro paths, could be unified with `resolve_ident_in_lexical_scope`. Wonder what is needed for that to happen.
`ScopeSet` -> All, AbsolutePath, Macro, Late
`record_use` and `force`, what are those things?
Called from `resolve_macro_path` and `finalize_macro_resolutions` in this file.
This is also called from `imports.rs`, thought it was only to resolve macros.

#### ranweiler

- Seems like a lot of work is around detecting ambiguity
- Interesting that it is intertwined with [fix suggestions](https://github.com/rust-lang/rust/blob/7734cb80782e65875994ee4709b7752623ea9ed2/compiler/rustc_resolve/src/macros.rs#L935-L945)
- Need to read `disambiguate_macro_rules_vs_modularized()`
- `Scope::DeriveHelpersCompat`... edition/proc macro support gore?

#### lqd

- what's "forcing" ?
- is "record_used" for some of the unused use lints ? (when would you not want it to be recorded ?)
- answer: various speculative uses, e.g. diagnostics, rustdoc, where you don't want to trigger side effects and things

#### HugoPeters1024

- When are derivings undetermined?
- Meta: Would this function be better if broken up into smaller pieces/is the current amout of documentation (especially one line comments) as desired or could there ideally be more?
- Probably not worth breaking it up, it mostly different arms of a match

#### doc

- Looking at the comments it looks like early resolve is imports and macros.
- Late resolve resolves the names that are not imports and macros.

noted: &mut self allows changes without taking ownership.

- Would like to know more about "getting a fresh section of AST". What precipitates? Is that the scope stuff? Feeling lost as usual. :)

#### your-handle-here

(copy and paste this to add your section)

## Deep section

Function to compare and contrast:

[rustc-resolve](https://github.com/rust-lang/rust/blob/5381442abe7d0ee7f2006867ae0dbf52ec9c1934/compiler/rustc_resolve/src/lib.rs#L1910)

#### nikomatsakis

* I found I wanted to dig in a bit to where the ribs come from and what they are used for
* Observations:
* still think my basic hunch was correct:
* `early_resolve_path_with_lexical_scope` manages names at the "module level"
* the ribs and things are used for other scopes beyond that "global" scope
* `resolve_path` -> `resolve_path_with_ribs` -> `resolve_path_with_lexical_scope`
* the middle function (`resolve_path_with_ribs`) walks down a path
* seems a bit surprising because `a::b::c`, resolving `a` is completely distinct from resolving `b` and `c`..?
* lots of complexity around `self::` and `super::` at start of path
* ribs are assembled by the `LateResolutionVisitor` (e.g., [here](https://github.com/rust-lang/rust/blob/5381442abe7d0ee7f2006867ae0dbf52ec9c1934/compiler/rustc_resolve/src/late.rs#L755-L772))

---

* ribs
* omg `ModuleKind::Block` is definitely a confusing name (I think Joshua referenced this...)
* also, interesting that there are modules for enums etc, even though name resolution doesn't search them normally; this is I guess how we manage `use Enum::Variant`; comment mentions associated types, but afaik resolution of associated types uses a different mechanism
* as we walk up the ribs:
* lots of kinds of ribs
* these seem to primarily be used to "shut off" different kinds of names
* e.g. labels from outer functon are not in scope within a closure
* `MacroDefinition` says "We passed through a `macro_rules!` statement", but is it also for macros 2.0
* falls back to "early resolve" with `ScopeSet::Late(ns, module, record_used_id)`
* caller `resolve_path_with_ribs`
* that is a fairly complex function! got a bit lost in it
* interesting comment I did not understand
* https://github.com/rust-lang/rust/blob/5381442abe7d0ee7f2006867ae0dbf52ec9c1934/compiler/rustc_resolve/src/late.rs#L774-L790

#### ranweiler

- Fallback final call to `early_resolve..`
- Surprised at `assert
- What's [`future_proof`](https://github.com/rust-lang/rust/blob/7734cb80782e65875994ee4709b7752623ea9ed2/compiler/rustc_resolve/src/late.rs#L905) about?
- How can a resolution be ["non-binding?"](https://github.com/rust-lang/rust/blob/7734cb80782e65875994ee4709b7752623ea9ed2/compiler/rustc_resolve/src/late.rs#L1753)

#### metajack

- shadowing example amused me greatly
- type ns vs value nst
- looking in validate_res_from_ribs
- jyn514: sometimes we allow you to name things that you're not allowed to actually use, e.g. from a closure you can't name things in out scope, and then we can give you a useful error
- what are upvars?
- answer: captured variables outside of rust
- blocks are anonymous modules, interesting. so are enums and traits apparently.
- really some kind of "name holder"
- checks for local, checks in modules, then if nothing goes to early_resolve_*

#### HugoPeters1024

- I've been seeing a lot of ``<'a>`` which hints at GATs, isn't that an unreleased feature?
- not GATs
- jyn514: https://doc.rust-lang.org/nomicon/hrtb.html
- if having a ribs list a common pattern? Wouldn't it make sense to abstract the traversal when searching for things?

trait Iterable {
    type Item;

    type Iterator<'a>: Iter<Item = &'a Self::Item>;
    //          ^^^^^
    
    fn iter(&'a self) -> Self::Iterator<'a>;
}

#### doc

- Focusing on collecting participant comments and observations to create blog post/retrospective for this session.
- Collecting your comments is helping me understand or record items for further reading
- One question: for the things the group identifies as improvements needed should we move those to issues in the repo? Or perhaps take the list to a compiler meeting?

#### spastorino

`early_resolve_ident_in_lexical_scope` can be called during expansion and import resolution. `resolve_ident_in_lexical_scope` must only be called during main resolution, not during import resolution.

`resolve_ident_in_lexical_scope`

fn main() {
    g(); // Since there are no local variables in scope yet, this resolves to the item.
    let g = || { println!("binding")};
    fn g() { println!("item") }
    g(); // This resolves to the local variable `g` since it shadows the item.
}

Called from `resolve_path_with_ribs` and in a bunch of places in late resolution. Called from `resolve_path`, called from `resolve_ast_path`. Comment that says "Resolve a path passed from rustdoc or HIR lowering.".

enum LexicalScopeBinding<'a> {
    Item(&'a NameBinding<'a>),
    Res(Res),
}

items are visible in their whole block, while `Res`es only from the place they are defined.

#### your-handle-here

(copy and paste this to add your section)

## could be a good candidate

* build reduced graph visitor

## Useful links

[Updates and retrospectives](https://mojosd.medium.com)

[Repo for the name resolver](https://rustc-dev-guide.rust-lang.org/name-resolution.html?highlight=name%20reso#basics)

[The rustc dev guide on name resolution](https://rustc-dev-guide.rust-lang.org/name-resolution.html)

[Repo for the Rustc Reading Club](https://github.com/rust-lang/rustc-reading-club): it may need some updating. :) You can submit topic suggestions and make requests of the organizers by adding an issue here.