2021-12-16

Pattern for this time

Broad / deep

• 10 minutes reading
• 5 minutes take notes
• 20 minutes of chat

Note-taking

While you read, take notes however you prefer. You can use this hackmd but you can also use paper or your brain.

At the end, we will take 5 minutes to leave some notes (in a section tagged with your handle) in this hackmd for others to persue/read. Note that while I don't plan to publish these notes, the hackmd is public and reachable by anyone who knows the url, so be polite.

We'll read them over and then I'll highlight particular people to expand on what they wrote.

Focus for this time

Early name resolution

Broad section

Prompts you might answer:

• Identify 3 interesting things that caught your attention.
• What is early vs late name resolution?
• What other pieces of code besides the resolver did you navigate to?
• What other sources of information (besides the code) did you find useful?
• What would be code you'd like to read more deeply?

Starting point for exploration

nikomatsakis

• What other pieces of code besides the resolver did you navigate to?
  • I spent a lot of time spelunking in the macro resolver. I like to have some idea of the "context" in which a function is used before I dive into what it does.
  • macro expansion appears to start at MacroExpander::expand_crate
    • which eventually calls resolve_macro_invocation
      • which eventually calls resolve_macro_path
        • which calls early_resolve_ident_in_lexical_scope
• What would be code you'd like to read more deeply?
  • I'd probably like to go deep on the fn we have and the data structures next, to understand how early_resolve_ident_in_lexical_scope works

---

• early_resolve_ident_in_lexical_scope is called from a number of places
• scope set parameter is interesting:
• callers
  • resolve_ident_in_lexical_scope–
    • walks up the "ribs" and then uses early_resolve_ident_in_lexical_scope
    • from this I infer that this is always a "module-level path"
    • and that this function is perhaps used for those
  • resolve_path_with_ribs
    • kind of the same
• resolve_macro_path
  • ah ha!
  • resolve_derives
  • smart_resolve_macro_path
    • resolve_macro_invocation
      • vscode can't find callers
      • in ResolverExpand trait
      • invoked by fully_expand_fragment
        • what is an ast fragment?
          • macro argument $x:expr I think
      • MacroExpander::expand_crate

estebank

• what are the differences with resolve_ident_in_lexical_scope? Why haven't they been unified yet?
• We don't memoize ident resolution within a scope? (This is likely not a material perf issue.)
• fn report_errors: we delay privacy errors for reporting after all resolutions have completed

(I gotta run)

jyn514

• "Late" seems strange given the comment "enum is not used for late resolution yet" - what's going on there?
• why are macros special-cased?
• AbsolutePath is giving me nightmares
• it confuses me every time that "Module" is not actually a module and just a scope - maybe it should be renamed?
  • see ModuleKind in resolve/lib.rs
  • why is there also ModuleOrUniformRoot aaaaaaahhhh
  • is ModuleData interned?? comparing pointers in same_def looks sketchy
• record_used is related to speculative resolution I think
• doesn't Ident contain a span? why is there a separate path_span argument?
  • seems to only be used for resolve_ident_in_module?
• what is the difference between record_used and force?
• why should self and super error?
• there are a lot more macro scopes than I expected (enum Scope in lib.rs)
• PROC_MACRO_DERIVE_RESOLUTION_FALLBACK is related to backwards compatibility hacks I think?
• oh, that's what "lexical" means - this is order-dependent
• it looks like early_resolve_ident and resolve_ident_in_module are mutually recursive
• maybe "resolve_ident_in_global_lexical_scope" is a better name? not always called in early pass

---

resolve_ident_in_lexical_scope

• why is the value ns treated differently for purposes of normalizing span?
• wish we had Alexis King on the call
• interesting that the compiler knows and disallows some usages (e.g. outer variables in a nested fn item)
• oh oof I didn't even think about constant names for const {} blocks
• looks like most of this is just a lookup to self.resolutions? that's constructed in build_reduced_graph_external - where does it happen for local items?
  • on-demand maybe? NameResolution is complicated
  • try_define looks promising
  • called from update_resolution, resolve_glob_import, define
  • define is called from resolve_import, visit_variant, … a whole bunch of stuff in BuildReducedGraphVisitor actually
    • build_reduced_graph_for_item is where most non-macro calls go through
    • why is resolve_visibility so complicated?

"ContainingScope" might be better than "Module"

AngelOnFira

(Cloning the Rust repo)

• Looking at visit_scopes
  • Matches scope type
  • Scope includes macros and modules
  • Match on the result
    • Ambiguity errors?
• Pipeline
  • Figure out where it is
  • Figure out if it's ambiguous
  • Resolve

resolve_ident_in_lexical_scope

• normalize_to_macros_2_0?
• At the end
  • Why map to an enum varient?

metajack

• what is difference between ScopeSet and ParentScope, does the former contain the latter?
• NameBindingKind::Res?, NameBinding ambiguity
• on error case, seems to add some error reporting but returns the inner binding; seems like maybe the "weakness" of bindings should be controlled more centrally have helpers
• visit pattern is nice here, since the code doesn't have to worry about how to walk the tree, just evaluate specific scopes. visit_scopes takes care of the ordering for us
• adjusting scopes? L788

spastorino

early_resolve_ident_in_lexical_scope -> resolve identifier in lexical scope.
Is used for resolving initial segments of macro paths, could be unified with resolve_ident_in_lexical_scope. Wonder what is needed for that to happen.
ScopeSet -> All, AbsolutePath, Macro, Late
record_use and force, what are those things?
Called from resolve_macro_path and finalize_macro_resolutions in this file.
This is also called from imports.rs, thought it was only to resolve macros.

ranweiler

• Seems like a lot of work is around detecting ambiguity
• Interesting that it is intertwined with fix suggestions
• Need to read disambiguate_macro_rules_vs_modularized()
• Scope::DeriveHelpersCompat… edition/proc macro support gore?

lqd

• what's "forcing" ?
• is "record_used" for some of the unused use lints ? (when would you not want it to be recorded ?)
  • answer: various speculative uses, e.g. diagnostics, rustdoc, where you don't want to trigger side effects and things

HugoPeters1024

• When are derivings undetermined?
• Meta: Would this function be better if broken up into smaller pieces/is the current amout of documentation (especially one line comments) as desired or could there ideally be more?
  • Probably not worth breaking it up, it mostly different arms of a match

doc

• Looking at the comments it looks like early resolve is imports and macros.
• Late resolve resolves the names that are not imports and macros.

noted: &mut self allows changes without taking ownership.

• Would like to know more about "getting a fresh section of AST". What precipitates? Is that the scope stuff? Feeling lost as usual. :)

your-handle-here

(copy and paste this to add your section)

Deep section

Function to compare and contrast:

rustc-resolve

nikomatsakis

• I found I wanted to dig in a bit to where the ribs come from and what they are used for
• Observations:
  • still think my basic hunch was correct:
    • early_resolve_path_with_lexical_scope manages names at the "module level"
    • the ribs and things are used for other scopes beyond that "global" scope
  • resolve_path -> resolve_path_with_ribs -> resolve_path_with_lexical_scope
  • the middle function (resolve_path_with_ribs) walks down a path
    • seems a bit surprising because a::b::c, resolving a is completely distinct from resolving b and c..?
    • lots of complexity around self:: and super:: at start of path
  • ribs are assembled by the LateResolutionVisitor (e.g., here)

---

• ribs
• omg ModuleKind::Block is definitely a confusing name (I think Joshua referenced this…)
  • also, interesting that there are modules for enums etc, even though name resolution doesn't search them normally; this is I guess how we manage use Enum::Variant; comment mentions associated types, but afaik resolution of associated types uses a different mechanism
• as we walk up the ribs:
  • lots of kinds of ribs
  • these seem to primarily be used to "shut off" different kinds of names
  • e.g. labels from outer functon are not in scope within a closure
  • MacroDefinition says "We passed through a macro_rules! statement", but is it also for macros 2.0
• falls back to "early resolve" with ScopeSet::Late(ns, module, record_used_id)
• caller resolve_path_with_ribs
  • that is a fairly complex function! got a bit lost in it
• interesting comment I did not understand
  • https://github.com/rust-lang/rust/blob/5381442abe7d0ee7f2006867ae0dbf52ec9c1934/compiler/rustc_resolve/src/late.rs#L774-L790

ranweiler

• Fallback final call to early_resolve..
• Surprised at `assert
• What's future_proof about?
• How can a resolution be "non-binding?"

metajack

• shadowing example amused me greatly
• type ns vs value nst
• looking in validate_res_from_ribs
  • jyn514: sometimes we allow you to name things that you're not allowed to actually use, e.g. from a closure you can't name things in out scope, and then we can give you a useful error
• what are upvars?
  • answer: captured variables outside of rust
• blocks are anonymous modules, interesting. so are enums and traits apparently.
  • really some kind of "name holder"
• checks for local, checks in modules, then if nothing goes to early_resolve_*

HugoPeters1024

• I've been seeing a lot of <'a> which hints at GATs, isn't that an unreleased feature?
  • not GATs
  • jyn514: https://doc.rust-lang.org/nomicon/hrtb.html
• if having a ribs list a common pattern? Wouldn't it make sense to abstract the traversal when searching for things?

trait Iterable {
    type Item;

    type Iterator<'a>: Iter<Item = &'a Self::Item>;
    //          ^^^^^
    
    fn iter(&'a self) -> Self::Iterator<'a>;
}

doc

• Focusing on collecting participant comments and observations to create blog post/retrospective for this session.
• Collecting your comments is helping me understand or record items for further reading
• One question: for the things the group identifies as improvements needed should we move those to issues in the repo? Or perhaps take the list to a compiler meeting?

spastorino

early_resolve_ident_in_lexical_scope can be called during expansion and import resolution. resolve_ident_in_lexical_scope must only be called during main resolution, not during import resolution.

resolve_ident_in_lexical_scope

fn main() {
    g(); // Since there are no local variables in scope yet, this resolves to the item.
    let g = || { println!("binding")};
    fn g() { println!("item") }
    g(); // This resolves to the local variable `g` since it shadows the item.
}

Called from resolve_path_with_ribs and in a bunch of places in late resolution. Called from resolve_path, called from resolve_ast_path. Comment that says "Resolve a path passed from rustdoc or HIR lowering.".

enum LexicalScopeBinding<'a> {
    Item(&'a NameBinding<'a>),
    Res(Res),
}

items are visible in their whole block, while Reses only from the place they are defined.

your-handle-here

(copy and paste this to add your section)

could be a good candidate

• build reduced graph visitor

Useful links

Updates and retrospectives

Repo for the name resolver

The rustc dev guide on name resolution

Repo for the Rustc Reading Club: it may need some updating. :) You can submit topic suggestions and make requests of the organizers by adding an issue here.