owned this note
owned this note
Published
Linked with GitHub
# 2019.11.06: RLS and rust-analyzer
Notes from discussion with matklad about libary-ification options on 2019.11.06
## “all in” on rust-analyzer
- only goal: bring rust-analyzer up to parity
- rustc is not actively developed while this happens
- danger: improving rustc moves us farther apart
- concern: rust-analyzer has no code-generating backend
## library-ification, deprecate RLS
- primary goal: extract into libraries that can be shared
- what happens to RLS?
- "freeze RLS"
- you must use rustc 1.39.0
- "minimal maintenance"
- just bug fixing and ICEs
- how much work is this? Igor can provide some input here
- "merging RLS, using rust-analyzer as a tool"
- matklad: this feels like a bad idea
- RLS maintenance is in part hard because it breaks together too many tools
- keeping them in sync is hard
- adding rust-analyzer will only make exacerbate this problem
-
- rust-analyzer focuses purely on IDE needs
- uses `cargo` or other builds to produce executable code
- goal: make rust-analyzer stop relying on `cargo check` to generate diagnostics; instead rust-analyzer is able to generate its own diagnostics (assuming appropriate support from libraries)
- candidates:
- query system (salsa etc) (75%)
- salsa:
- "multi-crate model", but this has potential perf costs
- references
- LRU
- rustc:
- zoxc work on streaming results
- rustc stores data to disk
- hashing approach that rustc uses
- parser (70%)
- syntax tree (either 0% or 75% understood)
- includes span information
- name resolution (0%)
- hygiene etc
- macro expansion (~90%??)
- hygiene
- type checker (50-75%)
- trait resolution (85%)
- longer term candidates:
- mir
- const-eval
- polonius
- question:
- mono repo vs poly repo
- do we try to blend RLS plus rust-analyzer?
- are people concerned about (excessive) factoring into library crates?
- can we architect things to enable quick checking of changes to crates like chalk etc
- e.g. so that you only have to rebuild chalk and relink (maybe recompile the driver)
- main features of RLS over rust-analyzer
- precise, inline error highlighting
- precise go-to definition and rename
- milestone: minimal viable rust-anayzer
- use "cargo check", but refactor so that it is run by the server itself, can deal with modified files
- milestone: rust feature completeness
- handle 90% of Rust
- for remaining 10%, no major architectural rewrites required. Currently missing things:
- name resolution and nested items is an example of a piece of missing design
- hygiene for declarative macros
- built-in macros like `concat_idents`
- cfg handling
- want to permit same crate but with two distinct sets of cfgs
- procedural macros
- but this interacts quite a lot with cargo, maybe worth pulling out
- build scripts (can be outsourced to cargo)
- have to figure out the output directory and make sure we supply it
- milestone: “rust-analyzer quality”
- proposal, pick some set of crates (e.g. regexp, servo, rustc)
- goal is that type checker should be >90% fidelity ?
- or 100% :)
- shortcoming:
- this does not address erroneous inputs
- milestone: rustc integration
- rustc uses librarified type-representation, at least for some subset of its type checking operation
question:
- do we expect long term to have two instantiations, one for throughput, one for latency?
- batch compilation:
- saving to disk, incremental updates
- interning of types
- IDEs:
- in-memory
- persistent memory across many compilations
- (=> must GC rather than intern forever)
## rustc replaces rust-analyzer
- i.e. rustc adopts lessons learned from rust-analyzer’s architecture
- key differences between rust-analyzer and rustc:
- rust-anayzer uses salsa,
- has an LRU mechanism for managing peak memory usage
- doesn’t store to disk
- exploratory step: queryifying save analysis
- exploratory step: creating queries for things like method completions
- things we need:
- persistent mode where rustc stays active to handle completions
- ability to parse source fragments (without starting from scratch for crate)
- multi-crate compilation world
- address interning + peak memory usage
- problems:
- long time until users of RLS see benefits ??
- maybe not true, depends a bit on the timing
- lots of work and hard to migrate compiler
- who is going to do this work?
- excitement level is lacking
# model one: two repositories that move together
libraries like type-checking that are "independent" of the context in which they are used, and there are two binaries
* batch context (rustc)
* IDE context (rust-analyzer)
* advantage: regular project, builds on stable, etc
* disadvantage: no test suite, etc etc
# model two: evolve rustc in place
there is one binary, we start out with a small "IDE-like" context (imagine an IDE that can do nothing but display a file's contents). and then incrementally "rebase" rest of compiler atop that context.
* parser + syntax tree
* then "rebase" name resolution (say) atop the syntax tree
* change parser so that it is independent of global state
* modify syntax tree
* but this model implies that we are building in the rustc repository
* with x.py, compiling llvm, building lots of stuff, etc
* advantage: test suite exists
# save-analysis
What parts of current RLS are worth salvaging?
What is role of save-analysis going forward?
* matklad: save-analysis is a useful piece of technology for various code-model tasks; but it is not a good solution for the IDE problem.
# ongoing efforts
* const generics, GATs
* definite interaction here
* mir optimizations
* adding specific optimizations
* adding debug info
* refactoring how MIR is represented
* parallel rustc + streaming work
* in model one, this isn't a problem because we can reconcile the two query systems later
* in model two, this is a problem
* (possible) bulding a semi-stable representation of rust sources
* macros 2.0 (stalled but...)
* polonius
*