# 2019.12.13 -- IDE / library-ification design meeting

## See also

* the compiler-team meeting request [rust-lang/compiler-team#222](https://github.com/rust-lang/compiler-team/issues/222)
* [Master hackmd](https://hackmd.io/9qrJYdSLTK2MWG4-pbmpWw?edit) that contains links to prior discussions

## Proposal

### Goal that we would be working towards

* Adopt rust-analyzer as the "official" IDE
* this would occur once certain milestones are met, likely early next year
* we freeze RLS afterwards
* Gradually address code duplication by extracting shared libraries used by both rustc and rust-analyzer
* initial candidates are described below
* it is not clear how merged rustc + rust-analyzer will eventually become, but certainly all "language logic" should be shared

### Steps towards adopting rust-analyzer as official LSP server

* Feature parity with RLS
* currently lacking errors, precise find-all-usages, and rename
* shortest route to achieve this is to integrate save-analysis
* longer term, we would be working on rust-analyzer having its own implementations, which would be available behind some kind of preference
* distribution via rustup (@matklad: not convinced it *must* be rustup. Distributing via VS Code marketplace might be better) (so that users don't have to build from source)

### Library-ification priorities

#### Near term

* Parser
* XXX matklad, how many details do we have worked out here? relatively few I think
* a possible appraoch is to make parser independent of the syntax tree, such that it produces events, and not a datastructure
* a known problem is that `rustc` token trees embed ast fragments (`TokenKind::Interpolated`)
* Chalk and core type inference logic (type equating, subtyping, etc)
* this will become a primary goal of #wg-traits
* plan is to be generic over precise representation of types via a trait called `TypeFamily`
* experimenting now to gain feedback, perhaps make this the topic of a dedicated future design meeting?
* one unknown: how will this interact with const generics and const evaluation?

#### Longer term

These are "upcoming candidates" for library-ification. It makes sense to wait on these until we have more progress on the ones above, as they are often building on those results. At present, rust-analyzer has "stubs" for them in any case.

* Macro expansion and name resolution
* Type checker
* MIR construction and optimization
* rust-analyzer (maybe?) doesn't need all of these
* though at least MIR construction may be important for later analyses; e.g., unsafety checks and borrow check are currently done on MIR
* Polonius
* exists, progressing slowly
* currently the set of "facts" are derived from MIR
* possible to derive them from HIR perhaps but

### Unknowns

* What should remain separate between rustc and rust-analyzer?
* It seems plausible that we will want to have the majority of language logic be shared, but offer different "shells" that are optimized for batch compilation vs IDE
* Example: rustc uses interned types, but that is not necessarily a good choice in an IDE
* Example: we are moving to a "streaming dep-graph model" in rustc, where the new dep-graph under construction never lives in memory and is streamed to disk; this is definitely not good for an IDE
* Library-ification will have to start here anyway, since we wish to extract libraries that are usable from both rustc + rust-analyzer as they exist.
* e.g. chalk is generic over how types are represented
* The only question then is how far we go

## Alternatives considered

* Start moving rustc as a whole to rust-analyzer architecture (persistent process, end-to-end fine-grained queries, multi-crate world)
* Upside: One source from the start
* Downside: It will be some time before users can derive value, whereas rust-analyzer is usable today
* Downside: rust-analyzer can move faster and be more experimental than current rustc, which makes it easier to figure out questions like "What should HIR look like in a highly incremental world"
* Downside: rust-analyzer currently contains "mock ups" of most parts of the compilation pipeline (e.g., name resolution, type checker, etc). This allows us to "library-ify" in any order and derive benefit (e.g., we can work on chalk without having done the type checker). If we tried to do everything in rustc, IDEs would only be able to benefit once we had a full incremental pipeline from the start
* Upside: due to "mock ups", rust-analyzer can't show errors even from reliable components.
For example, if chalk says "trait is not implmented", this might be due to a bug in macro exapnsion in rust-analyzer. rustc naturally has "compiler-grade" correctnesss from the start.
* However, we plan to report errors (initially) from rustc, and to be moving "towards" parity, with user opt-in, right (matklad: right!)? So it's not like the IDE won't show errors, is all I'm saying, but you won't get them as responsively as you might. --Niko

* Within current RLS architecture, replace `racer` with `rust-analyzer` as source of completions
* Upside: guarantee that we won't regress user experience
* Downside: a lot of engineering complexity
* Downside: large memory usage

## Discussion agenda

* Overview of the plan (15 minutes)
* Possible discussion topics (feel free to add ideas here, we'll review them before the meeting and try to choose the best ones)
* How much will making rust-analyzer "official IDE" interfere with its ability to move fast?
* Hope is to use "gates" to moderate this
* Who wants to co-lead the effort?
* Can/should we establish "quality criteria" for various milestones?
* e.g. to judge when we are ready to fully recommend rust-analyzer and remove RLS?
* or when to remove various "gates"?
* Naming and branding:
* Should we talk about RLS 2.0?
* At what point do we "advertise" this plan and how broadly?
* How much do we need to loop in core team, dev-tools team, etc?
* RLS maintenance:
* Should we keep RLS in a minimal working state in parallel? For how long?

## Follow-up items

* What is the best architecture to use to have rust-analyzer leverage rustc for precise queries, notably "find all usages"?
* Folks: eddyb, matklad, xanewok, maybe flodiebold etc
* This was a topic we spent a lot of the meeting on
* Library-ification specifics
* Parsers
* Chalk / type extraction