# pnkfelix:mw 1:1 meetings

[toc]

## 2022-02-14

### On DWARF

* pnkfelix's model, based on skim: looks complex, and yet maybe not sufficiently expressive
* big picture: DWARF specifies debuginfo in semi-language independent way, though there are extensions for many specific langauges and language families
* extensions are in the spec
* for example, some things in there look like they're just for C++
* looks like union of things needed for any language that's supported
* for thing specific to Rust, either identify something not currently used by us, and then ask debuggers to interpet it a certain way
* e.g. there is an inheritance tag, used by C++ and Java.
* we don't use it yet
* its used by Java to mean X implements an interface (and probably also class extension? mw not sure)
* so we might consider using that to mean X implements a trait.
* pnkfelix: is the grammar fixed?
* mw: not really a grammar. "DIE": a die can have these attributes
* e.g. would be nice if we could attach a name to a variant
* DWARF can specific discriminated unions, Pascal style.
* that's being reused in Rust in a way that's already supported in DWARF before.
* but Pascal-style unions don't have names
* pnkfelix: what about other languages with Algebraic Data Types, like ML or Haskell? Do any of them have support in DWARF
* mw: there is language tag for Swift.
* pnkfelix: do you know when DWARF support was implemented
* mw: I did stuff in Google SoC internship in 2013, but there was already some basic stuff there. I didn't start from scratch. Whole testing infrastructure was there in the form its still in today
* pnkfelix: might be nice to revisit that.
* re fixed grammar: semi-structured tree, but not level of Context-Free Grammar.
* things are quite uniform, at least at type level
* encoding of line-numbers and local variables, mw knows much less about this; its a black box within LLVM
* in LLVM, you just attach line number or source-file position at certain instructions, and LLVM does the rest for you.
* re variables: DWARF has its own lexical block structure for functions
* mw implemented that driven by AST structure
* pnkfelix: does DWARF have a notion of whether a variable is initialized or not
* mw: it definitely has a way of describing *how* a value should be decoded depending on position in control-flow
* mw: biggest complication there is getting the initialization information *through* all the layers
* mw: we produce *something* and then let LLVM optimizer run on it. So one has to keep the transformations that LLVM may perform in mind
* pnkfelix: are you saying that some of the info is effectively emitted on a side-channel that LLVM cannot observe? Or is all this stuff emitted in a way where one might expect LLVM to maintain the info, in an ideal world.
* mw: I think it works better for *clang*, which implies we're not using the APIs in the best way we could
* mw: not much work has been done on trying to achieve parity.
* What things to attack for Rust?
* mw: few things we don't describe at all, or not well
* which types implement traits
* description of fat ptrs is weird: we just say its a struct with two fields
* strictly speaking, doesn't *lose* information; a sufficiently smart debugger/pretty-printer is able to reconstruct what it needs
* pnkfelix: sounds super difficult to implement method dispatch with that foundtion
* (discussion of Greg from lldb's outline of implementation strategy)
* mw: things that make biggest differences for users
* make local variables show up
* nrc recently fwd'ed bug report of case where local variables are not showing up in unoptimized code
* going up the stack from a tail call, local variables not visible
* pnkfelix: might be inherent limitation of tail calls, at least for some contexts (e.g if stack frame was reused, those locals are not available to inspect anymore)
* worth investigating, in any case
* trait system support?
* mw: for high fidelity expression evaluation, probably need it
* mw: may not be high priority
* pnkfelix: counter-example: calling the Debug format routine from the debugger may be an important use case
* mw: may want something that's more reflective, suggested by niko
* pnkfelix: yes, I want this for e.g. hashtables
* mw: debuggers already support structured traversal. Advantage of reflection API is that the compiler automatically maintains the traversal/inspection code, versus approach of adding Python extensions where now every time you change the underlying data reprentation, you have risk of them going out of sync
* pnkfelix: counter-point: can the Python extension(s) be written in a robust way, to try to catch that?
* seeing concrete type of fat pointer is an important use case
* pnkfelix: a matter of mapping vtable to its corresponding type?
* mw: we already support *that*
* pnkfelix: so what's missing?
* mw: fat pointer already has a name that encodes what kind of fat pointer it is. So hypothetically the debugger *can* walk structure, but no one has implemented that (yet).
* mw: I want to do something like that for async support, but planning on doing it in Python for now.
* pnkfelix: I'd like to look at doing it on the native side.
* mw: I'd look first at whether gdb is doing this already, based on what tromney has said
* mw: also, beware: I removed support for this from some versions of the compiler, and it relanded only recently, PR 93503.
* mw: if DBT tool was easier to interface into CI, then you'd have something that would be easier to write unit tests for one's debugger extensions
* pnkfelix: CI in general is a huge miss for us
* mw: MCP to refactor some debuginfo in the compiler. Currently one big module that's grown messy over time.
* want tests that look at the actual DWARF, not the interpretation of that DWARF that certain debuggers produce
* the former is what LLVM does in its testing.
* pnkfelix: Greg Clayton has proposed doing more testing of Rust in the lldb CI
* pnkfelix: I'm very much in favor of this, but it will inject a new hurdle for any time we want to change our debugging strategy/output
* mw: yes. Tying down what our commitment should be important task for wg-debugging
* mw: part of that will be an RFC or maybe MCP when we make changes
* pnkfelix: pulling in stakeholder is an important
* pnkfelix: open question for me is how much difference in functionality there is between gdb and lldb

## Status of crashdump debugging

- Mostly clear what suspended async fns look like
- Mostly clear what tasks look like in smol/async-std
- just focused on smol (which async-std is built on top of) at first because Yosh is available for mw to talk to.
- haven't looked at smol/async-std in 4-5 months.
- much recent stuff is not dependent on executor: E.g. "what do Futures look like"; its largely coupled to the compiler, not the executor.
- Next immediate goal: Decode stack of suspended async fns (hope for Proof of Concept with four below sub-goals by end of February; WinDbg is important for internal goals, but gdb is also important for some users. Also good to have another debugger in the mix, just to guard against strong coupling.)
- Sub-goal: Map suspended fn (i.e. the "generator frame") to path of aysnc fn/block
- Sub-goal: Find awaitee in suspended fn ()
- Sub-goal: Find `file:line:col` of suspension point
- Sub-goal: Have visualizer/pretty printer that makes the above transparently available
- mw is working on fixing fundamental debuginfo blockers:
- Fat pointer debuginfo incomplete: (https://github.com/rust-lang/rust/pull/93006) (done)
- Type names of generic closures and async envs ambiguous: (https://github.com/rust-lang/rust/pull/93154) (waiting for review)
- VTable debuginfo not good enough (https://github.com/rust-lang/rust/pull/89597, https://github.com/rust-lang/rust/pull/93503)
- Suspended async fns/generators cannot inspected on MSVC (need to use same debuginfo-encoding as enums)
- aside: NatVis issue with niche layouts, but do generators even use niches? Not yet known.
- Awaitee field in async fns needs special name
- (i.e. the link to the value-receiver in the continuation)
- solution here is probably change the symbol chosen by the compiler to signal the purpose.

## Future topics not yet tackled

- `join!()` and `select!()` might profit from being moved to standard library, with debugger friendly implementation.
- Inter-task dependencies
- Dependencies on resources/reactors