Reading notes:

Link to document: https://cliffle.com/blog/lildb/

Attendance

• People: TC, zmitchell, eholk, yosh

Terminology

yosh: I'm surprised they're coming up with new terminology in this post. What they're referring to as an "await-trace" I've always known as a "logical trace"? As opposed to I guess "physical stack trace", which is the way "normal" stack traces work.

yosh: I think I first saw this pop up when Chrome added support for async debugging in devtools in ~2016 or so?

zmitchell: Are there places outside of async that a logical trace would show up?

yosh: Anything that suspends, moves, and then resumes somewhere else. You could imagine a trace across multiple threads being "logical". A lot of distributed tracing things are able to create traces across client/server/database verticals as well. The meaning of "trace" becomes a bit dependent on the context there, but I feel the term can be applied more broadly.

Missing pieces

zmitchell: What are the missing pieces preventing us from writing an async-debugger today, or adding support for async-specific debugging to something like gdb/rust-gdb?

• Query the existing tasks

yosh: This is the work mwoerster was working on before going on parental leave. It's in-progress. He should have a good overview of the exact state, but iirc you do indeed need to write debugger-specific plugins.

TC: Here's a link to him describing his work:

https://internals.rust-lang.org/t/async-debugging-logical-stack-traces-setting-goals-collecting-examples/15547

TC: As linked there, here's the document in our own repo about the debugging experience:

https://github.com/rust-lang/wg-async/blob/master/src/vision/submitted_stories/shiny_future/grace_debugs_a_crash_dump_again.md

TC: And here's a link to the initiative:

https://github.com/rust-lang/async-crashdump-debugging-initiative/tree/master
https://github.com/rust-lang/rust-dbg-ext

yosh; Wesley is the other expert on debuggers in Rust.

yosh: I wrote "Rust should own its debugger experience" to cover some of the shortcomings I see in making this a fully integrated experience. Right now we're somewhat dependent on upstreaming things like async integration, symbol translation, etc. to e.g. gdb/lldb/windbg, etc. In my opinion if we want to really provide a "first-class async debug experience", one of the required steps is for cargo to be able to run the debuggers (maybe even package them too), insert the right plugins, and then provide that for users. There are a lot of moving parts required to make this like, actually turnkey and easy to use.

"intrusive tracing" terminology

yosh: canonical terms I've seen used for this:

• intrusive tracing -> "static tracing" (requires annotations)
• non-intrusive tracing -> "dynamic tracing" (does not require annotations)

source: Brendan Gregg - "Systems Performance and the Cloud (first edition)"

How would you go about writing your own debugger?

Zach: How would you?

Eric: Debuggers already exist. We could probably extend those to be able to create the logical traces using their plugin API

Zach: Is there a specific module you could point me at where I could start looking at?

Eric: I believe "generator.rs"?

Eric: Here it is: https://github.com/rust-lang/rust/blob/master/compiler/rustc_mir_transform/src/generator.rs

Eric: That's pretty low level, so it may not be the best place to start if you're just trying to get a high level understanding, but it probably has the details you'd need to find the interesting debug info, or at least will point you in the right direction.

DWARF extensions?

To fix this, I think Rust needs to add some DWARF extensions, something they’ve managed to avoid so far (admirably!). But that’ll be a harder problem.

eholk: This was kind of an interesting claim, but something I'm currently skeptical of. Could Rust do better if it were willing to make what amounts to a custom symbol format and its own debugger?