Parallel documentation questions

Internals thread

Ask your questions here!

What are the major shared data structures that must be synchronized? (Niko)

How is interning pool synchronized? (Niko)

  • type interning
  • string interning (uses TLS…)

locked hashmaps, basically

simulacrum: do we know what the contention on these is like? Are they a bottleneck?

What is the role of the jobserver and how does integration work? (Niko)

How does the LLVM threading get kicked off, and how does that interact with the query threading? (Niko)

That too, please, e.g. from the discussion above it wasn't clear why -t and -j options are separate, I though the total number of "tasks" (both threads and processes) are controlled by the jobserver. (petrochenkov)

How does rayon request jobserver threads (simulacrum)?

​​​​* when do we release them back?

What happens if a threads asks for a query that is being computed elsewhere currently? (acrichto)

the thread blocks until the query completes

Deadlock handling: rayon detects when all threads have gone to sleeps and signals that a deadlock has occurred.

acrichto: shouldn't work stealing happen at this point?

niko: yes and no :) it's complicated, this is a current sol'n. Basically you have to avoid stealing the wrong thing, which could induce artificial deadlock.

How do we deal with thread-local state? What kind of thread-local state is passed around? How does that work with Rayon threadpool threads? (mw)

Documentation outline

Select a repo