---
tags: turbowish
---

# Tokio Console Rewind/Replay

The current Console allows a nice *live* view of the application that it is monitoring. But sometimes things happen quickly, and a user may not have been able to react to inspect the data they wanted. In such scenarios, the user wants to be able to *go back in time*: replay the Console those events again, with controls to pause, step forward or backward, and more.

We envision a rich temporal feature set, but present it here in stages. Each stage builds new functionality on top of the previous stage. We expect to [launch][] with pausing, recording, and replaying.

[launch]: #Launch-Deliverables-2021
[post launch features]: #Post-Launch-Deliverables-2022]

The [post launch features][] described here are also desired by customers, but are not planned as part of the initial launch. (Want to help add them? [Come join us!](https://github.com/tokio-rs/console)).

## Launch Deliverables (2021)

### Pause Command


Sometimes while watching the tasks play live, a user notices something out of the ordinary, and wants to select that task, and investigate its details. But with a live view, the task state will change, which may disrupt a user's attempt to interpret the output. Worse still, a single task's entry in the table of tasks competes with all the other tasks updating and adjusting relative orders in the table.

The first stage of the time-travel feature is a **pause** command: a user presses the spacebar and this pauses all updates from incoming program events. They can still adjust the table, scroll, and view details, all while the updates are paused. Pressing spacebar again will reconnect the update channel and re-establish the live running state. It won't resume in real time from where it left off, but rather jump forward in time to reflect the current state of the application.

### Replay Command


Pausing doesn't help when an interesting event has already passed. You would need to travel back in time to investigate it. When a user knows that they need that feature, they will initiate the subscriber enabling an option to **record** all events to a file as they happen. The Console application will be able to then read those recordings, and **replay** them as if viewing the user's application live (including the ability to pause, as described above).

This also allows users to share recordings with other developers using the same version of the console, to help debug together. The user can also enable recordings in a benchmark, in production, or similar places where one might not normally actively view via the Console. The user can then replay the recordings at a later time.

The replays will emulate the original timing of the events: if the recorded application took 30 minutes, then the replay will also be 30 minutes. (Other time-travel commands later in this document mitigate potential issues here.)

The user has two ways to initiate replay: on the initial `console` command line invocation, and interactively during a live `console` connection.

#### Command Line Replay

The user can invoke the console application with a command line option that includes the path to the recording: `console --replay <path-to-recording>`. When replaying a recorded file, the console will not connect to a live application.

During command line replay, resuming after a pause (i.e. hitting spacebar) runs the recording from the point where it was paused.

#### Interactive Replay

The user can issue a replay command to a live application that is running with the record option enabled. This will read the events from the start of the recording and display them in the console.

The user inputs the replay command while interacting with a recording console via a dedicated command palette at the bottom of the console interface, typing `:replay`.

This behavior is almost exactly the same as passing the recording via the command line; however, in this mode, one is still connected to the live application, and thus can skip ahead to the "current time".

Therefore, there are two paths to resume a paused recording while attached to a live application. One can continue from the paused point in time, via the `:go forward` command (also written `:continue`, via analogy with a typical debugger user interface). Alternatively, one can jump forward in time to the live application state, via the `:go live` command. When paused while replaying a recording and attached to a live application, hitting spacebar is synonymous with `:go live` (i.e., it unpauses, and jumps forward in time, just the same as when one is paused and *not* recording).

## Post-Launch Deliverables (2022)

### Rewind Command


If a notable event occurs late in the execution, replaying a recording from the beginning is tedious method of investigation. To address this, we allow the user to dynamically **rewind** while viewing a recording or live process.

For maximum control, the user inputs the command via a dedicated command palette at the bottom of the console interface, where one indicates whether to step backward and forward and specifies by how much to step: `:step back 2s` steps backward two seconds, and `:step forward 2s` steps forward two seconds.

After a step completes, the console enters a paused state at that point in time in the recording. If the user wishes to resume forward execution, they can issue the command `:go forward`.

### Keyboard Macros

A command palette maximizes expressive power, but typing long commands can be a burden, especially for repetitive actions like stepping backwards repeatedly while searching for an event of interest. To help the user avoid unnecessary typing, console will provide user-definable short-cuts, so that user-defined keystrokes like Ctrl-B can act as a shorthand for full commands like `:step back 2s`.

To add a new shortcut, one uses the command palette, like so:

`:define key Ctrl-B :step back 2s`

### Watch Command


Stepping backwards to search for an event of interest, even with keyboard short-cuts, is a tediuous and potentially error prone process for users.

To address this, the console provides a **watch** command in the Console. The watch command operates analogously to watchpoints (a.k.a data breakpoints) in a typical debugger: when some data of interest changes, the system pauses.

The user specifies a task and property of the task to "watch" (in the screenshot above, we are watching TID=286 and the Polls property, via the command `:watch 286 polls`). With a watchpoint in place, the process will automatically pause when the value has changed, or when the value matches a certain simple expression.

This way, a user could be sure to catch every "poll" of a specific task, or catch the first time its poll time goes over a millisecond, as examples.

Watchpoints compose with the time-travel commands: One set up a watchpoint, and then issue the command `:continue back` to run in reverse to see the most recent time that data cell changed.