###### tags: `DevNotes`

<style>
.ui-infobar, #doc.markdown-body { max-width: 1000px; }
</style>

# Notes on Realtime Clock, Anim playback, Depsgraph

## Questions
- Will it be necessary to step the realtime clock with substeps? If this were to work for the realtime clock it would also have to work for animation, which currently only steps in full frames. Substeps are currently only performed locally per-object.
_Substeps should be performed per object/modifier/simulation/etc. There is no need to update the entire scene in sync for each substep._
- Animation playback is currently enabled for just one screen at a time. Is this necessary for the realtime clock as well? What's the reason?
_Original idea was to have different states for each screen, but is not implemented. Realtime clock will (for now) share the timer with scene animation, so it will use the same Screen data._
- Should the realtime clock have a separate frame rate? For now it will use the same FPS setting as the animation clock.
- How to ensure that animation and realtime clock are in phase when using the same delta time?
- Can a single timer (`Screen.animtimer`) be used for both animation playback and realtime stepping? Yes if both are using the same frame rate, then they would automatically be synchronized (in phase).
Theoretical issue: Delay of up to 1 frame when starting one clock while the other is already running. Probably no problem in practice for reasonably high fps.
Technical problem: Existence of the timer is used to detect running animation, so we wouldn't be able to distinguish based on just that any more.
_No, realtime clock will use the same timer and be in sync with scene animation._
- Should simulations on the realtime clock be cached?
Probably not since the realtime clock is never really set back, other than for actually resetting the simulation, at which point the starting conditions might be totally different so caches would generally be invalid.
How long could caches get to avoid running out of space, since there is no built-in frame end?
Caching should still work if the sim is locked to the animation clock, or if baked (also uses anim frame range).
_For realtime clock only the previous frame is cached._

## :warning: HACKZ!!!

Questionable stuff i found:

- RNA functions for `Screen` don't use the PointerRNA and instead use the active window and its first screen with a running timer. Probably because only one screen should ever have a running timer? It hints at bad design ...
- `animtimer` in DNA is accessed outside of editors.
The animation timer _should_ be a pure editor feature, but there are two places where the `bScreen` DNA struct is accessed directly to look at the `animtimer`:
1. `BKE_sound_seek_scene`: Animation playing simplifies audio scrubbing (don't fully understand this yet).
2. `seq_prefetch_is_playing`: Sequencer prefetch is disabled during animation playback.

Both of these do not have access to the actual editors module, but they (incorrectly) use the DNA pointer as a shortcut.
- `ScreenAnimData` (and the new `ScreenRealtimeData`) has a `region` pointer which is used to identify the View3D that should be refreshed during animation playback. If the screen is redrawn (due to settings changes that trigger through RNA, see `rna_Screen_redraw_update`), the region is reset and arbitrarily picks the top left 3D view (`time_top_left_3dwindow`).
- `bScreen::scrubbing`, apparently a Cosmos Laundromat hack (yay!)
- `DURIAN_CAMERA_SWITCH` in `ED_update_for_newframe` (hooray!)

## How the animation update is scheduled during animation playback

- The animation play button is the `SCREEN_OT_animation_play` operator, which calls `ED_screen_animation_play` (as do many other functions, including py scripts through RNA).
- `ED_screen_animation_play` registers a timer with `ED_screen_animation_timer` => `WM_event_add_timer`.
- Timer update happens by a `wmEvent` of type `EVT_DATA_TIMER` being sent to the window. The event type is `TIMER0`.
- `SCREEN_OT_animation_step` operator is invoked through the keymap, which has a `"TIMER0"` entry (`blender_default.py`, `industry_compatible_data.py`, etc.) that matches the timer event.
- `ED_update_for_newframe` does the scene frame change through the animation_step operator. It is also called in case of `ND_FRAME` notifiers, which are handled after events.

## How the depsgraph performs updates on frame changes
- Animation clock updates only in full frame increments (timer delta is 1/FPS).
- Depsgraph does **not** store the last update time. Physics simulations have to keep track of the last update time themselves to calculate the delta. E.g.:
- `RigidBodyWorld::ltime`
- `ParticleSystem::cfra`
- `Cloth::last_frame`
- Relations to the `TimeSourceNode` are used to define time-dependent updates, usually through the use of a `TimeSourceKey`.
The `eUpdateSource` type of depgraph updates is not really used, i.e. `DEG_UPDATE_SOURCE_TIME` is not so relevant. It's only used to distinguish from `DEG_UPDATE_SOURCE_USER_EDIT` updates together with `RELATIONS` and `VISIBILITY` updates.
- May need a different time source that can be distinguished from timeline/animation time sources.

## How does "reset" work?
- `SCREEN_OT_animation_step` increments frame beyond the end frame (`scene.r.efra` or `scene.r.pefra`). The `ANIMPLAY_FLAG_JUMPED` flag in the timer customdata is set.
- Scene is tagged in the depsgraph with `ID_RECALC_FRAME_CHANGE`.
- When the `ID_RECALC_FRAME_CHANGE` flag is set `DEG_evaluate_on_refresh` will tag the scene time source node.
- Individual physics sims check of the current frame matches the start frame, e.g.
- Cloth: [`if (framenr == startframe) {...}`](https://projects.blender.org/blender/blender/src/commit/3ea5a8fbb919b05d035c92e5b7b1adae183dc9bb/source/blender/blenkernel/intern/cloth.cc#L359)
- DynPaint: [`if (int(scene->r.cfra) == surface->start_frame && !(cache->flag & PTCACHE_BAKED)) {...}`](https://projects.blender.org/blender/blender/src/commit/3ea5a8fbb919b05d035c92e5b7b1adae183dc9bb/source/blender/blenkernel/intern/dynamicpaint.cc#L2132)
- Particles: [`if ((cfra == startframe) || (psys->recalc & ID_RECALC_PSYS_RESET)) {...}`](https://projects.blender.org/blender/blender/src/commit/0256bfd3094ec0ff222977bdf47fd5619c509582/source/blender/blenkernel/intern/particle_system.c#L4482)

For the realtime clock there is no extra "start frame", but since the clock time only ever increases a value of zero could be used to detect a reset.

## Modifying `screen->animtimer` customdata to support both clocks
`screen->animtimer` is the timer currently used for regular animation steps during playback. To avoid unnecessary depsgraph updates it should also be used for the realtime clock, so both clocks run at the same `Scene.RenderData.frs_sec` fps value and only one timer event gets triggered per frame.

Consequences:
- Only one depsgraph update per frame (excluding extra updates through operators etc.)
- Starting anim playback or realtime clock may be delayed up to 1 frame due to the timer already running.
- Have to modify the `animtimer` customdata (`ScreenAnimData`) since it serves both clocks.
- Existence of the timer is used in many places to mean that the animation playback is running. This needs to be replaced to distinguish the animation and realtime clocks.

typedef struct ScreenAnimData {
// Used during the animation step operator to redraw the timeline region.
// Won't be needed for the realtime clock unless we also need regular redraws there?
// If this is not set to a specific region the redraws flag is used
// to determine which region types to redraw.
  ARegion *region; /* do not read from this, only for comparing if region exists */

// User-setting for which editors get redrawn during animation playback (see bScreen::redraws_flag)
// Probably also not needed?
  short redraws;

// REVERSE, SYNC, etc., animation playback specific
  short flag;                 /* flags for playback */

// Specific to animation playback
  int sfra;                   /* frame that playback was started from */

// Specific to animation playback
  int nextfra;                /* next frame to go to (when ANIMPLAY_FLAG_USE_NEXT_FRAME is set) */

// Might have something equivalent for realtime clock
// if updates take too long and the time step is larger
// than the nominal fixed step
  double lagging_frame_count; /* used for frame dropping */

// Specific to animation playback
// Used only to override some filters when finding the regions to redraw
// (so some region types always redraw during animation edit)
  bool from_anim_edit;        /* playback was invoked from animation editor */
} ScreenAnimData;

## Accessible DNA flags to indicate which clocks are running
With different clocks now using the same `wmTimer` this is not enough information to determine if animation playback is running. A flag inside the `ScreenAnimData` customdata won't be accessible outside of editors (needed for sound and sequencer, see [HACKZ](#-hackz).

A solution would be to put the flags for which clocks are running _outside_ of the timer customdata:

typedef struct bScreen {
  ...

  /** If set, screen has timer handler added in window. */
  wmTimer animtimer;
  /** Clocks that are currently running on the animdata timer. */
  int active_clock;
};

/** #bScreen.active_clock. */
enum {
  /** Animation playback. */
  ANIMTIMER_ANIMATION = (1 << 0),
  /** Realtime clock. */
  ANIMTIMER_REALTIME = (1 << 1),
};

The will indicate running clocks to code outside of the editor, without requiring in-depth knowledge of editor structs. It can be used in place of just checking the `animdata` pointer.

Old:

  if (screen->animtimer) {
    bool is_playing = true;
  }

New:

  if (screen->active_clock & ANIMTIMER_ANIMATION) {
    bool is_playing = true;
  }

The `animtimer` should always exist if `active_clock != 0`

## Screen functions that modify the timer but don't care about customdata

- `ED_screen_refresh` : general screen refresh
- `ED_screen_exit`: remove timer when closing screen (customdata is MEM_free'd)
- `ED_screen_change`: timer is moved to the new screen
- `ED_screen_state_toggle`, `ED_screen_state_maximized_create`: timer is moved to new screen state
- `ED_refresh_viewport_fps`: only uses generic timing info from `wmTimer`

## Functions that need changing

- `ED_screen_animation_timer`: General method of setting up the timer
- `ED_screen_animation_timer_update`: Modify the timer flags when settings are changed through RNA
- `frame_jump_exec`: Frame jump operator
- `screen_animation_step_invoke`: Main step function when the timer triggers to increment the frame

## `animtimer` in transform data

`TransInfo` also stores a copy of the `animtimer` pointer. It uses it to detect if playback is running and to check if the animation jumped, which causes some NLA ... stuff to happen (`animrecord_check_state`).

## `animtimer` for Panel animation
- `uiHandlePanelData` also has a `animtimer` but this is totally separate from animation playback.

## "Continue Physics" feature
This is an old feature that did something similar to a realtime clock, according to Brecht.

Removed here: https://projects.blender.org/blender/blender/commit/8b8d4ba7effcf73d72e224d2934b6ad0f3a5ee74
No mention of the feature in old 2.4 docs: https://archive.blender.org/wiki/index.php/Doc:2.4/Manual/Physics/Particles/Cache_And_Bake/

## Differences between DEG frame updates and other updates
After frame changes: `BKE_scene_graph_update_for_newframe_ex`
After other changes: `BKE_scene_graph_update_tagged`, `BKE_scene_graph_evaluated_ensure`

<style>
n { background-color: LightGray }
y { background-color: Orange }
</style>

||Frame update|Other update|
|---|---|---|
|can be skipped if DEG is fully evaluated|:x:|:heavy_check_mark:|
|callbacks (only on 1st pass)|`FRAME_CHANGE_PRE`, `FRAME_CHANGE_POST`|`DEPSGRAPH_UPDATE_PRE`, `DEPSGRAPH_UPDATE_POST`|
|`BKE_image_editors_update_frame`|:heavy_check_mark:|:x:|
|`BKE_sound_set_cfra`|:heavy_check_mark:|:x:|
|`DEG_graph_relations_update`|:heavy_check_mark:|:heavy_check_mark:|
|`prepare_mesh_for_viewport_render` for editmesh (BMesh-to-Mesh, tag ID)|:x:|:heavy_check_mark:|
|evaluate depsgraph|1st pass: `DEG_evaluate_on_framechange`, then `DEG_evaluate_on_refresh`|all passes: `DEG_evaluate_on_refresh`|
|`BKE_scene_update_sound`|:heavy_check_mark:|:heavy_check_mark:|
|`DEG_graph_relations_update` (to remove potential dead pointers from DEG)|:heavy_check_mark:|:heavy_check_mark:|
|break if fully evaluated (`DEG_is_fully_evaluated`)|:heavy_check_mark:|:heavy_check_mark:|
|After 1st pass: backup recalc flags and clear (`DEG_ids_clear_recalc`)|:heavy_check_mark:|:heavy_check_mark:|
|After >1 passes: restore recalc flags (`DEG_ids_restore_recalc`)|:heavy_check_mark:|:heavy_check_mark:|
|Update editors (`DEG_editors_update`)|:heavy_check_mark: (`time=true`)|:heavy_check_mark: (`time=false`)|
|Clear recalc flags again (`DEG_ids_clear_recalc`)|:heavy_check_mark: (optional)|:heavy_check_mark:|

## Depsgraph `frame` and `ctime`

These are the last frame/time values when a depsgraph was updated, copied from the scene. Since we now have multiple time sources it makes more sense to move this into the time source nodes, so each clock is stored separately. Setting the depsgraph time can also automatically tag the correct nodes.

## Notifiers for time-dependent depsgraph updates

Currently only `ND_FRAME` (in `NC_SCENE` category) exists to give operators a way to trigger time-dependent depsgraph updates. It is handled in `wm_event_do_notifiers` and calls `ED_update_for_newframe`.

The new realtime clock and potential other clocks need a similar mechanism to trigger updates when a respective time value changes. Currently it also uses the ND_FRAME notifier, which tags the wrong time-source node in depsgraph.

## Cache invalidation

The simulation nodes cache needs to be flagged as _invalid_ before it actually gets cleared (so it won't get cleared if the scene timeline jumps to frame 1). This happens in the depsgraph update by checking the `DEPSOP_FLAG_USER_MODIFIED` flag of the modifier (`DepsgraphNodeBuilder::build_object_modifiers`).

The realtime cache is only ever reset explicitly, through an operator/button. This operator needs to find all the relevant caches and flag them as invalid. It would be nicer if an observer pattern existed so that caches could subscribe to the reset event instead of putting logic for finding all caches into the operator, but this is not something Blender supports at this point.

The animation clock caches only get cleared when jumping back to the start frame, so the cache can be invalid yet still contain lots of old frames. The realtime cache is always cleared as soon as it is invalidated. This may be preferable behavior for the animation clock too?