# 2024 Workshop: Layered Actions

## Schedule

9:30
: Doors open at Blender HQ. Maybe also earlier, but Sybren will make sure he's there at 9:30.

10:00
: Start of the workshop. It'll be in the meeting room on the right when you enter.
- Introduce our workshop approach (based on [lessons from the previous workshop](https://code.blender.org/2023/07/lessons-from-the-2023-animation-workshop/)).
- Attitude towards gut feelings
- Topic Stack / "Stack it!" / silent
- "Durian!"
- Designated note taker (new Timmy every hour)

10:15
: Workshop Part 1
- Brainstorm/identify use cases, applications, and corresponding workflows.
- Which workflows do we aim for?
- Which workflow is going to be the main one for a MVP?

13:00
: Lunch time!

14:00
: Workshop Part 2
- UI/UX basics:
- Where/how do you see/select/create/organize/etc. layers? (E.g. side panel vs separate editor, and how does that work.)
- Do you work with multiple layers simultaneously, or only one?
- How do you see, filter, and work with the keys of multiple different layers?
- Consequences of our choices? Early limitations, most important focus points / features?
- How do tools interact?
- Which layer(s) do keys get inserted into?
- Which layer(s) do tools operate on?
- What do tools do when operating on multiple layers that mix together?
- Etc.
- How to deal with impossible situations (like working underneath a non-invertable layer)?
- What extra data is necessary, apart from "just bag of F-Curves"?

17:30
: Update BConf presentation slides with a quick summary of the workshop's outcomes.

18:00
: End of the workshop.

19:00
: Dinner at [Semai](http://semai.nl/). Not sure if Blender can cover the dinner costs, so we'll likely just split the bill with whoever joins.

## Notes

### Workshop Part 1

#### Use cases:

- Creating animation in layers:
- Base layer + shivering.
- Captured camera shake on top of base camera motion.
- Repeated motion / snippets + positioning in the world.
- Switchable/mixable animation (with controller) like games.
- Puppeteering / autokeying, new layer per loop
- Physics layer / not-keyframing
- Ragdoll physics (also for aligning snippets)
- Simulation with adjustments on top
- Secondary motion (wiggly bones)
- Animation-level Constraints
- Dynamic Overrides (but a bit of a cheat)
- Visualisation of caches etc. of current Blender simulation/cache/baking
- Import of data (USD/Alembic/FBX/glTF)
- Retiming of underlying layers
- Add-on-defined layer types
- Adjusting:
- ...someone else's messy animation (or your own a month later).
- ...mocap data.
- ...retargeted animation (mocap or otherwise).
- Sculpting animated stuff (layers for non-destructive workflow)
- F-Curve Modifiers
- Organizing:
- ...into separate concerns (e.g. body animation vs facial animation).
- ...into separate scene components (e.g. Mary vs Joe, or Christmas tree lights vs house lights).
- ...into separate takes / alternatives (for both animation and previs/layout).
- ...into "checkpoints" during the animation process (initial blocking, second pass blocking, final blocking, initial splining, etc.), so you can easily refer back to previous states.
- ...into separate exported anims (Grouping & marking layers for export).

#### Layers are... do...

- [x] are groups of keys
- [ ] are in control of other time-dependent values
- [ ] data-block chooser (replacing camera markers; choose in list of things)
- [ ] can emit events
- [x] can be active
- [x] can be selected
- [x] have a name
- [x] blended with underlying layers
- [x] replacing
- [x] combinator / conceptually additive
- [ ] other combine modes
- [x] data-type-aware (quaternion, colour, etc.) combinations
- [ ] have modifiers for its own data
- [ ] can be modifiers for underlying layers (procedural layers)
- [x] a way to do non-destructive editing / iterating
- [x] have animatable influence / weight
- [x] a way to reuse motion (like using mocap to get 'realism noise' on top)
- [ ] can be nested
- [x] can be baked down to underlying layer(s)
- [ ] merged (aka 'smart bake')
- [x] baked on every frame
- [x] have colours
- [x] have an order
- [x] can be re-ordered
- [ ] have evaluation order that is separate from display order (skypie / needs discussion)
- [x] are evaluated and drawn bottom to top
- [x] are not part of the existing anim UI (graph editor / dope sheet)
- [ ] can be time-warped
- [x] can be added & deleted
- [x] can be duplicated
- [ ] can be split, combined on a per-channel/control/object basis
- [ ] can be associated with controls / bones / objects
- [ ] and control their visibility (only show bones/controls on active layer, incl. motion paths)
- [ ] and control their visualisation (bone colour == layer colour, etc.)
- [x] can be locked (and unlocked)
- [x] can be muted
- [x] can be solo'ed (multiple layers can be solo'ed simultaneously)
- [ ] can be marked as 'up to here but no further'
- [ ] can be 'paused/frozen' at a specific point in time
- [ ] can define static 'rest pose' values of properties (probably only on bottomest layer)

## Big Questions

### Where do keys go?

- How do you select which layer to work on?
- How do you determine which keys go where?
- Press `I` to insert a key. What happens?
- Do we support selecting multiple layers?

### How does blending work?

- What happens when there is no underlying layer?
- Editing keys on middle layers
- What do the blend modes mean?
- How to do multi-value blending (transforms, colours, other stuff)
- What happens when you mute?

#### Workflows:

- Organization only.
- Layers are kept separate, and mix together to define the animation.
- Layers are used transiently to make adjustments, and are baked back down to a single layer.
- Non-dense workflow: don't want keys on every frame.
- Dense workflow: everything is keyed on every frame.

## General Topics

Potential questions for discussion:

- How much should we try to **pre-plan for strips** vs "we'll deal with that when we get there, let's just focus on layers now"?
- Same question for other topics as well:
- Animation-level constraints.
- Procedural animation.
- Etc.
- Do we go all in for "100%" solutions with a potentially more complex system, or aim more for "80%" solutions with a simpler system?
- Can we approach this with "80%" solutions that leave room to fill it in with "100%" or "95%" solutions down the road?
- How layer-aware should various tooling be? And how should the tooling behave then, and how does the user interact with that?
- Tools will have to be able to operate on **non-top layers**, and thus have some way to show their result in "fully evaluated space" while operating on "middle layer" space.
- Should tools work **across multiple layers**? And if so, what behavior(s) should that have? And how would an animator **control which layers** are (not) influenced?
- If we make the tooling as a whole generic in terms of layering, that may also be a good chance to transition to tooling that's generic across objects/bone/other properties.
- How should **animation mixing** work between layers? Should it be "dumb" and purely per F-Curve? Should it account for just a few special cases like quaternions? Should it have a broader awareness of spatial transforms, with different spaces?
- Also, what does that mean for mix modes? E.g. "add" is easy to define for per-fcurve mixing, but harder for fully transform-aware.
- How about colors?
- How to present **non-invertable** layer arrangements to the user?
- Example: if the user is trying to key on layer 3, but the layers above it are the "replace" mix mode with 100% influence, and therefore there is no way to map from the current scene values to key values on the layer.
- From a technical perspective, this is when the layer mixing is non-invertible for some reason (e.g. 100% replace, multiplying by zero, etc.).
- Can we have **strips of multiple types on the same layer**, or should each layer be for a specific strip type?
- This affects planned work on other Strip types. I.e. what does an additive layer mean for ID pointer animations. Think about animating the active camera on the scene.
- How should **hierarchical layers** work, and what are the implications for mixing, layer types, what is considered the active layer in various situations, etc.
- For example: should a group layer be its own special layer type that doesn't directly contain any animation data, and purely exists to group other layers? Or should it also be able to have its own animation data?
- How are Layers represented in **the GUI** and how does that work with existing editors like the Graph Editor.
- Show all layers in the Dope Sheet as summary channels?
- Separate layer editor to select the active Layer?