###### tags: `library-overrides`

# Library Overrides: Technical Overview

*This page gives a technical overview of how library overrides are
implemented. The goal is to only give an overview of what is already in
the code-base, but does not go into the exact details how it is
implemented.*

*More details on current development can be found on the [Overrides
project page](https://developer.blender.org/project/profile/141/).*

## File Layout {#file_layout}

- The core of library overrides are implemented in BKE\'s
\`BKE_lib_overrides.h\` and \`lib_overrides.c\`.
- The DNA structures are defined in \`DNA_ID.h\`.
- The RNA-level property flags are defined in \`RNA_types.h\`, and the
relevant API, in \`RNA_define.h\` and \`RNA_access.h\`.
- The RNA-level diffing and applying functionality is implemented in
\`rna_access_compare_override.c\`.

## What is a Library Override - Data Structure & Key Concepts {#what_is_a_library_override___data_structure_key_concepts}

Library Overrides are defined and stored at the ID level, in their own
sub-struct \`IDOverrideLibrary\`.

{{Note|Embedded and non-linkable IDs|Embedded (root node groups, master scene collections...) and non-linkable (shape keys...) IDs do not have their own library override struct. Instead, they are considered (from overrides point of view) as a sub-data of their owner ID, which will also store their overridden properties. They are flagged with `LIB_EMBEDDED_DATA_LIB_OVERRIDE`, and are also sometimes referred as 'virtual overrides' in the code.}}

Library overrides are essentially made of a reference to their linked ID
source (\`IDOverrideLibrary.reference\`), and of a list of
\`IDOverrideLibraryProperty\`, each representing a different RNA
property (stored as an RNA path) in the owner ID.

Each of these \'override properties\' in turn store a list of
\`IDOverrideLibraryPropertyOperation\`, each defining a specific
override operation to apply to that property.

Typically, there is one operation per property (usually simply replacing
the value of that property), but in some cases, like RNA collections
that support insertion of new items, there can be several ones.

**Library overrides only store override operations, not override
operands.** In other words, they do not actually store any data. Data is
stored in the local override ID, and this ID is used as source of the
override operations when re-generating the override from its reference
linked data (e.g. on file load).

What can or cannot be overridden is defined as part of the RNA
properties definition, in their own set of flags
\`PropertyOverrideFlag\`. Note that those flags also control the
[diffing behavior](#Detecting_Local_Changes_-_Diffing "wikilink")

### Hierarchies

Usually more than one ID needs to be overridden in some form of
\'group\'. A typical production character for example is made of
hundreds of IDs, and a lot of them need to be overridden. Since in most
cases their is a clear \'root\' to this dependency tree, it is also
called an \'override hierarchy\'.

All overrides must have a valid (non-NULL)
\`IDOverrideLibrary.hierarchy_root\` pointer (roots point to
themselves). Overrides that are **not** part of a hierarchy are flagged
with \`IDOVERRIDE_LIBRARY_FLAG_NO_HIERARCHY\`.

This strong hierarchy definition allows overrides from different
hierarchies to have relationships (e.g. one override of a linked
character can be parented to another override of the same linked
character), and generally help with complex hierarchical processes like
[resyncing](#Resync "wikilink").

### System Overrides vs. User Overrides {#system_overrides_vs._user_overrides}

System Overrides are overrides that are created because they are needed
to build the whole hierarchy, i.e. for technical reason.

The user does not need to actually edit those IDs. They are therefore
essentially non-editable in the UI, and are not expected to have any
override properties/operations besides the system-required ID pointer
ones.

System overrides are flagged with
\`IDOVERRIDE_LIBRARY_FLAG_SYSTEM_DEFINED\` in
\`IDOverrideLibrary-\>flag\`.

## Processes Overview {#processes_overview}

This section briefly describes the process of the most common library
overrides operations.

### Making Library Override {#making_library_override}

To make a local override of a linked data-block, a local copy of the
linked data is created and a library override structure is initialized
(see \`BKE_lib_override_library_create_from_id\`).

In almost all cases, overrides are not created in isolation, but as part
of an override hierarchy (either the whole hierarchy, or some sub-tree
being added to an existing override hierarchy).

{{Note|Note|This process effectively always makes the linked data 'directly linked' (i.e. its `LIB_TAG_INDIRECT` `ID.tag` is always cleared).}}

### Making Local Changes {#making_local_changes}

A user can work with the local override in a similar (though restricted)
way as with a regular local ID (using the UI, operators, the python API,
etc.).

One of the main restriction currently is that Edit modes are not allowed
on local overrides. Many operators are also checking for override case
in their \`poll\` callback.

#### Lists (RNA Collections) {#lists_rna_collections}

The general case is that lists (a.k.a. RNA collections) are not editable
for local overrides, so you cannot add, remove, or re-order their items.

There are a few exceptions (modifiers, constraints and NLA tracks),
where user can add new items, and re-organize those new items. Deletion
or re-organization of existing items from linked data is currently
forbidden.

Items which have been inserted in the local override (and are therefore
\'purely local\') must be tagged as such, so that editing code can
distinguish them from those coming from the linked ID. There is no
common way to do it currently, all use different flags
(\`eModifierFlag_OverrideLibrary_Local\` for modifiers\' flag,
\`CONSTRAINT_OVERRIDE_LIBRARY_LOCAL\` for constraints\' flags, etc.).

### Detecting Local Changes - Diffing {#detecting_local_changes___diffing}

Local changes are detected during Undo step generation (to provide UI
feedback to the user about what is overridden,
\`BKE_lib_override_library_main_operations_create\` called from
\`BKE_undosys_step_push_with_type\`), and before saving the file on
disk. This is called the \'diffing\' process.

For each RNA property that is different between the original linked data
and the local override one, an \`IDOverrideLibraryProperty\` and one or
more operations are stored in the override structure.

For simple data properties (like float, int, string, enum) a single
replace operation (\`IDOVERRIDE_LIBRARY_OP_REPLACE\`) is added. For
vector types (color, coordinates), there can be several operations for
some individual elements, or a single one for the whole array.

In case the diffing process finds a difference in RNA properties that
are not allowed to be overridden, it will attempt to restore the
value(s) from the linked reference data.

The default diffing process is implemented by
\`rna_property_override_diff_default\`. In case a specific RNA property
require non-standard diffing process, it can define its own callback,
and pass it to \`RNA_def_property_override_funcs\`.

#### Lists (RNA collections) {#lists_rna_collections_1}

When inserting a new item to a list the
\`IDOVERRIDE_LIBRARY_OP_INSERT_AFTER\` operation is used. It tracks the
name of the element to insert (from the local override data), and the
name of the one after which it will be inserted (the \'reference\', or
\'anchor\', item).

**Order of insertion operation is very important**, as a future insert
operation may reference (use as anchor) a previous inserted item, and
not necessarily an item coming from the linked ID.

### Writing Local Changes {#writing_local_changes}

When writing a blend file the local override ID is stored just like a
regular local data-block, with one key difference. The data stored on
disk is size-optimized by removing some potentially heavy data sets that
will never be overridden. E.g. with meshes, no geometry is written to
disk (neither vertices, edges and faces, nor their custom data layers,
can be overridden, see \`mesh.cc::mesh_blend_write\`).

The referenced linked ID is stored like any other directly linked
data-block (i.e. its data is not written on disk, only its name and
source library file).

During writing the local override is checked to ensure that it contains
all the necessary override operations
(\`BKE_lib_override_library_main_operations_create\` called from
\`wm_file_write\`).

### Reading Local Changes {#reading_local_changes}

When reading a blend file one of the last steps is to update the
overrides.

This is done by calling \`BKE_lib_override_library_main_update\`, at the
end of \`blo_read_file_internal\`. This function will go over all the
IDs that have a library override structure, and:

- Make a new local copy from their linked reference.
- Apply the override operations stored in the old override (just read
from disk) to the new copy, using the old override as source data.
This is handled in RNA code (see \`RNA_struct_override_apply\`).

This ensures that the local override data is always following as close
as possible the data from their linked reference.

If an override property RNA path is not valid anymore, it is ignored (a
message is printed in the console currently), and will be cleaned-up
during next diffing execution.

If the linked reference ID is not found, there will be no update and the
local override is kept as loaded from the .blend file. This allow to
preserve as best as possible the override data, when some library goes
missing.

{{Note|Note|The library overrides can be chained, i.e. an override from another .blend file can be linked, and a new override created from this linked data. This is handled in `BKE_lib_override_library_update`, by checking if the reference linked data is also an override, and if so, recursively calling `BKE_lib_override_library_update` on it first.}}

### Resync

Resync is an operation to update the local override hierarchy when it
does not match the hierarchy of its linked reference data anymore (i.e.
when relations between IDs have been added, changed or removed in the
linked library data). See \`BKE_lib_override_library_resync\` for manual
resync, and \`BKE_lib_override_library_main_resync\` for auto-resync.

Auto-resync is by default run as part of \`setup_app_data\`, after
reading the .blend file.

#### Resync Conflicts & Resolution {#resync_conflicts_resolution}

Resync conflicts will happen when reloading a work file with existing
library overrides, and the reference linked data does not match the
local overrides anymore.

The linked data-block remains the same, but its content changed (e.g. some modifiers were added or removed).

:\* The situation is similar to animation's FCurves or drivers: some
override properties may become invalid (their RNA path would not match
anything in override data).

:\* Currently, this is simply ignored, and invalid override properties
are cleaned up on next file save.

A new linked data-block is added, or its relationships to others are modified, and it should now become overridden as well.

:\* This is fully automatically handled by the partial resync code,
roughly:

:\*\* The affected parts of the hierarchy are re-created from the linked
data.

:\*\* Overrides from the old pre-resync data is ported over, as best as
possible, to the post-resync data.

An existing linked data-block is replaced by another one, or is removed.

:\* **System override:** If the override operation matching the usage of
the replaced/deleted ID is tagged with
\`IDOVERRIDE_LIBRARY_FLAG_IDPOINTER_MATCH_REFERENCE\` (i.e. this
operation is a [\'system override\'
one](#System_Overrides_vs._User_Overrides "wikilink")), the handling is
similar as above.

:\* **User override:** If the existing override was user-edited and
diverges from the linked-defined hierarchy, it will remain unchanged,
unless \`do_hierarchy_enforce\` is specified (only possible in manual
resync case).