# Bevy Preferences API

Games and apps need a way to store user settings such as window size, graphics options, sound and music volumes and so on. These settings are typically per-user, and are neither assets (in the traditional sense) nor saved games.

The term "preferences" or "settings" should not be confused with "configuration file" which is a more general concept. The scope of this design is strictly configuration files which are written by the game itself and used on subsequent runs, not something that is produced by a different program.

Preferences may be set by the end user *explicitly* (such as adjusting a music volume slider) or *implicitly* (like the fact that the user has already seen the tutorial and doesn't need to see it again), but in all cases are the result of user interaction.

### Preferences and the Bevy Editor

A reasonable question might be, "why are we talking about adding preferences to Bevy, when there are third-party solutions available?" The answer is that, like it or not, the Bevy Editor is going to need some kind of preference system and this can't be a third-party crate because that's not how Bevy rolls.

If we are going to do that, then we might as well make something flexible enough that it can be used for games and apps generally. However, there's a counter-argument: if the editor's preference framework has a complex API for supporting property overrides and file watching, which most games don't need, then it might be better for them to use a simpler solution.

(**Controversy**: It has been suggested that it might be better to have two different preferences frameworks: one for the Bevy editor, and one for games. The argument is that the requirements are different. For example, there's no concept of "project-specific" settings in a regular game, which means that the API can be much simpler.)

### A Word about VSCode

VSCode in particular, and IDEs in general, tend to have very complex settings files compared to other kinds of apps. And, moreover, just about every game developer uses an IDE and so is accustomed to working with complex settings files so it doesn't seem like a big deal. But VSCode and its ilk are outliers, and not typical apps.

I think we should be cautious about using VSCode's approach to settings as a model, as this will result in a very ambitious design.

### Previous Discussions

This has been discussed extensively here: https://github.com/bevyengine/bevy/pull/13312

In particular, cart has stated a set of requirements here:

https://github.com/bevyengine/bevy/pull/13312#issuecomment-2108921673

(Although I am in disagreement with one of cart's points, which we'll get to.)

## General Requirements

### Cross-Platform

All modern operating systems have designated locations for application preferences. For games that run in a browser, user preferences can either be saved in browser local storage, or stored on a server.

The preferences framework should adhere to the platform conventions for preference configuration. For example:

* **Linux**: `~/.config/APP_NAME`
* **Windows**: `~\AppData\Roaming\APP_NAME`
* **MacOS**: `~/Library/Preferences/APP_NAME`

The `directories` crate has methods to access all of the standard preference locations (and many other standard directory locations).

For browser-based games, the preferences framework should be able to abstract away the differences between filesystem-based configuration and browser storage APIs. This means that you can write a game that works on any platform, without having to use feature flags to control where preferences are stored.

On other platforms (console, handheld, and so on) it should be possible to have preferences automatically stored in "the proper place" for that platform.

What we want to avoid is having the developer have to write special saving code for each platform; particularly in light of the fact that oftentimes, solo developers only know about their own platform (the one they use for development). A Windows user might not know the proper place for storing preferences on Mac or Linux. Even professional companies get this wrong sometimes - linux home dirs tend to be littered with "dot-directories" that don't conform to the platform standards for configuration.

### Serialization Formats

**Controversy**: I'm of the opinion that we should be opinionated about formats in order to enforce consistency across the Bevy ecosystem; but others don't want to be restricted in this way.

An earlier version of this proposal supported the idea that the app developer could choose whatever serialization format they wanted; however, I think this makes the design of the preferences framework more complex for not much benefit.

Up to this point I have been limiting it to just two formats:

* For desktop platforms, TOML
* For web platforms, JSON

The reason for choosing JSON is that this is the format used on the web for just about everything. If we're going to be storing serialized strings in local storage, it's probably best to use a format that is easily parsed with built-in browser APIs.

TOML was chosen because of a couple reasons:
* Semantically, it's almost identical to JSON - that is, both formats support the same set of primitive types. This means we don't have to worry about different semantics on different platforms.
* At the same time, it's easier for humans to read and edit than either JSON or RON files.
* TOML is frequently used in desktop apps as a settings format, and somewhat resembles traditional config file formats such as Windows ".ini" files.
* TOML files are familiar to Rust programmers, since that is what Cargo uses.

A compromise solution is to have a "default" serializer that uses conditional compilation to choose the most idiomatic format for the platform (JSON on wasm targets, and TOML on desktop), but which can be replaced with a serializer of the user's choice. How difficult this is remains to be seen.

### Supporting Library Crates

It should be possible for libraries and plugins to define their own preference settings. These settings will usually be stored in the same configuration file as the preferences for the main application, albeit in a separate section.

For example, a crate which implements a "color picker" widget might want to store the list of recent colors. A debug inspector that has filtering might want to preserve the filter settings between runs. This preference should automatically be included in the app's preferences configuration as a side-effect of using the library, without requiring the app developer to explicitly include it.

This means that libraries and plugins will need a mechanism to save and load their own preference types with the framework.

Third-party crates can control what information gets stored in the preferences configuration, but they can't generally control where and when it gets stored, something that is usually determined by the application and the preferences framework.

It should also be possible for libraries which don't have support for preferences built-in, to be configured by the app-level preferences. Again, the typical example here is the `primary_window` field in Bevy's `DefaultPlugins` plugin group. We probably don't want `DefaultPlugins` to read/write preferences on its own, but it should be relatively easy for the app to read preferences for window size and mode and pass that information to the plugin.

It should also be possible for an app developer to choose *not* to have a settings file, even if they use a library plugin that supports settings. This means that library plugins have to be written in a way such that preferences are optional.

### Uniqueness Between Apps / Games

The app developer should be able to specify the root name of the preferences config file or directory, so that it doesn't collide with preference files from other games from different authors.

One idea is to recommended developers use the *reverse domain name* (e.g. "org.bevy") of a domain they own as the base directory name for the preferences path, similar to what is commonly done in Java with imports. So for example, on Linux my game might store preferences in `~/.config/com.mydomain.mycoolapp/settings.toml`. Using domain names is an easy way for developers to get a globally-unique name that's easy to read, without us having to maintain some kind of global registry.

Things get a bit more complex when we are talking about preferences for library crates. Consider something like the Bevy Inspector. Multiple games may include the inspector, but we generally don't want the inspector settings for game A to overwrite the inspector settings for game B. To avoid this, we can either store the inspector settings in the same config file as the game's settings, or another file in the same directory.

We want to avoid collision between preference keys defined by different libraries. This can be done by incorporating the crate name in the preference group keys. So for example, in the settings file for the game, you might see a section like:

[bevy_inspector]
filter = "camera"
sort_by = "type"
visible = true

### Multiple Namespaces

Although most preferences will be stored in a single configuration file, it may be desirable for certain groups of preferences to be stored in their own dedicated configuration file. An example might be `screenshots.json` or `characters.toml`. This would be stored in the same directory as the primary settings file.

### Fallback configurations / Inheritance

The vast majority of games and apps only require a single level of preferences, stored per user. However, editors often have "per-project" settings which can override global "per-user" settings. (There are also "per-machine" settings, but in practice these are rarely used in games and editors.)

This kind of inheritance significantly complicates the design of a preference system; ideally most apps should not have to pay the price of this additional complexity.

For example, take a game that has a setting for audio volume. This is something that you want to set once and forget about it; it wouldn't make sense to set it "per project" or "per character" or "per game save".

But if the preference system supports "per project" settings, then even a simple game has to specify, when setting a property, the scope of that property, because the API requires it. (Wendy Carlos's law: "Whatever you can control, you must control.")

Inheritance also means we have to use a sparse representation for the settings objects in memory; we can't just serialize regular rust structs, it's got to be a map of dynamic values or something.

### Bootstrapping

It must be possible to load preferences before plugin initialization. This is because some settings such as window size or display mode need to be initialized once, rather than being initialized to a default setting and then modified later.

Quoting from cart:

> Preferences must be available when a plugin initializes.

> This cannot/should not be initialized with defaults and then later fixed up when preference file(s) are loaded. Some "app init" things cannot / should not be "fixed up" later (as a "normal init flow"), such as GPU device configuration and window configuration.

This means that we will need to be able to load the complete set of preferences before the main game loop starts, or before the Bevy world is fully initialized.

**Unresolved** There are two different approaches we can take, which are described in the following sections:

#### Option 1: Loading Preferences before `App::new()`

In this design, preferences are loaded in `main` before the app is constructed. If preference loading is async, we'll have to `await` before creating the `App`.

This means we won't have access to ECS or the Bevy World during loading, so any loaded prefs data has to exist independently of the world. We can, however, insert this data into the world (as, say, a `Resource`) once the world exists, for convenient access from running systems.

For example, in `bevy_prefs_lite`, I have a `PreferenceStore` object which is loaded early in main, and then inserted as a `Resource` later. All preferences live in that one resource (which has implications for change detection.)

It also means that we won't have access to the `AppTypeRegistry` during loading (we can construct a new registry, but it will only contain the automatically derived types, not any types that are manually registered.)

An advantage of this approach is that it's relatively easy to pass in preferences as constructor parameters to plugins: for example, passing in window size and position to `DefaultPlugins` is trivial. Trying to modify these properties after the plugin has been constructed is significantly more involved.

The main downside of this approach is that we can't use any ECS infrastructure during the loading process - for example, we can't iterate through resources looking for the ones tagged as preference items. In fact, the preference loader can only deserialize into a generic, dynamically-typed tree (since it lacks type information to do more than this), which is then converted into Rust types as individual properties are accessed.

In this approach, there's no automatic saving of resources or other ECS items, all getting and setting of preference items is manual. This requires the app developer to write additional boilerplate code, however this code is fairly easy to write.

#### Option 2: Two-phase app initialization

The other option is to construct a minimal `World` that's not fully initialized, use that world to load preferences into it, and the complete the initialization.

This means that we can use ECS and the type registry when loading.

This approach carries some risks because it means that app and library developers have to be careful to do things in the right order. If a configuration setting is not "fixup-able" - meaning that it can only be set once - then we have to ensure that this happens after preferences are loaded, and not before.

In current Bevy, developers can add plugins, insert resources, add systems and observers, and so on, without worrying to much about the order in which these are done; they can group together things logically (adding plugins in one place, registering systems in another). In a two-phase world, however, developers need to start thinking about which phase a given initialization step needs to happen in. Here are some examples:

* Registration of generic types in the type registry needs to happen in the early phase so that those types can be used by the preference loader.
* Window size and position need to be set after preference are loaded.
* GPU configuration parameters that cannot be fixed up later also need to happen in the later phase.

This constraint on ordering gets even more involved if we decide to tightly integrate settings with resources. One design uses Bevy resources to represent individual sections or groups within the preferences file, with a reflect annotation to associate a resource with a given group. However, these resources have to be initialized to their default values and inserted in the world in the early phase, so that they can be queried during loading.

#[derive(Resource, Default, Reflect)]
#[reflect(Default, @PreferencesGroup("zoom"))]
pub struct Zoom {
    pub level: f32
};

This would produce a TOML file that has the following entry:

[zoom]
level = 0.0

### File Watching / Hot Loading

**Controversy**: whether we need this (my opinion is we don't).

As a general rule, file watching is only useful in cases where one app writes a file, and a different app reads it. If the same program is both reading and writing the file, there's no reason that the program needs to notify *itself* (it might have an internal notification, but there's no reason this has to go through the filesystem).

The vast majority of apps and games don't need to support external editing of preference files: the settings file is only written by the app itself, not by some external program. If the file is meant to be changed externally, then it's not really a settings file, it's a config file, and that should handled by a different framework. (It also means that the config file lives in a different directory.)

However, certain types of advanced editors (like VSCode) have complex hierarchical settings files that can be edited as text files (this is necessary because some VSCode settings are themselves complex data structures - for example, the only way to configure bracket colors or json-schema validation in VSCode is by editing the JSON directly.) If editing settings file with a text editor is expected to be common, then this argues in support of hot reloading.

This goes back to the VSCode feature parity issue: do we really need to support editing of the Bevy editor settings file from a text editor while the editor is running? This seems like gold-plating to me; "But Mom, VSCode does it" is not a good reason.

## Research projects

I have written two different preferences libraries, each taking a different approach:
* `bevy_basic_prefs` was an earlier attempt that used reflection and resources.
* [bevy_prefs_lite](https://github.com/viridia/bevy_prefs_lite) used a simpler strategy of manually getting and setting properties.

(Note: I am **not** proposing the upstreaming of either of these, but rather a greenfield design based on the lessons learned).

Here's some notes about the implementation of `bevy_prefs_lite`:

* The library uses the serde `toml::Value` and `json::Value` data types to store preference values once they have been loaded into memory.
* Which format is determined strictly by the compiler's platform target; it it's wasm, then it uses json, otherwise toml.
* There are three main data types:
* `PreferencesStore` - the map of all preference files currently loaded.
* `PreferencesFile` - represents the data from a single settings file
* `PreferencesGroup` - represents a logical section within a preference file
* Currently serde is used in two different ways:
* Converting Rust types to/from JSON or TOML values which are cached in memory.
* Reading/Writing the cached JSON or TOML values to disk or local storage.
* Currently values must implement Serialize/Deserialize, however there's hope we could use `Reflect` instead; however, this is hard because we need the type registry, which might not be around soon enough.
* Preference files have a dirty bit which is an atomic bool. This gets set whenever a preference property is set.
* Dirty files can be saved manually, or on a timer.
* Saving on a timer uses an asynchronous io task, so the game doesn't stutter.
* Saving uses filesystem operations which are crash resistant (save to a temp file and then rename) so there's no chance the settings file gets corrupted.
* Reading is synchronous; however in most cases we only read once (at startup) and then keep the preference settings in memory, as they tend to be small.
* It's not possible, in Bevy, to reliably "save on exit" due to the way that AppExit works. The most reliable way uses both methods:
* Save on a timer after a mutation
* Try to catch the AppExit event

## Alternate Approaches

`bevy_prefs_lite` currently takes a (somewhat unjustified) shortcut: it stores the preference settings in memory as either a `json::Value` or `toml::Value`, depending on platform, rather than using some neutral type. This only works by accident, due to the similarities between JSON and TOML.

A more robust approach would be to define our own dynamic type, such as `PreferenceValue` which is similar to `toml::Value`. This would need to support both serde and conversion to/from `PartialReflect`.

One approach that was considered is to use `bevy_reflect::DynamicStruct` and `DynamicMap` for this purpose, but that doesn't work, because you can't actually serialize these types - there's not enough type information. For example, in JSON there's only one number type (f64), so you don't know, at load time, whether the number should be converted into a u8 or an f32. This conversion has to be done later, when getting or setting the property, but `bevy_reflect` doesn't have a built-in way to do these lazy type conversions.

Another possible approach is to try and register all of the preference types up front. For example, you could have a reflection annotation:

#[reflect(@PrefsGroup("audio"))]
pub struct AudioSettings {
    music_volume: f32,
    speech_volume: f32,
    effects_volume: f32,
}

This does put some constraints on the developer, in that it requires each config group to be defined as a single struct. There will be times when the logical grouping in the config file doesn't match the logical grouping of the settings in memory: for example, you might want all of the "graphics" settings to be grouped together, but settings like MSAA might be handled by an entirely different graphics subsystem than chunk loading distance.

Also, this approach has a minor footgun in that the developer still has to load and save each of the registered types; it's not automatic. It could be automatic if these types were resources, but as we've already discussed, this is problematic if we want to access preferences before the Bevy world is completely initialized.