# The Datatypes Shit-List

This is a list of “difficult data-types" that bevy may need to deal with. The chief offenders are `!Send` and `!Sync` types, but there are many other constraints we may need to be aware of.

## Properties

Most “difficult types" are either `!Send`, `!Sync`, or both. These three possibilities each have unique limitations and need to be treated differently.

`!Sync` types cannot be read safely from multiple threads at the same time,
and thus `&T: !Send`. There is a workaround: [Exclusive](https://doc.rust-lang.org/std/sync/struct.Exclusive.html)
forces `!Sync` types to be `Sync` by only allowing unique borrows (`&mut`)
to access the values. A version of this currently available in Bevy's source
tree under the name of `SyncCell`.

`!Send` types are the ones that have the strongest set of restrictions. These
values cannot be constructed, accessed, or dropped from any thread other than
the one that owns them. They can reside inside `Send` types, but the owning
type cannot allow access to them or drop them on any other thread.

There are three main reasons why a type might be `!Send`:

- **Unsynchronized Shared Access** -`Rc<T>` is `!Send` because
it's reference counting is not synchronized and any value can be used
to clone another, even from another thread, if that was allowed.
- **Thread Local State** - The use of `thread_local` for correctness means
moving a value of that type to another thread may result in invalid internal
state.
- **Thread Locked Foreign State** - Some `!Send` types with FFI or OS-level
access may also need to be instantiated on a specific thread (eg. the main
process thread or the thread where the rendering device lives), or are only
allowed to be used on a specific thread.

In some cases, the implementation of these types may be platform dependant. This may result in types being `!Send` or even thread-locked on some platforms, but not on others.

Finally, there is the method by which the data needs to be accessed. Some typically need to be accessed within the normal flow of an ECS schedule, while others are controlled by a non-ECS event loop on the owning thread.

## The Shit-List

In no particular order, the types on the shit-list are:

### Windowing Types
+ The `winit` event-loop is `!Send + !Sync` but everything else is `Send + Sync`.
+ Locked to the main thread on MacOS/iOS/web.
+ Some data is controlled by the winit event loop.

We generally want to let winit run it's event loop on the main thread. The most important thing going forward is detaching this event loop from Bevy's main loop, and having the two event loops communicate by message passing.

### Wgpu Types
+ `Send + Sync` everywhere except on web where they are `!Send + !Sync`.
+ May need be created on to the main thread, only locked to the main thread on web.
+ Wants ECS scheduling (synced to the render world not the main world).

Rendering resources (from wgpu) are `!Send + !Sync` on web. On some platforms (like iOS/macos/web) they may need to be created on the main thread. They are are only locked to the main thread on web. Certain integrations with rendering backends (e.g. DLSS Vulkan) may not be thread safe.

### Audio

+ The audio context is `!Send + !Sync`.
+ Locked to the main thread on web.
+ Wants ECS scheduling.

Firewheel's audio context object is !Send and !Sync. This is not the audio processor -- it's really a bridge between the non-realtime (e.g. game code) and realtime code (i.e. audio processor). We frequently access this context directly from the ECS. This comes from `cpal`, the current audio backend of Firewheel, where stream management is not thread safe. Alternative backends can alleviate the problem (shameless plug to `interflow` which ultimately should have `Send + Sync` types wherever possible, but is definitely not ready yet).

For most platforms, we depend on cpal to handle the lowest level of audio I/O, which doesn't have a main thread requirement. The web audio api does however require the main thread.

#### Windows

WASAPI does not have any main thread restrictions, but is not thread-safe. All operations (except few specific cases not relevant to this document) must be done in the same thread as the underlying COM apartment that created them.

ASIO API is not thread safe and needs to be called on the main thread.

#### macOS

CoreAudio is thread-safe in the scope of functions that could indirectly be called from an ECS schedule.

#### Linux

ALSA is thread-safe, except for device management (e.g., opening devices and streams) which is `Send` but `!Sync`. No main thread restriction.

Pipewire (not implemented by `cpal` but added for future-proofing) has no main thread restriction, however the API is based around a main loop system which interacts with much of the API, therefore it is not thread-safe.

#### Web

Web APIs are not thread safe and need to be called on the main thread.

### Input and GilRs

TODO

### Interpreters and Virtual Machines
+ Usually `!Send + !Sync`.
+ Not thread-locked.
+ Wants ECS scheduling.

Scripting language VMs are almost always `!Send + !Sync` and are not typically thread-locked. You typically want to run your VM within a specific part of a schedule, so they also require ECS integration.
- `mlua` has [feature-flagged support for making their runtime Send](https://docs.rs/mlua/latest/src/mlua/types.rs.html#115).
- `Boa` requires [thread locals for their GC](https://github.com/boa-dev/boa/blob/728325b7c378eb1f4dab06505003ab8d65921087/core/gc/src/lib.rs#L45) and thus the entire runtime is `!Send` without exception.
- TODO: Check pyo3 (Python) (Probably `Send` due to the GIL)
- TODO: Check Steel (Scheme)

### Hardware SDKs
+ Usually `!Send + !Sync`.
+ May sometimes be locked to the process' main thread.
+ Wants ECS scheduling.

Hardware SDKs (like depth cameras for example) are often `!Send + !Sync` and users will likely want to use them in conjunction with the ECS world. These kinds of SDKs often have lifecycles that require creating several components (e.g. a handle to the camera, some configuration structs, and a stream) that are more complicated than sticking in a static and would benefit from ECS patterns. Avoiding unnecessary copies sending frame data to the render world, etc.

### OS-Level Accessibility

TODO: OS-level accessibility and UI preference reading: https://github.com/bash/mundy