Support for HDR and WCG in Bevy

This is a quick little note exploring the current state of High-Dynamic Range (HDR) and Wide-Color Gamuts (WCG) in Bevy.

Display-Referred and Scene-Referred Colors

Display-referred colors are defined by a display standard (typically sRGB, BT.709, BT.2020, BT.2100 or Display P3). sRGB and BT.709 are SDR, Display P3 and BT.2020 are WCG, BT.2100 is HDR + WCG. All display-referred colors have bounds on chromaticity and intensity, and usually have coefficients between 0 and 1.

Scene-referred colors are defined by quantities of light. They typically use the primaries from a display standard and are expressed in luminous units, such as lux or candela per square meter. They have bounded chromaticity but unbounded luminosity (meaning the light can be arbitrarily intense intense), which translates to allowing their coefficients to go above 1.

WebGPU

WebGPU allows us to create canvases in either the sRGB or Display P3 color spaces. Additionally, when the attachment format is set to rgba16float, WebGPU converts the output color values to the color space of the actual screen and then clamps them to the [0, 1] interval in that color space.

Canvas color values outside of the [0, 1] interval may be used to display colors outside of the gamut defined by the canvas' color space’s primaries, when permitted by the configured format and the user’s display capabilities. E.g, the color value (1.09,-0.23,-0.15) in an 'srgb' canvas will produce the same color as the color value (1,0,0) in 'display-p3' canvas.

Bevy

Currently, bevy's PBR shaders are built using the BT.709 primaries; they expect display-referred BT.709 (or sRGB) assets and output scene-referred BT.709 frames. When HDR is enabled, we output those frames directly as rgba16float and the OS compositor takes care of sending them to the HDR display in the correct format. Otherwise we tonemap the frames back to display-referred BT.709 and output them as rgba8-unorm-srgb which WebGPU then maps into sRGB for us.

All "color assets" (textures, UI swatches, whatever) are display referred and the vast majority are sRGB. Assets take one of two paths to the display: they are either (a) rendered directly without pbr shaders or (b) used in physically-based shading to produce scene-referred output data.

Recommendations

None of this needs to effect bevy_color much. It should do it's best to abstract away all this complex stuff. It's more about the render stack consumes the bevy_color api.

Our support for HDR requires some work to look similar to the SDR path. We still need to:

• Apply display mapping to the color-graded, scene-referred colors, taking the output display's max brightness into account (see this issue). This should ideally have a similar look to the chosen SDR tonemapper, just with improved dynamic range.
• Make better direct presentation (path a) for sRGB image assets in HDR mode.
• Avoid clamping colors ever, and trust the renderer to do it for us.
• Make it easier to mix HDR and SDR cameras (or impossible).

We currently have no support for WCG. To add support, we would need to:

• Switch all PBR code over to a WCG space like BT.2020.
• Make sure we can do tonemapping in the WCG space.
• Support assets in the WCG space.
• Convert assets to the canvas color space before sending them down path a.
• Convert assets into the WCG space before sending them down path b.
• Convert frames from the WCG space to the canvas space before they are presented.
• Ensure conversions do more than clip the data to the output color space, as that would lose detail. This is especially important if source assets are in WCG space but the canvas space isn't.