# Rebase of Firefox onto ANGLE's `chromium/5359` branch
## Current status
2022-11-23: Now in [review](https://phabricator.services.mozilla.com/D162655). Some relevant work remains outstanding, but this should be broken out as follow-up work.
## Rebasing Firefox patches
<div style="display:none;">
## Rebasing
</div>
This rebase of `mozilla/angle` will be marked as the [`firefox-111`](https://github.com/ErichDonGubler/angle/commits/angle-4) branch and based on upstream's [`chromium/5359`](https://chromium.googlesource.com/angle/angle/+/refs/heads/chromium/5359).
:::spoiler Main open questions from initial rebase
- [x] :warning: [`3a1d51f5`](https://github.com/mozilla/angle/commit/3a1d51f5784f13373f60e4138c80dcbebdab5524) was hard-blocking builds (`-Wunreachable-code`). @ErichDonGubler changed it to remove the remaining unreachable body of the function, since I feel like a theoretical merge conflict resolution would clearly present the intent.
- KG: I think we don't need this commit anymore, but also don't remove code, use `if (1) return;` or similar.
- [x] :interrobang: Dunno if we need [`400476a0b`](https://github.com/mozilla/angle/commit/400476a0b2e1ba6dfa8413375e20c15c2ec9188a)--it might be outmoded by [`58930a73ce`](https://github.com/mozilla/angle/commit/58930a73ce909537ebca498c8d9a27815915ee3a), with some migration?
- [x] Some new `gn` config got introduced for this. After a few minutes, I skipped analyzing it deeply in favor of leaning on your involvement from before.
- [x] Feeling super nervous about making a call about bitflags in `src/compiler/translator/SymbolTable_ESSL_autogen.cpp`, since I don't understand them.
- KG: Just trust the autogen'd output.
- [x] :information_source: Dropping [`33ffc1233`](https://github.com/mozilla/angle/commit/33ffc12336370c17ba8da02c8e1f0eeb44951a7b) in favor of [`b8d6f8aa93`](https://github.com/mozilla/angle/commit/b8d6f8aa936d643c788f0ed9088f2ae162defc38).
- [x] :information_source: Dropping [`c79c27ff2`](https://github.com/mozilla/angle/commit/c79c27ff20b6a13441807d54ce2b8a487cc57d4a), seems upstreamed by [`afda22b0`](https://github.com/mozilla/angle/commit/afda22b0856d8acfb35026a921bfafe4175278ce).
- [x] :information_source: Dropping [`23851a53`](https://github.com/mozilla/angle/commit/23851a53779dfd3239b4cceb7f93df9ea6efb9c3), seems upstream-fixed by [`59f496c0`](https://github.com/mozilla/angle/commit/59f496c099be5f4799ec3e0e5fe651f54da2736c).
- [x] :seedling: Added a commit to change `std::atomic<angle::Mutex*> g_{,Surface}Mutex(nullptr)` to `std::atomic<angle::Mutex*> g_{,Surface}Mutex{}`.
- [x] :question: Might not need [`9312f40a`](https://github.com/mozilla/angle/commit/9312f40add7e437c20e46f0f468036035268fa54) any more, if upstream fixed?
:::
<details><summary>Other items for discussion</summary>
- [x] 🛑 Definitely need to run codegen to resolve any divergences that manual editing may have caused, @ErichDonGubler is figuring this out, ATM.
- [x] :thinking_face: [`604cd6cae`](https://github.com/mozilla/angle/commit/604cd6caef56babd6956be442e8a06de0df88aec) gets listed in `cherry_picks.txt`, but...why?
- [x] TL;DA: @ErichDonGubler was confused, we changed the auto-generated script output a bit to be clearer.
- [x] :thinking_face: Do we think that regressions are likely from this point?
- KG: It kinda doesn't matter! (Because we don't have any other choice but to upgrade and find out)
</details>
- [ ] @ErichDonGubler experienced some issues with `git cl format` specifying too many arguments on Windows; patching to chunk into ~500 files was necessary. Upstream?
## Bug-hunting
### Compile issues
<details><summary>Fully resolved</summary>
#### `ShCompileOptions` changed
:::success
✔️ Found in [2022-11-15#1](https://hackmd.io/XxvU5HgHQVWw-kxKkKGA_A?both=#2022-11-15). Worked around compilation issues in [2022-11-15#2](https://hackmd.io/XxvU5HgHQVWw-kxKkKGA_A?both=#2022-11-15). Resolution: [exhaustively specify all flags](https://phabricator.services.mozilla.com/D162655).
:warning: Unclear whether or not the set of specified flags is what we need long-term. Will let further testing and [review](https://phabricator.services.mozilla.com/D162655) determine what refinements may be necessary.
:::
:::spoiler
Discovered that `ShCompileOptions` constants like `SH_VARIABLES` were no longer defined. `ShCompileOptions` has apparently been [migrated to a struct bitfields interface](https://github.com/mozilla/angle/commit/6f80f0f0373f9b4f014264e337443939021461b6), rather than a bitflags-based one.
Migration for setting individual flags is easy, but one hiccup: we currently rely on way to specify "all flags" without enumerating each flag individually (`options = -1;` from `dom/canvas/WebGLShaderValidator.cpp:ChooseValidatorCompileOptions`).
:::
#### `astcenc_vecmathlib_*` headers not found
:::success
✔️ Found in [2022-11-15#2](https://hackmd.io/XxvU5HgHQVWw-kxKkKGA_A?both=#2022-11-15). Resolution: define the new `ASTCENC_DECOMPRESS_ONLY` `#define` in `update-angle.py`.
:::
:::spoiler
`#include "astcenc_vecmathlib_sse_4.h"` failed to compile `checkout/third_party/astc-encoder/src/Source/astcenc_mathlib.h`, because these files were not discovered in the build graph when generated via `ninja`.
:::
#### Apple platforms failed to compile `astcenc` stuff
:::success
✔️ Resolution: define the new `ANGLE_ENABLE_APPLE_WORKAROUNDS` `#define` in `update-angle.py`.
:::
</details>
#### `exit-time-destructors` error in Windows
:::success
✔️ Found in [2022-11-15#1](https://hackmd.io/XxvU5HgHQVWw-kxKkKGA_A?both=#2022-11-15). Resolution: worked around in our fork.
:::
:::spoiler
`std::mutex` was incorrectly assumed to be trivially destructible in upstream's `Renderer11` API on Windows. This has been resolved by applying the `angle::base::NoDestructor` type wrapper around the `static` `Renderer11::gMutex` member.
:::
- [ ] Submit a patch to ANGLE upstream
### Pre-nightly run-time issues
:::success
Status: after a long and drawn-out set of issues, it appears we are finally good to try merging this rebase to mainline! :raised_hands:
:::
<details><summary>Fully resolved</summary>
#### Refcounting crash around `ID3DDevice`
<div style="display:none;">
#### Destructor crash for `rx::Renderer11`
#### Crash in `EglDisplay::fTerminate`
#### Destructor crash for `Swapchain11`'s `IDXGIKeyedMutex`
</div>
:::success
✔️ Discovered in [2022-11-29#1](#2022-11-29). Reported [here]([bug](https://bugzilla.mozilla.org/show_bug.cgi?id=1809216)). Mainline fix posted for review [here](https://phabricator.services.mozilla.com/D166385). See [here](https://hackmd.io/HYBz0_5pTAeP2QFG46R_YA) for instructions to reproduce.
:warning: This seems like a correct change, but there is still [some uncertainty](https://matrix.to/#/!wbEUPKmXGxodKqLlZL:mozilla.org/$69hU0zufRk1cyxcyfU4Jhr1LAOoxM-oxWLJLcCSYI7g?via=mozilla.org&via=matrix.org&via=igalia.com) about whether or not this change will not expose other problems.
:::
:::spoiler
An unbalanced set of refcount operations in Firefox led to consistent use-after-free crashes while the ANGLE/D3D11 back end was being torn down. This issue already existed in Firefox, but was not exposed until this update.
- [x] ~~Did the fix land?~~
:::
#### `ovr_multiview2_draw_buffers` test cases fail
:::success
✔️ Discovered in [2023-01-09#1](#2023-01-09). Worked around in [D162655](https://phabricator.services.mozilla.com/D162655).
:::
:::spoiler
Failures in the `gl2c` test group for Windows consistently failed in
```
TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-conf/generated/test_2_conformance2__extensions__ovr_multiview2_draw_buffers.html | the right edge of view 0 of color attachment 1 should be untouched
TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-conf/generated/test_2_conformance2__extensions__ovr_multiview2_draw_buffers.html | the left edge of view 1 of color attachment 1 should be untouched
TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-conf/generated/test_2_conformance2__extensions__ovr_multiview2_draw_buffers.html | the right edge of view 1 of color attachment 1 should be untouched
TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-conf/generated/test_2_conformance2__extensions__ovr_multiview2_draw_buffers.html | the left edge of view 2 of color attachment 1 should be untouched
TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-conf/generated/test_2_conformance2__extensions__ovr_multiview2_draw_buffers.html | the right edge of view 2 of color attachment 1 should be untouched
TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-conf/generated/test_2_conformance2__extensions__ovr_multiview2_draw_buffers.html | the left edge of view 3 of color attachment 1 should be untouched
TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-conf/generated/test_2_conformance2__extensions__ovr_multiview2_draw_buffers.html | the right edge of view 0 of color attachment 2 should be untouched
TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-conf/generated/test_2_conformance2__extensions__ovr_multiview2_draw_buffers.html | the left edge of view 1 of color attachment 2 should be untouched
TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-conf/generated/test_2_conformance2__extensions__ovr_multiview2_draw_buffers.html | the right edge of view 1 of color attachment 2 should be untouched
TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-conf/generated/test_2_conformance2__extensions__ovr_multiview2_draw_buffers.html | the left edge of view 2 of color attachment 2 should be untouched
TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-conf/generated/test_2_conformance2__extensions__ovr_multiview2_draw_buffers.html | the right edge of view 2 of color attachment 2 should be untouched
TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-conf/generated/test_2_conformance2__extensions__ovr_multiview2_draw_buffers.html | the left edge of view 3 of color attachment 2 should be untouched
```
:::
#### `EGL_FLEXIBLE_SURFACE_COMPATIBILITY_SUPPORTED_ANGLE` removed
:::success
✔️ This issue appears to have been conclusively resolved. We will keep an eye out for regressions. Originally found with [2022-11-21#1](2022-11-21).
:::
:::spoiler
Before this rebase, Fx's ANGLE back end for WebRender used [`EGL_FLEXIBLE_SURFACE_COMPATIBILITY_SUPPORTED_ANGLE`] as a context creation attribute. This has been removed in Chromium upstream in favor of the [`EGL_KHR_no_config_context`] extension, according to the [upstream commit message removing it](https://github.com/mozilla/angle/commit/ad8e4d99a977073074a36fe231477398843f1501). With this rebase, usage of the old attribute would return `EGL_BAD_ATTRIBUTE` (visible as `0x3004` in failure logs).
:::
[`EGL_FLEXIBLE_SURFACE_COMPATIBILITY_SUPPORTED_ANGLE`]: https://github.com/mozilla/angle/blob/fe3e46abbc08ecaa65a8ba656b94b0a5a2f29e11/extensions/EGL_ANGLE_flexible_surface_compatibility.txt
[`EGL_KHR_no_config_context`]: https://registry.khronos.org/EGL/extensions/KHR/EGL_KHR_no_config_context.txt
</details>
#### Stack overflow in `IntermNode.cpp`'s `PropagatePrecision` methods
:::success
✔️ Some AST traversal/manipulation changed in `angle` upstream, and it crashes with our current thread sizes. Resolved by increasing thread stack size of `CanvasRendererThread`. Discovered in [2022-11-29#1](#2022-11-29).
:::
- [ ] Looks resolvable in ANGLE upstream. Get this fixed?
- [ ] Get MVRE in here.
- [ ] Either stack sizes will need to be adjusted in client code (docs plz), or make a new thread under the hood on the ANGLE side.
- Maybe related: [upstream issue 4042](https://bugs.chromium.org/p/angleproject/issues/detail?id=4042&q=stack+overflow&can=2)
- [ ] Investigate why stack sizes increased so much between old and new versions in this upgrade.
- [ ] This seems to reveal some architecture issues with Fx code; bringing down WebGL context operations because of a shader compilation crash is not an acceptable regression.
- [ ] @UAJMTBhySlWm5X4K8GRAgg stated that this is likely to be a DoS problem for Mac, if we use ANGLE for shader compilation; there's no isolated GPU process on that platform.
- KG: This kind of crash-on-bad-enough-shader isn't a hard-blocker for us to update, but it should be on our radar and receive its own prioritization.
## Issues discovered after landing
<div style="display:none;">
## Post-update action items
</div>
### Snap builds broken
:::success
✔️ [Bug 1812260](https://bugzilla.mozilla.org/show_bug.cgi?id=1812260): snap builds broken after missing. Should be resolved with [D167815](https://phabricator.services.mozilla.com/D167815).
:::
## Try build history
At @jgilbert's recommendation, @ErichDonGubler has been using the union of the two `./mach try fuzzy` queries:
* `!asan !tsan !plat !js !talos 'mochi 'webgl`
* `!asan !tsan !plat !js !talos 'reftest`
:::spoiler Old builds
### 2022-11-15
* [1](https://treeherder.mozilla.org/jobs?repo=try&revision=0aee74a20b37c324b154df66536f01919d45e8c1); somewhat behind `tip` at time of pushing.
* [2](https://treeherder.mozilla.org/jobs?repo=try&revision=941da06dba3565769a36f4a13726c0ec502158ff): rebased onto latest `tip` as of 2022-11-15
### 2022-11-16
* [1](https://treeherder.mozilla.org/jobs?repo=try&revision=296f95ca48970c628e9dffc12a3820891aeb5888)
* [2](https://treeherder.mozilla.org/jobs?repo=try&revision=972ee68135a0d730a68a38f163577347a7c4ee6b)
* [3](https://treeherder.mozilla.org/jobs?repo=try&revision=a3d801179db686de9411fb2e2a4e61091fde4be1): fixed builds for Windows and Mac
* [4](https://treeherder.mozilla.org/jobs?repo=try&revision=e17a508924f345c50cc6ec03972c25250521c2f1)
### 2022-11-16
* [1](https://treeherder.mozilla.org/jobs?repo=try&revision=df9076fd340586c6d9c011749698c4da93487f09): attempt to fix non-spurious Mac test failures
### 2022-11-18
* [1](https://treeherder.mozilla.org/jobs?repo=try&revision=df9076fd340586c6d9c011749698c4da93487f09): rebased onto latest `tip` from `mozilla-central`.
### 2022-11-21
* [1](https://treeherder.mozilla.org/jobs?repo=try&revision=0927dd49871b43dc6d10fb6babad7bdbe6d3ee1b): First non-`mach try auto` build, based on the first version of the [last revision's review](https://phabricator.services.mozilla.com/D162655?id=650032#inline-896784).
* Failures:
* The vast majority of failures in this build occurred because of [an issue with `EGL_FLEXIBLE_SURFACE_COMPATIBILITY_SUPPORTED_ANGLE` being removed](#EGL_FLEXIBLE_SURFACE_COMPATIBILITY_SUPPORTED_ANGLE). These will not be noted explicitly.
* Linux WebRender opt
* `bc2` appears to be intermittent.
* `bc15` appears to be intermittent.
* Windows 10 x86 WebRender
* `opt`
* :warning: `Mochitest with WebGL over IPC`'s `gl2c` is intermittent.
### 2022-11-29
* [1](https://treeherder.mozilla.org/jobs?r=try&revision=be0f4ca1969232848a088fd72f9275d986a7f37a&selectedTaskRun=YzVZiNDpQr6RENiW9qUkAQ.0): remove the now-invalid `EGL_FLEXIBLE_SURFACE_COMPATIBILITY_SUPPORTED_ANGLE` attribute
* Failures
* Windows 10 x64 WebRender opt:
* `M` fails on:
* `gl1c` (retried, failed)
* :warning: `gl1e` (retried, failed): appears to be [an ANGLE destructor issue](#Destructor-crash-for-Swapchain11’s-IDXGIKeyedMutex)
* :warning: `gl2c` (retried, failed): appears to be [an ANGLE destructor issue](#Destructor-crash-for-Swapchain11’s-IDXGIKeyedMutex)
* :warning: `gl2e3` (retried, failed): appears to be [an ANGLE destructor issue](#Destructor-crash-for-Swapchain11’s-IDXGIKeyedMutex)
* `M-gli` fails on:
* `gl1c` (retried, failed)
* :warning: `gl2c` (retried, failed): [failed to fetch WebGL rendering context](#Unable-to-fetch-WebGL-rendering-context-for-Canvas)
* :warning: `gl1e` (retried, failed): unexpected failures in `shader-uniform-packing-restrictions.html` caused by a [stack overflow](#Stack-overflow-in-IntermNodecpp’s-PropagatePrecision-methods).
* :warning: `gl2e3` (retried, failed): appears to be [an ANGLE destructor issue](#Destructor-crash-for-Swapchain11’s-IDXGIKeyedMutex)
* :warning: `gl2e4` (retried, failed): unexpected failures in `shader-uniform-packing-restrictions.html` caused by a [stack overflow](#Stack-overflow-in-IntermNodecpp’s-PropagatePrecision-methods)
* Windows 10 x64 WebRender debug:
* `M` fails on:
* `gl1c` (retried, failed)
* `gl2c` (retried, failed)
* `gl2e3` (retried, failed)
* `M-gli` fails on:
* `gl1c` (retried, failed)
* `gl2c` (retried, failed)
* `gl2e3` (retried, failed)
* `gl2e4` (retried, failed)
* Linux 18.04 x64 WebRender opt and debug:
* :warning: All of these appear to be a [stack overflow bug](#Stack-overflow-in-complex-glsl-does-not-crashhtml).
* `gl1e` (retried, failed)
* `gl2e3` (retried, failed)
* `gl2e4` (retried, failed)
#### 2022-12-02
* [1](https://treeherder.mozilla.org/jobs?repo=try&revision=43d316d199effa7e64bba149008e0c1573539988): [2022-11-29#1](#2022-11-29), but rebased onto latest `tip` from `mozilla-central`.
#### 2022-12-16
* [1](https://treeherder.mozilla.org/jobs?repo=try&revision=8e078b45b35b4f36785a8970e22805319da77ee4): rebase [2022-12-02#1](#2022-12-02) onto `mozilla-central`'s `tip` with minor unrelated dev fixes from @ErichDonGubler.
* Windows 10 x64 2004 WebRender debug `gl2e3`: found a [crash calling `EglDisplay::fTerminate`](#Crash-in-EglDisplayfTerminate).
#### 2023-01-09
<div style="display:none;">
#### Later builds
</div>
Finely tracking failures were stopped, for a while, since attention was devoted solely to the fix that this build ended up providing.
* [1][2023-01-09#1]: include a fix for a [refcounting crash](#Refcounting-crash-around-ID3DDevice).
[2023-01-09#1]: https://treeherder.mozilla.org/jobs?repo=try&revision=cc94fbad4532e682f9044a9586c136b24e09f090
#### 2023-01-17
* [1](https://treeherder.mozilla.org/jobs?repo=try&revision=5cb31f208d4170a6f2c1c2174637e7fafbca4646): Trying (and failing) to mark the [failing `dom/canvas/test/webgl-conf/generated/test_2_conformance2__extensions__ovr_multiview2_draw_buffers.html` test case](#ovr_multiview2_draw_buffers-test-cases-fail) as an expected failure.
* An intermittent `gl1c` failure happened, but @ErichDonGubler doesn't think it's a problem? 🤔
* [2](https://treeherder.mozilla.org/jobs?repo=try&revision=604e5ba8658ac25fde0b36021a7c4225389c3377), [3](https://treeherder.mozilla.org/jobs?repo=try&revision=83ca0817ec428d4d7e4c93a071786a8660cbb250): Marks the [`ovr_multiview2_draw_buffers` test failure](#ovr_multiview2_draw_buffers-test-cases-fail) as expected for Windows.
* D'oh, missed the `debug` clause we needed for `mochitest-errata.ini`. Will fix that up.
* [4](https://treeherder.mozilla.org/jobs?repo=try&revision=38d1089f2a011e4c913bc6fc6ccf853e8a892e4b&selectedTaskRun=bBdoJdz4ShaZgE7sBy1cdA.0): Add a `debug` clause to the [expected `ovr_multiview2_draw_buffers` test failure](#ovr_multiview2_draw_buffers-test-cases-fail).
* No remaining [persistent](https://treeherder.mozilla.org/jobs?repo=try&revision=38d1089f2a011e4c913bc6fc6ccf853e8a892e4b&selectedTaskRun=bBdoJdz4ShaZgE7sBy1cdA.0) problems found! :raised_hands:
#### 2023-01-18
* [1](https://treeherder.mozilla.org/jobs?repo=autoland&revision=84a5f26fe60beb0dd1e87d8a87b5cc3b27d2688c): First attempt to `autoland`. Revealed some more issues:
* The `PoolAlloc` allocator makes `asan` angry on Linux! TODO: issue entry above?
* [2](https://treeherder.mozilla.org/jobs?repo=try&revision=9e3ad5e565a4ca44a76d23ea06e875c093dab4f7): attempting to fix `asan` violations in `PoolAlloc` on Linux, per [@jgilbert's suggestion](https://matrix.to/#/!EwRyHDhpoUHFNjRwIJ:mozilla.org/$FsCaUErPkXXJJOtmOy9O6RWiCKBAKKISLd77bGM16Tg?via=mozilla.org&via=matrix.org). This failed, but nicely demonstrates failures from `1` above.
* [3](https://treeherder.mozilla.org/jobs?repo=try&revision=8179d2a2bce0da39f84366d5444e0f96af1f49f2): _actually_ try to fix things; @ErichDonGubler hadn't propagated the `ANGLE_DISABLE_POOL_ALLOC` compilation flag properly in `2`.
:::
## Patches
Patch stacks in Lando:
- Original rebase: https://lando.services.mozilla.com/D162655/
- Snap build fix: https://lando.services.mozilla.com/D167815/
- Snap build fix: https://lando.services.mozilla.com/D167815/