# Updating ANGLE

We aim to update ANGLE every two Firefox cycles.
Examples will be given based on 2019aug16's ANGLE update for Firefox 70. (our last ANGLE update was in 68, though we took some cherry-picks in 69)

## 0. Ingredients

- Windows system
- depot_tools checkout: https://chromium.googlesource.com/chromium/tools/depot_tools.git
- CMD shell with depot_tools in PATH (consider depot_tools.bat below)
- I use depot_tools.bat (see below) instead of putting depot_tools in my permanent PATH, because depot_tools has its own git and friends that I don't want polluting my system.
- You can also just put it in PATH temporarily, though I find this annoying.
- ANGLE checkout with remotes:
- `origin`: https://chromium.googlesource.com/angle/angle
- `moz`: [e-mail]:mozilla/angle.git

### `depot_tools.bat`:

@title depot_tools cmd
set PATH=C:\dev\depot_tools;%PATH%;
set DEPOT_TOOLS_WIN_TOOLCHAIN=0
set GYP_MSVS_VERSION=2019
@cmd

## 1. Choose a merge-base

For our merge-base we want to choose an upstream branch that matches what Chromium is targetting, generally for their Beta channel.

OmahaProxy is their tool for seeing what's shipping: https://omahaproxy.appspot.com/

We're looking for `win`+`beta` row, `true_branch` column. (Today that's `3865`)

## 2. Rebase our top-of-tree ANGLE on the merge-base

Checkout our previous top-of-tree ANGLE branch from the `moz` remote.
`git fetch origin` will grab all the new numbered branches.
Rebase onto the new merge-base: `git rebase -i origin/chromium/3865`
Check that the rebase looks sane! (gfx/angle/cherry_picks.txt will give you an idea of what to expect)
Rebasing might turn up conflicts that either:

- Are for csets that are redundant or no longer necessary, or
- Are from bitrot, and need to be repaired.

Good luck! We try upstream our changes into ANGLE aggressively, so ideally all our cherry_picks disappear in subsequent ANGLE updates.

## 3. `scripts/bootstrap.py` and `gclient sync`

Checking out the ANGLE git repo is only part of the equation. We'll also need to download/update its dependencies, which is done with `gclient sync`.

In your depot_tools shell:

py scripts\bootstrap.py
gclient sync

### I got "Exception: Visual Studio Version 2017 (from GYP_MSVS_VERSION) not found."
Make sure VS 2019 is installed, set GYP_MSVS_VERSION, and re-run:

set GYP_MSVS_VERSION=2019
gclient sync

## 4. Vendor into Gecko

In your gecko checkout, checkout something recent, like Central's tip. (both git and hg should work for this, but I use git)

Update the merge-base file at `gfx/angle/MERGE_BASE` with `chromium/3865`.

In your depot_tools shell:

cd path\to\angle-repo
py ..\path\to\gecko-repo\gfx\angle\update_angle.py origin

You should see output like:

C:\dev\angle>py ..\mozilla\gecko4\gfx\angle\update-angle.py origin
Importing graph
  ('gn', 'gen', 'out')
Done. Made 172 targets from 70 files in 3270ms
  ('gn', 'desc', '--format=json', 'out', '*')

Processing graph
  static_library //:translator
  static_library //:angle_common
  source_set //:angle_system_utils
  source_set //:angle_version
  action //:commit_id
  source_set //:includes
  source_set //:xxhash
  static_library //:preprocessor
  shared_library //:libEGL
  source_set //:libEGL_egl_loader
  shared_library //:libGLESv2
  source_set //:libANGLE
  static_library //:angle_gpu_info_util
  static_library //:angle_image_util
  copy //:copy_compiler_dll
  group //build/config:shared_library_deps
  group //build/config:common_deps

7 libraries:
  //:angle_common
  //:libEGL
  //:angle_image_util
  //:libGLESv2
  //:angle_gpu_info_util
  //:translator
  //:preprocessor

Logging cherry picks to ..\mozilla\gecko4\gfx\angle\cherry_picks.txt.
  ('git', 'merge-base', 'HEAD', 'origin/chromium/3865')
  ('git', 'log', '08b97da894b278c10e70b6888ac65e3972a13b31~1..08b97da894b278c10e70b6888ac65e3972a13b31')
  ('git', 'log', '08b97da894b278c10e70b6888ac65e3972a13b31..')

Run actions
  ('ninja', '-C', 'out', ':commit_id')
ninja: Entering directory `out'
ninja: no work to do.

Export targets
    Writing ..\mozilla\gecko4\gfx\angle\targets\angle_common\moz.build
    Writing ..\mozilla\gecko4\gfx\angle\targets\libEGL\moz.build
    Writing ..\mozilla\gecko4\gfx\angle\targets\angle_image_util\moz.build
[libGLESv2] Unrecognized define: ANGLE_CAPTURE_ENABLED
    Writing ..\mozilla\gecko4\gfx\angle\targets\libGLESv2\moz.build
    Writing ..\mozilla\gecko4\gfx\angle\targets\angle_gpu_info_util\moz.build
    Writing ..\mozilla\gecko4\gfx\angle\targets\translator\moz.build
    Writing ..\mozilla\gecko4\gfx\angle\targets\preprocessor\moz.build

Migrate required files
  Copying 1035/1035

Done

### Standalone builds

At this point, you can also build ANGLE standalone:

ninja -C out

update_angle.py configures some `gn args` to match what we build, including making it possible to build at all with is_clang = true.

If it doesn't build, message jgilbert!

### Handling `Unrecognized define`

In this case, we see a warning: `Unrecognized define: ANGLE_CAPTURE_ENABLED` from the `libGLESv2` target.
Open up `gfx/angle/update_angle.py` and locate `REGISTERED_DEFINES`.
Add `ANGLE_CAPTURE_ENABLED` with a `True` (meaning "use ANGLE's value for this define"). `False` would mean "ignore ANGLE's value, we'll define it ourselves if we want".

### Handling `Invalid include`

This particular update doesn't have this problem, but ANGLE upstream will sometimes forget to add new header files to their .gni build files. Since we try to copy only the files we know we need, we'll be missing any files not mentioned in the build files.

If you were to go to `angle-repo/src/libGLESv2.gni`, and comment out the line containing `passthroughdepth2d11ps.h`, `update_angle.py` will output the following:

C:\dev\angle>py ..\mozilla\gecko4\gfx\angle\update-angle.py origin
Importing graph
  ('gn', 'gen', 'out')
Done. Made 172 targets from 70 files in 3283ms
  ('gn', 'desc', '--format=json', 'out', '*')

Processing graph
  static_library //:translator
  static_library //:angle_common
  source_set //:angle_system_utils
  source_set //:angle_version
  action //:commit_id
  source_set //:includes
  source_set //:xxhash
  static_library //:preprocessor
  shared_library //:libEGL
  source_set //:libEGL_egl_loader
  shared_library //:libGLESv2
  source_set //:libANGLE
Warning in //:libANGLE: src\libANGLE\renderer\d3d\d3d11\Blit11.cpp: Invalid include: b'libANGLE/renderer/d3d/d3d11/shaders/compiled/passthroughdepth2d11ps.h'
Traceback (most recent call last):
  File "..\mozilla\gecko4\gfx\angle\update-angle.py", line 308, in <module>
    libraries = gather_libraries(ROOTS, descs)
  File "..\mozilla\gecko4\gfx\angle\update-angle.py", line 303, in gather_libraries
    dag_traverse(roots, fn)
  File "..\mozilla\gecko4\gfx\angle\update-angle.py", line 134, in dag_traverse
    recurse(x)
  File "..\mozilla\gecko4\gfx\angle\update-angle.py", line 127, in recurse
    recurse(x)
  File "..\mozilla\gecko4\gfx\angle\update-angle.py", line 127, in recurse
    recurse(x)
  File "..\mozilla\gecko4\gfx\angle\update-angle.py", line 119, in recurse
    t = pre_recurse_func(key)
  File "..\mozilla\gecko4\gfx\angle\update-angle.py", line 297, in fn
    assert has_all_includes(target_name, descs), target_name
AssertionError: //:libANGLE

Don't panic, this is only fatal because if we don't fix it, our *Gecko* build will fail very eventually, and we'd rather fail-fast.
All we need to do is add the file to the proper .gni file, try `update_angle.py` again.

After we add all the missing includes, commit it and *upstream it immediately!*. (see "Upstreaming changes" below)

Here's an example: https://github.com/mozilla/angle/commit/ad5a21320c011cacda9ca4b0f48e1c779427e511

## 5. Record it into the mozilla/angle GitHub repo

To keep things easy for ourselves, push the firefox-N (here firefox-70) branch up to the `moz` remote, so we have easy access to it for next time.

## 6. Make the Gecko commits

Here's the bug for the Firefox 68 ANGLE update:
https://bugzilla.mozilla.org/show_bug.cgi?id=angle-68

ANGLE updates usually have 2 or 3 commits:

1. Update our updater (gfx/angle/* except for gfx/angle/checkout and gfx/angle/targets)
2. Vendored ANGLE update (from update_angle.py) (gfx/angle/[checkout,targets])
3. Mark new unexpected-fails and unexpected-passes (sometimes not needed)

Get review from jgilbert for all of these, though the review on the (huge) commit#2 will just be a sanity check. (We're not going to re-review all the upstream changes!)

You'll want to do a Try run to figure out if you need anything for commit#3, or if there's anything else wrong. While we don't use ANGLE's GLES driver for any platform but Windows, all platforms use the ANGLE shader translater for WebGL, so new and exciting build failures can happen on any platform.

"That's all!" :)

## Appendix: Upstreaming changes

ANGLE has an explainer, though it's a little out-of-date:
https://chromium.googlesource.com/angle/angle/+/master/doc/ContributingCode.md#getting-started-with-gerrit-for-angle
Use the depot_tools shell (thus depot_tools's git hooks) so that you can ignore the `Install the Gerrit commit_msg hook` section.

1. File a bug on Chromium's angleproject sub-tracker:
- https://bugs.chromium.org/p/angleproject/issues/entry
2. Write the fix. Ideally add a test.
3. `git cl format`
4. Use the bug number from #1 in the commit body next to the `Change-Id` line: `Bug: angleproject:3330`
- Example: https://github.com/mozilla/angle/commit/41ec3fab29e097be566490530590bc97ce235ab6
5. Setup Chromium commit access
6. `git cl upload`
7. Ask for review, generally from [e-mail] and [e-mail].
8. Pass review and have them land the change! \o/