CI Overhaul - esp-hal

For context, please read the relevant issue:

• https://github.com/esp-rs/esp-hal/issues/2969

Additional issues with the CI label:

• https://github.com/esp-rs/esp-hal/issues/2080
• https://github.com/esp-rs/esp-hal/issues/2440
• https://github.com/esp-rs/esp-hal/issues/2971

Problems

• Some checks are performed multiple times when they only need to be run once.
  • e.g. The current CI workflow runs when a PR is opened, when the PR is added to the merge queue, and then again when merged into the main branch.
• Checks are not performed consistently accross packages.
  • MSRV checks are not run for all packages, only a select few
    • (It's hard to provide MSRV guarantees when we don't check this :D)
  • Documentation checks could be improved
    • When #3026 is merged, perhaps this can be run on a nightly basis or something to ensure that everything is in good shape
• nightly toolchain checks often result in errors, but these are easy to ignore
  • Opening an issue on failure might be adequate to resolve this
  • BQ: Maybe interesting: https://blog.rust-lang.org/inside-rust/2025/01/17/this-development-cycle-in-cargo-1.85.html#plugin-of-the-cycle
• We still do not have any sort of HIL test for Wi-Fi/Bluetooth
  • We may be fairly limited in our options here, but if we're at least able to confirm we can connect to an AP and send/receive a simple request that's probably better than nothing
  • BQ: I'd love to see that. Ideally we would have a dedicated access-point for testing. For BLE it might be already good to just check we can initialize it and the controller reponds
• It's slow

Brainstorming

• It would be nice to run the documentation workflow automatically when a release is created
  • The major blocking issue here is, how do we know which packages need documenting? Building everything may result in out-of-sync documentation in the situation that one or more packages have received updates since their last release, but they are not being published as part of the current release
  • SM: I think this can be solved with tagging (even if it is a bit laborious)
• It might be worth investigating whether using our self-hosted runners is advantageous/desirable here
  • SM: It's a shame we had to switch to RPI's for HIL, instead of the VPS set up we had at the start, if we were still doing that we could probably reuse (or expand) the runners there quite easily
  • SM: At the very least the runners we use for compiler builds often sit idle, we can try and incorp those into our CI system (for linux self hosted runner, this should be easy. Not sure if we need any extra steps for the windows/mac runners)
• Look into automated semver checks (… again)
  • SM: Definitely, I think with the beta.0 release out of the door we'll want to do this
• See if we can reduce the runtime of the workflow and overall reduce congestion when many PRs are being opened/updated
  • This may be at least partially solved by simply running fewer checks, as mentioned above in the Problems sections
• It would be nice to try and minimize the amount of YAML in the workflow(s) and try to delegate to our xtask package as much as possible
  • BQ: That's something I'd really like to have. In the past I experimented with using justfiles and shell scripts - but delegating more to the xtask would be better
  • SM: I think we can collect the individual jobs we do per chip clippy/docs/builds/examples into xtask ci --chip ${CHIP}, which I think would remove the need for https://github.com/esp-rs/esp-hal/blob/main/.github/actions/check-esp-hal/action.yml?
  • SM: Better way of doing the MRSV? It doesn't even check all packages at the moment: https://github.com/esp-rs/esp-hal/blob/6c2fd59e0b7a348dbf83ffb261b615510294cd45/.github/workflows/ci.yml#L113-L163
    • Maybe we only need to MSRV check stable packages?
• Can we automate anything else from the release process?
  • Creation of the next migration-guides files
  • Updating CHANGELOG headers

Planning

1. Explore using self hosted runners to speed up CI

2. Explore automating docs deployment with tagged releases

3. Reducing number of tasks run

   • don't run on main

4. Issue creation for nightly failures

5. Reduce YAML (covert to xtask), to aid running CI locally

   • MSRV

6. Automate release document rollover (migration guide, changelog templates)

7. esp-wifi test