changed 3 years ago  
Linked with GitHub

homu rewrite

The Rust BORS clone homu is crucial to the daily development of Rust infrastructure (compiler, miri, clippy etc.).

Sadly, the code of homu is quite old and messy, it is currently basically unmaintainable and it is very difficult to implement new features for it.
It also probably contains a lot of legacy code for CI systems that aren't used anymore (e.g. Travis).

It would be nice if we could rewrite homu (preferably in Rust), with the following goals:

  1. Make the code more maintainable/testable.
    • Writing it in Rust should by itself improve maintainability and readability vs the current Python code that is very dynamically typed.
    • Prepare/extend the test suite for homu.
  2. Get rid of legacy systems, tailor the code for the current GitHub CI.
    • In the process, possibly change some of the workflows to be better integrated with GitHub.
  3. Implement new feature requests (once the rewrite is done) that are now stuck (homu issue list).

What does homu do?

First, it would be nice to document what does homu do at its current state:

  • Web server
    • homu provides a web server.
    • Displays basic help (which GH commands are available).
    • Displays current merge queue.
    • Allows creating rollup PRs.
    • Synchronizes PR state on startup.
  • GitHub integration
    • The server also has webhooks for GH events.
    • Reads commands from GitHub comments on PRs.
  • Merge queue
    • Downloads information about PRs and commits, creates merge commits, schedules tests and performs the actual merges.
    • Handles PR priority and tree closing.
  • Database
    • Stores information about current PR states, repositories, closed trees, priorities etc.
    • Uses SQLite.
  • Configuration
    • Configured through a TOML file.

What do we want to keep/change?

  • Web server
    • We need to reimplement the backend, but the frontend can probably stay the same for now.
  • Database
    • We can provide an interface that will let us abstract the DB, and then implement backends e.g. for SQLite, DynamoDB or others.

Alternatives to a full rewrite

Use existing tools:

  • bors-ng
    • A lot of momentum and activity, active development (+).
    • Written in Elixir (-), probably wouldn't be friendly to rustc developers for maintenance.
    • As any other tool, it's opinionated and doesn't necessarily match the commands used currently by homu. Compare: https://bors.tech/documentation/ vs https://bors.rust-lang.org/. Things like treeclosed seem to be missing. We could add them, but the tool is used by a lot of users and it's unclear whether we could fit its needs (and get our changes in) for the specific use cases of rustc (clippy, miri, …) projects.
  • bors-rs
    • Written in Rust (+).
    • Seems similar in scope and features to homu (+).
    • Unmaintained (-).
    • Implemented as a GitHub App (could be moved to a self-deployed app).
  • original bors
    • Unmaintained, written in Python, wouldn't improve the current situation (- - -).

If we adapt an existing tool, we would probably either need to fork it to add commands that are currently offered by homu, or generalize it enough so that we can add custom commands to it.

There seem to be two ways forward:

  • Use and integrate bors-ng.
  • Rewrite homu in Rust (while potentially taking some code from bors-rs).

More detailed comparison of homu vs bors-ng commands

Note: aliases (like r+ vs merge) are not included, only commands that
differ semantically.

homu bors-ng Description Do we need it?
r+ r+ Reviewed, merge x
r+ <SHA> N/A Reviewed, merge with SHA guard
r+ p=<prio> r+ p=<prio> Set PR priority, merge x
r=<name> r=<list of names> Reviewed by name(s), merge x
r- r- Cancel merge x
p=<prio> p=<prio> Set PR priority x
try try Try build x
N/A (?) try- Cancel a try build
delegate+ delegate+ Delegate rights to PR author x
delegate=<name> delegate=<list of names> Delegate to specific user(s)
delegate- N/A Undelegate PR author
rollup=never single on Disable rollup/batching x
rollup=always single off Enable/force rollup/batching x
rollup=iffy N/A iffy rollup
rollup=maybe N/A maybe rollup
rollup- N/A Unmark as rollup
squash N/A Squash before merge
squash- N/A No squash before merge
force N/A Stop all builds, move to next PR
treeclosed=<prio> N/A Close tree for PRs below <prio>
treeclosed- N/A Undo tree closure
clean N/A Clean up previous build results
retry N/A Retry build (?)
N/A retry Retry previous command

There are also some other debug/maintenance commands like ping.

Select a repo