GitOps for packaging

Overview

Packaged Linux distributions require a lot of very specialized tooling and services which makes it harder to onboard people to packaging and to maintain infrastructure services for it.

The goal of the project is to refactor packaging workflows and pipelines so that they converge on using GitOps approach as much as possible.

As Git and Merge Request (Pull request, PatchSet, ReviewRequest) became de-facto the standard of the software development there is entire ecosystem of tools, orchestration engines, dashboards, visualizations around them, which gives us some freedom of choice of a particular implementation.

What is GitOps?

There are many hand-wavy descriptions, but for the purpose of this initiative, let's define it as:

Every action in the workflow is triggered by a Merge Request to a certain Git repository, which then would trigger the relevant automation.

Possible candidates

Build-related:

• Build of a package in a Koji sidetag
• Tag a package to a Koji tag
• Submit a Bodhi Update
• Build a new compose
• Mass rebuild
• Create new branch

Organizational:

• Request a new package
• Request a new SIG
• Request a new event
• Submit a calendar entry for a Meeting

Reverse GitOps

In the GitOps workflow Merge requests trigger the workflow steps. I'd like to propose the term Reverse GitOps for the system where every workflow action triggers a Merge Request.

The idea is that before we adopt the GitOps workflow we create a "Git-shadow" of a current process. And we let it run for a while to see how real-world events are represented in the Git repo we have created.

Let's consider the example:

I'd like to turn every Bodhi update into a Merge Request to a Git repo.

I already have a stream of Bodhi updates. I setup a Git repo on a side and create the event listener which for every real Bodhi update creates a Merge Request to my repo which represents it. I merge the MR as soon as Bodhi update is shipped. I can then iterate on the exact content of the repo and the form of the MR - yaml schema and so on, while keeping it aligned with the existing process.

Using this approach we can test GitOps workflow before making any disruptive changes to the main pipeline.

Questions

Does it mean we will write yaml files all day long?

Yes. There is of course nothing stopping us from adding a higher level tools which would provide shortcuts for yaml editing and MR submissions. Think Emacs YASnippet, or whatever cli aliases. Yet the expectation should be that the entire workflow is doable without any special tools like fedpkg just with the basic knowledge of Git.

Does Git Forge matter?

No. The basics - Git repo, Merge Request workflow and the possibility to do code review and CI on "MR created" and "MR merged" events is the shared functionality of all Git Forge tools.

Does CI engine matter?

Somewhat. We don't need it for the design of the workflow, to define what the automation on MR should do. We do need it to run the automation.