How to build OKD in 2024

Prow

This approach would continue to leverage Prow as the OKD build system.
Images would be built pre-submit (as CI on PRs) and post-submit (on merge, promoted to OKD ImageStream)

Pros

• Direct feedback to devs (on status) through GitHub UI
• Source of truth build metadata from Prow
• Prow has ability to replace FROM directives in Dockerfiles
• Continuous builds (ie. components are rebuilt immediately after they've been changed)

Cons

• Need to touch/create configs for each repo (~200)
  • [vrutkovs] we wouldn't need to if we keep UBI base
• Build configs obscured in release repo, making build reproduction hard
• [vrutkovs] CentOS9 Stream may introduce new bugs, so OKD bugs are no longer OCP bugs

Questions?

What's still needed

Doozer + Prow

Doozer + Konflux

This approach would use Doozer to generate OKD sources from ocp-build-data metadata. The generation step would manipulate Dockerfiles to replace FROM directives referencing builder and base images.
As a second step, these generated sources would have to be fed into a separate system for building the images.

Pros

• Maintenance shared with ART team
• Source of truth build metadata from ocp-build-data

Cons

• ocp-build-data does not reference OKD-specific Dockerfiles (e.g. Dockerfile.okd or Dockerfile.scos) that exist for a couple of images. When generating the sources with Doozer, these images would have to be accounted for.
• No direct feedback to devs when a build breaks
• Discreet nightly builds

Questions?

• How to get from generated sources to built images?
• Where would the builds happen? How would they be scheduled?
• Could images be built only-if-changed?
• How can feedback be provided to devs?
• Does each repo require a .tekton dir with PipelineRun definitions for this approach?

What's still needed

okd-payload-pipeline

This approach would leverage https://github.com/okd-project/okd-payload-pipeline for OKD builds end-to-end.

Pros

• Based on upstream operators like Argo Workflows and Shipwright
• Responds to the community request for "reproducible builds"
• Ability to replace FROM directives in Dockerfiles
• Enable partners and contributors to produce their own builds (replacing one or more components or rebuilding the payload entirely)
• Except for the infrastructure resources availability, multiarch is already possible

Cons

• No direct feedback to devs when a build breaks
• Discreet nightly builds
• Static metadata updated manually: there is still some manual work to be done in-between branch-cut dates to sync Dockerfiles images replacements and other changes in the Prow configs
• How would we build multi-arch for all arches? we do not have a single cluster setup that supports all arches.
  • Other architectures can be achieved through remote workers joining the cluster
• [vrutkovs] Not practical to keep up the pace of building nightlies

Questions

• Can we provide feedback to devs?

What's still needed

• Can we automatically derive the build configs from the ocp-buildmetadata or from some other source (metadata in the CVO?) and prow?
• Run the pipeline periodically and add a step to provide feedback about overall build failures
• Push the release back to app.ci or quay.io and create a releasestream in the okd release controller (this can also allow running origin through Prow)
• Produce different builders for each required Golang version

Konflux only

This approach would build component images with Konflux via Tekton with config contained in .tekton dir in each repo.

Pros

• same system for OKD and future (4.17?) OCP production builds
• could run builds as CI or as scheduled Nightly
• build configs co-located with sources, making the process transparent
• Direct feedback to devs (on status) through GitHub UI

Cons

• involves touching each repo and creating pipeline config
• longer adoption time given that konflux is in its nascent stages
• Need to touch/create configs for each repo (~200), branch, variant (okd/ocp)

Questions?

What's still needed