Hook add-determinism into the RPM build process

submitted as https://fedoraproject.org/wiki/Changes/ReproduciblePackageBuilds

Summary

This change integrates add-determinism to the RPM build process such that RPMs released by Fedora will not have non-deterministic components which prevent reproducability.

Owner

Name: Davide Cavalca, Neil Hanlon,
Email: dcavalca@fedoraproject.org, neil at shrug.pw,

Current status

Detailed description

As of 2023 there is an ongoing effort to implement Reproducible builds in Fedora. Reproducible builds will allow our users to be able to independently verify that the RPMs have not been tampered with (either maliciously or hardware/software fault): someone can do an independent rebuild of a package and confirm that they get identical binaries when building with the same versions of the compiler and other tools. This Change allows us to move forward in this direction by removing a source of irreproducibility issues.

add-determinism is a rust program which, as its name suggests, adds determinism to files that are given as input by attempting to standardize metadata contained in binary or source files to ensure consistensy and clamping to $SOURCE_DATE_EPOCH in all instances. add-determinism can be considered the "fedora version" of strip-nondeterminism from the Debian project. Since strip-nondeterminism is written in perl, it is undesirable for use in Fedora, as we don't want to pull perl in the buildroot for every package.

It's worth noting that this Change does not intend to impose any specific reproducibility requirements on Fedora packages.

Clamp build mtimes to SOURCE_DATE_EPOCH

Feedback

(pending discussion)

Benefit to Fedora

Adding determinism (i.e., removing non-determinsim) enables the Fedora community to have confidence that, if given the same source code, build environment, build instructions, and metadata from the build artifacts, any party can recreate copies of the artifacts that are identical except for the signatures and some parts of metadata.

As proposed, this change does add a small amount of time to the processing of RPMs at the end of a build. Accordingly, packages containing large quantities or sizes of files would take longer to process than smaller packages. In addition, add-determinism takes steps to ensure it does not interfere with other buildroot post processors, like mangle-shebangs, python-hardlink, python-bytecompile, others.

Scope

Proposal Owners:
- Integrate add-determinism as a BuildRoot Policy script to run after all packages
Other Developers:
- Test their packages with added determinism, report problems
- Potentially integrate changes to packages to enable reproducibility
Release Engineering: mass rebuild required (?)
Policies and Guidelines: Fedora Packaging Guidelines should be updated to include information on the add-determinism BuildRoot Policy. User documentation should be amended to include instructions on how to verify reproducibility for a given package, and what packages are known to be non-reproducible.
Trademark approval: N/A (not needed for this Change)
Alignment with Community Initiatives:

All software and requests are consistent with the decision process and similar across other groups in Fedora. The Fedora Reproducibility Working group begin at Flock 2023 in Cork.

Upgrade/compatibility impact

No impact is expected.

How To Test

build a local copy of redhat-rpm-config with https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/293
install add-determinism
build packages ;)

User Experience

No impact is expected.

Dependencies

This Change requires integration into redhat-rpm-config (see https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/293).

Contingency Plan

The redhat-rpm-config PR would be reverted.

Documentation

See the add-determinism README at https://github.com/keszybz/add-determinism

Release Notes

Fedora package builds are now more deterministic, bringing the distribution closer to the goal of achieving fully reproducibile builds for all of its packages.