owned this note   changed 5 years ago  
Published Linked with GitHub

Unifying rpm building process in CI

Problem Statement

In TripleO Deployment, in upstream and downstream all the tripleo & infrared CI jobs deploys tripleo using
RPM.
in Tripleo CI, we have build-test-packages role https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/build-test-packages.
It is an Ansible role for generating custom RPMSs from upstream Gerrit changes in the TripleO project using DLRN. This repo then can be injected in the tested environment, a repo file created and a yum update should start using the built RPMs.

For example: A developer working on tripleo-ansible patch and tripleo-heat-templates patch and wanted to test it together then, He pushed the tripleo-ansible patch
and then add Depends-On: <tht patches> in the commit message. During TripleO CI run, build-test-packages extracts the tripleo-ansible commit message and depends-on tht patches
using zuul deps.py. Then it uses rdoinfo to find the right project and then same information is passed to DLRN to build both packages from that commit. Once packages are built.
It creates a yum repo and passed it tripleo jobs where yum update operation updates the latest package.

The same thing is not possible in downstream. It makes downstream developers hard to test the patches in downstream dist-gits.

We have https://pagure.io/zuul-distro-jobs (maintained by PCD team) which contains all building blocks for above scenario. So now we have multiple tools doing same things.
We want to remove the maintaince and work on using one tool to solve the above problem.

Solution

Things to discuss

 
<migi> chandankumar: https://docs.google.com/presentation/d/1YPJCn_94z2ryLJi5sWfD_-RqvHxYQXWp3WPWR-Odlxo/edit#slide=id.g70426dd9e3_0_412
<migi> https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/osp/job/osp-rpm-build
<migi> https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/osp-jobs.git;a=blob;f=zuul.d/jobs-base.yaml;h=6e14c619e8ec55ddd8f2f28e7661c56e05f268ff;hb=HEAD#l38
<migi> https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/osp-jobs.git;a=blob;f=playbooks/rdopkg/build.yaml;h=7b91c38f145ce3df37bd583d527792f341f54191;hb=HEAD
<migi> https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/osp-jobs.git;a=tree;f=roles;h=967073cfa70f8758cb3e84a7ed453251a60fc2d4;hb=HEAD
<migi> https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/osp-jobs.git;a=blob;f=roles/rdopkg_patch_components_setup_environment/tasks/main.yaml;h=ea223826a21eef7d815690ef46178b64ae2748c4;hb=HEAD
<migi> https://github.com/rhos-infra/patch-components/tree/master/roles
<migi> https://github.com/rhos-infra/patch-components/blob/master/roles/patch_rpm/files/patch.sh
<migi> https://github.com/rhos-infra/patch-components/blob/master/roles/patch_rpm/tasks/patch_component.yml
<migi> https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/osp-jobs.git;a=blob;f=playbooks/rdopkg/build.yaml;h=7b91c38f145ce3df37bd583d527792f341f54191;hb=HEAD
<chandankumar> https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/build-test-packages/defaults/main.yml
<chandankumar> https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/build-test-packages/defaults
<chandankumar> https://opendev.org/openstack/tripleo-quickstart/src/branch/master/roles/repo-setup
<migi> https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/osp-jobs.git;a=blob;f=playbooks/rdopkg/fetch.yaml;h=6afca0abaec6983a97ee555f6ec9da63bd0daeff;hb=HEAD
<migi> https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/osp/build/c78bb11166e247fda046264179c2c62c/console
<migi> https://github.com/rhos-infra/patch-components/
<migi> https://github.com/rdo-packages/nova-distgit/blob/rpm-master/openstack-nova.spec#L117
<migi> https://opendev.org/openstack/nova/src/branch/master/bindep.txt
<migi> pcre-devel [platform:rpm test]
<migi> pcre-devel [platform:rhel rpmunit rpmbuildrequires]
* Disconnected ()
<chandankumar> migi sorry power cut
<chandankumar> back now
<migi> chandankumar: ack
<migi> https://code.engineering.redhat.com/gerrit/#/c/192830/51/zuul.d/osp-tox-jobs.yaml
<migi> https://github.com/rdo-packages/nova-distgit/blob/rpm-master/openstack-nova.spec#L161
<migi> https://github.com/rdo-packages/nova-distgit/blob/rpm-master/openstack-nova.spec#L255
<migi> https://github.com/rdo-packages/nova-distgit/blob/rpm-master/openstack-nova.spec#L33-L40
<migi> pcre-devel [platform:rhel rpmunit service-nov]
<chandankumar> migi https://github.com/rdo-packages/nova-distgit/blob/rpm-master/openstack-nova.spec#L728
<chandankumar> https://github.com/rdo-packages/nova-distgit/blob/rpm-master/openstack-nova.spec#L692
Select a repo