# Component and Intergation CI for Release Delivery
###### tags: `Documentation`
### What we heard in the sync meeting..
* rdo promoted components -> candidate compose..
* compose is triggered on http://osp-trunk.hosted.upshift.rdu2.redhat.com/rhel8-osp16-2/current-tripleo/
#### Answer from Shreshtha
:::info
* Candidate composes is from OSP promoted components.
* Trunk compose is triggered by RDO current ( http://osp-trunk.hosted.upshift.rdu2.redhat.com/rhel8-osp16-2/current/
:::
### action items
* AI: build jobs that validate changes to rhos-release... However we do not think the coverage is adequate to run a job or two here.
* Attempting to ru w/o candidate repo - using rhos-release args: "-P 17-nightly ceph-4 -r 8.3"
* We heard the nightly compose is built from RDO promoted components.. is it RDO promoted components or OSP promoted components
:::warning
* Should our intergration pipeline pick up the nightly compose?
* If it's built on RDO promoted components, can it be built w/ OSP Promoted Components
:::
### rhos-17
* traditionally we use trunk-candidate ( please define ) [trunk-candidate](http://download.eng.bos.redhat.com/rel-eng/repos/rhos-17.0-rhel-8-trunk-candidate/tagged-builds.lst)
* on 10/13/2020 - lon/rlandy
```
rlandy|rover> http://download.eng.bos.redhat.com/rel-eng/repos/rhos-17.0-rhel-8-trunk-candidate/tagged-builds.lst
[14:23:00] <rlandy|rover> <rlandy> nothing provides python3-network-runner >= 0.2.2 needed by python3-networking-ansible-4.0.1-0.20201008110557.6f56fd4.el8ost.noarch
```
* at the meeting.. trunk-candidate does not trigger all the time, we instead use the nightly compose.
* rlandy|rover> https://rcm-tools-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/Composes II - Nightly/job/nightly-compose-rhosp-17.0
* Also Pungi was mentioned.
* Project Pungi Compose for RHOSP 17.0 nightly (nightly)
* rhos_release_args: "-P 17-nightly ceph-4 -r 8.3"
* using Pungi we failed to build containers
----
## What is the basic information both rel-del and ci needs to understand
### Glossary of terms:
#### Delivery
**brew tags**: Applided during koji/brew builds that help identify and group packages.
**OSP Trunk**: The internal DLRN instance(s) used to sync and perform brew builds internally.
**Trunk Compose**: A pungi compose based on the latest builds from OSP Trunk for a given release.
**Non-Trunk Compose**: A pungi compose based on release candidate brew tags.
**dist-git**: A set of git repositories internally that hold rpm specfiles or the container dockerfiles.
**pungi** a tool to create composes.. makes the build look like what would be available via sub-man. provides repo closure
**odcs** on demand compose service. wrapper around pungi. Single use compose mainly for things like qcow builds. They have a limited lifespan.
**koji** rpm builder, internally know as brew.
#### CI
**Upstream TripleO CI** - tripleo ci deployments that execute against [tripleo repos](https://opendev.org/openstack/?tab=&sort=recentupdate&q=tripleo)
**3rd Party RDO CI** - CI that executes from [rdo softwarefactory](https://review.rdoproject.org/zuul/status). Can vote, but not prevent upstream changes from merging. Can execute against a variety of sources, upstream, github, non-openstack etc..
**Integration Pipeline** - Originally only used to protect upstream jobs from non-tripleo patches breaking tripleo ci. Now is both upstream and downstream. 20+ jobs created to represent *any* voting upstream job.
* upstream: look for "openstack-periodic-integration" in [rdo softwarefactory](https://review.rdoproject.org/zuul/status)
* Monitoring: [link](http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?orgId=1)
* internal: look for "openstack-periodic-integration" in [internal softwarefactory](https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/status)
* Monitoring: [link](http://tripleo-cockpit.usersys.redhat.com/d/KyHCwLHMk/rhos-16-2-full-component-pipeline?orgId=1)
**Component Pipeline** - CI to check and test a known good build with changes from ind
### Workflows
* where are the dockerfiles src for the compose. (rlandy)
## rhos-17 plan
* non trunk compose
* run diff
* align centos8 and rhel8 rpms
* run diff
* try to ci
----
* What we heard the workflow is..
```graphviz
digraph hierarchy {
nodesep=0.1
node [color=darkgreen,fontname=Courier,fontcolor=black,shape=box]
edge [color=darkgreen]
"RDO compute/consistent"->{ "RDO compute/component-ci-testing" }
"RDO security/consistent"->{ "RDO security/component-ci-testing" }
"RDO cinder/consistent"->{ "RDO cinder/component-ci-testing" }
"RDO compute/component-ci-testing"->{ "RDO compute/promoted-components" }
"RDO cinder/component-ci-testing"->{ "RDO cinder/promoted-components" }
"RDO security/component-ci-testing"->{ "RDO security/promoted-components" }
node [color=darkgreen,fontname=Courier,fontcolor=black,shape=box]
edge [color=darkgreen]
"RDO compute/promoted-components"->{ "RDO promoted-components" }
"RDO cinder/promoted-components"->{ "RDO promoted-components" }
"RDO security/promoted-components"->{ "RDO promoted-components" }
->" RDO --> Import to OSP Candidate Compose (where does this go?) "
->"wait for " -> "OSP current-tripleo"
}
```
-----
# Full CI workflow
Full picture, how things were designed to work.
Pasted to [googlejam board](https://jamboard.google.com/d/1p4ODk5MxuvJZyL42Wzm45v_3KY9_catxuUi4Z1cbYfQ/viewer?f=0) for notes.
```graphviz
digraph hierarchy {
nodesep=0.1
node [color=darkgreen,fontname=Courier,fontcolor=black,shape=box]
edge [color=darkgreen]
"RDO compute/consistent"->{ "RDO compute/component-ci-testing" }
"RDO security/consistent"->{ "RDO security/component-ci-testing" }
"RDO cinder/consistent"->{ "RDO cinder/component-ci-testing" }
"RDO compute/component-ci-testing"->{ "RDO compute/promoted-components" }
"RDO cinder/component-ci-testing"->{ "RDO cinder/promoted-components" }
"RDO security/component-ci-testing"->{ "RDO security/promoted-components" }
node [color=darkgreen,fontname=Courier,fontcolor=black,shape=box]
edge [color=darkgreen]
"RDO compute/promoted-components"->{ "RDO promoted-components" }
"RDO cinder/promoted-components"->{ "RDO promoted-components" }
"RDO security/promoted-components"->{ "RDO promoted-components" }
->{" RDO --> Import to OSP Delorean "}
->"OSP DLRN builds from import" ->"OSP compute/consistent" -> "OSP compute/component-ci-testing" -> "OSP compute/promoted-components" ->{ "OSP promoted-components" }
"OSP DLRN builds from import" -> "OSP cinder/consistent" -> "OSP cinder/component-ci-testing" -> "OSP cinder/promoted-components" ->{ "OSP promoted-components" }
"OSP DLRN builds from import" -> "OSP security/consistent" -> "OSP security/component-ci-testing" -> "OSP security/promoted-components" ->{ "OSP promoted-components" }
"OSP promoted-components" -> "OSP tripleo-ci-testing" -> "OSP Integration Tests" -> "OSP current-tripleo"
"OSP current-tripleo" -> "OSP Trigger Compose"
}
```
## RHOS-RELEASE and YUM REPOS
### OSP - 16.2
DLRN = http://osp-trunk.hosted.upshift.rdu2.redhat.com/rhel8-osp16-2/
Dependencies 16.2 [link](http://git.app.eng.bos.redhat.com/git/tripleo-environments.git/tree/config/release/rhos-16.2.yml)
* rhos-releaes
* osp_release: rhos-16.2
* rhos_release_args: "ceph-4 -r 8.3"
* yum repos
```
- type: generic
reponame: osptrunk-candidate-deps
filename: osptrunk-candidate-deps.repo
baseurl: http://download.eng.bos.redhat.com/rel-eng/repos/{{ osp_release }}-rhel-8-trunk-candidate/x86_64/
priority: 30
enabled: 1
gpgcheck: 0
- type: generic
reponame: rhosp-ansible-2.9
filename: rhosp-ansible-2.9.repo
baseurl: http://pulp.dist.prod.ext.phx2.redhat.com/content/dist/layered/rhel8/x86_64/ansible/2.9/os/
priority: 30
enabled: 1
gpgcheck: 0
- type: generic
reponame: osp-trunk-deps
filename: osptrunk-deps.repo
baseurl: http://download.devel.redhat.com/brewroot/repos/{{ osp_release }}-rhel-8-trunk-build/latest/x86_64
priority: 30
module_hotfixes: 1
enabled: 0
gpgcheck: 0
```
### OSP - 17.0
DLRN = http://osp-trunk.hosted.upshift.rdu2.redhat.com/rhel8-osp17/
Dependencies OSP-17 [link](http://git.app.eng.bos.redhat.com/git/tripleo-environments.git/tree/config/release/rhos-17.yml)
```
- type: generic
reponame: osptrunk-candidate-deps
filename: osptrunk-candidate-deps.repo
baseurl: http://download.eng.bos.redhat.com/rel-eng/repos/rhos-17.0-rhel-8-trunk-candidate/x86_64/
priority: 30
enabled: 1
gpgcheck: 0
- type: generic
reponame: rhosp-ansible-2.9
filename: rhosp-ansible-2.9.repo
baseurl: http://pulp.dist.prod.ext.phx2.redhat.com/content/dist/layered/rhel8/x86_64/ansible/2.9/os/
priority: 30
enabled: 1
gpgcheck: 0
- type: generic
reponame: osp-trunk-deps
filename: osptrunk-deps.repo
baseurl: http://download.devel.redhat.com/brewroot/repos/rhos-17.0-rhel-8-trunk-build/latest/x86_64
priority: 30
module_hotfixes: 1
enabled: 0
gpgcheck: 0
```
:::info
A real time view into the above promotions can be found here:
* [high level view of all promotions](http://dashboard-ci.tripleo.org/d/QrkFP-zGz/upstream-and-rdo-promotions?orgId=1)
* [component level promotions](http://dashboard-ci.tripleo.org/d/FE8Hf29Wz/component-pipeline)
:::
# meeting notes
## 10/29/2020
Compare 16.2 builds
- Control
- https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-16.2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-internal-rhos-16.2/c3ecaf6/logs/undercloud/var/log/extra/all_available_packages.txt.gz
- Test
- http://rhos-ci-logs.lab.eng.tlv2.redhat.com/logs/rcj/phase1-16.2_director-rhel-8.3-virthost-1cont_1comp_1ceph-ipv4-geneve-ceph/68/undercloud-0/var/log/extra/all_available_packages.txt.gz
Diff: https://termbin.com/ou35
Into the compose..
dump
versions.csv
delorean.repo
delreoan.md5
Ignore list:
*.debuginfo
*.debugsource
*-devel
*-doc
blosc-*
golang.*
hdf5-.*
jjoyce to check erlang
openstack-selinux-*
openstack-tripleo-heat-templates-compat
openstack-tripleo-validations-tests
python2.*
python3-heat-agent
python3-heat-agent-*
python3-kuryr-*
python3-matplotlib-*
pyxattr-*
yaml-cpp-static
Checked into https://code.engineering<foo>/gerrit/#/c/216562/1/compare_rpms/rhos-16_2.txt
## How patches are imported from upstream to internal
DLRN is working as expected here. Due to its design constraints, the import from RDO is done in several steps:
* a) Copy the RDO spec as-is.
* b) Rebase the downstream source git on top of the upstream commit being built, so our downstream-only commits are always up to date.
* c) Add patches from the downstream git via rdopkg (rhospkg actually). This also updates the changelog, and
sets the specific Release number in NVR, which is defined as -<a>.<timestamp>.<commit hash>.el8.ost, where
<a> is a OSP release-specific number (2 for 16.2).
* d) rhpkg import / rhpkg build
This ensures that any spec change done in RDO is brought to OSP without manual intervention, and the downstream
distgit -trunk-patches branch should never be manually modified.
Sign in with Wallet
Connect another wallet