# Component CI tldr;

## Resources

* Per component DLRN design [jpena]
* https://docs.google.com/drawings/d/1Qvp5kEI8C1XaBK19-7OQfpD5M7hAM2AlBunkYt4zkDs/edit
* <weshay> notes this is not up to date atm</weshay>

* Use rdoinfo / ospinfo to tag each package to a component
* https://github.com/redhat-openstack/rdoinfo/tree/master/tags
* e.g. https://review.rdoproject.org/r/#/c/22394/11/tags/stein.yml
* git.app<snip>/git/ospinfo.git/tree/

* Repos in component based promotion [jpena]
* https://docs.google.com/drawings/d/11FPV_0sb0jZx9758lDfs86BqnlIWMWnWixqf_TO_6Go/edit
* <weshay>phase 2 is accurate
* phase 1 will have consistent -> tripleo-ci-testing </weshay>

* Component CI timeline [panda]
* https://docs.google.com/drawings/d/1Es4doXw2UEafZ0yxg6m4sMeCghCl2eS-OtxzB5pTAfc/edit
* <weshay> we just about have all the info we need now to fill this out </weshay>

* Full Internal Production chain design [migi]
* https://docs.google.com/drawings/d/1KcKh-WzRNrLAVF70w1aXtBDlaVp_TyVdJbxxd4sMSi4/edit

* Taiga user story
* https://tree.taiga.io/project/tripleo-ci-board/us/1376
* current general promotion: https://drive.google.com/file/d/1p9UFJNFKKKiOQB7N08TTHfdox1HghL35/view
* new component promotion: https://drive.google.com/file/d/1Ry8Mlvl-XqdqxOshtQo4NYf6jbK6I9ce/view

* Component CI repo to use:
* https://trunk-staging.rdoproject.org/centos7-master/
* API
* https://trunk-staging.rdoproject.org/api-centos-master-uc/api/report.html
* rdoinfo patch seperating component
* https://review.rdoproject.org/r/#/c/22394/

* Other's meeting notes:
* https://hackmd.io/n-VUTo3jQ0iVvki9I_a4AQ

## Glossary

* Component
* A set of packages supporting a single purpose set of services
* for compute component
* It would be nova, python-novaclient
* Component can have a set of one to many source repos
* upstream source repos are verified on each gerrit change and merge. Internal code.engineering repos may or may not have check/gate jobs verifying rebase or patches.
events using upstream jobs defined in .zuul.yaml or zuul.d dir in source repo
* A component can be verified by a set of tripleo / OSP jobs
* RDO is a basically a set of yum repositories.

* Common
* every package that does not belong to any other component.

## Questions

* Where components are defined?
* As per this review: https://review.rdoproject.org/r/#/c/22394/
in rdoinfo project under rdo.yml file, component: <component name>
tag and component: common is added.

* What is the common component?
* common means "every package that does not belong to any other component"
* There is arbitrary definition for a number of components e.g. compute, network, storage, labels are assigned to the packages,
we're left with some packages that do not clearly fit into any particular component,
***those go into the fallback "common" component***
* for example, oslo libraries do not seem to belong logically to any defined component. Oslo will be defined in the common component.

* What are the phases of promotion?
* There are two phases here:
* 1st phase also known as **component promotion**:
* current-tripleo dlrn.repo + component_consistent drln.repo => component-<component_name>-current-tripleo-compute
* the component hash from component_consistent drln.repo gets promoted.
* 2nd Phase also known as **integration promotion**
* Each promoted component
* e.g. current-tripleo-compute
* The unpromoted / untested common component
* e.g. dlrn-server/component/common/consistent/delorean.repo

* What are the repo priorities in phase-1 and phase-2 jobs?
* in phase-1:
* component-repo priority one
* current-tripleo repo priority two
* <weshay> I'll note that once all the components are built out, we will treat and test the common component the same as the other components</weshay>
* in phase-2:
* Each promoted components repo - priority one
* untested common **<weshay>will be untested only in POC</weshay>** consistent repo - priority one

* Who is going to set the priority of the repos?
* DLRN always set the priority of all the repos to 1. the consumer
needs to set the priority to depending upon their usecases.

* What is going to change in component promotion from current promotion?
* In current promotion, we go to https://trunk.rdoproject.org/centos7-master
* from here: grab consistent/dlrn.repo -> tripleo-ci-testing/dlrn.repo
-> current-tripleo/dlrn.repo
* In component promotion, we go to https://trunk-staging.rdoproject.org/centos7-master/
* from here: grab **component/compute/consistent/dlrn.repo
-> component/compute/tripleo-ci-testing-compute/dlrn.repo
-> component/compute/current-tripleo-compute/dlrn.repo
Note: we need to send a dlrn api call to update the dlrn hash in current-tripleo-component/dlrn.repo.
-> (once all components promotes) grab the centos7-master/current-tripleo-component/dlrn.repo
-> current-tripleo/dlrn.repo
** <weshay>this deviates from the current model in that a new directory is created. In any deviation I would have to ask if there is a good </weshay>
* then all current-tripleo-{all components combined + consistent_common} will give -> current-tripleo
* once we promote a commit in the compute component using "tripleo-ci-testing-compute",
the DLRN API will automatically create centos7-master/component/compute/tripleo-ci-testing-compute
and centos7-master/tripleo-ci-testing-compute
* Note: in future: https://trunk-staging.rdoproject.org/centos7-master/current-tripleo-component/dlrn.repo to grab
all the promoted component from one place

## THIS NEEDS CONFIRMATION FROM TRIPLEO-CI + RDO TEAM
* The yum repos will be defined as
* https://trunk-staging.rdoproject.org/centos7-master/consistent/delorean.repo
* Confirm w/ jpena whether or not the above url path can be use to achieve the following. **We are fine w/ another path OR constructing the following in CI**

# This is a set of untested rpms from the common component.
# Again any rpm that does not fit into a component definition.
[delorean-component-common]
name=delorean-openstack-tobiko-4d14b44332bff4b83dfccfe96f31250218bb1f96
baseurl=https://trunk-staging.rdoproject.org/centos7/component/common/4d/14/4d14b44332bff4b83dfccfe96f31250218bb1f96_638b9ca3
enabled=1
gpgcheck=0
priority=1

# This must be a promoted version of the component-compute
[delorean-component-compute]
name=delorean-openstack-nova-0138fb1ada9957c21427c5680dc47d6527a5a703
baseurl=https://trunk-staging.rdoproject.org/centos7/component/compute/01/38/0138fb1ada9957c21427c5680dc47d6527a5a703_a545aa4b
enabled=1
gpgcheck=0
priority=1

^^- To get the above, we should use different repos:
* https://trunk-staging.rdoproject.org/centos7-master/consistent/delorean.repo (priority: 2) would give us the latest untested rpms from the common component
* https://trunk-staging.rdoproject.org/centos7-master/component/compute/component-ci-passed/delorean.repo (priority: 1) would give us the latest set of packages from the compute component that passed component CI.
* https://trunk-staging.rdoproject.org/centos7-master/component-ci-passed/delorean.repo (priority: 1) would instead give us the latest set of packages from _all_ components that passed component CI.

# Question: Does the component pipeline take the latest tripleo changes?

[delorean-current]
name=delorean-current
baseurl=http://mirror.dfw.rax.opendev.org:8080/rdo/centos7/d7/e7/d7e7abe63f66da722ed5cc6457a85cae53bb8629_6682698d
gpgcheck=0
enabled=1
priority=10
includepkgs=ansible-role-container-registry,ansible-role-tripleo*,ansible-tripleo-ipsec,instack,instack-undercloud,openstack-tripleo-*,os-apply-config,os-collect-config,os-net-config,os-refresh-config,puppet-*,python*-tripleo*,python*-paunch*,paunch-services,tripleo-ansible,ansible-config_template

* Adv... this would allow the latest changes from tripleo like container changes into the component pipeline
* DisAdv... today the promotion jobs DO NOT take delorean-current and this would a diversion from defined pipelines we use today

In the end this provides the ideal set of packages to achieve the following goals in the integration jobs.
### 1. test known good components with any untested rpms.
### 2. minimize the set of changes tested in any given job
### 3. Ensure the promoted components work together

^^- This is entirely a CI design decision. From the repo perspective, we could:
* Include the latest consistent set of packages from the tripleo component on each job (https://trunk-staging.rdoproject.org/centos7-master/component/tripleo/consistent/delorean.repo). With this, we no longer need to specify the includepkgs option, just play with repo priorities.
* Use the top-level consistent repo (as above) and keep the includepkgs directive.
* Promote the tripleo component as we will do for the rest, and use the component-promoted tripleo repo.
* Any other option?

* Where is component consistent dlrn.repo defined?
* https://trunk-staging.rdoproject.org/centos7/component/compute/consistent/delorean.repo

* Question: Any technical blockers regarding starting the internal component pipeline w/ a dlrn-trunk-staging?

* ~~Where is all consistent dlrn.repo defined?~~
* ~~https://trunk-staging.rdoproject.org/centos7-master/consistent/delorean.repo~~
* ~~It contains delorean-component-common and delorean-component-compute repos~~
* ~~delorean-component-common is the repo file for common component~~
* ~~In first phase, we can disable this repo.~~
* ~~It is there so that anyone who wants to test with the latest common components~~
* ~~delorean-component-compute is the repo file for compute component~~
* ~~It will be used in phase-1 promotion job~~

## Sprint 18 Goals for the component pipeline
1. What are the results from multiple runs against a known good build + new nova bits
2. What is the best way to build containers just for nova
* single job?
* multiple jobs with push / pull of containers?
* don't build, just update?
3. What's the best way to promote a component?

### What has been accomplished in Sprint 18
1. We are able to build containers per component
2. We can promote

## Sprint 19 Goals for the component pipeline

1. Do we understand and have enough data to make a decision with regards to dlrn-current in the component pipelines?

## passed components that will move to integration

This is the dlrn.repo that defines the promoted compnents

dlrn-staging/current-tripleo-component/dlrn.repo

# This is a promoted component.
[delorean-component-keystone]
name=delorean-openstack-tobiko-4d14b44332bff4b83dfccfe96f31250218bb1f96
baseurl=https://trunk-staging.rdoproject.org/centos7/component/keystone/4d/14/4d14b44332bff4b83dfccfe96f31250218bb1f96_638b9ca3
enabled=1
gpgcheck=0
priority=1

# This is a promoted component-compute
[delorean-component-compute]
name=delorean-openstack-nova-0138fb1ada9957c21427c5680dc47d6527a5a703
baseurl=https://trunk-staging.rdoproject.org/centos7/component/compute/01/38/0138fb1ada9957c21427c5680dc47d6527a5a703_a545aa4b
enabled=1
gpgcheck=0
priority=1

## Downstream component CI promotion

* What are the phases of downstream component CI promotion?

* Once a component gets promoted upstream
* then there is a rebase of the same component in code.eng.

* During rebase CI jobs will run
* If rebase is clean i.e. once code is successfully merged in code.eng.
* DLRN will build the package then again component pipeline will start
* [need to decide] if component pipeline get blocked for longer time may be 3-4 week
* How to deal with the rebase?

* Once component promotion happens upstream, how we can signal downstream for rebase?
* How rebase will be done?
* Send the rebase as a gerrit patch? need to explore how it works?
* if the rebase is frequent, the downstream cherry-pick gets out of sync, how to deal with that?