owned this note   changed 6 years ago  
Linked with GitHub

Component CI tldr;

Resources

Glossary

  • Component

    • A set of packages supporting a single purpose set of services
      • for compute component
        • It would be nova, python-novaclient
    • Component can have a set of one to many source repos
      • upstream source repos are verified on each gerrit change and merge. Internal code.engineering repos may or may not have check/gate jobs verifying rebase or patches.
        events using upstream jobs defined in .zuul.yaml or zuul.d dir in source repo
    • A component can be verified by a set of tripleo / OSP jobs
    • RDO is a basically a set of yum repositories.

  • Common

    • every package that does not belong to any other component.

Questions

  • Where components are defined?

  • What is the common component?

    • common means "every package that does not belong to any other component"
    • There is arbitrary definition for a number of components e.g. compute, network, storage, labels are assigned to the packages,
      we're left with some packages that do not clearly fit into any particular component,
      those go into the fallback "common" component
    • for example, oslo libraries do not seem to belong logically to any defined component. Oslo will be defined in the common component.

  • What are the phases of promotion?

    • There are two phases here:
    • 1st phase also known as component promotion:
      • current-tripleo dlrn.repo + component_consistent drln.repo => component-<component_name>-current-tripleo-compute
      • the component hash from component_consistent drln.repo gets promoted.
    • 2nd Phase also known as integration promotion
      • Each promoted component
        • e.g. current-tripleo-compute
      • The unpromoted / untested common component
        • e.g. dlrn-server/component/common/consistent/delorean.repo

  • What are the repo priorities in phase-1 and phase-2 jobs?

    • in phase-1:
      • component-repo priority one
      • current-tripleo repo priority two
      • <weshay> I'll note that once all the components are built out, we will treat and test the common component the same as the other components</weshay>
    • in phase-2:
      • Each promoted components repo - priority one
      • untested common <weshay>will be untested only in POC</weshay> consistent repo - priority one

  • Who is going to set the priority of the repos?

    • DLRN always set the priority of all the repos to 1. the consumer
      needs to set the priority to depending upon their usecases.

  • What is going to change in component promotion from current promotion?

    • In current promotion, we go to https://trunk.rdoproject.org/centos7-master
      • from here: grab consistent/dlrn.repo -> tripleo-ci-testing/dlrn.repo
        -> current-tripleo/dlrn.repo
    • In component promotion, we go to https://trunk-staging.rdoproject.org/centos7-master/
      • from here: grab **component/compute/consistent/dlrn.repo
        -> component/compute/tripleo-ci-testing-compute/dlrn.repo
        -> component/compute/current-tripleo-compute/dlrn.repo
        Note: we need to send a dlrn api call to update the dlrn hash in current-tripleo-component/dlrn.repo.
        -> (once all components promotes) grab the centos7-master/current-tripleo-component/dlrn.repo
        -> current-tripleo/dlrn.repo
        ** <weshay>this deviates from the current model in that a new directory is created. In any deviation I would have to ask if there is a good </weshay>
      • then all current-tripleo-{all components combined + consistent_common} will give -> current-tripleo
    • once we promote a commit in the compute component using "tripleo-ci-testing-compute",
      the DLRN API will automatically create centos7-master/component/compute/tripleo-ci-testing-compute
      and centos7-master/tripleo-ci-testing-compute
    • Note: in future: https://trunk-staging.rdoproject.org/centos7-master/current-tripleo-component/dlrn.repo to grab
      all the promoted component from one place

THIS NEEDS CONFIRMATION FROM TRIPLEO-CI + RDO TEAM

# This is a set of untested rpms from the common component.
# Again any rpm that does not fit into a component definition.
[delorean-component-common]
name=delorean-openstack-tobiko-4d14b44332bff4b83dfccfe96f31250218bb1f96
baseurl=https://trunk-staging.rdoproject.org/centos7/component/common/4d/14/4d14b44332bff4b83dfccfe96f31250218bb1f96_638b9ca3
enabled=1
gpgcheck=0
priority=1

# This must be a promoted version of the component-compute
[delorean-component-compute]
name=delorean-openstack-nova-0138fb1ada9957c21427c5680dc47d6527a5a703
baseurl=https://trunk-staging.rdoproject.org/centos7/component/compute/01/38/0138fb1ada9957c21427c5680dc47d6527a5a703_a545aa4b
enabled=1
gpgcheck=0
priority=1

^^- To get the above, we should use different repos:

Question: Does the component pipeline take the latest tripleo changes?

[delorean-current]
name=delorean-current
baseurl=http://mirror.dfw.rax.opendev.org:8080/rdo/centos7/d7/e7/d7e7abe63f66da722ed5cc6457a85cae53bb8629_6682698d
gpgcheck=0
enabled=1
priority=10
includepkgs=ansible-role-container-registry,ansible-role-tripleo*,ansible-tripleo-ipsec,instack,instack-undercloud,openstack-tripleo-*,os-apply-config,os-collect-config,os-net-config,os-refresh-config,puppet-*,python*-tripleo*,python*-paunch*,paunch-services,tripleo-ansible,ansible-config_template
  • Adv… this would allow the latest changes from tripleo like container changes into the component pipeline
  • DisAdv… today the promotion jobs DO NOT take delorean-current and this would a diversion from defined pipelines we use today

In the end this provides the ideal set of packages to achieve the following goals in the integration jobs.

1. test known good components with any untested rpms.

2. minimize the set of changes tested in any given job

3. Ensure the promoted components work together

^^- This is entirely a CI design decision. From the repo perspective, we could:

Sprint 18 Goals for the component pipeline

  1. What are the results from multiple runs against a known good build + new nova bits
  2. What is the best way to build containers just for nova
  • single job?
  • multiple jobs with push / pull of containers?
  • don't build, just update?
  1. What's the best way to promote a component?

What has been accomplished in Sprint 18

  1. We are able to build containers per component
  2. We can promote

Sprint 19 Goals for the component pipeline

  1. Do we understand and have enough data to make a decision with regards to dlrn-current in the component pipelines?

passed components that will move to integration

This is the dlrn.repo that defines the promoted compnents

dlrn-staging/current-tripleo-component/dlrn.repo

# This is a promoted component.
[delorean-component-keystone]
name=delorean-openstack-tobiko-4d14b44332bff4b83dfccfe96f31250218bb1f96
baseurl=https://trunk-staging.rdoproject.org/centos7/component/keystone/4d/14/4d14b44332bff4b83dfccfe96f31250218bb1f96_638b9ca3
enabled=1
gpgcheck=0
priority=1

# This is a promoted component-compute
[delorean-component-compute]
name=delorean-openstack-nova-0138fb1ada9957c21427c5680dc47d6527a5a703
baseurl=https://trunk-staging.rdoproject.org/centos7/component/compute/01/38/0138fb1ada9957c21427c5680dc47d6527a5a703_a545aa4b
enabled=1
gpgcheck=0
priority=1

Downstream component CI promotion

  • What are the phases of downstream component CI promotion?

  • Once a component gets promoted upstream

    • then there is a rebase of the same component in code.eng.

      • During rebase CI jobs will run

    • If rebase is clean i.e. once code is successfully merged in code.eng.

      • DLRN will build the package then again component pipeline will start

  • [need to decide] if component pipeline get blocked for longer time may be 3-4 week

    • How to deal with the rebase?

  • Once component promotion happens upstream, how we can signal downstream for rebase?

  • How rebase will be done?

    • Send the rebase as a gerrit patch? need to explore how it works?
    • if the rebase is frequent, the downstream cherry-pick gets out of sync, how to deal with that?
Select a repo