owned this note   changed 5 years ago  
Published Linked with GitHub

Content Provider / Parent-Child work items

tags: Design

changes

https://projects.engineering.redhat.com/browse/TRIPLEOCI-210

to-do

  • build a poc job ( sagi )
  • get review from tripleo, infra-team ( wes )
  • socialize w/ ci team ( wes )

Documentation

Glossary / Terms / Naming conventions

Templates

Create a new template w/ an additional "-pipeline" template name

Parent of consumer job

(when creating new parent) tripleo-ci-undercloud-content-consumer-centos-8

Doc

Plan

  • (sshanidm) Define a job that can expose containers registry link and YUM repo link
  • (chandan)Define a test job (i.e. scenario001) that consumes and uses these links
  • Create a job that builds RPM and containers and pushing containers to local registry. Make these steps optional with parameters.
  • To configure child job (scenario001) to use correctly these artifacts if need. Other branchful jobs should be able still to build/download containers by their own.
  • demo DF.
  • rename standalone-provider –>
    • tripleo-ci-centos-8-content-provider (sagi)
    • tripleo-ci-centos-7-content-provider (sagi)
  • communicate plan (wes)
  • move all rdo third party jobs to the rdo registry. (wes w/ sagi's review)
  • plan out branchful jobs.. ( see below )
  • Test multinode jobs w/ a provider job (sagi)
  • push ceph containers to openstack zuul registry or quay or both ( manual ) (sagi / chandan)
  • podman pull on the provider for non-tripleo containers (need code ) ( sagi / chandan )
    • Need to add redundant registries to pull from quay, rdo-registry, docker.io, in that order
  • migrate ussuri ( sagi / chandan )
  • migrate train centos8 ( sagi / chandan )
  • migrate train centos7 ( pending decision re: Alex on c7 train )
  • work out provider parent for changes that have both c7 and c8 jobs.
  • migrate stein centos7 ( pending results from just using quay.io)
  • migrate the rest ( pending results from just using quay.io )

Planning meeting Agenda, Discussion Items:

address the current issues

  • atm we have gate jobs running that are not in check.

to-do

Branchful parent / child jobs Plan

  • Idea
    • Add a content provider job for each release
    • Put it as a dependencies on the branchful job
    • Structure:
      • centos-7-train-content provider
      • standalone-7-train:
        • dependencies:
          • centos-7-train-content provider
      • centos-7-stein-content provider
      • standalone-7-stein:
        • dependencies:
          • centos-7-stein-content provider

meeting notes

  • content provider for each release
    • question: how complicated is that zuul config.. every branchful job will have a dependency
    • really this is just for the tripleo-ci repo.
    • concern about time:
  • rocky, stein, queens
    • rocky.. let's drop all but one job
      • remove branchful
    • stein.. let's drop all but one job
      • remove branchful
    • queens.. need to keep what we have.
TO-DOs

CentOS -7 based content provider jobs

  • Idea
    • run runv3 playbook to set the env and perform build test packages
      then call tipleo-ci container-builds role and push the contents to local registry and pause th job and consume it in same fashion

meeting notes

  • c7 is following the same pattern as c8 jobs w/ a content provider, however implementation jobs is different.

  • We may choose not to use c7 content providers and just pull containers from key.

Confirm Promoter Server push docker.io && Quay.io

update and upgrade jobs [Need discussion]

Before that: How updates and upgrade jobs work?
Ideas:

  • There is no issue with update jobs as there will be package updates only? so no changes required there
  • for upgrade job, we might to have two content provider for example
    • upgrade job from train to ussuri
      • we need to build containers for both train and ussuri and put both
        jobs as deps
  • layout
- c8-content-provider-previous-current-tripleo
- c8-content-provider-current-tripleo
- c8-update job:
    dependencies:
      - c8-content-provider-previous-current-tripleo
      - c8-content-provider-current-tripleo
- c8-standalone job:
    dependencies:
      - c8-content-provider-current-tripleo

meeting notes

  • deploys w/ previous-current-tripleo updates to current-tripleo
  • two content providers for update / upgrade jobs
    • n + 1 jobs
      • n content provider
        • build gerrit changes for n
        • provide yum repo for n
        • provide containers for n
      • n +1
        • build gerrit change for n+1
        • provide yum repo for n+1
        • provide container for n+1
      • in CI after deployment, prior to upgrade switch the yum repos, and update the containers-prepare-params.yml

update / upgrade TO-DO

Focus here should be soley on CentOS-8. Once we have validated this w/ C8, come back to CentOS-7

  • create two content-provider jobs
    • one for previous-current-tripleo
    • one for current-tripleo ( exists )
    • ensure the switch from previous-build-content-provider
      • yum repo change (this may exist)
      • switch container-prepare-params
    • review and validate this work
  • call 22 sep:
    • update job should be OK sshnaidm already fixed - he will verify
    • standalone-upgrade - pull for tripleo-u deployment and then re-use the built containers for the upgrade (master current-tripleo) from provider
    • then can workout the building of tripleo-u too…
      • TBD if there will be one provider job with different namespace in same registry for example
      • or just build in the standalone-upgrade job… since no-one else will consume tripleo-u (maybe the multinode upgrade job can re-use.)

For rdo software factory jobs

meeting notes

Handling jobs defined in zuul layouts outside of tripleo-ci.

  • Ensure any job outside of tripleo is using standalone w/ build-containers flag set.
    • victoria, ussuri, train c8
  • stein, rocky, queens
    • set the jobs to use quay.io
    • consider a non-tripleo job template that is the standard for non-standalone jobs.
    • consider asking other projects to drop tripleo jobs if quay.io is a problem

Any changes to component pipeline?

  • these jobs should all use the rdo-registry

Templates

tripleo-ci templates that need to switch to the new content provider:

tripleo-repos

https://projects.engineering.redhat.com/browse/TRIPLEOCI-210

Switch to content provider jobs/templates

Q&A (Sprint #34)

gerrit-topic: new-ci-job

(NOT new-ci-jobs)
https://review.opendev.org/#/q/branch:master+topic:new-ci-job

Original provider patch for reference

https://opendev.org/openstack/tripleo-ci/commit/f360f4168f7933bb561093a04c5c7f3c26200cd5

How does content provider jobs work?

https://zuul-ci.org/docs/zuul/reference/jobs.html#pausing-the-job

Where is it implemented in our jobs?

https://github.com/openstack/tripleo-ci/blob/7acd24cec0d0ab97a7c5d17c5d2ab1dc85df3106/playbooks/tripleo-ci/run-provider.yml#L14

Which jobs are using content provider parent ?

standalone* and multinode* jobs parent content provider job (or any job that has dependencies on content provider)

            dependencies:
              - tripleo-ci-centos-8-content-provider

Clarify what needs to be done in this task: Wire-up content provider pipeline into tripleo repos

Switch to content-provider jobs in all tripleo/ci repos
See https://projects.engineering.redhat.com/browse/TRIPLEOCI-210

Select a repo