# parent/child zuul container/rpm builds and artifacts
###### tags: `Design`

# Work Items tracked [here](https://hackmd.io/vRMVeZXZRgK5Vxqi6ENUDg)

# In check and gate build containers and rpms

* per gerrit change..
* parent job
* builds containers
* builds rpms
* share artifacts

# Reasoning behind this proposal
* mirror timeouts in upstream causing undercloud and standalone failures
* https://bugs.launchpad.net/tripleo/+bug/1889122
* https://trello.com/c/DcPnhDoj/1608-cixlp1889122tripleociproa-mirror-timeouts-in-upstream-causing-undercloud-and-standalone-failures
* Short summary
* During tripleo deployment we perform container image prepare, at
this step, we pull the containers from registry and modify the container image using tripleo-modify-image role, re-tag it and push it to local registry.
* In Upstream CI, we pull images from docker.io at that time it was hitting HTTPError: 429 Client Error: Too Many Requests for url leading to docker hub rate limit issue.
* https://www.docker.com/blog/scaling-docker-to-serve-millions-more-developers-network-egress/
* Free plan – anonymous users: 100 pulls per 6 hours
* Free plan – authenticated users: 200 pulls per 6 hours
* Pro plan – unlimited

## Implementation
* Registry options
* https://opendev.org/openstack/ansible-role-container-registry
* https://hub.docker.com/_/registry
* insecure.registry opendev.org
* open firewall for registry

* rpm builds / yum repo
* use gating.repo via build-test-packages
* expose via httpd
* open firewall for yum repo.. port 80
* see.. https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/install-built-repo/templates/install-built-repo.sh.j2
* this should have the ip of the node in the parent job

* Connection between jobs
* update release files to point to dynamic ip of parent job
* default back to docker.io

* in parent job update zuul to hold the node in parent
* zuul option "vars.pause = true"
* https://zuul-ci.org/docs/zuul/reference/jobs.html#pausing-the-job

## what do we need to change
* put option on build-test-packages not to run on the child jobs
* update container-prepare-parameters.yaml to point to node in parent job

## benefits of this design
* jobs won't have to update each container w/ tripleo-image-modify role. The containers will be built from current-tripleo + patch(s)
* parent job will fail and not run child jobs if
* containers fail to build
* rpm fails to build
* unit test failures in rpm build
* net sum is fail faster :)

## drawbacks of this design
* perhaps less feedback as child jobs fail

## alternatives
* See conversation on #openstack-infra
* http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2020-07-28.log.html#t2020-07-28T16:12:40
* data in proxy
* http://paste.openstack.org/show/796392/
* http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2020-07-29.log.html#t2020-07-29T18:16:36

# Current progress

Sagi is working w/ local reproducer to create parent child jobs w/ container builds.

Poc reviews:
* https://review.opendev.org/#/q/topic:new-ci-job+(status:open+OR+status:merged)

parameter_defaults:
    ContainerImagePrepare:
    -   set:
            ceph_alertmanager_image: alertmanager
            ceph_alertmanager_namespace: docker.io/prom
            ceph_alertmanager_tag: v0.16.2
            ceph_grafana_image: grafana
            ceph_grafana_namespace: docker.io/grafana
            ceph_grafana_tag: 5.4.3
            ceph_image: daemon
            ceph_namespace: docker.io/ceph
            ceph_node_exporter_image: node-exporter
            ceph_node_exporter_namespace: docker.io/prom
            ceph_node_exporter_tag: v0.17.0
            ceph_prometheus_image: prometheus
            ceph_prometheus_namespace: docker.io/prom
            ceph_prometheus_tag: v2.7.2
            ceph_tag: v4.0.10-stable-4.0-nautilus-centos-7-x86_64
            default_tag: true
            name_prefix: centos-binary-
            name_suffix: ''
            namespace: 198.72.124.111:5001/tripleomaster
            neutron_driver: ovn
            rhel_containers: false
            tag: f31fc1ea72ab1f45eb1b56d06e3a1640
        tag_from_label: rdo_version

## standalone job and option to build containers locally.
* Variables in play
* zuul.build_containers_images
* native tqe variable, build_container_images

* do not enable build_container_images in fs052
* keep the tqe build_container_images variable so I can build and deploy locally
* CI jobs.. if needed will require zuul.build_container_images

Action Items:
* just ensure zuul.build_container_images is working upstream and internally.
* https://review.opendev.org/#/c/748426/

## Work items achieved till now
* tripleo-ci-centos-8-content provider job getting consumed in standalone and scenario job as well as multinode job under tripleo-standalone-scenarios-pipeline project template against tripleo-ci and for multinode
* tripleo-multinode-tripleoci - for c8 s000 and s007 multinode
* tripleo-multinode-container-full-pipeline
* tripleo-multinode-container-minimal-pipeline
* https://review.opendev.org/#/c/747591/15/zuul.d/multinode-jobs.yaml
* c7 based buildah container build job have got the functionality to push containers to local registry
* building non-tripleo containers in content provider job itself
* https://review.opendev.org/750119 - pull non tripleo containers in content provider and push it to local registry
* https://review.opendev.org/750119 - use quay for ceph containers

#### meeting notes
* pipeline is running against tripleo-ci, patch up to execute against triple-quickstart.
* https://review.opendev.org/749645

* non-tripleo built containers will be pulled from quay and pushed into content-provider registry

*