TripleO Repos - single source for repos/modules/versions
========================================================

###### tags: `Design`

This document describes the current options in use for setting up repos, modules and versions and the proposal to use tripleo-repos as the single source for these settings upstream and rhos-release downstream.

[TOC]

## Proposed primary functions of tripleo-repos
* single source to install non-baseos (non centos) repos for tripleo
* what is in repo-setup that is not currently in tripleo-repos
* it will use get-hash as a python module/cli (which we need to write )
* creating repo from templates and files
* example:

              - type: file
                filename: delorean.repo
                down_url: "https://trunk.rdoproject.org/{{ distro_ver }}-{{ release }}/${DLRN_PATH_TAG}/delorean.repo"

              - type: generic
                reponame: quickstart-centos-base
                filename: quickstart-centos-base.repo
                baseurl: ${NODEPOOL_CENTOS_MIRROR}/8/BaseOS/x86_64/os/

* before cmd after cmd
* leave that in the config/where it is?
* options in tripleo-repos (dnf clean, update)
* like rhos-release - will not work with internal (python module, etc.)
* proxy handling (release file, mirrors, rdoproject)
* dlrn_hash_newest etc.
* other repos (centos base, rename, proxy mirrors all done in release files right now) sed enable, disable
* repo-setup stays - invokes tripleo-repos ... all options tripleo-repos require should be in the release file
* keep interface the same - work on branch, diff version of cli v2, unit tests on this branch. Could flip CI to run in the new branch
* support ansible collections from galaxy?
* yum modules manifest e.g. container-tools, $other
* get and transform dlrn hashes
* transform human readable dlrn links ( e.g current-tripleo )
* to a url w/ a pinned hash
* (dlrn new) args: centos8, master, current-tripleo
* RETURN: https://trunk.rdoproject.org/centos8-master/current-tripleo/19/ac/19ac8a26034e818128be95835e02f49f/
* RETURN: 19ac8a26034e818128be95835e02f49f
* RETURN: 19/ac/19ac8a26034e818128be95835e02f49f
* (dlrn component) args: centos8, master, compute, current-tripleo
* RETURN: https://trunk.rdoproject.org/centos8-master/component/compute/96/e9/96e9124ad705df82fb4866c9241fac5e5e98b761_73a51637/
* RETURN: 96e9124ad705df82fb4866c9241fac5e5e98b761_73a51637
* RETURN: 96/e9/96e9124ad705df82fb4866c9241fac5e5e98b761_73a51637

* (classic dlrn)args: centos7, stein, current-tripleo
* RETURN: https://trunk.rdoproject.org/centos7-stein/03/78/037897775d2d628562fb259a7f4c7d5f7f78ea54_8d9f1861/
* RETURN: /03/78/037897775d2d628562fb259a7f4c7d5f7f78ea54_8d9f1861
* RETURN 037897775d2d628562fb259a7f4c7d5f7f78ea54_8d9f1861
* creating repostiries from templates
* creating repos from files
* injecting an additional one off repos: e.g. dep-pipeline, component-line

## Background Info
Reviewing the code base, there are multiple places where repos are set up and versions are set. Some of these include:

* tripleo repos
* https://github.com/openstack/tripleo-repos
* repo setup role
* https://github.com/openstack/tripleo-quickstart/tree/master/roles/repo-setup
* release config file
* https://github.com/openstack/tripleo-quickstart/tree/master/config/release/tripleo-ci/CentOS-8
* container (base tcib file)
* https://github.com/openstack/tripleo-common/blob/master/container-images/tcib/base/base.yaml#L59
* tripleo-ansible
* container tooling (tcib)
* rdo config
* https://review.rdoproject.org/r/31439 (example)
* rhos-release (downstream)
* Alex's work on a tool for customers
* https://github.com/mwhahaha/rhos-bootstrap
* https://bugzilla.redhat.com/show_bug.cgi?id=1916329

## Design Requirements
* The current cli interface must be maintained.
* additional options are welcome
* The tooling used for the cli can change
* The current unit tests must continue to pass on the new branch

Since having various locations for setting is confusing and error prone, there should be one source of truth for how and which repos are installed for a tripleo deployment and that source of truth should be tripleo-repos. Having a single source of truth for repo config, modules, etc will make things more consistent, reliable and easier to debug.

Finding and setting the dlrn hash (from a dlrn link) is another area where the functionality is duplicated in various forms and in numerous places including:
* promoter
* get_hash role
* container role
* zuul post run files

Tripleo repos doesn't do what we require. Suggestion is to start something new?

## Implementation Options

* Initial implementation from Alex: https://review.opendev.org/c/openstack/tripleo-repos/+/767214

## Proposal

### https://review.opendev.org/c/openstack/tripleo-specs/+/772442 Add spec for single sourcing repo setup

### Blueprint
* Design doc

### Rewrite of tripleo-repos on a separate branch
* Integrate tripleo-repos into tq/repo-setup and tq/config/release
* Single source of truth for yum modules?
* What needs to be done to fix up tripleo-repos / rewrite

### What will remain in tripleo-quickstart/config/release files?
* [DRAFT HLD for Tripleo repos and repo-setup role
](https://hackmd.io/BlDF_hMiSFCPXPVPn6iTlQ)

### Will the repo-setup role disappear?
* no

### How will tripleo-repos work with downstream?

### Proposed work and/or feature requests for rhos-release

### Obtaining the full dlrn hash
* Needs to get the full hash from a dlrn link

### Installation
* pip module, available via pypy
* ansible module
* rpm install

### Usage:
* via cli
* via ansible ( ansible library built on the python module )
* as a python library

## Decision Points

### What module is supported in what release?
container_tools_version:
- master
- update: false
- disable: rhel8
- enable: rhel8
- victoria
- update: false
- disable: rhel8
- enable: rhel8
- train
- update: true
- disable: rhel8
- enable: 2.0

container_tools_pin:
master:
victoria:
train:
- disable: rhel8
enable: 2.0

### Example settings for release config file
(parallel to rhos-relase)

## Questions

* Should it support python2?
* no
* Should it support YUM and centos7?
* yes

## TODO
* (marios) consider the metadata config format that we will use describing all the info in a patch, may be in json or yaml, jinja
* It should be overriden as per ci needs
* Inheritance should work
* explore do we re-use alex info and carry extra metadata (full uri to repos) can we do it neatly... or we keep our own info
* needs to consider 3 main use-cases/personas: upstream/rhos-release dstream and customer (subscription manager)
* (sagi)tripleo-repo should work with config file as well as expanded cli
* Add new v2 interface in setup.cfg for the 'new'tripleo-repos : https://review.opendev.org/c/openstack/tripleo-repos/+/779489
* Re-using dnf.py from rhos-bootstrap or using native dnf module
* (chandan)dlrn hash module
* write down the clear expection of this module
* ansible module for tripleo-repos

## Notes 3/09

* new tripleo-repos will embed upstream urls into the code base
* internal urls will be overridden in the internal ci release files.

*