# TripleO CI Configuration

## Goals
* Understand the TripleO quickstart configs
* update and install the config properly

## Some rules:
* Put your questions at the bottom section of hackmd

## We just deploy tripleo using quickstart.sh script

./quickstart.sh -R centosci/train-current-tripleo \
--nodes config/nodes/1ctlr_1comp.yml \
-E config/environments/default_libvirt.yml \
-c config/general_config/minimal.yml.yml \
-w /var/tmp/bootcamp/ \
-p quickstart.yml \
--no-clone \
--tags all \
127.0.0.2

## Acronyms used
* TQ - tripleo-quickstart
* TQE - tripleo-quickstart-extras
* TOCI - tripleo-ci
* tht - tripleo-heat-templates
* upstream jobs - tripleo based upstream jobs running on zuul.opendev.org
* rdo jobs - zuul jobs running on rdo-cloud and vexxhost using review.rdoproject.org zuul tenant

## TripleO CI related configurations

All the config lies under TQ/config dir.
* environments - All the environments related files
* Like: libvirt host
* general-config - Contains all the featureset files
* nodes - contains undercloud and overcloud nodes config
* release - release files specific vars and stages of content flows during content promotion

### Release files
* Release files helps to specify which version of OpenStack to deploy on which distro version
and also which types of tested contents needs to consume.
* Under config/release dir:
* tripleoci -> Used in rdo periodic jobs and upstream CI
* centosci -> Used in CentOS CI and used by users (It is mostly stable and tested contents)
* Under config/release/tripleoci/CentOS-8
* promotion-testing-hash-master.yml -> used in periodic jobs which runs on multiple times in a day.
* master.yml -> used in upstream jobs

#### Understanding Release files
* Open https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/release/tripleo-ci/CentOS-8/master.yml
* dlrn_hash_tag

# for periodic jobs, it is tripleo-ci-testing and for usptream -> current-tripleo
# it can be found here: https://trunk.rdoproject.org/centos8-master/
dlrn_hash_tag: current-tripleo 

* repo_cmd_before and repo_cmd_after

Before tripleo deployment,
Any shell commands defined under repo_cmd_before will be executed before repo setup.
Once repo setup is done then repo_cmd_after gets executed
It is done by repo_setup role. https://opendev.org/openstack/tripleo-quickstart/src/branch/master/roles/repo-setup

* repos -> list of repos needs to be created before the deployment
* images -> Address of undercloud and overcloud images
:::info
By Using -R flag we pass release file.
Note: Release files help us not only specify which version of OpenStack to deploy but also the stages of qualification. This is why you'll find tripleo-ci, ci-centos and OSP release files in different directories.

:::

### Nodes

:::info
[Upstream TripleO Quickstart Node Documentation](https://docs.openstack.org/tripleo-quickstart/latest/node-configuration.html)
:::

* It stands for nodes configuration files through which user can configure nodes
* In Tripleo deployment, we have one undercloud and multiple overclouds
* An overcloud nodes can be *N* of controller, compute or ceph nodes
* Nodes configuration yaml files should be meaningful so that it can identified easily
* different default vars can be set via for different nodes
* <node_name>_memory
* <node_name>_disk
* <node_name>_vcpu
* where node_name is undercloud, control, compute, disk
* we use node_name to default for setting default values for all the nodes
* node_count is needed to specify how many nodes we want to deploy.
* Take a look at this file: https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/nodes/1ctlr_1comp_64gb.yml
* **overcloud_nodes** vars are used for change the number and type of nodes deployed in your overcloud.

overcloud_nodes:
  - name: control_0 # Name of the node and it's naming starts with <node_name>_0
    flavor: control # type of flavor to use which is defined in the environment files
    virtualbmc_port: 6230 #  used to create a virtual BMC to manage a virtual machines using the IPMI protocol via virtualbmc
    default_name: overcloud-controller-0 
    hostnamemap_override: overcloud-controller-foo-0 #setting FQDN for the overcloud’s public VIP

* https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/overcloud-deploy is responsible for overcloud deployment.
* --nodes param is used to pass the node configuration file.

### Environments

* Nodes can be created basically on libvirt hosts, openstack cloud or on baremetals.
* In order to create nodes on different environment, we have different environment files living under config/environment folder
* libvirt - https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/environments
* rdo-cloud and other environments used in CI - https://opendev.org/openstack/tripleo-ci/src/branch/master/toci-quickstart/config/testenv
* Since we are going to deploy it on libvirt so
* https://github.com/openstack/tripleo-quickstart/blob/master/config/environments/default_libvirt.yml
* It contains vars specific to that environment.
* For setting up libvirt hosts: https://opendev.org/openstack/tripleo-quickstart/src/branch/master/roles/environment
* -E param is used to pass the environment files

### FeatureSet Configuration
* featureset files are yaml files containing different roles vars and tripleo heat templates parameters.
* Featureset configuration enables or disables various features of TripleO.
* Each of these featureset contains a customized list of openstack services deployed.
* The matrix can be found here: https://docs.openstack.org/tripleo-quickstart/latest/feature-configuration.html
* A list of services are defined by tripleo-heat-templates
* Here is the list of that environment files: https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/ci/environments

For example enabling SSL connections, or what type of networking to use, or what tempest tests should be executed.

Looking at [featureset001 as an example:](https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/general_config/featureset001.yml)

Turn SSL on:

undercloud_generate_service_certificate: true
ssl_overcloud: true

Networking Config:

network_isolation: true
network_isolation_type: "multiple-nics"
network_isolation_args: >-
  -e {{ overcloud_templates_path }}/ci/environments/network/multiple-nics/network-isolation-absolute.yaml
  -e {{ overcloud_templates_path }}/ci/environments/network/multiple-nics/network-environment.yaml

Tempest Configurations:

# Settings for os_tempest
run_tempest: >-
  {% if release not in ['pike', 'queens', 'rocky', 'stein'] -%}
  false
  {%- else -%}
  true
  {%- endif -%}

use_os_tempest: >-
  {% if release not in ['pike', 'queens', 'rocky', 'stein'] -%}
  true
  {%- else -%}
  false
  {%- endif -%}

# It will create a public network name 'public' using os_tempest
tempest_interface_name: public

# In order to have a public network with external connectivity, we need to use
# flat network type
tempest_public_net_provider_type: flat

# It is the physical network name through which public network will be created
# having connectivity with external world.
tempest_public_net_physical_name: datacentre

# Setting the tempest_cidr as it is required while creating public subnet from which
# floating IPs gets assigned
tempest_cidr: '10.0.0.0/24'

# In order to create a private network, fs01 is based on OVN, geneve should be used
# as private network type
tempest_private_net_provider_type: geneve

tempest_private_net_seg_id: ''

tempest_install_method: distro

# Having tempest_network_ping_gateway set to true allows to ping any of the IP from
# router to find out network related issue in the deployment early
tempest_network_ping_gateway: true

# It is the python-tempestconf profile which also consumes tempest-deployer-input file
tempest_tempestconf_profile:
  debug: true
  create: true
  deployer-input: "{{ ansible_user_dir }}/tempest-deployer-input.conf"
  os-cloud: "{{ tempest_cloud_name }}"
  out: "{{ tempest_workspace }}/etc/tempest.conf"
  network-id: "{{ tempest_neutron_public_network_id }}"
  overrides: "{{ tempest_tempest_conf_overrides | default({}) }}"

tempest_tempest_conf_overrides:
  auth.use_dynamic_credentials: 'True'

test_white_regex: ''
tempest_whitelist:
  - 'tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops'

# For periodic pipeline run tempest *.api tests in fs001
# See https://tree.taiga.io/project/tripleo-ci-board/task/1383
tempest_periodic_whitelist:
  - 'tempest.api'
  - 'tempest.scenario.test_network_basic_ops.TestNetworkBasicOps'

tempest_check_whitelist:
  - 'tempest.scenario.test_network_basic_ops.TestNetworkBasicOps'

tempest_test_whitelist: >-
  {% if ('periodic' in zuul.pipeline and not job.force_non_periodic|default(false)|bool) or (job.force_periodic|default(false)|bool) -%}
  {{ tempest_periodic_whitelist }}
  {%- else -%}
  {{ tempest_check_whitelist }}
  {%- endif -%}

#### What is scenarios in a featureset?
* The Overcloud usually consists of nodes in predefined roles such as Controller nodes, Compute nodes, and different storage node types.
* Each of these default roles contains a set of services defined in the core Heat template collection.
* By using composable roles, we can create our own custom scenario for example
* Deploying tripleo with glance service with ceph, networking with ovn as an example
* https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/general_config/featureset010.yml
* composable_scenario: multinode-containers.yaml
* https://opendev.org/openstack/tripleo-heat-templates/src/branch/stable/train/ci/environments/multinode-containers.yaml

* -c param is used to pass the featureset files

### `-p` for playbooks
* since we have environment files, nodes configuration, featureset and release files
* All of them are containing ansible vars, we need playbooks which will call specific roles
* to deploy openstack on libvirt hosts
* Here quickstart.yaml playbook is used

### `-t` to pass tags
* tags are used for running a specific piece of tasks from a playbook or role
* -t = all then all the default tags will be passed
* https://github.com/openstack/tripleo-quickstart/blob/master/quickstart.sh#L15
* --teardown = all, It means it will remove virthost and nodes
* you can specify virthost to delete virthost

## Working with tripleo quickstart

* clone the tripleo-quickstart

$ git clone https://opendev.org/openstack/tripleo-quickstart
$ cd tripleo-quickstart

* Make sure all the required packages installed for running quickstart.sh

# Note it calls install-deps.sh script and install all the required package using bindep.txt
# In TripleO-CI, it is invoked here: https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/run-test/templates/toci_quickstart.sh.j2#L25
$ bash quickstart.sh --install-deps

* Let's create the quickstart workspace

# In TripleO-CI, it is invoked here: https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/run-test/templates/toci_quickstart.sh.j2#L30
$ bash quickstart.sh --bootstrap --no-clone --working-dir $LOCAL_WORKING_DIR --playbook noop.yml --retain-inventory 127.0.0.2

What it is going to do?
* --bootstrap
* runs --install-deps
* It re-creates venv in working dir
* Run python setup.py install
* Behind the scene also uses -r quickstart-extras-requirements.txt install these in venv using (pip install -r <requirements.txt>)
* --playbook to run particular playbook
* for example: in noop.yaml playbook, we are just making sure env is up and ready for further execution
* --retain-inventory - retaining the ansible inventory for re-running the quickstart on same env

* Let's use quickstart.sh script to develop or integrate other roles
* For example: https://github.com/openstack/tripleo-operator-ansible
* It is a collection of playbooks & roles to automate tripleo deployment, let's integrate it
* repo link: https://github.com/openstack/tripleo-operator-ansible

* with `-u <git+https://github.com/openstack/tripleo-operator-ansible#egg=tripleo-operator-ansible>`
* Or we can add this line in quickstart-extras-requirements.txt
* Let's try that

    $ bash quickstart.sh --no-clone -u git+https://github.com/openstack/tripleo-operator-ansible#egg=tripleo-operator-ansible -w $LOCAL_WORKING_DIR -y 127.0.0.2

* Let's check whether this role is installed or not

    $ source ~/devel_env/bin/activate
    $ pip list | grep tripleo-operator-ansible

* Let's create a playbook to reuse that role
* for example, let's create tripleo_operator.yaml

- hosts: undercloud
  gather_facts: true
  collections:
    - tripleo.operator
  tasks:
    - name: Create dummy interface
      command: ip link add prov type dummy
      become: true
      when: not 'prov' in ansible_facts.interfaces

* Let's run it using tripleo quickstart

$ bash quickstart.sh --no-clone -p tripleo-operator.yaml -w ~/devel_env -y 127.0.0.2

* It got termintated, let's see the error

'[' '!' -f /root/devel_env/playbooks/tripleo-operator.yaml '

The playbook needs to be in workspace folder.

* Let's take a look at setup.cfg

[files]
data_files =
    usr/local/share/tripleo-quickstart/roles = roles/*
    usr/local/share/tripleo-quickstart/library = library/*
    usr/local/share/tripleo-quickstart/test_plugins = test_plugins/*
    playbooks = playbooks/*
    config/ = config/*

Ah, playbooks need to go under playbooks directory
* Let's fix that and run

$  ~/devel_env/bin/python setup.py install

* Let's try again with quickstart

$ bash quickstart.sh --no-clone -p tripleo-operator.yaml -w ~/devel_env -y 127.0.0.2

* Ah, it works

### Let's extend further
* Let's add these two tasks

    - name: Create directory with tag
      file:
        state: directory
        path: "{{ ansible_user_dir }}/foobar"
      tags:
        - createdir

    - name: Create directory with no tag
      file:
        state: directory
        path: "{{ ansible_user_dir }}/{{ dir_name }}"
      when: dir_name is defined

* Install it

$ ~<path to workspace>/bin/python setup.py install

* Try to relist the tasks

bash quickstart.sh --no-clone -p tripleo_operator.yaml -w /home/ironman/tocitraining/workspace -y 127.0.0.2

So two tasks got liste, third one is not listed as dir_name is not defined,
* Let's run only those taska having tags

bash quickstart.sh --no-clone -p tripleo_operator.yaml -w /home/ironman/tocitraining/workspace -t createdir 127.0.0.2

* Now run the tasks by skipping a particular tag and passing extra vars

bash quickstart.sh --no-clone -p tripleo_operator.yaml -w /home/ironman/tocitraining/workspace --skip-tags createdir --extra-vars dir_name=corona 127.0.0.2

* These things are used here: http://codesearch.openstack.org/?q=--extra-vars&i=nope&files=&repos=

## Let's do some homework!
* Follow the above steps and integrate https://opendev.org/openstack/openstack-ansible-openstack_openrc role
* create a playbook which tells that role actually exists.
* Try to run this playbook https://opendev.org/openstack/tripleo-ci/src/branch/master/toci-quickstart/config/collect-logs.yml
* and see the output
* Sync fs01 and minimal.yaml featureset for master release
* Add a playbook in tripleo-quickstart to execute tripleo-ci containers build and image build role
* To further extend the knowledge, please review ussuri branching patches

## Links
* https://docs.openstack.org/tripleo-quickstart/latest/configuration.html
* https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config
* https://docs.openstack.org/tripleo-quickstart/latest/feature-configuration.html
* https://docs.openstack.org/tripleo-quickstart/latest/node-configuration.html
* https://docs.openstack.org/tripleo-quickstart/latest/working-with-extras.html

## Questions
Note: Put your name followed by queries: