# tripleo-ansible testing gen2

Because triple-ansible repository was created before ansible collection spec was ready running the molecule tests on it is not as simple as one would expect. This document aims to identify actions that we can take in order to make it as simple as possible to run tests on both zuul and locally by anyone.

Numbers below are not about order of implementation, they are used only to quickly reference different ideas. Most of the points are indepenendent of each-other.

## 1. adopt collection layout
Molecule and ansible-lint do know how to automatically install collections including their dependencies, reducing the need for most preparations.

Adopting collection layout does not mean that we can no longer use that code outside a collection.

## 2. enable molecule to auto execute a prerun playbook

Currently we rely on execution of `scripts/run-local-test` which makes several operations. If we add a feature to molecule to
detect presence of such file and run it before running a scenario
we can avoid having to do this manually.

Once issue with this is that this would also run on CI, where we may not want to do the same steps. Still instead of making molecule aware of where is run, it would better to add logic inside that playbook itself.

## 3. Use tox-ansible

tox-ansible plugin makes extreamly easy to run individual molecule schenarios without having to worry about installing ansible, molecule or managing environments inside tox.ini file.

It is as easy as calling `tox -e scenario-name`

We already started to use this plugin on other less-complex repositories.

## 4. Make molecule find all scenarios inside a repo

One common inconvenience with molecule is that it only looks inside `$cwd/molecule/` folder for scenarios. It would be much easier if it would look inside the entire repository and list all of them, with the remark that before executing commands from a particular scenario it would temporary change current directory to match current behavior.

Once done it would mean that it does not matter from which sub-directory you call the tool, because location of scenario will determing which cwd will be used when executing it. This behavior also matches tox one, which allows user to call it from any sub-directory from within the project.

One drawback on this feature will be that molecule will report an error if any scenario name is duplicated inside a repository, so
we may need to rename 'default' scenarios.

## 5. Molecule itself can product HTML report

We no longer need to make use of pytest-html in order to save excusion report of molecule scenario as molecule itself can produce as HTML that contains its entire output (with colors).

The only issue we have is that molecule report does not contain multiple scneario runs, it is only about one scenario. If we do not have a problem of having one html file per scenario, we should be fine.

## 6. How to combine multiple scenarios in a single zuul job?

I observed that we have cases where we run multiple scenarios inside a single CI job. This may make sense in order to save resources as we can easily reach limits of our CI/CD systems if we keep every single one separated.

One example was the

Still, this opens few questions:
- how to we map zuul job name to molecule scenario being run, bundling multiple scenarios under one job, making it harder to identify which command you want to run locally to reproduce the same issue
- at this moment tox-ansible does not know how to group multiple schenarios, basically you need to list them like `tox -e scenario1,scenario2` to run multiple ones.

Ideally we should be able to find a way where we can use the same naming for zuul jobs, tox-ansible and molecule calls. If a user looks at a zuul job named `tox-molecule-@foobar`, he should be able to assume he can run the same locally with either `molecule -s @foobar` or `tox -e @foobar`.