# Validation-Framework CI
###### tags: `Design`
Proposal to modify how the validation-framework is currently working in CI.
Noting that the validation-framework itself is a DF/tripleo responsibility, the actual tests and results from those tests are owned by other projects and DFG's
## TripleO Validations
And if you don't know, now you know.
## Fix the TripleO gate
turn off validate_services that are currently running in upstream check or gate patch that can break the tripleo gate [patch1](https://review.opendev.org/#/c/705051/) [patch2](https://review.opendev.org/#/c/705178/) [patch3](https://review.opendev.org/#/c/705177/)
## Execute validations in the promotion pipeline
Since the validations are owned by non-tripleo projects ensure the validations are run in the promotion pipelines where non-tripleo content is validated prior to entering upstream.
Note, I see validations were turned on upstream but not in the promotion jobs, this should never be the case.
* upstream - https://review.opendev.org/#/c/705051/3/zuul.d/standalone-jobs.yaml
* promotion job - https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/standalone-jobs.yaml#L153-L166
The pattern should be exactly reversed, the validations should be turned on in promotion but not upstream check or gate.
## Define a single scenario and standalone to test the validation framework itself.
* The validation team can pick a current scenario and copy it to a new number. This way they can edit the services they want as needed. [tripleo deployment scenarios](https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/ci/environments)
* Define a standalone job that executes the above scenario in upstream and promotion.
Name the job "tripleo-ci-validations-something-something"
This should give the validation framework team the room and flexibility to test the framework as they see fit while also assuring the tripleo gate does not fail.
## Future points.
The CI team is building out a [component pipeline](https://hackmd.io/5uYRmLaOTI2raTbHWsaiSQ)
As we define jobs in this pipeline I think it would be a great idea to add as much of the validation framework as possible here per component.
#### Customizing component pipeline jobs
We will be adjusting the tempest tests per component to heavily hit only the specific component under test. E.G. if the we're running a compute component job expect a lot of:
It would be a really good idea to follow the same pattern for the validation-framework.
* add strict validations back in
* ensure validations are configurable
* ensure failing validations can work w/ a skip list (may already be avail.)
#### The component pipeline will run both upstream and internally w/ osp.
A mock up of a component pipeline dashboard is viewable [here](http://tripleo-cockpit.usersys.redhat.com/d/Wj-ao_sWz/full-component-pipeline?orgId=1)
## Summary / tldr
It's not appropriate for validations to fail in the tripleo upstream gate unless the validation framework itself has failed.
* Limit the exposure upstream
* Create a specific job for the validation team to work with upstream.
* Heavily test validations in the promotion pipeline
* Incorporate validations into the component pipeline planning.