# Testing strategy - Redgate Clone


Cloning Capability:
- Consists of multiple components communicating with each other over NATS and HTTP
- Unit tests cannot provide enough coverage, because most of the major risks are related to integration
- Integrates with several third-party services
- Tests has to include third-party services
- Relies on Kubernetes for internal configuration, communication and security
- Tests has to include Kubernetes deployment
- Proper Kubernetes deployment is relatively costly
- Reusing an existing Kubernetes cluster for multiple test runs can be flaky
- Uses rook-ceph as a core of it's cloning implementation
- Tests ideally had to test rook-ceph operations
- Rook-Ceph deployment is quite costly
- Relies on KOTS for cluster deployment and updates
- KOTS deployment is very costly
Overall neither unit tests (too narrow in their scope) nor end-to-end tests (too slow and/or flaky) can provide a satisfying coverage as a part of CI.

## Proposed solution
Cloning Capability should rely on 4 main types of tests:

1. **Unit test** can be used for testing isolated complex business logic (e.g. complex validation)
2. **In-memory hosted tests** are grey box tests that host one or several components in-memory to allow for fast integration testing of a limited scope
- For ASP.NET services WebHost factory can be used (see existing tests)
- It should possible to easily host [NATS in-memory for tests](https://github.com/nats-io/nats.net/blob/v0.6.0/NATSUnitTests/UnitTestUtilities.cs#L45)
- We can use contract testing (e.g. pact.io ) to ensure that different components follow each others expectations.
- **Investigation needed**: How can we easily run Go services in this scenario?
3. **Simplified Kubernetes tests** (???) hosts most of the application on a simplified Kubernetes platform. Such tests can provide coverage for Kubernetes related scenarios that would be fast enough to run in the CI
- This tests require rook-ceph to be mocked out, since it won't run well on a simplified Kubernetes set up
- **Investigation needed**: What is the best simplified K8s platform to run this? (minikube? Alternatively kind may be a somewhat better option if the setup doesn't require ceph)
- **Investigation needed**: How can we mock CephWorker while providing realsistic setup otherwise?
- This tests won't be deployed using KOTS, so there should be a realistic configuration for non-KOTS deploytments
4. **End-to-end tests**: Full system tests using proper K8s cluster, KOTS, and rook-ceph. This is the only kind of tests that can ensure that everything is integrated corectly however they are too slow to be run in CI. This tests should be run on a slower schedule (daily?) or on-demand before a release.

Note: There are many more questions about best ways to test specific integration points and scenarios, but we should first have a high-level outline of the tests that makes sense for Cloning Capability considering its technical context

## Next steps
### E2E tests
- [x] Make them run for AKS (and Minikube) using run.sh test
- [x] Look into parallelizing them
- [ ] How do we test upgrade scenarios? -- _Manually at present, might be good enough for Technical Beta_
- [x] Make them "one click"
- Which language we want to use?
- Make them run in an Azure Pipeline

### Go
- Implement basic unit tests as an example

### K8s
- Add basic validation for kustomize and manifests
- Ask 9 3/4 about testing k8s manifests