Tasks moved to https://github.com/cncf/cnf-conformance/issues/24#issue-567199720

# Tasks moved to https://github.com/cncf/cnf-conformance/issues/24#issue-567199720 --- ### Issue 24: Tasks [Acceptance Criteria](https://github.com/cncf/cnf-conformance/issues/24#issuecomment-587938706) ### Scalability Test: Can CNF increase and decrease capacity? EPIC: CNF Conformance Test Suite Release: v0.0.1 Test Category: Scalability Type of test (static or runtime) = Runtime Tasks: - [ ] see https://hackmd.io/Be2pyIA4Tue-HMT3KdfOPA - [ ] example code: https://github.com/cncf/cnf-conformance/commit/6a7224390f3ceb0dbae94c964e5a24d39b4efd73 **Environment set up tasks: ** - [ ] Copy [admin.conf (the kubeconfig)](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/getting-started.md#accessing-kubernetes-api) to new file name $HOME/shared/conformance01.kubeconfig (for accessing the shared conformance cluster) - [ ] Get SSH access to one of the shared dev machines to use as your test workstation - Ask @denverwilliams, @wavell, @nupejosh or @taylor - [ ] Talk with @denverwilliams, @wavell to get access to the KUBECONFIG file - [ ] Set the KUBECONFIG environment variable to point to the shared kubeconfig file for accessing the target K8s cluster - [ ] Follow the [installation instructions](https://github.com/cncf/cnf-conformance#installation) to set up the the conformance suite for development and testing - [ ] Update installation instructions to cover new setup and existing workstation updates. Example: - You should have a working conformance suite that compiles and runs correctly. (eg. crystal-lang installed) - [ ] If you have it installed, pull down latest version of CNF conformance test code from develop branch (eg. `git fetch ; get rebase` from the cnf-conformance directory) - [ ] Run `shard install` to get dependencies **Upstream tool set up tasks: (conformance suite + upstream tools)** - N/A **CNF setup Tasks** - [x] Use the https://github.com/cncf/cnf-conformance/tree/master/cnfs for adding target CNFs to be tested - [ ] Create example conformance configuration file for configuring the use of the target CNF - [ ] Create example environment configuration that has the CNF container name list - [ ] Create code to use the conformance CNF configuration and target folder for testing the CNF - [ ] Add documentation for testing a target CNFs with the cnformance suite (eg. add to usage md). Uncludes - Path to Helm chart under cnfs folder - Options to pass helm chart - [ ]Add documentation for using target CNFs with the cnformance suite (eg. add to usage md), Include - The name of the containers needs to be configured **Sample CNF tasks:** - [ ] Create a sample-cnf folder - [ ] Create a subfolder for the sample cnf (eg. dns-cnf1) - [ ] Create a sample CNF using a minimal container which requires no host or vanilla K8s modifications (no extensions) --> eg. CoreDNS container - [ ] Create sample CNF conformance configuration file for using the sample CNF - [ ] Create sample environment configuration that has the sample CNF container name list - [ ] Add TL;DR doc on using the sample in the sample folder **Code implementation tasks:** - [ ] If one test blows up (eg. CNF code not found for static) the suite should continue to run - [ ] Add util task which can deploy a CNF that has "valid conformance deploy configuration" - helm chart v3.0 - helm chart path - containers name - NOTE: does not run when tests run (include `all`) - [ ] Create new increase test which increases the replica set - [ ] Check and save current replica count - [ ] At start set replica to 1 (overriding CNF defaults) - [ ] Show data from cluster about container - [ ] Do increase - [ ] Show data from cluster about container - [ ] Reset replica back to original saved replica count - [ ] Show pass or fail for doing increase - [ ] Show green for pass and red for fail - [ ] Create new decrease test which decreases the repica set - [ ] Check and save current replica count - [ ] At start set replica to more than 1 (Eg. 3; overriding CNF defaults) - [ ] Show data from cluster about container - [ ] Do decrease - [ ] Show data from cluster about container - [ ] Reset replica back to original saved replica count - [ ] Show pass or fail for doing decrease - [ ] Show green for pass and red for fail - [ ] Update existing increase_decrease test to call the individual increase and decrease - [ ] Think about and document how we can roll up a summary for sets of tests => https://github.com/cncf/cnf-conformance/issues/30 - [ ] Scalability category should call the increase and decrease test set - [ ] The all test command should run the increase and decrease tests - [ ] Add the increase test to the "runtime test set" - [ ] Add the decrease test to the "runtime test set" - [ ] Optionally, Add the "increase+decrease test set" to the "runtime test set" **Documentation tasks:** - [ ] Update [Test Categories md](https://github.com/cncf/cnf-conformance/blob/master/TEST-CATEGORIES.md#scaling-tests) if needed - [ ] Update [Pseudo Code md](https://github.com/cncf/cnf-conformance/blob/master/PSEUDO-CODE.md#scaling-tests) if needed - [ ] Update [USAGE md](https://github.com/cncf/cnf-conformance/blob/master/USAGE.md#scaling-tests) if needed - [ ] Update [installation instructions](https://github.com/cncf/cnf-conformance#installation) if needed **QA tasks** Dev Review: - [ ] walk through A/C - [ ] do you get the expected result? - [ ] if yes, move to `Needs Peer Review` column - [ ] if no, document what additional tasks will be needed Peer review: - [ ] walk through A/C - [ ] do you get the expected result? - [ ] if yes, move to `Reviewer Approved` column - [ ] if no, document what did not go as expected, including error messages and screenshots (if possible)