# Test progress monitoring and control
###### tags: `ISTQB` `SQA`
### Test progress monitoring
Having developed the test plan, the activities and timescales determined within it need to be constantly reviewed against what is actually happening.

**The data required to monitor progress**
- can be collected manually.
- test management tools automatic collect the data .

==The progress data is also used to measure exit criteria such as test coverage==, for example 50 per cent requirements coverage achieved.

Common test metrics include:

- Percentage of work done in test case preparation (or percentage of planned test cases prepared).
- Percentage of work done in test environment preparation.
- ==Test case execution== (e.g. number of test cases run/not run, and test cases passed/failed).
- ==Defect information== (e.g. defect density, defects found and fixed, failure rate and retest results).
- Test coverage of requirements, risks or code.
- Subjective confidence of testers in the product
- Dates of test milestones.
- ==Testing costs, including the cost compared with the benefit of finding the next defect or to run the next test.==

Ultimately test metrics are used to track progress towards the completion of testing, which is determined by the exit criteria. ==So test metrics should relate directly to the exit criteria.==

- Dashboard
which reflect all of the relevant metrics on a single screen or page, ensuring maximum impact.

For a dashboard, and generally when delivering metrics, **it is best to use a relatively small but impact-worthy subset of the various metric options available.**

### Test reporting
Test reporting is the process where by test metrics are reported in a summarized format to **update the reader regarding the testing tasks undertaken.**

The information reported can include:
- **What has happened during a given period of time**, for example a week, a test level or the whole test endeavour, or when exit criteria have been met.
- Analysed information and metrics required to support recommendations and decisions about future actions, such as:
- ==an assessment of defects remaining.==
- the economic benefit of continued testing, for example additional tests are exponentially more expensive than the benefit of running.
- **outstanding risks**
- the level of confidence in tested software, for example defects planned versus actual defects found.

**The IEEE 829 standard includes an outline of a test summary report that could be used for test reporting.**


The information gathered can also be used to help with any process improvement opportunities. This information could be used to assess whether:
- the goals for testing were correctly set (were they achievable; if not why not?)
- the test approach or strategy was adequate (e.g. did it ensure there was enough coverage?)
- the testing was effective in ensuring that the objectives of testing were met.

### Test control
We have referred above to the collection and reporting of progress data.
Test control uses this information to decide on a course of action to ensure control of the test activities is maintained and exit criteria are met.

==This is particularly required when the planned test activities are behind schedule.== The actions taken could impact any of the test activities and may also affect other software life cycle activities.
- Make decisions based on information from test monitoring.
- Reprioritise tests when an identified project risk occurs (e.g. software delivered late).
- Change the test schedule due to availability of a test environment.
- ==Set an entry criterion requiring fixes to be retested (confirmation tested) by a developer before accepting them into a build (this is particularly useful when defect fixes continually fail again when retested).==
- Review product risks and perhaps change the risk ratings to meet the target.
- ==Adjust the scope of the testing== (perhaps the amount of tests to be run) to manage the testing of late change requests.

The following test-control activities are likely to be outside the test leader’s responsibility. However, this should not stop the test leader making a recommendation to the project manager.

- Descoping of functionality, that is removing some less important planned deliverables from the initial delivered solution to reduce the time and effort required to achieve that solution.
- Delaying release into the production environment until exit criteria have been met.
- Continuing testing after delivery into the production environment so that defects are found before they occur in production.