# THE TEST DEVELOPMENT PROCESS

###### tags: `ISTQB` `Software Testing` `ISTQB` `SQA`

## The main steps
- It's the 2^nd^ step in the fundamental test process (FTP)
- Include 3 main steps
1. Identify test conditions
2. Specify test cases
3. Specify test procedures

digraph fundemental_test_process{
    splines=ortho
    node[
        shape=box,
        fixedsize=true,
        width=2,
        fontsize=12
    ];
    
    subgraph cluster_1 {
        label = "Test analysis and design"
        color=blue
		node [
            style=filled,
        ];  
        "Identify test conditions"
		->"Specify test cases"
        ->"Specify test procedure";      
	};
    
    "Test planning and control"
    ->"Identify test conditions";
    
    "Specify test procedure" 
    ->"Test implementation \nand execution" 
    ->"Evaluating exit criteria \nand reporting"
    ->"Test closure activities";   
}

## Terms
:::warning
:bulb: Term's definitions help us to communicate with **the same understanding**, which is fundemental of effective communication.
:::

### A test condition
:::info
:star: **Definition**
***An*** item or event of a component or system that could be verified by <span style="color:red;">***one or more</span> test cases***, for example a function, transaction, feature, quality attribute or structural elemen.
:::

- In other words, a test condition is some characteristic of our software that we can check with a test or a set of tests.
- Typically be a small section of the specification for our SUT.

### A test case
:::info
:star: **Definition**
***A*** set of *input values, execution preconditions, expected results and execution postconditions*, developed for ***a particular objective or test condition***, such as to exercise a particular program path or to verify compliance with a specific requirement.
:::
- Verify the test condition.
- In other words, a test case:
1. gets the system to some starting point for the test (**execution preconditions**)
2. then applies a set of **input values** that should achieve a given outcome (**expected result**)
3. and leaves the system at some end point (**execution postcondition**)

- 4 elements of a test case:
- execution preconditions
- input values
- expected result
- execution postcondition

- **What does a good test case need?**
1. **Traceable** back to the test condiction and the element of the specification that is testing.
2. Add a **specific** value for the **input**.
3. **Specify** where the system should move to after input.

### A test procedure specification
:::info
:star: **Definition**
***A*** sequence of actions for the execution of ***a*** test.
:::
- The test procedure would need to add some details.
- Actions that get SUT into the right starting state, input the values, and check the outcome.

### A test excution schedule
:::info
:star: **Definition**
A document collects all the individual **test procedures** in the right sequence and sets up the system so that they can be run.
:::
- The test execution schedule collects together all the tests and sequences them, taking into account any **priorities**
- example of considering piorities:
A test of the password security needs to be done at the beginning of a sequence of tests.

### Mapping relation example

digraph condiction_case_procedure_execution{
    rankdir="LR"
    
    // 'A' test condiction can be verified by 'one or more' test cases.
    "a specificaion1"
    ->{
        "a test condiction1"
        "a test condiction2"
    }
    
    "a specificaion2"
    ->{
        "a test condiction3"
        "a test condiction4"
    }
    
    "a test condiction1"
    ->{
        "a test case1-1"
        "a test case1-2"
        "a test case1-3"
    }
    
    "a test condiction2"
    ->{
        "a test case2-1"
        "a test case2-2"
    }
    
    "a test condiction3"
    ->{
        "a test case3-1"
        "a test case3-2"
    }
    
    "a test condiction4"
    ->{
        "a test case4-1"
        "a test case4-2"
    }
    
    // 'A' test procedure specification – a sequence of actions for the execution of 'a' test.
    "a test case1-1"
    ->"a test procedure\n specification1-1"
    "a test case1-2"
    ->"a test procedure\n specification1-2"
    "a test case1-3"
    ->"a test procedure\n specification1-3"
    
    "a test case2-1"
    ->"a test procedure\n specification2-1"
    "a test case2-2"
    ->"a test procedure\n specification2-2"
    
    "a test case3-1"
    ->"a test procedure\n specification3-1"
    "a test case3-2"
    ->"a test procedure\n specification3-2"
    
    "a test case4-1"
    ->"a test procedure\n specification4-1"
    "a test case4-2"
    ->"a test procedure\n specification4-2"
    
    // A test execution schedual collects a set of tests or test procedure specifications.
    {
        "a test procedure\n specification1-1",
        "a test procedure\n specification1-3",
        "a test procedure\n specification2-1",
        "a test procedure\n specification2-2",
        "a test procedure\n specification3-1",
        "a test procedure\n specification4-2"
    }->"a test execution schedual"
    
    
}

- according to the project condition
In some situations it may be appropriate to produce very little documentation and in others a very formal and documented process may be appropriate.
在某些情況下，生成很少的文檔可能是適當的，在其他情況下，非常正式且有文檔記錄的過程可能是合適的。
Safety-critical systems, for example, will normally require a more formal test process.

- In preparation for execution the test execution
the test execution schedule collects together all the tests and sequences them, **taking into account any priorities** (highest priority tests would be run first) and any **dependencies between tests**.

### THE IDEA OF TEST COVERAGE
**it provides a quantitative assessment of the extent and quality of testing.**

- important 2 reason
- It provides a quantitative measure of the quality of the testing that has been done by measuring what has been achieved.
- It provides a way of estimating how much more testing needs to be done. Using quantitative measures we can set targets for test coverage and measure progress against them.

If we apply coverage measures to testing based on priorities, which are themselves based on the risks addressed by individual tests, we will have a reliable, objective and quantified framework for testing.

### CATEGORIES OF TEST CASE DESIGN TECHNIQUES
- The test case design techniques three categories
- Black-box specification-based
- based on specification or model of a system or proposed.
- black-box techniques are based on an analysis of the test basis documentation, including both functional and non-functional aspects.
They do **not use** any information regarding the **internal structure of the component or system** under test.
- White-box structure-based
- based on structure of a component or system
- experience-based techniques
- test case from the tester’s experience

### SPECIFICATION-BASED (BLACK-BOX) TECHNIQUES
The main thing about specification-based techniques is that **they derive test cases directly from the specification or from some other kind of model of what the system** should do.
:::spoiler 如果定義的好的益處
If a test basis is well defined and adequately structured we can easily identify test conditions from which test cases can be derived.
:::

The most important point about specification-based techniques is that
- specifications or models do not (and should not) define how a system should achieve the specified behaviour when it is built
- it is a specification of the required (or at least desired) behaviour.

:::danger
If they have misunderstood the specification or chosen to change it in some way without telling anyone then their implementation will deliver behaviour that is different from what the model or specification said the system behaviour should be.
Our test, based solely on the specification, will therefore fail and we will have uncovered a problem.
:::

:::info
:star: **If there is no specification at all**
the tester may have to build a model of the proposed system, perhaps by interviewing key stakeholders to understand what their expectations are.
:::

#### five specification-based techniques for the Foundation Certificate
- Equivalence partitioning.
- Boundary value analysis.
- Decision table testing. :question:
- State transition testing.
- Use case testing.

#### Equivalence partitioning
- Input partitions
- Equivalence partitioning is based on a very simple idea: it is that in many cases the inputs to a program can be ‘chunked’ into groups of similar inputs.
**For example, a program that accepts integer values can accept as valid any input that is an integer(i.e. a whole number) and should reject anything else (such as a real number or a character).**
:::spoiler example
If we imagine a program that separates numbers into two groups according to whether they are positive or negative the total range of integers could be split into three ‘partitions’: the values that are less than zero; zero; and the values that are greater than zero.
:::

:::info
**excercise 4.1**
Suppose you have a bank account that offers variable interest rates: 0.5 per cent for the first £1,000 credit; 1 per cent for the next £1,000; 1.5 per cent for the rest. If you wanted to check that the bank was handling your account correctly what valid input partitions might you use?
:::

:::success
Answer: split three partitions: 0~1000 ,1001~2000 , over 2001
:::

:::info
**excercise 4.2**
A mail-order company selling flower seeds charges £3.95 for postage and packing on all orders up to £20 value and £4.95 for orders above £20 value and up to £40 value. For orders above £40 value there is no charge for postage and packing.
If you were using equivalence partitioning to prepare test cases for the postage and packing charges what valid partitions would you define?
What about non-valid partitions?
The answer can be found at the end of the chapter.
:::

:::success
Answer: split three partitions: 1~20 , 21~40 , over 40
non-valid: 0 , negative
:::

#### Boundary value analysis
**One thing we know about the kinds of mistakes that programmers make is that errors tend to cluster around boundaries.**

For example, if a program should accept a sequence of numbers between 1 and 10, the most likely fault will be that values just outside this range are incorrectly accepted or that values just inside the range are incorrectly rejected.

:::info
For this variant, which is the one documented in BS 7925-2, we include one more value at each boundary when we use boundary value analysis: **the rule is that we use the boundary value itself and one value (as close as you can get) either side of the boundary.**
:::
:::info
**BS 7925-2**: the software component testing standard. Publisher: IEEE
:::
:::info
**Exercise 4.3**
A system is designed to accept scores from independent markers who have marked the same examination script.
Each script should have 5 individual marks, each of which is out of 20, and a total for the script.
Two markers’ scores are compared and differences greater than three in any question score or 10 overall are flagged for further examination.
Using equivalence partitioning and boundary value analysis identify the boundary values that you would explore for this scenario.
:::

:::success
**Equivalence partitioning Answer:**
each scores 0 - 20
total 0 - 100
different 0 - 3 and over 3
total different 0-10 and over 10

**Boundary analysis Answer:**
each score -1 0 1 and 19 20 21
total -1 0 1 and 99 100 101
different -1 0 1 and 2 3 4
total different -1 0 1 and 9 10 11
:::