Software Normalization

Homework

HW1 - 10/16

unsigned int clz(unsigned int t)
{
    unsigned int r = 0;
    if (!t)
        return 32;
    for (;; t <<= 1) {
        if (!(0x80000000 & t))
            r++;
        else
            break;
    }
    return r;
}
/**
 * clz - count leading zero
 * 
 * A naive implementation of the clz for 32-bit value.
 * 
 * Return amount of the leading zeros.
 */

Test case

test case 1
1) Input values: t 0x40000000
2) expected result: 1
3) test program's result: 1
Line coverage: 1 3 4 6 7 8 9 10 12
--------------------
test case 2
1) Input values: t 0
2) expected result: 32
3) test program's result: 32
Line coverage: 1 3 4 5

HW2 - 10/23

Control flow

Edges

12
23
24
45
47
56
64

Initial Node: 1
Final Nodes: 3, 7

test case 1
1) Input values: t 0x40000000
2) expected result: 1
3) test program's result: 1
Edge coverage: 12 24 45 47 56 64
--------------------
test case 2
1) Input values: t 0
2) expected result: 32
3) test program's result: 32
Edge coverage: 12 23

Edge-pair

123
124
245
247
456
564
645
647

test case 1
1) Input values: t 0x40000000
2) expected result: 1
3) test program's result: 1
Edge-pair coverage: 124 245 247 456 564 645 647
--------------------
test case 2
1) Input values: t 0
2) expected result: 32
3) test program's result: 32
Edge-pair coverage: 123

Requirement

• control flow is required
  • Hackmd native feature
    • Graphviz is also supported natively
  • draw.io
• 100% edge coverage
• 100% edge-pair coverage
• test path coverage (both with-loop and without-loop are required)

Test path

123
1247
12{456}47

test case 1
1) Input values: t 0x40000000
2) expected result: 1
3) test program's result: 1
Test path coverage: 1245645647
--------------------
test case 2
1) Input values: t 0x80000000
2) expected result: 0
3) test program's result: 0
Test path coverage: 1247
--------------------
test case 3
1) Input values: t 0
2) expected result: 32
3) test program's result: 32
Test path coverage: 123

HW4 - 10/30

Stress test

0xffffffff
0x1
0x0

test case 1
1) Input values: t 0xffffffff
2) expected result: 0
3) test program's result: 0
--------------------
test case 2
1) Input values: t 0x1
2) expected result: 31
3) test program's result: 31
--------------------
test case 3
1) Input values: t 0x0
2) expected result: 32
3) test program's result: 32

Requirement

• stress test

HW5 - 11/6

Factorial calculator:

#define THRES 10000
 
// Evaluate factorial result of %n, with a flag determines whether to enable
// threshold-checking.
//
// Return -1 when negative %n or %n exceeds the threshold with the flag set,
// otherwise, correct result is returned.
int factorial(int n, bool limit) {
  if ((limit && (n > THRES)) || (n < 0))
        return -1;
 
  int result = 1;
  for (int i = 1; i <= n; i++) {
    result *= i;
  }
 
  return result;
}

Related header file:

#ifndef _FACTORIAL_HPP
#define _FACTORIAL_HPP
 
int factorial(int, bool);
 
#endif // _FACTORIAL_HPP

Predicate Coverage

((limit && (n > THRES)) || (n < 0)) = true, false


test case 1
1) Input values: n -5, limit false
2) expected result: -1
3) test program's result: -1
--------------------
test case 2
1) Input values: n 5, limit false
2) expected result: 120
3) test program's result: 120

Clause Coverage

limit          = true, false
(n > BOUNDARY) = true, false
(n < 0)        = true, false


test case 1
1) Input values: n 5, limit true
2) expected result: 120
3) test program's result: 120
--------------------
test case 2
1) Input values: n -9, limit false
2) expected result: -1
3) test program's result: -1

Combinatorial Coverage

// limit = true
// (n > BOUNDARY) = true
// (n < 0) = true
// Logically unreachable
--------------------
// limit = true
// (n > BOUNDARY) = true
// (n < 0) = false

test case 1
1) Input values: n 10001, limit true
2) expected result: -1
3) test program's result: -1
--------------------
// limit = true
// (n > BOUNDARY) = false
// (n < 0) = true

test case 2
1) Input values: n -5, limit true
2) expected result: -1
3) test program's result: -1
--------------------
// limit = true
// (n > BOUNDARY) = false
// (n < 0) = false

test case 3
1) Input values: n 5, limit true
2) expected result: 120
3) test program's result: 120
--------------------
// limit = false
// (n > BOUNDARY) = true
// (n < 0) = true
// Logically unreachable
--------------------
// limit = false
// (n > BOUNDARY) = true
// (n < 0) = false

test case 4
1) Input values: n 5, limit false
2) expected result: 120
3) test program's result: 120
--------------------
// limit = false
// (n > BOUNDARY) = false
// (n < 0) = true

test case 5
1) Input values: n -5, limit false
2) expected result: -1
3) test program's result: -1
--------------------
// limit = false
// (n > BOUNDARY) = false
// (n < 0) = false
// Logically unreachable
--------------------


for (int i = 1; i <= n; i++)

test case 6
1) Input values: n 0, limit false
2) expected result: 1
3) test program's result: 1
--------------------
test case 7
1) Input values: n 1, limit false
2) expected result: 1
3) test program's result: 1
--------------------
test case 8
1) Input values: n 5, limit false
2) expected result: 120
3) test program's result: 120
--------------------
test case 9
1) Input values: n 0, limit true
2) expected result: 1
3) test program's result: 1
--------------------
test case 10
1) Input values: n 1, limit true
2) expected result: 1
3) test program's result: 1
--------------------
test case 11
1) Input values: n 5, limit true
2) expected result: 120
3) test program's result: 120

Input Domain Characterization

System level

Input value {<0, 0, >0}
Limit flag {true, false}

Unit level

Parameters F(int n, bool limit)
Possible values:
n: {-555, 0, 5}, limit: {true, false}


non-combinational test:
F(-555, true), F(0, false), F(5, false)

test case 1
1) Input values: n -555, limit true
2) expected result: -1
3) test program's result: -1
--------------------
test case 2
1) Input values: n 0, limit false
2) expected result: 1
3) test program's result: 1
--------------------
test case 3
1) Input values: n 5, limit false
2) expected result: 120
3) test program's result: 120
--------------------


combinational test:
F(-555, true), F(-555, false), F(0, true), F(0, false), F(5, true), F(5, false)

test case 1
1) Input values: n -555, limit true
2) expected result: -1
3) test program's result: -1
--------------------
test case 2
1) Input values: n -555, limit false
2) expected result: -1
3) test program's result: -1
--------------------
test case 3
1) Input values: n 0, limit true
2) expected result: -1
3) test program's result: -1
--------------------
test case 4
1) Input values: n 0, limit false
2) expected result: 1
3) test program's result: 1
--------------------
test case 5
1) Input values: n 5, limit true
2) expected result: 120
3) test program's result: 120
--------------------
test case 6
1) Input values: n 5, limit false
2) expected result: 120
3) test program's result: 120

Test driver

#include <cstdio>
#include <gtest/gtest.h>
#include "factorial.hpp"
 
namespace {
 
TEST(factorialTest, predicateCoverage) {
        // ((limit && (n > THRES)) || (n < 0)) -> true
        EXPECT_EQ(-1, factorial(-5, false));
        
        // ((limit && (n > THRES)) || (n < 0)) -> false
        EXPECT_EQ(120, factorial(5, false));
}
 
TEST(factorialTest, clauseCoverage) {
        // limit -> true
        // (n > BOUNDARY) -> true
        // (n < 0) -> false
        EXPECT_EQ(120, factorial(5, true));
 
        // limit -> false
        // (n > BOUNDARY) -> false
        // (n < 0) -> true
        EXPECT_EQ(-1, factorial(-9, false));
}
 
TEST(factorialTest, clauseCombiCoverage) {
        // limit -> true
        // (n > BOUNDARY) -> true
        // (n < 0) -> true
        EXPECT_EQ(-1, factorial(10001, true));
 
        // limit -> true
        // (n > BOUNDARY) -> true
        // (n < 0) -> false
        EXPECT_EQ(-1, factorial(10001, true));
 
        // limit -> true
        // (n > BOUNDARY) -> false
        // (n < 0) -> true
        EXPECT_EQ(-1, factorial(-5, true));
       
        // limit -> true
        // (n > BOUNDARY) -> false
        // (n < 0) -> false
        EXPECT_EQ(120, factorial(5, true));
 
        // limit -> false
        // (n > BOUNDARY) -> true
        // (n < 0) -> true
// Logically unreachable
 
        // limit -> false
        // (n > BOUNDARY) -> true
        // (n < 0) -> false
        EXPECT_EQ(120, factorial(5, false));
 
        // limit -> false
        // (n > BOUNDARY) -> false
        // (n < 0) -> true
        EXPECT_EQ(-1, factorial(-5, false));
 
        // limit -> false
        // (n > BOUNDARY) -> false
        // (n < 0) -> false
// Logically unreachable
}
 
 
TEST(factorialTest, inputDomainNonCombi) {
        EXPECT_EQ(-1, factorial(-555, true)); // F(-555, true)
        EXPECT_EQ(1, factorial(0, false)); // F(0, false)
        EXPECT_EQ(120, factorial(5, false)); // F(5, false)
}
 
TEST(factorialTest, inputDomainCombi) {
        EXPECT_EQ(-1, factorial(-555, true)); // F(-555, true)
        EXPECT_EQ(-1, factorial(-555, false)); // F(-555, false)
        EXPECT_EQ(1, factorial(0, true)); // F(0, true)
        EXPECT_EQ(1, factorial(0, false)); // F(0, false)
        EXPECT_EQ(120, factorial(5, true)); // F(5, true)
        EXPECT_EQ(120, factorial(5, false)); // F(5, false)
}
} // namespace
 
GTEST_API_ int main(int argc, char **argv) {
        printf("Running main() from %s\n", __FILE__);
        testing::InitGoogleTest(&argc, argv);
        return RUN_ALL_TESTS();
}

Gtest result

Requirement

• predicate coverage
  result of the whole statement
• clause coverage
  each clause must be tested with one of true and false
• combinatorial coverage
  true/false of each clause must be tested
• Input Domain Characterization
  categorization
  combinatorial

HW6 - 11/20

Factorial calculator:

#define THRES 10000
 
// Evaluate factorial result of %n, with a flag determines whether to enable
// threshold-checking.
//
// Return -1 when negative %n or %n exceeds the threshold with the flag set,
// otherwise, correct result is returned.
int factorial(int n, bool limit) {
  if ((limit && (n > THRES)) || (n < 0))
        return -1;
 
  int result = 1;
  for (int i = 1; i <= n; i++) {
    result *= i;
  }
 
  return result;
}

Test case

Mutant #1:

int factorial(int n, bool limit) {
--  if ((limit && (n > THRES)) || (n < 0))
++  if ((limit || (n > THRES)) || (n < 0))
        return -1;

  int result = 1;
  for (int i = 1; i <= n; i++) {
    result *= i;
  }

  return result;
}

test case 1
1) Input values: n 5, limit true
2) expected result: 120
3) test program's result:
    a) original result: 120
    b) actual result: -1

Mutant #2:

int factorial(int n, bool limit) {
  if ((limit && (n > THRES)) || (n < 0))
--        return -1;
++        return 1;

  int result = 1;
  for (int i = 1; i <= n; i++) {
    result *= i;
  }

  return result;
}

test case 1
1) Input values: n 10001, limit true
2) expected result: -1
3) test program's result:
    a) original result: -1
    b) actual result: 1

Mutant #3:

int factorial(int n, bool limit) {
  if ((limit && (n > THRES)) || (n < 0))
        return -1;

--  int result = 1;
++  int result = 0;
for (int i = 1; i <= n; i++) {
    result *= i;
  }

  return result;
}

test case 1
1) Input values: n 5, limit true
2) expected result: 120
3) test program's result:
    a) original result: 120
    b) actual result: -1

Mutant #4:

int factorial(int n, bool limit) {
  if ((limit && (n > THRES)) || (n < 0))
        return -1;

  int result = 1;
--  for (int i = 1; i <= n; i++) {
++  for (int i = 1; i <= n; i--) {
        result *= i;
  }

  return result;
}

test case 1
1) Input values: n 5, limit true
2) expected result: 120
3) test program's result:
    a) original result: 120
    b) actual result: 0

Mutant #5:

int factorial(int n, bool limit) {
  if ((limit && (n > THRES)) || (n < 0))
        return -1;

  int result = 1;
  for (int i = 1; i <= n; i++) {
--    result += i;
++    result -= i;
  }

  return result;
}

test case 1
1) Input values: n 5, limit true
2) expected result: 120
3) test program's result:
    a) original result: 120
    b) actual result: 0

Mutant #6:

int factorial(int n, bool limit) {
  if ((limit && (n > THRES)) || (n < 0))
        return -1;

  int result = 1;
  for (int i = 1; i <= n; i++) {
      result += i;
  }

--  return result;
++  return n;
}

test case 1
1) Input values: n 5, limit true
2) expected result: 120
3) test program's result:
    a) original result: 120
    b) actual result: 5

Requirement

• mutation test
  Mutation should be performed at least once per statement (line), e.g. modify return value, modify arithmetic operators…

HW7 - 11/27

#define LOWERCASE_MASK  0x20
#define ASCII_NUM_START 0x30
#define ASCII_NUM_END   0x9
#define MAX_LEN         8
 
// It is assumed that the input is a null-terminated string, you
// are on YOUR OWN if you feed me a non-null-terminated string
//
// Constraints:
//      - first word should lays within A-Z with case-insensitive
//      - subsequent words should lays only within digits 0-9
//      - max length of the string is of 8
bool isIdValid(const char *str)
{
        if (!str && !str[0])
                return false;
 
        size_t len = strlen(str);
 
        // validate the first word and the length
        if (!((str[0] | LOWERCASE_MASK) >= 'a' &&
              (str[0] | LOWERCASE_MASK) <= 'z') ||
               len > MAX_LEN)
                return false;
 
        for (int i = 1; str[i]; i++) {
                if (((str[i] | LOWERCASE_MASK) <= 'a' &&
                     (str[i] | LOWERCASE_MASK) >= 'z') ||
                     (unsigned char)(str[i] - ASCII_NUM_START) <= ASCII_NUM_END)
                        continue; // passed, move to the next character
                else
                        return false; // invalid character detected
        }
 
        return true;
}

Equivalence partition

Test case

test case 1
1) Input values: str s1234
2) expected result: true
3) test program's result: true
covered class(es): (5) (6) (7)
--------------------
test case 2
1) Input values: str @4321
2) expected result: false
3) test program's result: false
covered class(es): (1) (6) (7)
--------------------
test case 3
1) Input values: str sws@##
2) expected result: false
3) test program's result: false
covered class(es): (2) (5) (7)
--------------------
test case 4
1) Input values: str (empty string)
2) expected result: false
3) test program's result: false
covered class(es): (1) (2) (3)
--------------------
test case 5
1) Input values: str a121221222
2) expected result: false
3) test program's result: false
covered classes: (4) (5) (6)

GoogleTest

Requirement

• Equivalence partition (Valid Invalid)
  • Valid Invalid
  • Software Testing UT12 p25

HW8 - 12/4

bool isIdValid(const char *str)
{
    if (!str && !str[0])
            return false;
 
    size_t len = strlen(str);
 
    // validate the first word and the length
    if (!((str[0] | LOWERCASE_MASK) >= 'a'  &&
          (str[0] | LOWERCASE_MASK) <= 'z') ||
           len > MAX_LEN)
        return false;
 
    for (int i = 1; str[i]; i++) {
        if (((str[i] | LOWERCASE_MASK) <= 'a' &&
             (str[i] | LOWERCASE_MASK) >= 'z') ||
             (unsigned char)(str[i] - ASCII_NUM_START) <= ASCII_NUM_END)
                continue; // passed, move to the next character
        else
                return false; // invalid character detected
    }
 
    return true;
}

Input boundary value

Test case

test case 1
1) Input values: str "\x40543553"
2) expected result: false
3) test program's result: false
--------------------
test case 2
1) Input values: str "\x41543553"
2) expected result: true
3) test program's result: true
--------------------
test case 3
1) Input values: str "\x42543553"
2) expected result: true
3) test program's result: true
--------------------
test case 4
1) Input values: str "\x59543553"
2) expected result: true
3) test program's result: true
--------------------
test case 5
1) Input values: str "\x5a543553"
2) expected result: false
3) test program's result: false
--------------------
test case 6
1) Input values: str "\x5b543553"
2) expected result: false
3) test program's result: false
--------------------
test case 7
1) Input values: str "z\x2f43553"
2) expected result: false
3) test program's result: false
--------------------
test case 8
1) Input values: str "z\x3043553"
2) expected result: true
3) test program's result: true
--------------------
test case 9
1) Input values: str "z\x3143553"
2) expected result: true
3) test program's result: true
--------------------
test case 10
1) Input values: str "z\x3843553"
2) expected result: true
3) test program's result: true
--------------------
test case 11
1) Input values: str "z\x3943553"
2) expected result: true
3) test program's result: true
--------------------
test case 12
1) Input values: str "z\x3a43553"
2) expected result: false
3) test program's result: false
--------------------
test case 13
1) Input values: str ""
2) expected result: false
3) test program's result: false
--------------------
test case 14
1) Input values: str "z"
2) expected result: false
3) test program's result: false
--------------------
test case 15
1) Input values: str "z5"
2) expected result: true
3) test program's result: true
--------------------
test case 16
1) Input values: str "z54355"
2) expected result: true
3) test program's result: true
--------------------
test case 17
1) Input values: str "z543553"
2) expected result: true
3) test program's result: true
--------------------
test case 18
1) Input values: str "z5435533"
2) expected result: false
3) test program's result: false
--------------------

Gtest

#include <cstdio>
#include <gtest/gtest.h>
#include "factorial.hpp"
 
namespace {
TEST(isIdValidTest, boundaryValueTest) {
        char str[MAX_LEN] = "z543553";
        char s;
        for (int i=0; i<MAX_LEN; i++) {
                s = str[i];
                str[i] = 0;
                EXPECT_EQ(!i ? false: true, isIdValid(str));
 
                str[i] = 0xff;
                EXPECT_EQ(false, isIdValid(str));
                str[i] = s;
        }
}
 
 
} // namespace
 
GTEST_API_ int main(int argc, char **argv) {
        printf("Running main() from %s\n", __FILE__);
        testing::InitGoogleTest(&argc, argv);
        return RUN_ALL_TESTS();
}
 

Requirement

• boundary value

HW9 - 12/11

Used sources

unsigned int clz(unsigned int t)
{
    unsigned int r = 0;
    if (!t)
        return 32;
    for (;; t <<= 1) {
        if (!(0x80000000 & t))
            r++;
        else
            break;
    }
    return r;
}
 
int factorial(int n, bool limit) {
  if ((limit && (n > THRES)) || (n < 0))
        return -1;
 
  int result = 1;
  for (int i = 1; i <= n; i++) {
    result *= i;
  }
 
  return result;
}
 
bool isIdValid(const char *str)
{
        if (!str && !str[0])
                return false;
 
        size_t len = strlen(str);
 
        // validate the first word and the length
        if (!((str[0] | LOWERCASE_MASK) >= 'a' &&
              (str[0] | LOWERCASE_MASK) <= 'z') ||
               len > MAX_LEN - 1) // '\0' is excluded to make sure all bytes
                                 //  lays within the buffer (i.e. no OOB access)
                return false;
 
        for (int i = 1; str[i]; i++) {
                if (((str[i] | LOWERCASE_MASK) <= 'a' &&
                     (str[i] | LOWERCASE_MASK) >= 'z') ||
                     (unsigned char)(str[i] - ASCII_NUM_START) <= ASCII_NUM_END)
                        continue; // passed, move to the next character
                else
                        return false; // invalid character detected
        }
 
        return true;
}

GoogleTest - test driver

//#include <limits.h>
#include <cstdio>
#include <gtest/gtest.h>
#include "factorial.hpp"
 
namespace {
 
TEST(clzTest, lineCoverage) {
	EXPECT_EQ(1, clz(0x40000000));
	EXPECT_EQ(32, clz(0));
}
 
TEST(clzTest, edgeCoverage) {
	EXPECT_EQ(1, clz(0x40000000));
	EXPECT_EQ(32, clz(0));
}
 
TEST(clzTest, testPathCoverage) {
	EXPECT_EQ(1, clz(0x40000000));
	EXPECT_EQ(0, clz(0x80000000));
	EXPECT_EQ(32, clz(0));
}
 
TEST(clzTest, stressTest) {
	EXPECT_EQ(0, clz(0xffffffff));
	EXPECT_EQ(31, clz(0x1));
	EXPECT_EQ(32, clz(0));
}
 
TEST(factorialTest, predicateCoverage) {
        // ((limit && (n > THRES)) || (n < 0)) -> true
        EXPECT_EQ(-1, factorial(-5, false));
        
        // ((limit && (n > THRES)) || (n < 0)) -> false
        EXPECT_EQ(120, factorial(5, false));
}
 
TEST(factorialTest, clauseCoverage) {
	// limit -> true
	// (n > BOUNDARY) -> true
	// (n < 0) -> false
	EXPECT_EQ(120, factorial(5, true));
 
	// limit -> false
        // (n > BOUNDARY) -> false
        // (n < 0) -> true
	EXPECT_EQ(-1, factorial(-9, false));
}
 
TEST(factorialTest, clauseCombiCoverage) {
	// limit -> true
        // (n > BOUNDARY) -> true
        // (n < 0) -> true
        EXPECT_EQ(-1, factorial(10001, true));
 
        // limit -> true
        // (n > BOUNDARY) -> true
        // (n < 0) -> false
        EXPECT_EQ(-1, factorial(10001, true));
 
        // limit -> true
        // (n > BOUNDARY) -> false
        // (n < 0) -> true
        EXPECT_EQ(-1, factorial(-5, true));
       
        // limit -> true
        // (n > BOUNDARY) -> false
        // (n < 0) -> false
        EXPECT_EQ(120, factorial(5, true));
 
        // limit -> false
        // (n > BOUNDARY) -> true
        // (n < 0) -> true
// Logically unreachable
 
        // limit -> false
        // (n > BOUNDARY) -> true
        // (n < 0) -> false
        EXPECT_EQ(120, factorial(5, false));
 
        // limit -> false
        // (n > BOUNDARY) -> false
        // (n < 0) -> true
        EXPECT_EQ(-1, factorial(-5, false));
 
        // limit -> false
        // (n > BOUNDARY) -> false
        // (n < 0) -> false
// Logically unreachable
}
 
TEST(factorialTest, inputDomainNonCombi) {
	EXPECT_EQ(-1, factorial(-555, true)); // F(-555, true)
	EXPECT_EQ(1, factorial(0, false)); // F(0, false)
	EXPECT_EQ(120, factorial(5, false)); // F(5, false)
}
 
TEST(factorialTest, inputDomainCombi) {
	EXPECT_EQ(-1, factorial(-555, true)); // F(-555, true)
	EXPECT_EQ(-1, factorial(-555, false)); // F(-555, false)
	EXPECT_EQ(1, factorial(0, true)); // F(0, true)
	EXPECT_EQ(1, factorial(0, false)); // F(0, false)
	EXPECT_EQ(120, factorial(5, true)); // F(5, true)
	EXPECT_EQ(120, factorial(5, false)); // F(5, false)
}
 
TEST(factorialTest, mutation) {
	puts("\n\e[1;41mStarting mutation test\e[0m\n");
	EXPECT_EQ(120, m1(5, true)); // mutation #1
	EXPECT_EQ(-1, m2(10001, true));
	EXPECT_EQ(120, m3(5, true));
	EXPECT_EQ(120, m4(5, true));
	EXPECT_EQ(120, m5(5, true));
	EXPECT_EQ(120, m6(5, true));
	puts("\n\e[1;41mDone mutation test\e[0m\n");
}
 
TEST(isIdValidTest, equivalencePartition) {
        EXPECT_EQ(true, isIdValid("s1234"));
        EXPECT_EQ(false, isIdValid("@4321"));
        EXPECT_EQ(false, isIdValid("sws@##"));
        EXPECT_EQ(false, isIdValid(""));
        EXPECT_EQ(false, isIdValid("a121221222"));
}
 
TEST(isIdValidTest, boundaryValueFirstChar) {
        char str[MAX_LEN] = "z543553";
        for (int i=0; i<3; i++) {
                str[0] = 0x40 + i;
                EXPECT_EQ(!i ? false: true, isIdValid(str));
        }
	str[0] = 'z';
	for (int i=0; i<3; i++) {
                str[0] = 0x59 + i;
                EXPECT_EQ(i==2 ? false: true, isIdValid(str));
        }
}
 
TEST(isIdValidTest, boundaryValueSubsequentChar) {
        char str[MAX_LEN] = "z543553";
        for (int i=0; i<3; i++) {
                str[1] = 0x2f + i;
                EXPECT_EQ(!i ? false: true, isIdValid(str));
        }
        for (int i=0; i<3; i++) {
                str[1] = 0x38 + i;
                EXPECT_EQ(i==2 ? false: true, isIdValid(str));
        }
}
 
TEST(isIdValidTest, boundaryValueLengthOfInput) {
        EXPECT_EQ(false, isIdValid(""));
        EXPECT_EQ(true, isIdValid("z"));
        EXPECT_EQ(true, isIdValid("z5"));
        EXPECT_EQ(true, isIdValid("z54355"));
        EXPECT_EQ(true, isIdValid("z543553"));
        EXPECT_EQ(false, isIdValid("z5435533"));
}
 
} // namespace
 
GTEST_API_ int main(int argc, char **argv) {
	printf("Running main() from %s\n", __FILE__);
	testing::InitGoogleTest(&argc, argv);
	return RUN_ALL_TESTS();
}

test results

Running main() from driver.cpp
[==========] Running 13 tests from 3 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from clzTest
[ RUN      ] clzTest.edgeCoverage
[       OK ] clzTest.edgeCoverage (0 ms)
[ RUN      ] clzTest.testPathCoverage
[       OK ] clzTest.testPathCoverage (0 ms)
[ RUN      ] clzTest.stressTest
[       OK ] clzTest.stressTest (0 ms)
[----------] 3 tests from clzTest (0 ms total)

[----------] 6 tests from factorialTest
[ RUN      ] factorialTest.predicateCoverage
[       OK ] factorialTest.predicateCoverage (0 ms)
[ RUN      ] factorialTest.clauseCoverage
[       OK ] factorialTest.clauseCoverage (0 ms)
[ RUN      ] factorialTest.clauseCombiCoverage
[       OK ] factorialTest.clauseCombiCoverage (0 ms)
[ RUN      ] factorialTest.inputDomainNonCombi
[       OK ] factorialTest.inputDomainNonCombi (0 ms)
[ RUN      ] factorialTest.inputDomainCombi
[       OK ] factorialTest.inputDomainCombi (0 ms)
[ RUN      ] factorialTest.mutation

Starting mutation test

driver.cpp:109: Failure
Expected equality of these values:
  120
  m1(5, true)
    Which is: -1
driver.cpp:110: Failure
Expected equality of these values:
  -1
  m2(10001, true)
    Which is: 1
driver.cpp:111: Failure
Expected equality of these values:
  120
  m3(5, true)
    Which is: 0
driver.cpp:112: Failure
Expected equality of these values:
  120
  m4(5, true)
    Which is: 0
driver.cpp:113: Failure
Expected equality of these values:
  120
  m5(5, true)
    Which is: -14
driver.cpp:114: Failure
Expected equality of these values:
  120
  m6(5, true)
    Which is: 5

Done mutation test

[  FAILED  ] factorialTest.mutation (4211 ms)
[----------] 6 tests from factorialTest (4211 ms total)

[----------] 4 tests from isIdValidTest
[ RUN      ] isIdValidTest.equivalencePartition
[       OK ] isIdValidTest.equivalencePartition (0 ms)
[ RUN      ] isIdValidTest.boundaryValueFirstChar
[       OK ] isIdValidTest.boundaryValueFirstChar (0 ms)
[ RUN      ] isIdValidTest.boundaryValueSubsequentChar
[       OK ] isIdValidTest.boundaryValueSubsequentChar (0 ms)
[ RUN      ] isIdValidTest.boundaryValueLengthOfInput
[       OK ] isIdValidTest.boundaryValueLengthOfInput (0 ms)
[----------] 4 tests from isIdValidTest (0 ms total)

[----------] Global test environment tear-down
[==========] 13 tests from 3 test suites ran. (4211 ms total)
[  PASSED  ] 12 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] factorialTest.mutation

 1 FAILED TEST

File 'factorial.cpp'
Lines executed:100.00% of 7
Creating 'factorial.cpp.gcov'

File 'hw1.cpp'
Lines executed:100.00% of 9
Creating 'hw1.cpp.gcov'

File 'idVerifier.cpp'
Lines executed:100.00% of 14
Creating 'idVerifier.cpp.gcov'

File 'driver.cpp'
Lines executed:100.00% of 91
Creating 'driver.cpp.gcov'

line coverage result file

Following snippet marks which line of source code is/are ran by the test:

       23:   16:bool isIdValid(const char *str)
        -:   17:{
      23*:   18:        if (!str && !str[0])
    #####:   19:                return false;
        -:   20:
       23:   21:        size_t len = strlen(str);
        -:   22:
        -:   23:        // validate the first word and the length
       23:   24:        if (!((str[0] | LOWERCASE_MASK) >= 'a' &&
       19:   25:              (str[0] | LOWERCASE_MASK) <= 'z') ||
        -:   26:               len > MAX_LEN - 1) // '\0' is excluded to make sure all bytes
        -:   27:                                 //  lays within the buffer (i.e. no OOB access)
        7:   28:                return false;

where:
- ##### indicates the line is not covered
- number at the beginning of line (e.g. 23:) indicates how much times the line is executed
- * after the execution count indicates that the statement has unexecuted block(s)

Final exam

• black-box & white-box testing
  slide: Testing_l1.ppt p30-31

• various testing methods
  

  

• regression test

  • slide: Testing_l1.ppt p27
    
    

• 1_PropLogic.ppt

  • 
    • not p and(^) q
    • not q -> p
    • p ->q
      above statements are all equivalent to each other
  • DeMorgans
    
    

• 4_predCalculus.ppt

  • 

• $\mathrm{\forall }X\mathrm{\forall }Y$
  $T(X,Y)\equiv \mathrm{\forall }X,Y$
  $T(X,Y)$

• 4_predCalculus.ppt p17

  $\mathrm{\exists }x\mathrm{\exists }y\equiv \mathrm{\exists }xy$
  
  
  secondary expression:
  
  $\mathrm{\exists }xyt\{x\ne y\wedge x\ne t\wedge y\ne t\wedge L(Lynn,x)\wedge L(Lynn,y)\wedge L(Lynn,t)\wedge \mathrm{\forall }z[L(Lynn,z)\to (z=x\vee z=y\vee z=t)]\}$

  third expression:
  maybe same with the first one but

  $\mathrm{\exists }x\mathrm{\exists }y$ changed to
  $\mathrm{\exists }xy$

• 

Final project

• introduce selected tool
• how is the tool used for testing the program (maybe show some result the tool printed)

topics

• gtest installation, how to write test with gtest, and compiling the test with gtest
  • compilation command:

    ​​​​​​​​g++ -o driver $(SRCS) -Wall -lgtest -lpthread
    

• (1) gtest feature intro.
• (2) gtest feature intro.
  • death tests
  • user-defined assertions
• (3) gtest feature intro.
• gtest output result explanation (test result of all HWs)

Memo

• Test suite
  Refer to a set of related test cases.
• '<<' has name stream insertion operator, whereas '>>' has name stream extraction operator

Gtest

• Both test suite and test case name must not contain underscore character
• Assertion ASSERT_* would abort the current function when it fails, whereas failure of assertion EXPECT_* would record the failure but keep the test running
• Beware that a failure assertion may incur memory leak
• when possible, ASSERT_EQ(actual, expected) is preferred to ASSERT_TRUE(actual == expected), since it tells you actual and expected's values on failure
• To assert that a C string is NULL, use ASSERT_STREQ(c_string, NULL). Consider using ASSERT_EQ(c_string, nullptr) if c++11 is supported. To compare two string objects, one should use ASSERT_EQ
• In order to deal with floating point numbers, one should use some variation macros, see here for details. This prevents one from bothering by the rounding

Reference

• Escape sequence in string literal

tags: ntou