Iter 2 & 3 testing

# Iter 2 & 3 testing ## Equivalence Partitions ### Take note: - Invalid cases should be underlined ### Todo: - [x] Follows, Follows* - [ ] Parent, Parent* - [x] ModifiesP - [x] ModifiesS - [ ] UsesP - [ ] UsesS - [x] Calls, Calls* - [x] Next - [x] Pattern A - [x] Pattern If - [x] Pattern W - [ ] With clause - [ ] Next * - [ ] Affects - [ ] Affects* ### Test case creation: - [x] Move all the single clause queries to iter1-test1 - [x] Move the simple prog for iter1-test1 - [ ] Create test source prog for iter2-test1 (should have multiple procedures) - [ ] Copy the queries from iter1-test1 into iter2-test1 - [ ] create iter2-test2 for the new relationships, like next, calls* etc. - [ ] Create source prog ### Follows(param 1, param 2) & Follows*(param 1, param 2) param 1 & param 2 -> synonym | _ | INTEGER #### Equivalence Partition for param 1 & param 2 * synonym: [s1], [a1], [p1], [r1], [w1], [if1], [c1], [proc1] * \_: [_], * INTEGER: [-infinity ... -1], [0], [1 ... MAX_INT] | param 1 | param 2 | | ----------- | ----------- | | s1 | a1 | | a1 | a1 | | r1 | p1 | | w1 | if1 | | c1 | p1 | | _ | s1 | | proc1| s1 | | - 1 | s1 | | 0 | s1 | | a1 | _ | | s1 | proc1| | a1 | -1 | | a1 | 0 | | a1 | 1 | ### ModifiesP(param 1, param 2) param 1 & param 2 -> synonym | ‘\_’ | ‘"’ IDENT ‘"’ #### Equivalence Partition for param 1 * synonym: [proc1] * \_: [_], * ‘"’ IDENT ‘"’: ["9"], [""], ["valid"], ["invalid_string"] #### Equivalence Partition for param 2 * synonym: [v1] * \_: [_], * ‘"’ IDENT ‘"’: ["9"], [""], ["valid"], ["invalid_string"] | param 1 | param 2 | | ----------- | ----------- | | p1 | v1 | | p1 | _ | | p1 | "valid" | | _ | v1 | | _ | _ | | _ | "valid" | | validString | v1 | | validString | _ | | validString | "valid" | |"invalid_string"| v1 | |"invalid_string"| _ | |"invalid_string"| "valid" | | p1 |"invalid_string"| ### ModifiesS(param 1, param 2) param 1 -> synonym | ‘\_’ | INTEGER param 2 -> synonym | ‘\_’ | ‘"’ IDENT ‘"’ #### Equivalence Partition for param 1 * synonym: [s1], [a1], [p1], [r1], [w1], [if1], [c1], [proc1] * \_: [_], * INTEGER: [-infinity ... -1], [0], [1 ... MAX_INT] #### Equivalence Partition for param 2 * synonym: [v1] * \_: [_], * ‘"’ IDENT ‘"’: ["9"], [""], ["valid"], ["invalid_string"] | param 1 | param 2 | | ----------- | ----------- | | s1 | v1 | Select s1 such that Uses() | a1 | _ | | p1 | "valid" | | r1 | s1 | | w1 | s1 | | if1 | s1 | | c1 | s1 | | proc1| s1 | | - 1 | s1 | | 0 | s1 | | s1 | proc1| | s1 | "9" | | s1 | "" | | s1 | "invalid_string" | ### Calls & Calls* (param 1, param 2) param 1 & 2 -> synonym | ‘_’ | ‘"’ IDENT ‘"’ #### Equivalence Partition for param 1 & 2 * synonym: [proc1], [a1], [c1], [prog_line] * \_: [_], * ‘"’ IDENT ‘"’: ["9"], [""], ["valid"], ["invalid_string"] | param 1 | param 2 | | ----------- | ----------- | | proc1 | proc1 | | proc1 | proc2 | | a1 | proc1 | | c1 | proc1 | | proc1 |prog_line| | proc1 | _ | | _ | _ | | _ | proc1 | | proc1 | "valid" | | "valid" | proc2 | | "valid" | "valid" | | proc1 | "9" | | proc1 | "" | | proc1 | "invalid_string" | | "valid" | _ | | _ | "valid" | ### Next(param 1, param 2) & Next*(param 1, param 2) #### note: - next/* is valid for Statement (stmt / read / print / call / while / if / assign) & Program line (prog_line) synonyms. param 1 & param 2 -> synonym | _ | INTEGER #### Equivalence Partition for param 1 & param 2 * synonym: [s1], [a1], [p1], [r1], [w1], [if1], [c1], [proc1], [prog_line] * \_: [_], * INTEGER: [-infinity ... -1], [0], [1 ... MAX_INT] | param 1 | param 2 | | ----------- | ----------- | | s1 | a1 | Note: You may swap the following synonyms around | a1 | a1 | as long as they all return at least 1 valid result. | r1 | p1 | | w1 | if1 | | c1 | p1 | <- Up to here. | _ | s1 | | _ | _ | | s1 | _ | Note: Feel free to change any of these s1 to a specific type of | s1 | -1 | statement synonym. | s1 | 0 | | s1 | 1 | | 3 | s1 | | 4 | 7 | Note: These 2 values should be valid line# | proc1| s1 | |prog_line | s1 | | 2 | _ | | _ | 1 | ### note: for if, while and assign pattern, synonyms must be variable type. ### Pattern a (param1, param 2) param 1 -> synonym | ‘\_’ | ‘"’ IDENT ‘"’ param 2 -> exact match | partial match | ‘\_’ #### Equivalence partition for param 1 * synonym: [v1] * \_: [_] * ‘"’ IDENT ‘"’: ["9"], ["valid"], ["invalid_string"] #### Equivalence partition for param 2 * exact match: ["x"], ["9"], ["x + y * 3"] * partial match: [\_"x"\_], [\_"9"_], [\_"x + y * 3"\_] * \_: [_] | param 1 | param 2 | | ----------- | ----------- | | v1 | exact | | v1 | _ | | v1 | partial | | _ | exact | | _ | _ | | _ | partial | | validString | exact | | validString | \_ | | validString | partial | |invalidString| exact | |invalidString| \_ | |invalidString| partial | ### Pattern while (param1, param2) param 1 -> synonym | ‘\_’ | ‘"’ IDENT ‘"’ param 2 -> _ #### Equivalence partition for param 1 * synonym: [v1] * \_: [_] * ‘"’ IDENT ‘"’: ["9"], [""] ["alphanumeric"], ["invalid_string"] #### Equivalence partition for param 2 * \_: [_] | param 1 | param 2 | | ----------- | ----------- | | v1 | _ | | _ | _ | | "" | _ | | "9" | _ | | validstring | _ | | invalidstring | _ | ### Pattern if (param1, param2, param3) param 1 -> synonym | ‘\_’ | ‘"’ IDENT ‘"’ param 2 -> _ #### Equivalence partition for param 1 * synonym: [v1] * \_: [_] * ‘"’ IDENT ‘"’: ["9"], [""] ["alphanumeric"], ["invalid_string"] #### Equivalence partition for param 2 * \_: [_] | param 1 | param 2 | param 3 | | ----------- | ----------- | -----------| | v1 | _ | - | | _ | _ | | | "" | _ | | | "9" | _ | | | validstring | _ | | | invalidstring | _ | | ### note: if and while pattern have the same combination of cases for pql. ### With (param 1, param 2) param 1 & 2 -> ‘”’ IDENT ‘”’ | INTEGER | attrRef | synonym (prog_line) #### Equivalence Partition for param 1 & 2 * attrRef: procedure.procName, call.procName, variable.varName, read.varName, print.varName, constant.value, stmt.stmt#, read.stmt#, print.stmt#, call.stmt#, while.stmt#, if.stmt#, assign.stmt# * ‘"’ IDENT ‘"’: ["9"], [""], ["valid"], ["invalid_string"] * INTEGER: [-infinity ... -1], [0], [1 ... MAX_INT] | param 1 | param 2 | | ----------- | ----------- | | | | ## Multi Clause Design Objective & Heuristics - Test Case Design Objectives - Query with at least 2 double syn clause AND both syn are the same. - Query with at least 3 chained clause - Query with at least 2 chained (clause with 2 syn) and at least 1 chained (clause with 1 syn) - Query with chain with Pattern to be evaluated first, then such that [Note: this is a short case] - Query with chain with such that evaluated first, then pattern [Note: this is a short case too] - Query with alternate such that/pattern ^. - Random mix, but with tuple and BOOLEAN tests. - Boundary Tests -> Clauses could have first, last, non-existing statements - Load/Stress Testing -> Many Clauses & different types - Test Case Design Heristics - A given combination of clause pair (e.g. follows + parent) must appear in at least 1 test. - Order of Clauses in tests can be changed without affecting correctness. - don't prioritise simple queries involving old relationships alr tested in iter 1. - After testing pairwise, most of the multi-clause testing efforts should focus on grouping & Table merging - Independent, All dependent vs mix of common synonyms in multi-cl queries - Different target synonym type (e.g. Select variable, prog_line, procedure...)