Dynamic Testing Techniques

Dynamic Testing Techniques
The process of evaluating a system or component based upon its behaviour during execution. “

Black Box Testing:
Test case selection that is based on an analysis of the specification of the component without
reference to its internal workings.”

Testing based on an analysis of the specification of a piece of software without reference to its
internal workings. The goal is to test how well the component conforms to the published requirements for the component
 

• It attempts to find:
• Incorrect or missing functions
• Interface errors
• Errors in data structures or external database access
• Performance errors
• Initialization and termination errors

Black-box test design treats the system as a "black-box", so it does not explicitly use knowledge of the internal structure. Black box testing is based solely on the knowledge of the system requirements. Black-box test design is usually described as focusing on testing functional requirements. In comparison, White-box testing allows one to peek inside the "box", and it focuses specifically on using internal knowledge of the software to guide the selection of  test data.

Black box testing focuses on testing the function of the program or application against its
specifications. Specifically, this technique determines whether combinations of inputs and
operations produce expected results.
 

• Test Case design Techniques under Black Box Testing:
• Equivalence class partitioning
• Boundary value analysis
• Comparison testing
• Orthogonal array testing
• Decision Table based testing
• Cause Effect Graph

Equivalence Class Partitioning
 

Equivalence class: A portion of the component's input or output domains for which the component's behaviour is assumed to be the same from the component's specification.

Equivalence partition testing: A test case design technique for a component in which test cases are designed to execute representatives from equivalence classes.

Determination of equivalence classes
 

Examine the input data.
 

Few general guidelines for determining the equivalence classes can be given
 

If the input data to the program is specified by a range of values:
 

e.g. numbers between 1 to 5000.
One valid and two invalid equivalence classes are defined.
 

If input is an enumerated set of values:
o e.g. {a,b,c}
 

one equivalence class for valid input values
· Another equivalence class for invalid input values should be defined.
 

Example
A program reads an input value in the range of 1 and 5000:
computes the square root of the input number
There are three equivalence classes:
the set of negative integers,
set of integers in the range of 1 and 5000,
Integers larger than 5000.
The test suite must include:
representatives from each of the three equivalence classes:
A possible test suite can be: {-5, 500, 6000}.
 

Boundary Value Analysis
Boundary value: An input value or output value which is on the boundary between equivalence classes, or an incremental distance either side of the boundary.

Boundary value analysis: A test case design technique for a component in which test cases are designed which include representatives of boundary values.

Example
· For a function that computes the square root of an integer in the range of 1 and 5000:
· Test cases must include the values: {0, 1, 5000, and 5001}.

Cause and Effect Graphs
A graphical representation of inputs or stimuli (causes) with their associated outputs (effects),
which can be used to design test cases”

Cause-effect graphing attempts to provide a concise representation of logical combinations and corresponding actions. A Cause-and-Effect Diagram is a tool that helps identify, sort, and display possible causes of a specific problem or quality characteristic (Viewgraph 1). It graphically illustrates the relationship between a given outcome and all the factors that influence the outcome.

Causes (input conditions) and effects (actions) are listed for a module and an identifier is
assigned to each.

A cause-effect graph developed.

• Graph converted to a decision table.
• Decision table rules are converted to test cases.

The C&E diagram is also known as the Fishbone/Ishikawa diagram because it was drawn to
resemble the skeleton of a fish, with the main causal categories drawn as "bones" attached to the spine of the fish, as shown below
 

Example C&E diagram for a Server crash issue:

Advantages

• Helps determine root causes
• Encourages group participation
• Indicates possible causes of variation
• Increases process knowledge
• Identifies areas for collecting data

Comparison Testing

• In some applications, the reliability is critical.
• Redundant hardware and software may be used.
• For redundant s/w, use separate teams to develop independent versions of the software.
• Test each version with same test data to ensure all provide identical output.
• Run all versions in parallel with a real-time comparison of results.
• Even if will on run one version in final system, for some critical applications can develop
• independent versions and use comparison testing or back-to-back testing.
• When outputs of versions differ, each is investigated to determine if there is a defect.
• Method does not catch errors in the specification.
• Exercise on Live Application

White-Box Testing:
“Test case selection that is based on an analysis of the internal structure of the component.”
Testing based on an analysis of internal workings and structure of a piece of software. Also known as Structural Testing / Glass Box Testing / Clear Box Testing. Tests are based on coverage of code statements, branches, paths, conditions

Aims to establish that the code works as designedExamines the internal structure and implementation of the program Target specific paths through the programNeeds accurate knowledge of the design, implementation and code
 

Test Case design techniques under White Box Testing:

• Statement coverage
• Branch coverage
• Condition coverage
• Path coverage
• Data flow-based testing
• Mutation testing

Statement Coverage:
Test case design technique for a component in which test cases are designed to execute
statements.”

Design test cases so that every statement in a program is executed at least once. Unless a
statement is executed, we have no way of knowing if an error exists in that statement
 

Example:
Euclid's GCD computation algorithm:

int f1(int x, int y)

{
  while (x != y)

  {
   if (x>y) then
   x=x-y;
  else y=y-x;
  }
  return x; 

}

By choosing the test set {(x=3, y=3), (x=4, y=3), (x=3, y=4)} all statements are executed at least
once

Branch Coverage:
Branch : A conditional transfer of control from any statement to any other statement in a
component, or an unconditional transfer of control from any statement to any other statement in
the component except the next statement, or when a component has more than one entry point, a transfer of control to an entry point of the component.
 

Branch Testing: A test case design technique for a component in which test cases are designed to execute branch outcomes.

Branch testing guarantees statement coverage
 

Example
Test cases for branch coverage can be: {(x=3, y=3), (x=4, y=3), (x=3, y=4)}

Condition Coverage:
Condition: “A Boolean expression containing no Boolean operators. For instance, A<B is a
condition but A and B is not.”

Test cases are designed such that:
Each component of a composite conditional expression given both true and false values.
 

Example
Consider the conditional expression ((c1.and.c2).or.c3):
Each of c1, c2 and c3 are exercised at least once i.e. given true and false values.  

• Condition testing stronger than branch testing. 
• Branch testing stronger than statement coverage testing

Path Coverage:
Path: A sequence of executable statements of a component, from an entry point to an exit point.
Path testing: A test case design technique in which test cases are designed to execute paths of a component.

A testing mechanism proposed by McCabe

Test cases which exercise basic set will execute every statement at least once. Aim is to
derive a logical complexity measure of a procedural design and use this as a guide for defining a basic set of execution paths.
 

Cyclomatic Complexity:The Cyclomatic complexity gives a quantitative measure of the logical complexity. Introduced by Thomas McCabe in 1976, it measures the number of linearly-independent paths through a program module. This measure provides a single ordinal number that can be compared to the complexity of other programs. Cyclomatic complexity i s often referred to simply as program complexity, or as McCabe's complexity.
 

This value gives the number of independent paths in the Basis set, and an upper bound for
the number of tests to ensure that each statement is executed at least once. An independent path is any path through a program that introduces at least one new set of processing statements or a new condition (i.e., a new edge)

 Cyclomatic complexity (CC) = E - N + p
Where
E = the number of edges of the graph
N = the number of nodes of the graph
p = the number of connected components

Cyclomatic complexity provides upper bound for number of tests required to guarantee coverage of all program statements. As one of the more widely-accepted software metrics, it is intended to be independent of language and language format.
 

Deriving Test Cases
1. Using the design or code, draw the corresponding flow graph.
2. Determine the Cyclomatic complexity of the flow graph.
3. Determine a basis set of independent paths.
4. Prepare test cases that will force execution of each path in the basis set.

Data Flow-Based Testing:
“Testing in which test cases are designed based on variable usage within the code.”

Selects test paths of a program:
According to the locations of definitions and uses of different variables in a program.
For a statement numbered S,

DEF(S) = {X/statement S contains a definition of X}
USES(S) = {X/statement S contains a use of X}
 

Example: 1: a=b; DEF (1) = {a}, USES (1) = {b}.
 

Example: 2: a=a+b; DEF(1)={a}, USES(1)={a,b}.
 

A variable X is said to be live at statement S1, if
X is defined at a statement S:
there exists a path from S to S1 not containing any definition of X.
 

DU Chain Example
1 X(){
2 a=5; /* Defines variable a */
3 While(C1) {
4 if (C2)
5 b=a*a; /*Uses variable a */
6 a=a-1; /* Defines variable a */
7 }
8 print(a); } /*Uses variable a */
 

Definition-use chain (DU chain)
[X,S,S1],
S and S1 are statement numbers,
X in DEF(S)
X in USES(S1), and
the definition of X in the statement S is live at statement S1.
Every DU chain in a program is covered at least once. It is very useful for selecting test paths of a program containing nested if and loop statements
 

1X(){
2 B1; /* Defines variable a */
3 While(C1) {
4 if (C2)
5 if(C4) B4; /*Uses variable a */
6 else B5;
7 else if (C3) B2;
8 else B3; }
9 B6 }
[a,1,5]: a DU chain.
 

Assume:
DEF(X) = {B1, B2, B3, B4, B5}
USED(X) = {B2, B3, B4, B5, B6}
There are 25 DU chains.
However only 5 paths are needed to cover these chains.

Mutation Testing: 

The software is first tested using an initial testing method based on white-box strategies we already discussed. After the initial testing is complete, mutation testing is taken up. The idea behind mutation testing is to make a few arbitrary small changes to a program at a time, each time the program is changed it is called a mutated program; the change is called a mutant.A mutated program tested against the full test suite of the program. If there at least one test case in the test suite for which a mutant gives an incorrect result, then the mutant is said to be dead.

If a mutant remains alive even after all test cases have been exhausted, the test suite is enhanced to kill the mutant. The process of generation and killing of mutants can be automated by predefining a set of primitive changes that can be applied to the program.
 

The primitive changes can be:
Altering an arithmetic operator,
Changing the value of a constant,
Changing a data type, etc.
 

A major disadvantage of mutation testing:
computationally very expensive,
A large number of possible mutants can be generated.
 

Grey Box Testing
Grey box Testing is the new term, which evolved due to the different architectural usage of the system. This is just a combination of both Black box & White 

box testing. Tester should have the knowledge of both the internals and externals of the function. 

Tester should have good knowledge of White Box Testing and complete knowledge of
Black Box Testing.
 

Grey box testing is especially important with Web and Internet applications, because the
Internet is built around loosely integrated components that connect via relatively welldefined
interfaces
K.