Black Box Testing

Summary

• Advantages
• Approach
• Random Tests
• Partition testing
  • Example
• Category-Partition Method
  • Identify Categories
  • Partition Categories Into Choices
  • Identify Constraints Among Choices
  • Produce and Evaluate Test Case Specifications
  • Generate Test Cases from Test Case Specifications
• Model Based Testing
  • Finite State Machine

Advantages

• Focus on the domain
• No need for code
  • Early test design
• Catches logic defects
• Applicable at all granularity levels

Approach

How to go from functional specification to test cases?

1. Identify independently testable features
2. Identify relevant inputs
3. Derive test case specifications
4. Generate test cases

Random Tests

Pick inputs at random

• Provides uniform input
• All inputs considered equal
• No designer bias
  • Write tests assuming behavior of user that may not be true

Needle in a haystack

Partition testing

Failures are generally sparse, but dense in some subdomains

1. Identify partitions of domain
2. Select inputs from each partition

Example

split(String str, int size)

Possible Partitions:

• size < 0
• size = 0
• size > 0
• str with length < size
• str with length in [size, size * 2]
• str with length > size * 2

Possible Inputs:

• size = -1
• size = 0
• size = MAXINT
• string with length = (size -1)
• string with length = size

Category-Partition Method

6 steps to go from specification to test cases, created by Ostrand & Balcer, CACM in June 1988

1. Identify independently testable features
2. Identify categories
3. Partition categories into choices
4. Identify constraints among choices
5. Produce/Evaluate test case specifications
6. Generate test cases from test case specifications

Identify Categories

Categories := characteristics of each input element

Partition Categories Into Choices

Interesting cases/subdomains

Identify Constraints Among Choices

This keeps us from generating nonsensical test cases later and to reduce number of test cases.

3 types:

• Property … if
• Error
  • Only consider special characters if length is not 0
• Single

Produce and Evaluate Test Case Specifications

Can be automated

Produces test frame

Test Frame #36
  input str
    length: size - 1
    content: special characters
  input size
    value: > 0

Generate Test Cases from Test Case Specifications

Simple instantiation of frames

Final Result : Set of concrete tests

Test Case #36
  str = "ABCC!\nt0"
  size = 10

Model Based Testing

Different ways to identify relevant inputs and derive test case specifications

Finite State Machine

Graph where nodes represents states and edges represent state transitions and labels mean event/action.

Test cases can cover state transitions