General Concepts
Summary
Failure, Fault, and Error
Failure := an observable incorrect behavior
Fault (aka bug) := incorrect code
Error := cause of a fault
Verification
- Testing (Dynamic verification)
- Static verification
- Inspections
- Manual, group activity
- Formal proofs of correctness
- Formally verify that code implements a formal specification
Pros and Cons
| Technique | Pros | Cons |
|---|---|---|
| Testing | No false postives | Incomplete |
| Static | Considers all program behaviors | Can generate false positives |
| Inspection | Systematic, Thorough | Informal, Subjective |
| Formal Proof | Strong guarantees | Complex, Expensive, Require mathematical specification |
Testing
Executing a program on a sample of the input domain
- Dynamic
- Optimistic approximation
A test is successful if the program fails
Granularity Levels
- Unit testing
- Testing of individual modules in isolation
- Integration testing
- Testing multiple modules and their interaction
- System testing
- Test the entire system as a whole
- Acceptance testing
- Test that the system does what the customer wants it to do
- Regression testing
- Test that changes made behave as expected
Black Box and White Box Testing
- Black box testing
- “Closed box”
- Based on description/specification of software
- Cover as much specified behavior as possible
- Cannot reveal errors due to implementation details
- White box testing
- “Open box”
- Based on the code
| Type | Based on | Coverage | Limitation |
|---|---|---|---|
| Black box | description/specification | Cover as much specified behavior as possible | Cannot reveal errors due to implementation details |
| White box | code | Cover as much coded behavior as possible | Cannot reveal errors due to missing paths |