Professor Cem Kaner J.D., PhD explains the “Impossibility of Complete Testing” in his lecture notes. This lecture mathematically demonstrates how impossible it to test all the options of a given set of enhancements. The testing straggly is to identify the areas that are most likely to pose issues and to develop and apply test cases / scripts to these areas. Professor Kaner points out that planning is really the development of creating a sampling strategy. Any set of tests that a tester executes can only be a tiny sample of the set that could be run. The goal is to come up with a powerful enough sample to find the most defects.
He likewise proposes this “rule of thumb”.
- Cover every line of code with at least one test case.
- Include tests for every variable (input / output).
- Check wether the variable can handle extreme values and special cases. (If the system can handle extreme values, then it can probably handle simpler ones.)
- Include tests for every data flow or for every pair of variables that can possibly occur.
Professor Kaner concludes his notes by further recommending the following:
- Develop the habit of “Forward Scanning”. (Anticipate issues.)
- Ongoing extension of testing depth.
- Heuristics for sampling. (Allow the testers to learn how to improve their tests as they test.)
- Holding risk-orientated discussions
- Creating obvious stopping rules
Here is how I apply these recommendations:
- Anticipate issues as you put together test strategy.
- Each test round should result in more effective tests.
- Testers should be allowed to improve their tests as they test.
- Keep all stake holders in the loop as to the every changing set of risks during the test round.
- Offer the stake holders exact criteria for a “go or no go” for testing to be complete. Proceed with their agreement.
Now that we understand each test cycle contains the risk of wrongly identifying the tests which should be included in the plan, let’s discuss how to identify, manage and report risk.
Click her for Part Two – Assessing Quality Assurance Risk
