Managing Software Quality Assurance Test Rounds – Part One (Understanding the Nature of Software Testing)

Professor Cem Kaner J.D., PhD explains the “Impossibility of Complete Testing” in his lecture notes. This lecture mathematically demonstrates how impossible it to test all the options of a given set of enhancements. The testing straggly is to identify the areas that are most likely to pose issues and to develop and apply test cases / scripts to these areas. Professor Kaner points out that planning is really the development of  creating a sampling strategy. Any set of tests that a tester executes can only be a tiny sample of the set that could be run. The goal is to come up with a powerful enough sample to find the most defects.

He likewise proposes this “rule of thumb”.

  1. Cover every line of code with at least one test case.
  2. Include tests for every variable (input / output).
  3. Check wether the variable can handle extreme values and special cases. (If the system can handle extreme values, then it can probably handle simpler ones.)
  4. Include tests for every data flow or for every pair of variables that can possibly occur.

Professor Kaner concludes his notes by further recommending the following:

  1. Develop the habit of “Forward Scanning”. (Anticipate issues.)
  2. Ongoing extension of testing depth.
  3. Heuristics for sampling. (Allow the testers to learn how to improve their tests as they test.)
  4. Holding risk-orientated discussions
  5. Creating obvious stopping rules

Here is how I apply these recommendations:

  1. Anticipate issues as you put together test strategy.
  2. Each test round should result in more effective tests.
  3. Testers should be allowed to improve their tests as they test.
  4. Keep all stake holders in the loop as to the every changing set of risks during the test round.
  5. Offer the stake holders exact criteria for a “go or no go” for testing to be complete. Proceed with their agreement.

Now that we understand each test cycle contains the risk of wrongly identifying the tests which should be included in the plan, let’s discuss how to identify, manage and report risk.

Click her for Part Two – Assessing Quality Assurance Risk

About Larry

Passionate, experienced Agilest with a love for Scrum and Kanban. Huge fan of Lean, Theory of Constraints and of course the Toyota Production System. Avid learner. Deeply influenced by the works of Jeff Sutherland, Ken Schwaber, Eliyahu Goldratt, Jeff Patton and countless others. Active in both the PMI and Agile communities in Southern California.
This entry was posted in Managing Software Quality Assurance Test Rounds, Quality & Value and tagged , , , , . Bookmark the permalink.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.