Managing Software Quality Assurance Test Rounds – Part Three (Developing the Project Matrix)

Now that we understand how to assess the project risk, we are ready to actually plan and manage our test rounds. Since software development is a dynamic process, each level of testing must allow a great amount of flexibility. Offering all testing methods and types at each level gives the quality assurance manager the ability to tailor each test round to challenge the unique risks each system change represents.

I recommend the following matrix be applied to each test level (unit, integration, system and acceptance).

TypeMethodTest Case IDTest Case Risk ScoreTests Use Care IDAssign to
Smoke or Sanity
A/B Alternate Route

This matrix lists the test types in rows. It then lists the variables, which may impact the test types, in columns. Offering a great amount of flexibility, this table also allows any test method (white, gray or black box) to be applied to any test type. Each test case is assigned a risk score indicating the risk posed to the application if the case is not run. This value allows for rapid prioritization in case the test round must be cut short. The use case is indicated insuring all use cases are tested at least once. Indicating the owner of the test allows for quick followup.

I also find it helpful to add a column entitled special instructions and another entitled complete. If no other management tools are available, these additional columns should suffice in creating a brief overview the test round’s progress.

If using Excel or Numbers to organize test rounds, I find it helpful to associate a note containing a short question with each test type. If I don’t know the answer, I should run the test.

  • Installation – Does the application run after installation?
  • Compatibility – Does system support the application?
  • Smoke or sanity – Are there serious problems? Is it safe to proceed with further testing?
  • Regression – Have any old bugs resurfaced?
  • Acceptance – Does the application support the new business requirements?
  • Alpha – Are the new features stable enough for a simple end user review?
  • Bata – Do the new features initially meet the end users expectations?
  • Functional – Does the application support the new technical requirements?
  • Destructive – Do unexpected inputs cause the system to fail?
  • Performance – (the numbers given serve as an example only)

–Does the system support a load of 100,000 users?
–Is the system scalable when adding 1 to 100,000 users?
–Does the system display endurance when 100,000 users hit the system non-stop for 48 hours?

  • Usability – Is the system user friendly?
  • Accessibility – Does the system meet the requirements for special needs users?
  • Security – Does the system satisfy organization’s required security standards?
  • Internationalization – Does the system support the expectations of international users?
  • Development – Did the system pass the code review? (Was the best technical approach used for this solution?)
  • A/B Alternate Route – Have alternative workflows been identified and tested?

Click her for Part Four – Test Round Scenario

About Larry

Larry Lawhead is a passionate Agilest with a love for coaching, mentoring and organizational transformation. Huge fan of the Lean Enterprise, LeSS, Theory of Constraints, systems thinking and of course the Toyota Production System. Avid learner. Deeply influenced by the works of Jeff Sutherland, Eliyahu Goldratt, James Womack, Craig Larman, Faith Fuller, Marita Fridjhobn and countless others. He is likewise a member of the team that brings the Agile Open to the Southern California. Above all else, he loves coaching teams and individuals. He believes creating a safe and empowered place for individuals and self-organized teams to develop brilliant solutions, shared vision, demonstrate compassion and desire to continuously improve brings life’s greatest rewards. Watching clients unfold into a greater sense wholeness, purpose and direction is the miracle coaching enables. This satisfaction leads to continuous self-improvement, curiosity and personal growth that Larry passionately embraces.
This entry was posted in Managing Software Quality Assurance Test Rounds, Quality & Value and tagged , , , , , , , , , , , , . Bookmark the permalink.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.