Managing Software Quality Assurance Test Rounds – Part Two (Assessing Quality Assurance Risk)

“What gets measured gets managed and what gets managed gets done.” – Peter Drucker

Before the project is organized and critical numbers are calculated, I recommend running a risk assessment exposing weaknesses in your quality assurance planning and processes. This analysis is a great tool that identifies issues well in advance. The assessment is much easier than it may sound. Here is what you need:

  1. List of known quality assurance risks
  2. Risk stake holders
  3. Category (optional)
  4. Assigned Risk Score

I quantify the risk by assigning each risk a number. One represents the least amount of risk and five the highest. You may quantify risk by assigning any number you like, but this system of 1-5 works well for me. Assigning the score comes from experience. The more projects you manage, the more accurate your estimations will become. Insure you inform those receiving the report the risks are based on your experience. The more they interact with you and your numbers, the more they will grow to respect your judgment.

I assembled a list of risks by searching the web for typical software testing issues. I continued searching and reading about testing issues until they began to repeat themselves. This may sound rather unscientific, but blog posts, book reviews and community discussions are about as close to “ground zero” as one can get. If you want to collect a list of risks which reflect the project you manage, deriving them from others managing the same type of project is extremely beneficial. The list is not static. As you become accustom to identifying and managing your own project risk, you may want to add or remove items from this list, until you have a catalog that works for you.

The following example contains 23 of the software testing issues most often reported by other software quality assurance managers. I organized these into categories and assigned them to the team which represents the risk. I then assigned risk scores based on a hypothetical situation.

Here is what the numbers reveal. The QA team is generally experienced, but has had little training on the application they are testing. Testing and reporting processes are in place and experience has confirmed they are solid. Post deployment bug management requires improvement and there is some scope issues with the development team. Along with not being familiar with the application sufficient unit testing and having sufficient test cases or scripts pose the greatest risk to this project. Keeping this in mind, the overall average of these scores is 2.28 which is moderate. If the quality assurance manager mitigates all tasks with a risk score of four, the project or test round should result in few bugs finding their way into production. If time allows, resolving issues with a score of three will greatly enhance the effectiveness of the test round.

Risk Assessment Model

RiskRisk CategoryRisk Stake HoldersRisk Score
Sufficient QA experienceTrainingQA Team2.5
Sufficient application experienceTrainingQA Team4
Sufficient AuthorityPolicyCross Team2
Sufficient bug reporting solutionPolicyQA Team1
Sufficient bug reporting processPolicyCross Team2
Sufficient bug mgt. process (acceptance testing)PolicyCross Team1
Sufficient bug mgt. process (post deployment)PolicyCross Team3
Appropriate QA LocationPolicyQA Team1
Sufficient QA EnvironmentPolicyQA Team1
Sufficient Dev Scope DefinedPolicyCross Team3
Sufficient QA Scope DefinedPolicyQA Team2
Sufficient ÒHand OffÓ PolicyPolicyCross Team1
Sufficient Unit TestingPolicyCross Team4
Sufficient testingPolicyCross Team3
Sufficient team communicationPolicyQA Team1
Sufficient domain experience (acceptance testing)PolicyCross Team3
Sufficient testing time availablePlanningCross Team3
Sufficient test level appliedPlanningQA Team2
Sufficient test method appliedPlanningQA Team1
Sufficient set of test types appliedPlanningQA Team3
Sufficient test cases developedPlanningQA Team3
Sufficient test cases / scripts selectedPlanningQA Team2
Sufficient test case / script updates.PlanningQA Team4
Risk Score for this test round2.28

Now that the general quality assurance risk is assessed, you may turn your attention toward planning your projects or test rounds and working your critical numbers.

Click her for Part Three – Developing the Project Matrix

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.