Managing Software Quality Assurance Test Rounds – Part Four (Test Round Scenario)

Now that we understand the overall project risk and have our project matrix in place, we are ready to actually plan our tests and follow the critical numbers.

The purpose of this example is not to be entirely accurate in every detail. Exact test case and use case matching was not attempted. Such granularity was beyond the scope of this example. Likewise the amount and types of tests conducted at each level, may not reflect a perfectly balanced test round. Such discretion is the responsibility of the quality assurance manager based on the nature of the enhancements and the risk they pose to the system.

Here is the background to our sample test:

  1. New functionality has been added to an existing application.
  2. A new release is expected once every six weeks. For this reason no test cases or test scripts with a risk score of 1 or 2 are used for mission critical test types. Those with a risk score of three are used sparingly.
  3. Each use case is tested by at least one method at some level. The same use case may be tested at different levels.
  4. Test methods (white, gray or black box) are isolated by level. This may not apply to each test round. (At times a black box test may be required at the integration level, and some gray box testing at the unit test level etc.)
  5. A/B testing is done at both the integration and acceptance levels. The acceptance testers may be aware of unusual execution paths based on their interaction with end users. The quality assurance team should become aware of these unusual interactions and insure new discoveries are documented in new or updated test cases.

This test assumes the following agreements have been made:

  1. Use Cases from the business unit are available.
  2. The quality assurance team has received some level of training from the business unit.
  3. The quality assurance team has obtained from the development team the technical requirements.
  4. Agreements have been struck with the development team for unit testing, some business analysts, customer support and end user stakeholders to assist with acceptance testing.

The critical path for this sample scenario is as follows:

  1. Obtain agreement for participation. (see assumptions)
  2. Begin unit testing.
  3. Complete unit testing.
  4. Begin integration alpha testing.
  5. Complete integration alpha testing.
  6. Begin acceptance beta testing.
  7. Begin integration smoke testing.
  8. Complete integration smoke testing.
  9. Begin full integration testing and system testing.
  10. Complete acceptance beta testing.
  11. Complete full integration and system testing.
  12. Begin acceptance smoke testing.
  13. Complete acceptance smoke testing.
  14. Begin full acceptance testing.
  15. Successful completion of acceptance testing indicates the system is deployment ready and may move to the staging environment.

Unit Test

All tasks are assigned to the lead developer Mark. He will assign the test cases to individuals on his team.

Planing Matrix Example - Unit Test

TypeMethodTest IDRiskCase IDAssigned
Smoke or Sanityn/a
RegressionWhite Box1425129Mark
FunctionalWhite Box1114271Mark
UsabilityWhite Box1355178Mark
DevelopmentWhite Box1294.5372Mark
A/B Alternate RouteWhite Box8955441Mark

Integration Test

Acceptance testers asked to beta test Test Cases 875 – 878. For this reason after unit testing is complete, Charles begins his alpha tests.

Planing Matrix Example - Integration Test

TypeMethodTest Case IDTest Case Risk ScoreTests Use Care IDAssign to
InstallationGrey Box1214976Joyce
InstallationGrey Box1225977Joyce
InstallationGrey Box1233978Joyce
CompatibilityGrey Box249576Reanna
CompatibilityGrey Box250577Reanna
CompatibilityGrey Box251578Reanna
CompatibilityGrey Box252479Reanna
Smoke or SanityGrey Box764588Daniel
Smoke or SanityGrey Box765589Daniel
Smoke or SanityGrey Box766590Daniel
Smoke or SanityGrey Box767491Daniel
RegressionGrey Box3254547Donald
RegressionGrey Box3265548Donald
RegressionGrey Box3275549Donald
RegressionGrey Box3284550Donald
RegressionGrey Box3294551Donald
RegressionGrey Box3403552Donald
RegressionGrey Box3414553Donald
AcceptanceGrey Box3123743William
AcceptanceGrey Box3135744William
AcceptanceGrey Box3143645William
AcceptanceGrey Box3154746William
AcceptanceGrey Box3165747William
AcceptanceGrey Box3175748William
AcceptanceGrey Box3184749William
AlphaGrey Box8755658Charles
AlphaGrey Box8765659Charles
AlphaGrey Box8775670Charles
AlphaGrey Box8785671Charles
FunctionalGrey Box4293538Daniel
FunctionalGrey Box4305539Daniel
FunctionalGrey Box4315540Daniel
FunctionalGrey Box4325541Daniel
FunctionalGrey Box4335542Daniel
FunctionalGrey Box4345543Daniel
FunctionalGrey Box4354544Daniel
DestructiveGrey Box352352Joyce
DestructiveGrey Box353553Joyce
DestructiveGrey Box354554Joyce
DestructiveGrey Box355455Joyce
DestructiveGrey Box356566Joyce
UsabilityGrey Box4435442Reanna
UsabilityGrey Box4444443Reanna
UsabilityGrey Box4454444Reanna
UsabilityGrey Box4463445Reanna
AccessibilityGrey Box1042304William
AccessibilityGrey Box1051305William
AccessibilityGrey Box1063306William
SecurityGrey Box3064307William
SecurityGrey Box3073308William
SecurityGrey Box3085309William
SecurityGrey Box3095310William
InternationalizationGrey Box9203400Reanna
InternationalizationGrey Box9212401Reanna
InternationalizationGrey Box9222402Reanna
A/B Alternate RouteGrey Box2255389Daniel
A/B Alternate RouteGrey Box2265390Daniel
A/B Alternate RouteGrey Box2274391Daniel
A/B Alternate RouteGrey Box2285392Daniel

System Test

After Charles finishes his alpa testing, he may begin his system tests. System tests are run on an isolated system, having no impact on regression tests. A fully automated system is used. For this reason, one resource is sufficient to create new scripts and run the tests.

Planing Matrix Example - System Test

TypeMethodTest Case IDTest Case Risk ScoreTests Use Care IDAssign toAdditional Instructions
CompatibilityGrey Box1249576Charles
CompatibilityGrey Box1250477Charles
CompatibilityGrey Box1251478Charles
CompatibilityGrey Box1252579Charles
CompatibilityGrey Box1253580Charles
Smoke or Sanityn/a
RegressionGrey Box13254547Charles
RegressionGrey Box13265548Charles
RegressionGrey Box13275549Charles
RegressionGrey Box13284550Charles
FunctionalGrey Box14293538Charles
FunctionalGrey Box14302439Charles
FunctionalGrey Box14315540Charles
FunctionalGrey Box14324541Charles
FunctionalGrey Box14333542Charles
FunctionalGrey Box14345543Charles
DestructiveGrey Box1352352Charles
DestructiveGrey Box1353353Charles
DestructiveGrey Box1354554Charles
PerformanceGrey Box19005n/aCharlesload test
PerformanceGrey Box19015n/aCharlesscalability test
PerformanceGrey Box19025n/aCharlesendurance test
A/B Alternate Routen/a

Acceptance Test

After alpha tests pass, Emma starts her bata testing. Full acceptance testing begins after acceptance smoke testing is finished.

Planing Matrix Example - Acceptance Test

TypeMethodTest Case IDTest Case Risk ScoreTests Use Care IDAssign to
Smoke or SanityBlack Box774588Jason
Smoke or SanityBlack Box775589Jason
Smoke or SanityBlack Box776590Jason
Smoke or SanityBlack Box777491Jason
Smoke or SanityBlack Box778492Jason
Smoke or SanityBlack Box779593Jason
RegressionBlack Box9034547Brooke
RegressionBlack Box9045548Brooke
RegressionBlack Box9055549Brooke
RegressionBlack Box9064550Brooke
RegressionBlack Box9074551Brooke
RegressionBlack Box9085552Brooke
RegressionBlack Box9093553Brooke
AcceptanceBlack Box8735743Hannah
AcceptanceBlack Box8745744Hannah
AcceptanceBlack Box8755745Hannah
AcceptanceBlack Box8765746Hannah
AcceptanceBlack Box8775747Hannah
AcceptanceBlack Box8785748Hannah
AcceptanceBlack Box8795749Hannah
AcceptanceBlack Box8905750Hannah
AcceptanceBlack Box8915751Emma
AcceptanceBlack Box8925752Emma
AcceptanceBlack Box8935753Emma
AcceptanceBlack Box8945754Emma
AcceptanceBlack Box8955755Emma
BetaBlack Box5274529Emma
BetaBlack Box5285530Emma
BetaBlack Box5295531Emma
BetaBlack Box5305532Emma
BetaBlack Box5315533Emma
DestructiveBlack Box701452Alice
DestructiveBlack Box702553Alice
DestructiveBlack Box703454Alice
DestructiveBlack Box704555Alice
DestructiveBlack Box705456Alice
DestructiveBlack Box706557Alice
DestructiveBlack Box707458Alice
DestructiveBlack Box708459Alice
DestructiveBlack Box709460Alice
DestructiveBlack Box710361Alice
DestructiveBlack Box711562Alice
UsabilityBlack Box6934442Frank
UsabilityBlack Box6943443Frank
UsabilityBlack Box6954444Frank
UsabilityBlack Box6964445Frank
UsabilityBlack Box6973446Frank
AccessibilityBlack Box2094304Laura
AccessibilityBlack Box2103305Laura
AccessibilityBlack Box2113306Laura
InternationalizationBlack Box7775444Jennifer
InternationalizationBlack Box7784445Jennifer
InternationalizationBlack Box7793446Jennifer
InternationalizationBlack Box7803447Jennifer
InternationalizationBlack Box7813448Jennifer
A/B Alternate RouteBlack Box9025389Jodie
A/B Alternate RouteBlack Box9034390Jodie
A/B Alternate RouteBlack Box9043390Jodie
A/B Alternate RouteBlack Box9055392Jodie
A/B Alternate RouteBlack Box9064393Jodie

Click her for Part Five – Critical Numbers (KPI)

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.