Integration tests the stability of the system.
Integration testing is the most extensive level and the one which requires the largest portion of the test plan. At this point you should plan for a full regression test as well as functional testing to insure new features work as expected. This level of testing requires running all existing test cases from your script library and the new cases which have been created from new use cases. Few tests impact the quality of the release as much as integration testing. It goes without saying the quality assurance manager needs a robust automated system to run this level. There are numerous tools on the market. Check here for a 2011 list of best automation tools from a Tools Journal post.
More than anywhere the test case and test script is vital to the success at this level of testing. An automated tool is vitally important, but the effectiveness of the tool is directly impacted by the quality of the test cases. I have a copy of my standard test case here. This article also defines my approach to creating test cases.
It goes without saying managing and prioritizing the library of test cases and test scripts is vitally important. This library must be updated by adding every new script created for each new cycle. As indicated earlier, I recommend each script be assigned a risk score. The score is assigned in relationship to the risk the application is exposed to if the script is not run. One means not running the script represents very low risk and five very high risk. Any script with a risk score of four or five should be run for each smoke test. If there is “absolutely no time for testing at all”, then run scripts with a risk score of five. Scripts with a score of five through three should be run during alpha testing and for a full regression test scripts with a score of five through one must be run. I use these risk assessment values to track and report all my critical numbers.
In addition to good cases, the approach is important. In quality assurance circles there is some discussion on the “direction” of integration testing. Wikipedia likewise details on this discussion. I prefer mixing the so-called “Big Bang” and the “focused functional approaches”. The Big Bang tests the system as it would be employed by the end user; focusing on reproducing the load the users bring to the system. The focused functional test is where the quality assurance manager is able to sign-off that the application does what the new use cases require. Creating the test plan using this hybrid approach, tests the system as a whole and offers criteria by which the application may be certified as having the new functionality.
My previous essay on “Architecting the Environment” details my approach to system testing. The only addition to those thoughts is that system testing is not required to be part of each release. However, it should be scheduled on a regular basis such as quarterly or after the system undergoes a number of major enhancements or stands before an important business cycle such as “Open Enrollment” for medicare applications.
Click here for Part Three – Acceptance Testing