A Balanced Approach to Software Testing

Avoid "new entanglements"

Avoid “new entanglements”

In his reconstruction speech given to Congress on April 11, 1865, Abraham Lincoln warned the nation that creating inflexible plans will, “…become a new entanglement”. The quality assurance manager can draw a number of lessons from the broader context  of the speech. Here is the complete thought.

…so new and unprecedented is the whole case, that no exclusive, and inflexible plan can safely be prescribed as to details and colatterals. Such exclusive, and inflexible plan, would surely become a new entanglement. Important principles may, and must, be inflexible.

In the present “situation” as the phrase goes, it may be my duty to make some new announcement to the people of the South. I am considering, and shall not fail to act, when satisfied that action will be proper.

Abraham Lincoln – April 11, 1865

I strongly recommend these thoughts from Abraham Lincoln be kept in mind as test plans are forged,

Unprecedented case

Each test cycle is a completely new experience. Software development is amazing because the combination of functionalities represents endless possibilities. Like introducing a child into a family, introducing a list of new features into an application causes change and sometimes shock. The QA team should have their list of standard test cases, but this list must grow. Using the standard set of cases alone will lead to insufficient testing. Don’t disappoint your end users with problems the that result from insufficient testing. Plan each test as if it were the first time the application was tested.

Important and sudden changes

As mentioned in an earlier essay, because of the issues which occur during development, the quality assurance cycle is usually cut short. Depending on the software project management process used by your company, the product that finally makes it to the quality assurance team may contain 25% more features than originally planned. Plan for this change as you design your schedule. Plan to quickly reprioritize and rearrange your milestones activities and deliverables. Also keep in mind the project management principle that if change is introduced into the schedule the “project balance” will be disrupted. This balance consists of scope, resources, due date and budget. Also keep in mind adding new team members slows down the project as detailed in this article from Wikipedia.

The quality assurance manager should always be abreast of the following:

  1. The priorities of the organization.
  2. The priorities of the development project.
  3. The priorities of each quality assurance test and how these support the development project.

A new entanglement – 

These changes come with a ”Yin and Yang” balance. First the Yin – Keeping these priorities constantly in mind helps when you must quickly shift tests around to support

development changes. Rigid adherence to your initial test plan will only serve to frustrate the development process. This will result in changes not being tested and blaming the development team when production bugs are discovered. Agreeing to development scope before the project begins is essential, but the reality of shifting priorities and ever tighter deadlines requires a flexible mindset.

The “Yang” is simple and includes two principles. The first is attentive project management. As priorities shift the project balance is disturbed. If more features must be tested, the more the quality assurance deadline must be extended. If the deadline cannot be extended, then lower priority features must be dropped from the release. If no feature may be dropped, your situation becomes more intense. If this is the case you have two additional options which will not save much time, if any. These are to add additional testing tools and or adding additional team members. Most project management guidelines and mathematical models demonstrate the net gain to these two steps is practically null due to “ramp-up time”. These final two options also cause the test cycle to go over budget.

Important principles may, and must, be inflexible –

If the test cycle cannot go over budget, cannot drop features and not extend the deadline then the “Yang” principle must be applied. As a very basic part of quality assurance management, an acceptable rate of production “bugs” must be established. This should be calculated in one of two ways; percentage or ratio of bugs to use cases. This is one of the quality assurance manager’s critical numbers and should be calculated at the same time after each release. Another critical number is the rate at which new bugs are reported after a release.

Make some new announcement to the people of the South – 

I recommend running the failure rate (percentage or ratio) at the average time that the rate of reported bugs begins to decline. Knowing the critical numbers gives the quality assurance manager a lot of power to push back. If the enterprise has decided the acceptable rate of production bugs is 10% and you are relatively sure, this will be higher due to changing priorities, stake holders must be informed. At this point the quality of the release becomes an enterprise management issue and not one of quality assurance management.

Abraham Lincoln was right to establish that Such exclusive, and inflexible plan, would surely become a new entanglement. However managing flexible principles does not dismiss responsibility either on the part of the quality assurance manager or the enterprise.

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.