Wednesday, 15 March 2006
 Sunday, 07 August 2005

Testing Levels

In previous versions of Visual Studio, a tester had to resort to many different tool vendors for his testing equipment. The release of Microsoft Visual Studio 2005 Team System will mark an important milestone in testing land since it marks the recognition of the tester as a first class citizen in Visual Studio. It will provide testers with tools that support testing throughout the entire software development and maintenance lifecycles. Does this mean that every tool a tester ever dreamt of or even really needs is in Team System? No, but it’s a great start. In this post I’ll focus on the test levels that are defined in the “V” model and the Visual Studio 2005 and Team System features that support them. The “V” model has become an industry wide standard for visualizing the levels of tests. Figure 1 is an illustration of the “V” model.



I regularly notice that there is a lot of confusion on the what, how and when wile discussing these levels. Before we can start to define the different testing levels it’s probably wise to define what a unit and a component is:

Unit:
A unit is the smallest compilable component. It does not include any communicating components and it’s generally created by one programmer.
Component:
A unit is a component. The integration of one or more components is a component. The reason for "one or more" as contrasted to "Two or more" is to allow for components that call themselves recursively.

On the right-leg of the “V” model you’ll find these levels:
Unit Testing:
During unit-testing the developer should always make sure the unit is tested in isolation, and that it is the only possible point of failure. In unit testing communicating components and called components should be replaced with stubs, simulators, or trusted components. Calling components should be replaced with drivers or trusted super-components.
Component Testing:
During component testing the same scenarios are tested as during unit testing but all stubs and simulators are replaced with the real thing.
Integration Testing:
Integration testing identifies problems that occur when components are combined. Component A and B are two components for which A calls B. Figure 2 illustrates integration testing for Components A and B:
Test Suite A contains the component level tests of component A.
Test Suite B consits the component level tests of component B.
Tab are the tests in A’s suite that cause A to call B.
Tsab are the tests in B's suite for which it would be possible to replace the code that is written to test component B by a call from Component A as input for the tests.
When you combine the test suites Tsab and Tab you will have a set of component tests that you can use after you modify Component B. When you modify Component B or A, you will be able to verify that will still function correctly together.


System Testing:
In system testing the tester will verify if the developed system or subsystems still meet the requirements that were set in the functional and quality specifications.
Acceptance Testing:
In acceptance testing the user and system manager will verify if the developed system still meets the requirements that were set in the functional and quality specifications. This level of testing is done in an environment that simulates the operational environment in the greatest possible extent.
Release Testing:
Prior to a public release of a program you must ensure that all bugs that were intended to be fixed were actually fixed. In release testing following aspects will be verified:

A mixture of previously failed-and-fixed tests and tests that have always passed;

Virus checking of the final installation package. Too many cases of distribution of viruses have been reported to not take this additional precaution.
A comparison of all features actually working reliably with prepared documentation. It is crucial that the documentation reflects all design decisions made during development and testing.

There are many variations on these definitions and the “V” model, but the key point is that the abovementioned testing levels are formally defined. A wise man once said: “Even when laws have been written down, they ought not always to remain unaltered.” Despite the fact that we are not talking about laws here, you can always leave a comment when you have another understanding of these definitions.

Team System | Testing
08/07/2005 15:43:44 UTC  #  Comments [1]