Continuing the series The Many Facets of Software Testing – A Hierarchical Approach, it is time to examine the second domain of a testing matrix – the integration level at which a test can be performed. Clearly, as the integration level increases, more pieces are requested to be in a functional state; and this will force the testing to be done later in the cycle.
Improving the speed that one can get feedback [by virtue of tests passing or failing] has many benefits; it is easier to take corrective action while the work is still fresh. Fewer people are impacted by any defects. The smaller the item under test, the easier it is to isolate the source of the problem; finding a logic error in 20-50 lines of code is much easier than figuring out what went wrong while looking at tens of thousands of lines of code.
Using Test Driven Development [TDD] or similar approaches provides a set of highly granular tests that are written concurrently with the code. A test is created which defines one thing the code should do, then the code is written to meet this goal. Then another test is written, followed by extending the code to also meet this goal. Typically these cycles are measured in minutes. This type of testing is referred to as “Unit Testing” because it is focused on testing the smallest “unit” of code that can possibly be tested. For object oriented languages [C#, VB.NET, C++] this typically equates to having tests for each non-private method. As units of code start to interact [e.g. methods in one class begin to invoke methods in a different class], additional testing can be added to validate that the these interactions function properly.
Up to this point the tests are being defined and written by the developers are they are writing the code. As the library of tests grows in size it provides protection against future work (even by the same person) having unintended effects on existing functionality. Execution of the tests occur while the developer is working, prior to the code being added to the source repository, and upon each update to the source repository.
As the process of integration continues additional tests can focus on the capabilities provided specifically by the integration, this reduces the size and scope of tests that are required at the full system level.
A growing number of organizations are following this process to the point where full end-to-end system testing has been eliminated. The complexity, fragility, and expense of these deep tests against a complete system no longer provide sufficient return on investment [ROI] to justify their usage. Of course, it is a significant journey to get to this point; yet, just knowing that successful organizations have achieved it can be a significant motivation towards starting the journey.