Pesticide Paradox: Variation Over Repetition
The increasing complexity of systems under test and short project life cycles make software testing a complex undertaking
. QA Testing is not merely the execution of test cases. Its about understanding the product and striving to enhance its quality, keeping the cost, resource and time ratio in mind. There are many reasons that deter us in shipping a high quality and reliable software on time. One of the reasons is the ineffective test suite suffering from pesticide paradox. When farmers use the same pesticide repeatedly, the insects develop immunity against them. Similarly, repeatable tests make the product code resistant to test code. The immediate effect is that the bugs are revealed with decreasing frequency. The point to highlight is that we need to update and maintain test suite to increase its coverage and bug detecting ability. Test creation is not a set it, forget it activity. Test cases that explore new branches in the application and have high probability of unveiling bugs should be regularly added to existing test suite.There are several reasons for the gradual degradation and decay of test suite. Some of the reasons are listed below:
Software under test is dynamically evolving. The changed code must have changed effect. Even a simple change could bring the whole house of cards crashing down. The point is that the test suite loses its effectiveness with time if new test cases are not written for added functionality or the tests for existing features are not modified, even if they are only slightly affected by new additions. Test suites lose effectiveness because of the designed assumptions about the test environment. Another reason is that test suites require intricate skills for operation and maintenance. Frequent team transitions make the task of updating and maintenance cumbersome, leaving test suite in an inconsistent state. Another reason could be attributed to the fact that exhaustive qa and testing is not possible. Even in simple applications, the number of possible inputs, possible outputs and the number of paths through the software is very large. In order to verify all possible scenarios and data combinations, unrealistic numbers of tests are required. Whatever structured approach we employ to design test cases (Black Box or White Box); our test suite will never be rich enough to explore all paths of the application. The cause for this ineptness is that sometimes tests are added to the suite with a short-term perspective. If not removed later, these tests become a liability because of the impractical scenarios they cover. Also, more and more projects are following agile methodology. The drawback is the lack of documentation which results in data loss and requirements that cannot be properly mapped to test cases. Another reason is redundant tests which make test suite bulky and add no value.
As the test cycles are usually time boxed, the need of the hour is effective test suite. Repetitive tests do more harm than good. Iteration like friction is likely to generate heat than progress. Some of the recommended solutions to prevent stagnation of test suite are as follows:
1. Test Suite Augmentation
Test suite augmentation techniques address the code elements that are affected by changes and are not covered by regression test suite. After the identification of code elements, the test cases to explore them are generated using appropriate algorithms, such as genetic or concolic.
2. Discontinue Ineffective Tests
Periodic cleanup of test suite is very important to do away with the tests that are no longer in sync with the project. Obsolete, broken and redundant tests need to be identified and discarded.
3. Model Based Testing
In MBT, a model of the system under test is created which is a true reflection of the systems behaviour. The model can be a decision table, FSM, Markov chain, etc., depending upon the applications characteristics. The test cases are generated dynamically (graph traversal algorithms employed) unlike the fixed set of tests in static automation. This approach incorporates changes easily and provides good coverage.
4. Synthesize Test Data
Efficient testing requires a comprehensive test data set consisting of both valid and invalid data. Many bugs in the application are data specific so relying solely on standard production data would be like putting all the eggs in one basket. Depending upon the test environment and test type, new data needs to be created and appended to existing test data. If test data undergoes modifications during
qa and testing
, then post testing a clear approach for repairing of altered test data needs to be implemented.
5. Usage of Metrics
Its been rightly said that we cannot improve what we cannot measure. By measuring the requirements covered by test case, a reasonable estimate of the fault detecting capability of test case can be made. Test suite effectiveness metrics help us in determining the test escape count, which in turn, helps in improving test design process. Metrics that determine the pre-ship fault density to post-ship fault density are of utmost importance in uncovering the effectiveness of test suite.
To summarize, I would say that the maintenance of test suite is of paramount importance in dealing with pesticide paradox. Just as flossing prevents dental decay, maintenance prevents test suite decay. The three properties of test suite: size, coverage and fault detecting ability need to be monitored for efficient utilization of organizational resources, shipping reliable high quality software on time and reaping benefits.
by: Knowledge Sharing Team Dogs For Sale 5 Keys To Better Understanding Your Dog Pvc Pet Beds For Dogs Weight Loss Program: How It Helps To Shape The Body Did You Know That Your Pet Can Also Have Allergy? Self Catering Holidays in the Bodrum Peninsula by:Teddy Lupain Learn To Alleviate Your Night Time Panic Attacks 5 Top Tips For Keeping Your Pet Safe During The Summer Heat Bedbugfleadog How does Clopidogrel Bisulfate (Generic Plavix) work? The Benefits Of Home Shopping Catalogues by:Charlotte Bleasdale The Spencer Institute's Online Mind Body Fitness Certification Program Provides Holistic Training Fo Signs You Are Choosing the Best Online Pet Store for your Pet Needs