This paper presents an empirical study of the mutation techniques used by the tool PASTE. This tool allows the automation of our passive testing methodology for systems that present stochastic-time information. In our proposal, invariants express the fact that each time the implementation under test performs a given sequence of actions, then it must exhibit a behavior according to the probability distribution functions reflected in the invariant. We briefly review the theoretical framework of our methodology and the main features of our tool. Next, we present in detail the Mutants module that provides us with a functionality to test the effectiveness of invariants for detecting errors. Finally, we present a study of the results obtained from the performed experiments.

Available soon...

Available soon...

If a mutation is not killed by a test suite, this usuallymeans that the test suite is not adequate. However, itmay also be that the mutant keeps the program’s seman-tics unchanged—and thus cannot be detected by any test.We found such equivalent mutants to be surprisingly com-mon: In an experiment on the JAXEN XPATH query engine,8/20 = 40% of all mutations turned out to be equivalent.Worse, checking the equivalency took us 15 minutes for asingle mutation. Equivalent mutants thus make it impossi-ble to automatically assess test suites by means of mutationtesting. To identify equivalent mutants, we are currently investi-gating the impact of a mutation on the execution: the morea mutation alters the execution, the higher the chance of itbeing non-equivalent. First experiments assessing the im-pact on code coverage are promising.

Available soon...

Aspect-oriented programming introduces new challenges for software testing. Inparticular the pointcut descriptor (PCD) requires particular attention fromtesters. The PCD describes the set of joinpoints where the advices are woven.In this paper we present a tool, AjMutator, for the mutation analysis of PCDs.AjMutator implements several mutation operators that introduce faults in thePCDs to generate a set of mutants. AjMutator classifies the mutants accordingto the set of joinpoints they match compared to the set of joinpoints matchedby the initial PCD. An interesting result is that this automaticclassification can identify equivalent mutants for a particular class of PCDs.AjMutator can also run a set of test cases on the mutants to give a mutationscore. We have applied AjMutator on two systems to show that this tool issuitable for the mutation analysis of PCDs on large AspectJ systems.

The rise of Web Services and their WS-BPEL compositions in recent years makes necessary to pay special attention to testing in this context. Mutation testing is a white box testing technique that has been applied successfully to programs written in different languages. In order to apply it we need a set of mutation operators and a system for mutant generation. This paper introduces a set of mutation operators for the WS-BPEL 2.0 language and a framework, based in genetic algorithms, for automatic mutant generation without rendering all possible mutants. This framework can also detect potentially equivalent mutants.

Available soon...

This paper introduces a new paradigm for Mutation Testing, which we call Higher Order Mutation Testing (HOM Testing). Traditional Mutation Testing considers only first order mutants, created by the injection of a single fault. Often these first order mutants denote trivial faults that are easily killed. Higher order mutants are created by the insertion of two or more faults. The paper introduces the concept of a subsuming HOM; one that is harder to kill than the first order mutants from which it is constructed. By definition, subsuming HOMs denote subtle fault combinations. The paper reports the results of an empirical study of HOM Testing using ten programs, including several non trivial real–world subjects for which test suites are available.

This paper presents a theory of testing that integrates into Hoare and He’s Unifying Theory of Programming (UTP). We give test cases a denotational semantics by viewing them as specification predicates. This reformulation of test cases allows for relating test cases via refinement to specifications and programs. Having such a refinement order that integrates test cases, we develop a testing theory for fault-based testing. Fault-based testing uses test data designed to demonstrate the absence of a set of pre-specified faults. A well-known fault-based technique is mutation testing. In mutation testing, first, faults are injected into a program by altering (mutating) its source code. Then, test cases that can detect these errors are designed. The assumption is that other faults will be caught, too. In this paper, we apply the mutation technique to both, specifications and programs. Using our theory of testing, two new test case generation laws for detecting injected (anticipated) faults are presented: one is based on the semantic level of UTP design predicates, the other on the algebraic properties of a small programming language.