Although powerful, mutation is a computationally very expensive testing technique. In fact, its three main stages (mutant generation, mutant execution and result analysis) require many resources to be successfully accomplished. Thus, researchers have made important efforts to reduce its costs. This paper represents an additional effort in this sense. It describes the results of two experiments in which, by means of combining the original set of mutants and therefore obtaining a new set of mutants - each one with two faults - the number of mutants used is reduced to half. Results lead to believe that mutant combination does not decrease the quality of the test suite, whereas it supposes important savings in mutant execution and result analysis.

We present a model-based approach to testing access control requirements. By using combinatorial testing, we first automatically generate test cases from and without access control policies—i.e., the model—and assess the effectiveness of the test suites by means of mutation testing. We also compare them to purely random tests. For some of the investigated strategies, non-random tests kill considerably more mutants thanthe same number of random tests. Since we rely on policies only, no information on the application is required at this stage. As a consequence, our methodology applies to arbitrary implementations of the policy decision points.

Available soon...

Mutation testing has been shown to be one of the strongest testing criteria for the evaluation of both programs and test suites. Comprehensive sets of mutants require strong test sets to achieve acceptable testing coverage. Moreover, mutation operators are valuable for the evaluation of other testing approaches. Although its importance has been highlighted for Aspect-Oriented (AO) programs, there is still a need for a suitable set of mutation operators for AO languages. The quality of the mutation testing itself relies on the quality of such operators. This paper presents the design of a set of mutation operators for AspectJ-based programs. These operators model instances of fault types identified in an extensive survey. The fault types and respective operators are grouped according to the related languages features. We also discuss the generalisation of the fault types to AO approaches other than AspectJ and the coverage that may be achieved with the application of the proposed operators. In addition, a cost analysis based on two case studies involving real-world applications has provided us feedback on the most expensive operators, which will support the definition of further testing strategies.

Mutants are automatically-generated, possibly faulty variants of programs. The mutation adequacy ratio of a test suite is the ratio of non-equivalent mutants it is able to identify to the total number of non-equivalent mutants. This ratio can be used as a measure of test effectiveness. However, it can be expensive to calculate, due to the large number of different mutation operators that have been proposed for generating the mutants. In this paper, we address the problem of finding a small set of mutation operators which is still sufficient for measuring test effectiveness. We do this by defining a statistical analysis procedure that allows us to identify such a set, together with an associated linear model that predicts mutation adequacy with high accuracy. We confirm the validity of our procedure through cross-validation and the application of other, alternative statistical analyses.

Available soon...

Available soon...

Available soon...

Available soon...

Aspect-Oriented Programming (AOP) provides new modularization of software systems by encapsulating cross-cutting concerns. AspectJ, an AOP language, uses abstractions such as pointcuts, advice, and aspects to achieve AOP’s primary functionality. Faults in pointcuts can cause aspects to fail to satisfy their requirements. Hence, testing pointcuts is necessary in order to ensure correctness of aspects. In mutation testing of pointcuts (a type of fault-based pointcut testing), the number of mutants (i.e., variations) for pointcuts is usually large due to the usage of wildcards. It is tedious to manually identify effective mutants that are of appropriate strength and resemble closely the original pointcut expression, reflecting the kind of mistakes that developers may make. To reduce developers’ effort in this process, we have developed a new framework that automatically identifies the strength of each pointcut and generates pointcut mutants with different strengths. Developers can inspect the pointcut mutants and their join points for pointcut correctness or choose the mutants for conducting mutation testing. We conducted an empirical study on applying our framework on pointcuts from existing AspectJ programs. The results show that our framework can provide valuable assistance in generating effective mutants that are close to the original pointcuts and are of appropriate strength.