Modelling tools play a crucial role in model-driven software development methods. A particular difficulty in testing such software systems is the generation of adequate test cases because the test data are structurally complicated. This paper proposes an approach called data mutation to generating a large number of test data from a few seed test cases. It is inspired in mutation testing methods, but differs from them in the way that mutation operators are defined and used. In our approach, mutation operators transform the input data rather than the program under test or the specification of the software. It is not a test adequacy measurement. Instead, it generates test cases. The paper also reports a case study with the method on testing a modelling tool and illustrates the applicability of the proposed method. Experiment data clearly demonstrated that the method can achieve a high test adequacy. It has a high fault detecting ability and good cost effectiveness.

The need for testing-for-diagnosis strategies has been identified for a long time, but the explicit link from testing to diagnosis (fault localization) is rare. Analyzing the type of information needed for efficient fault localization, we identify the attribute (called Dynamic Basic Block) that restricts the accuracy of a diagnosis algorithm. Based on this attribute, a test-for-diagnosis criterion is proposed and validated through rigorous case studies: it shows that a test suite can be improved to reach a high level of diagnosis accuracy. So, the dilemma between a reduced testing effort (with as few test cases as possible) and the diagnosis accuracy (that needs as much test cases as possible to get more information) is partly solved by selecting test cases that are dedicated to diagnosis.

Increase in the size and complexity of the software developed has made software testing a challenging exercise. A number of testing techniques are available but they differ in terms of statement coverage, condition coverage and particularly in fault detection capabilities. The size of the test suite also differs from one technique to other. Fault that has propagated into the system inadvertently, especially into the branch statements, have severe effects as they affect the logic of the program. In this paper, an experimental evaluation of the popular branch-testing techniques (Elmendorf's method, Boolean Operator (BOR), Modified Condition/Decision Coverage (MCDC), and Reinforced Criteria/Decision Coverage (RCDC)) is presented. These techniques are evaluated on the basis of types of faults they identify, size of the test suite and their effectiveness in fault detection. For experiments, various branch statements used and referred in literature are selected. Test cases and mutants were prepared for these branch statements. Mutants were prepared by seeding single operator and operand faults into the statements. The results indicate that for a subset of fault types BOR is effective. A variant of MCDC and RCDC demonstrate better performance on the full class of faults and are only slightly worse than Elmendorf's (CEG) method test suite.

The mutation testing focuses on the most possible mistakes of the software and so it has high ability to expose mistakes. However, at present it is still mainly used in procedure-oriented program testing. In recent years, object-oriented programming becomes more and more popular. This paper sets up the concepts for two new kinds of class level mutants for object-oriented program testing. One kind is of attribute mutants. The other is of method mutants. The formal description for the concepts is presented and the algorithms for generating attribute mutants and method mutants are proposed in this paper. A tool prototype for the algorithms is implemented and the related testing adequacy is also analyzed.

Available soon...

SESAME (Software Environment for Software Analysis by Mutation Effects) is a fault injection tool using mutation as the target fault model. It has been used for 15 years to support dependability research at LAAS-CNRS. A salient feature of SESAME is that it is multi-language. This made it possible to inject faults into software written in assembly languages, procedural languages (Pascal, C), a data-flow language (LUSTRE), as well as in a declarative language for temporal planning in robotics. This paper provides an overview of the tool, and reports on its use in experimental research addressing either fault removal or fault tolerance topics.

A set of mutation operators is "sufficient" if it can be used for most purposes to replace a larger set. We describe in detail an experimental procedure for determining a set of sufficient C language mutation operators. We also describe several statistical analyses that determine sufficient subsets with respect to several different criteria, based on standard techniques for variable reduction. We have begun to carry out our experimental procedure on seven standard C subject programs. We present preliminary results that indicate that the procedure and analyses are feasible and yield useful information.

This paper presents results from empirical studies of object-oriented, class level mutation operators, using the auto- mated analysis and testing tool MuJava. Class mutation operators modify OO programming language features such as inheritance, polymorphism, dynamic binding and encapsulation. This paper presents data from 866 classes in six open-source programs. Several new class-level mutation operators are defined in this paper and an analysis of the number of mutants generated is provided. Techniques for eliminating some equivalent mutants are described and data from an automated tool are provided. One important result is that class-level mutation operators yield far more equivalent mutants than traditional, statement-level, operators. Another is that there are far fewer class-level mutants than statement-level mutants. Together, these data suggest that mutation for inter-class testing can be practically affordable.

Temporal correctness is crucial for real-time systems. Few methods exist to test temporal correctness and most methods used in practice are ad-hoc. A problem with testing real-time applications is the response-time dependency on the execution order of concurrent tasks. Execution order in turn depends on execution environment properties such as scheduling protocols, use of mutual exclusive resources as well as the point in time when stimuli is injected. Model based mutation testing has previously been proposed to determine the execution orders that need to be verified to increase confidence in timeliness. An effective way to automatically generate such test cases for dynamic real-time systems is still needed. This paper presents a method using heuristic-driven simulation to generate test cases.

This paper presents an abstract view of mutation analysis. Mutation was originally thought of as making changes to program source, but similar kinds of changes have been applied to other artifacts, including program specifications, XML, and input languages. This paper argues that mutation analysis is actually a way to modify any software artifact based on its syntactic description, and is in the same family of test generation methods that create inputs from syntactic descriptions. The essential characteristic of mutation is that a syntactic description such as a grammar is used to create tests. We call this abstract view grammar-based testing, and view it as an interface, which mutation analysis implements. This shift in view allows mutation to be defined in a general way, yielding three benefits. First, it provides a simpler way to understand mutation. Second, it makes it easier to develop future applications of mutation analysis, such as finite state machines and use case collaboration diagrams. The third benefit, which due to space limitations is not explored in this paper, is ensuring that existing techniques are complete according to the criteria defined here.