With the increasing use of models for software development and the emergence of model-driven engineering, it has become important to build accurate and precise models that present certain characteristics. Model transformation testing is a domain that requires generating a large number of models that satisfy coverage properties (cover the code of the transformation or the structure of the metamodel). However, manually building a set of models to test a transformation is a tedious task and having an automatic technique to generate models from a metamodel would be very helpful. We investigate the synthesis of models based on plans. Each plan comprises of a sequence of model synthesis rules (or mutation operators) specified as graph grammar (GG) rules. These mutation operators are primitive GG rules , automatically obtained from any meta-model. Such plans can be evolved by various artificial intelligence techniques to generate useful models for different tasks including model transformation testing.

In current hardware design flow, functional verification is widely acknowledged as the crucial step. This paper presents a new contribution to reduce the cost of this step by automating it. We address here, one of the principal challenges of dynamic verification, by providing a new approach for automatic test generation. This approach combines mutation-based test techniques and genetic algorithms to produce stimuli for design under test. The feasibility of the proposed approach is assessed with a preliminary implementation, and some framework has been tested

The current version of Java (J2SE 5.0) provides a high level of support for concurreny in comparison to previous versions. For example, programmers using J2SE 5.0 can now achieve synchronization between concurrent threads using explicit locks, semaphores, barriers, latches, or exchangers. Furthermore, built-in concurrent data structures such as hash maps and queues, built-in thread pools, and atomic variables are all at the programmer's disposal. We are interested in using mutation analysis to evaluate, compare and improve quality assurance techniques for concurrent Java programs. Furthermore, we believe that the current set of method mutation operators and class operators proposed in the literature are insufficient to evaluate concurrent Java source code because the majority of operators do not directly mutate the portions of code responsible for synchronization. In this paper we will provide an overview of concurrency constructs in J2SE 5.0 and a new set of concurrent mutation operators. We will justify the operators by categorizing them with an existing bug pattern taxonomy for concurrency. Most of the bug patterns in the taxonomy have been used to classify real bugs in a benchmark of concurrent Java applications.

Current mutation analysis tools are primarily used to compare different test suites and are tied to a particular programming language. In this paper we present the ExMAn experimental mutation analysis framework - ExMAn is automated, general and flexible and allows for the comparison of different quality assurance techniques such as testing, model checking, and static analysis. The goal of ExMAn is to allow for automatic mutation analysis that can be reproduced by other researchers. After describing ExMAn, we present a scenario of using ExMAn to compare testing with static analysis of temporal logic properties. We also provide both the benefits and the current limitations of using our framework.

The empirical assessment of test techniques plays an important role in software testing research. One common practice is to seed faults in subject software, either manually or by using a program that generates all possible mutants based on a set of mutation operators. The latter allows the systematic, repeatable seeding of large numbers of faults, thus facilitating the statistical analysis of fault detection effectiveness of test suites; however, we do not know whether empirical results obtained this way lead to valid, representative conclusions. Focusing on four common control and data flow criteria (block, decision, C-use, and P-use), this paper investigates this important issue based on a middle size industrial program with a comprehensive pool of test cases and known faults. Based on the data available thus far, the results are very consistent across the investigated criteria as they show that the use of mutation operators is yielding trustworthy results: generated mutants can be used to predict the detection effectiveness of real faults. Applying such a mutation analysis, we then investigate the relative cost and effectiveness of the above-mentioned criteria by revisiting fundamental questions regarding the relationships between fault detection, test suite size, and control/data flow coverage. Although such questions have been partially investigated in previous studies, we can use a large number of mutants, which helps decrease the impact of random variation in our analysis and allows us to use a different analysis approach. Our results are then; compared with published studies, plausible reasons for the differences are provided, and the research leads us to suggest a way to tune the mutation analysis process to possible differences in fault detection probabilities in a specific environment.

Testing interfaces of an embedded system is important since the heterogeneous layers such as hardware, OS and application are tightly coupled. We propose the mutation operators in three respects, "When?', "Where?' and "How?' in order to inject a fault into RTOS program when testing interface between RTOS and application. Injecting a fault without affecting the RTOS in run-time environment is the core of proposed mutation operators. We apply the mutation operators to interface testing during the integration of RTOS and application in the industrial programmable logic controller.

Available soon...

Available soon...

This paper attempts to subsume the existing great variety of mutation operations to two basic operations, insertion and omission and their combinations. These basic operations are applied to different elements of graph-based models of increasing representation power. A case study applies the approach to these models for generating mutants of different features and compares the fault detection capacity of the mutants generated.

We present a candidate fault model for pointcuts in AspectJ programs. The fault model identifies faults that we believe are likely to occur when writing pointcuts in the AspectJ language. Categories of fault types are identified, and each individual fault type is described as categorized. We argue that a fault model that focuses on the unique constructs of the AspectJ language is needed for the systematic and effective testing of AspectJ programs. Our pointcut fault model is a first step towards such a model.

Modelling tools play a crucial role in model-driven software development methods. A particular difficulty in testing such software systems is the generation of adequate test cases because the test data are structurally complicated. This paper proposes an approach called data mutation to generating a large number of test data from a few seed test cases. It is inspired in mutation testing methods, but differs from them in the way that mutation operators are defined and used. In our approach, mutation operators transform the input data rather than the program under test or the specification of the software. It is not a test adequacy measurement. Instead, it generates test cases. The paper also reports a case study with the method on testing a modelling tool and illustrates the applicability of the proposed method. Experiment data clearly demonstrated that the method can achieve a high test adequacy. It has a high fault detecting ability and good cost effectiveness.

The need for testing-for-diagnosis strategies has been identified for a long time, but the explicit link from testing to diagnosis (fault localization) is rare. Analyzing the type of information needed for efficient fault localization, we identify the attribute (called Dynamic Basic Block) that restricts the accuracy of a diagnosis algorithm. Based on this attribute, a test-for-diagnosis criterion is proposed and validated through rigorous case studies: it shows that a test suite can be improved to reach a high level of diagnosis accuracy. So, the dilemma between a reduced testing effort (with as few test cases as possible) and the diagnosis accuracy (that needs as much test cases as possible to get more information) is partly solved by selecting test cases that are dedicated to diagnosis.

Increase in the size and complexity of the software developed has made software testing a challenging exercise. A number of testing techniques are available but they differ in terms of statement coverage, condition coverage and particularly in fault detection capabilities. The size of the test suite also differs from one technique to other. Fault that has propagated into the system inadvertently, especially into the branch statements, have severe effects as they affect the logic of the program. In this paper, an experimental evaluation of the popular branch-testing techniques (Elmendorf's method, Boolean Operator (BOR), Modified Condition/Decision Coverage (MCDC), and Reinforced Criteria/Decision Coverage (RCDC)) is presented. These techniques are evaluated on the basis of types of faults they identify, size of the test suite and their effectiveness in fault detection. For experiments, various branch statements used and referred in literature are selected. Test cases and mutants were prepared for these branch statements. Mutants were prepared by seeding single operator and operand faults into the statements. The results indicate that for a subset of fault types BOR is effective. A variant of MCDC and RCDC demonstrate better performance on the full class of faults and are only slightly worse than Elmendorf's (CEG) method test suite.

The mutation testing focuses on the most possible mistakes of the software and so it has high ability to expose mistakes. However, at present it is still mainly used in procedure-oriented program testing. In recent years, object-oriented programming becomes more and more popular. This paper sets up the concepts for two new kinds of class level mutants for object-oriented program testing. One kind is of attribute mutants. The other is of method mutants. The formal description for the concepts is presented and the algorithms for generating attribute mutants and method mutants are proposed in this paper. A tool prototype for the algorithms is implemented and the related testing adequacy is also analyzed.

Available soon...

SESAME (Software Environment for Software Analysis by Mutation Effects) is a fault injection tool using mutation as the target fault model. It has been used for 15 years to support dependability research at LAAS-CNRS. A salient feature of SESAME is that it is multi-language. This made it possible to inject faults into software written in assembly languages, procedural languages (Pascal, C), a data-flow language (LUSTRE), as well as in a declarative language for temporal planning in robotics. This paper provides an overview of the tool, and reports on its use in experimental research addressing either fault removal or fault tolerance topics.

A set of mutation operators is "sufficient" if it can be used for most purposes to replace a larger set. We describe in detail an experimental procedure for determining a set of sufficient C language mutation operators. We also describe several statistical analyses that determine sufficient subsets with respect to several different criteria, based on standard techniques for variable reduction. We have begun to carry out our experimental procedure on seven standard C subject programs. We present preliminary results that indicate that the procedure and analyses are feasible and yield useful information.

This paper presents results from empirical studies of object-oriented, class level mutation operators, using the auto- mated analysis and testing tool MuJava. Class mutation operators modify OO programming language features such as inheritance, polymorphism, dynamic binding and encapsulation. This paper presents data from 866 classes in six open-source programs. Several new class-level mutation operators are defined in this paper and an analysis of the number of mutants generated is provided. Techniques for eliminating some equivalent mutants are described and data from an automated tool are provided. One important result is that class-level mutation operators yield far more equivalent mutants than traditional, statement-level, operators. Another is that there are far fewer class-level mutants than statement-level mutants. Together, these data suggest that mutation for inter-class testing can be practically affordable.

Temporal correctness is crucial for real-time systems. Few methods exist to test temporal correctness and most methods used in practice are ad-hoc. A problem with testing real-time applications is the response-time dependency on the execution order of concurrent tasks. Execution order in turn depends on execution environment properties such as scheduling protocols, use of mutual exclusive resources as well as the point in time when stimuli is injected. Model based mutation testing has previously been proposed to determine the execution orders that need to be verified to increase confidence in timeliness. An effective way to automatically generate such test cases for dynamic real-time systems is still needed. This paper presents a method using heuristic-driven simulation to generate test cases.

This paper presents an abstract view of mutation analysis. Mutation was originally thought of as making changes to program source, but similar kinds of changes have been applied to other artifacts, including program specifications, XML, and input languages. This paper argues that mutation analysis is actually a way to modify any software artifact based on its syntactic description, and is in the same family of test generation methods that create inputs from syntactic descriptions. The essential characteristic of mutation is that a syntactic description such as a grammar is used to create tests. We call this abstract view grammar-based testing, and view it as an interface, which mutation analysis implements. This shift in view allows mutation to be defined in a general way, yielding three benefits. First, it provides a simpler way to understand mutation. Second, it makes it easier to develop future applications of mutation analysis, such as finite state machines and use case collaboration diagrams. The third benefit, which due to space limitations is not explored in this paper, is ensuring that existing techniques are complete according to the criteria defined here.