Available soon...

Available soon...

Available soon...

Available soon...

Amongst the various testing strategies, mutation testing has been empirically found to be the most effective in detecting faults. However, mutation often imposes unacceptable demands on computing and human resources due to the large number of mutants that need to be compiled and executed on one or more test cases. In addition, the tester needs to examine many mutants and analyze these for possible equivalence with the program under test. For these reasons, mutation is generally regarded by the practicing test engineer as too expensive to use. As one significant component of the cost of mutation is the execution of mutants against test cases, we believe that the cost can be reduced dramatically by reducing the number of mutants that need to be examined. We report the results from a case study designed to investigate two alternatives to reduce the cost of mutation. The alternatives considered are: (1) the randomly selected x'% mutation, and (2) the constrained mutation. We provide experimental data indicating that both alternatives lead to test sets that distinguish a significant number of all mutants and provide high all-uses coverage.

Fault-based testing attempts to show that particular faults cannot exist in software by using test sets that differentiate between the original program (hypothesized to be correct) and faulty alternate programs. The success of this approach depends on a number of assumptions, notably that programmers are competent insofar as they only commit relatively trivial faults, and that faults only couple infrequently. Fault coupling occurs when test sets are able to differentiate between the original program and faulty alternate programs when faults occur in isolation, but not when they occur in combination; it is a complicating factor in fault-based testing. Fault coupling is studied here within the context of finite bijective functions. A complete mathematical solution of the problem is possible in this simplified case; the results indicate that fault coupling does indeed occur infrequently, and are thus in agreement with the empirical results obtained by others in the field. One surprising result is that certain kinds of test set are able to avoid fault coupling altogether.

Available soon...

Mutation-based software testing, or \em mutation testing, is a powerful testing technique applied primarily at the unit software level. Central to mutation testing is the need to analyze a test set to determine a quality measure called the \em mutation adequacy score\,; this assessment process is called \em mutation analysis. Unfortunately, the conventional method of performing mutation analysis, which requires interpreting many slightly different versions of the same program, has significant problems. Automated mutation analysis systems based on the conventional interpretive method are slow, laborious to build, and usually unable to completely emulate the intended operational environment of the software being tested. This research presents a solution to these problems: the \underlineMutant \underlineSchema \underlineGeneration (MSG) method. Rather than mutating an intermediate form of the program that then must be interpreted, this new method describes how to encode all mutations into one source-level program, a ``metamutant''\@. This program is then compiled (once) with the same compiler used during development and is executed in the same operational environment at compiled-program speeds. Since mutation systems based on mutant schemata do not need to provide run-time semantics and environment, they are significantly less complex and easier to build than interpretive systems, as well as more portable. An approach to automatically generating metamutants using attribute grammars is also presented. An MSG-based prototype mutation analysis system, \tt TUMS, was designed and implemented to demonstrate the automated generation of metamutants and to allow empirical performance studies to be conducted. Benchmarks show \tt TUMS significantly faster than \tt Mothra, a conventional interpretive mutation analysis system, with speed-ups as high as an order-of-magnitude observed. Additional studies are reported that contrast the performance of \tt TUMS to a hypothetical ``ideal'' mutation analysis system. We conclude that high performance mutation analysis is possible through the \mboxcreation and instantiation of mutant schemata and that the MSG method described in this dissertation is a viable and desirable approach for building automated mutation analysis systems.

We report results from an experiment to compare the fault detection effectiveness of mutation, its variants and the all-uses data flow criteria. Adequate test sets were generated randomly, as opposed to by human testers as in some previous studies. We view our results in the light of those from earlier studies comparing mutation with path-oriented testing strategies. We identify and discuss factors that one might consider while evaluating an adequacy criterion for use in practice. Results from our experiments strengthen a hypothesis that an adequacy criterion based on one of the two variants of mutation has superior fault detection effectiveness than that of the all-uses criterion.

The paper addresses the issue of testing programs written in the synchronous data flow language LUSTRE. We define a mixed strategy which combines statistical testing and deterministic extremal values testing. Statistical testing involves exercising a program with random inputs, the input distribution and the number of test data being determined according to test criteria. Three criteria based on the finite state automaton produced by the LUSTRE compiler are studied, the feasibility of the method for designing test sets according to them being exemplified on a real-case study. Then, mutation analysis (here, specific to LUSTRE) is used to assess the efficiency of the test sets. The results allow us (i) to define the most cost-effective criterion for designing efficient statistical test sets of reasonable size, and (ii) to show the high fault revealing power of the corresponding mixed strategy, killing the whole set of 310 mutants involved in the experiments.