SESAME (Software Environment for Software Analysis by Mutation Effects) is a fault injection tool using mutation as the target fault model. It has been used for 15 years to support dependability research at LAAS-CNRS. A salient feature of SESAME is that it is multi-language. This made it possible to inject faults into software written in assembly languages, procedural languages (Pascal, C), a data-flow language (LUSTRE), as well as in a declarative language for temporal planning in robotics. This paper provides an overview of the tool, and reports on its use in experimental research addressing either fault removal or fault tolerance topics.

The paper addresses the issue of testing programs written in the synchronous data flow language LUSTRE. We define a mixed strategy which combines statistical testing and deterministic extremal values testing. Statistical testing involves exercising a program with random inputs, the input distribution and the number of test data being determined according to test criteria. Three criteria based on the finite state automaton produced by the LUSTRE compiler are studied, the feasibility of the method for designing test sets according to them being exemplified on a real-case study. Then, mutation analysis (here, specific to LUSTRE) is used to assess the efficiency of the test sets. The results allow us (i) to define the most cost-effective criterion for designing efficient statistical test sets of reasonable size, and (ii) to show the high fault revealing power of the corresponding mixed strategy, killing the whole set of 310 mutants involved in the experiments.

The fault revealing power of different test patterns derived from ten structural test criteria currently referred to in unit testing is investigated. Experiments performed on four programs that are pieces of a real-life software system from the nuclear field are reported. Three test input generation techniques are studied: (1) deterministic choice, (2) random selection based on an input probability distribution determined according to the adopted structural test criterion, and (3) random selection from a uniform distribution on the input domain. Mutation analysis is used to assess the test set efficiency with respect to error detection. The experimental results involve a total of 2914 mutants. They show that structural statistical testing, which exhibits the highest mutation scores, leaving alive only six from 2816 nonequivalent mutants within short testing times, is the most efficient. A regards unit testing of programs whose structure remains tractable, the experiments show the adequacy of a fault removal strategy combining statistical and deterministic test patterns