One of the main concerns in safety-critical software is to ensure sufficient reliability because proof of the absence of systematic failures has proved to be an unrealistic goal. fault-tolerance (FT) is one method for improving reliability claims. It is reasonable to assume that some software FT techniques offer more protection than others, but the relative effectiveness of different software FT schemes remains unclear. We present the principles of a method to assess the effectiveness of FT using mutation analysis. The aim of this approach is to observe the power of FT directly and use this empirical process to evolve more powerful forms of FT. We also investigate an approach to FT that integrates data diversity (DD) assertions and TA. This work is part of a longer term goal to use FT in quantitative safety arguments for safety critical systems.

The injection of faults has been widely used to evaluate fault tolerance mechanisms and to assess the impact of faults in computer systems. However, the injection of software faults is not as well understood as other classes of faults (e.g., hardware faults). In this paper, we analyze how software faults can be injected (emulated) in a source-code independent manner. We specifically address important emulation requirements such as fault representativeness and emulation accuracy. We start with the analysis of an extensive collection of real software faults. We observed that a large percentage of faults falls into well-defined classes and can be characterized in a very precise way, allowing accurate emulation of software faults through a small set of emulation operators. A new software fault injection technique (G-SWFIT) based on emulation operators derived from the field study is proposed. This technique consists of finding key programming structures at the machine code-level where high-level software faults can be emulated. The fault-emulation accuracy of this technique is shown. This work also includes a study on the key aspects that may impact the technique accuracy. The portability of the technique is also discussed and it is shown that a high degree of portability can be achieved.

Fault-based testing is a technique where testers anticipate errors in a system under test in order to assess or generate test cases. The idea is to have enough test cases capable of detecting these anticipated errors. This paper presents a theory and technique for generating fault-based test cases for concurrent systems. The novel idea is to generate test purposes from faults that have been injected into a model of the system under test. Such test purposes form a specification of a more detailed test case that can detect the injected fault. The theory is based on the notion of refinement. The technique is automated using the TGV test case generator and an equivalence checker of the CADP tools. A case study of testing web servers demonstrates the practicability of the approach.

This paper presents a novel approach to unit testing that lets users of deployed software assist in performing mutation testing of the software. Our technique, MUGAMMA, provisions a software system so that when it executes in the field, it will determine whether users' executions would have killed mutants (without actually executing the mutants), and if so, captures the state information about those executions. In the absence of bug reports, knowledge of executions that would have killed mutants provides additional confidence in the system over that gained by the testing performed before deployment. Captured information about the state before and after execution of units (e.g., methods) can be used to construct test cases for use in unit testing when changes are made to the software. The paper also describes our prototype MUGAMMA implementation along with a case study that demonstrates its potential efficacy.

Software testing is an important technique for assurance of software quality. Mutation testing has been identified as a powerful fault-based technique for unit testing, and there has been some research on automatic generation of test data for mutation testing. However, existing approaches to this kind of test data generation usually generate test data according to one mutant at one time. Thus, more test data that are needed for achieving a given mutation score. In this paper, we propose a new approach to generating one test data according to multiple mutants that are mutated at the same location at one time. Thus, our approach can generate smaller test suite that can achieve the same mutation testing score. To evaluate our approach, we implemented a prototype tool based on our approach and carried out some preliminary experiments. The experimental results show that our approach is more cost-effective.

Software testing is one of the most relevant activities regarding system development but, at the same time, it is a difficult topic to learn or teach without the appropriate supporting mechanisms. A learning process of software testing should involve the cooperation of theoretical and experimental knowledge with related tools in order to facilitate the apprenticeship of specific testing theories and skills. In previous works we investigated the establishment of supporting mechanisms for developing educational modules. The idea was contribute to the improvement of learning processes in general by producing quality educational products, capable of motivating the learners and effectively contribute to their knowledge construction process. In this paper we illustrate the application of our ideas in the context of software testing, more specifically by the development of an educational module for mutation testing. We consider mutation testing since it is a sound and powerful approach, but its use is still limited, mainly due to its high cost of application and, also, due to the need of an adequate learning process.

Available soon...

In MDE, model transformations should be efficiently tested so that it may be used and reused safely. Mutation analysis is an efficient technique to evaluate the quality of test data, and has been extensively studied both for procedural and object-oriented languages. In this paper, we study how it can be adapted to model oriented programming. Since no model transformation language has been widely accepted today, we propose generic fault models that are related to the model transformation process. First, we identify abstract operations that constitute this process: model navigation, model’s elements filtering, output model creation and input model modification. Then, we propose a set of specific mutation operators which are directly inspired from these operations. We believe that these operators are meaningful since a large part of the errors in a transformation are due to the manipulation of complex models regardless of the concrete implementation language.

Along with growing popularity of agile methodologies and open source movement, unit testing has become one of the core practices in modern software engineering. It is particularly important in eXtreme Programming [1], which explicitly diminish the importance of other artifacts than source code and tests cases. In XP unit test cases not only verify if software meets functional requirements, but also enable refactoring, alleviate comprehension and provide guidance on how the production code should be used. Therefore, they contribute to many other important practices of XP, which explicitly or implicitly rely on their ability to effectively discover bugs.

Available soon...