Available soon...

Available soon...

Available soon...

Available soon...

There is a perceived lack of mutation tools for the C language and information about the tools that are available was not widely disseminated until recently. This has lead to the development of a small set of programs to support the use of mutation on C language programs as part of a research effort into the use of automatically generated test sets. This paper describes a lightweight C language mutation tool and some initial results from its application to sorting algorithms. The paper describes how kill rates could be improved using a different encoding for Boolean values and details how the use of CPU time can be used as an additional means of detecting mutants.

Available soon...

This article presents two tools to generate test cases, one for Java programs and one for .NET programs, as well as a structured testing process whereby such tools can be used in order to help in process automation. The main innovation in this work is the joint use of diverse techniques and technologies, which have been separately applied to test automation: reflection to extract the class structure; regular expressions to describe test templates and test cases; JUnit and NUnit as test case execution frameworks; mutation and MuJava as test case quality measurers; serialization to deal with the parameters of complex data types; and once more, reflection, to facilitate the test engineer in the task of writing the oracle. Finally, the article presents an experiment carried out to validate the work. Copyright © 2006 John Wiley & Sons, Ltd.

In this paper, we study how mutation analysis can be adapted to qualify test cases aiming at testing a security policy. The objective is to make test cases efficient to reveal erroneous implementations of a security policy. The notion of security policy testing is studied and mutation operators are defined in relation with the security rules. To make the approach applicable in practice we discuss and empirically rank the security mutation operators from the most to the least difficult to kill. The empirical study is a library software, which is implemented with a typical 3-tier architecture.

To increase confidence in the correctness of specified policies, policy developers can conduct policy testing by supplying typical test inputs (requests) and subsequently checking test outputs (responses) against expected ones. Unfortunately, manual testing is tedious and few tools exist for automated testing of access control policies. We present a fault model for access control policies and a framework to explore it. The framework includes mutation operators used to implement the fault model, mutant generation, equivalent-mutant detection, and mutant-killing determination. This framework allows us to investigate our fault model, evaluate coverage criteria for test generation and selection, and determine a relationship between structural coverage and fault-detection effectiveness. We have implemented the framework and applied it to various policies written in XACML. Our experimental results offer valuable insights into choosing mutation operators in mutation testing and choosing coverage criteria in test generation and selection.

Available soon...

While important efforts are dedicated to system functional testing, very few works study how to test specifically security mechanisms, implementing a security policy. This paper introduces security policy testing as a specific target for testing. We propose two strategies for producing security policy test cases, depending if they are built in complement of existing functional test cases or independently from them. Indeed, any security policy is strongly connected to system functionality: testing functions includes exercising many security mechanisms. However, testing functionality does not intend at putting to the test security aspects. We thus propose test selection criteria to produce tests from a security policy. To quantify the effectiveness of a set of test cases to detect security policy flaws, we adapt mutation analysis and define security policy mutation operators. A library case study, a 3-tiers architecture, is used to obtain experimental trends. Results confirm that security must become a specific target of testing to reach a satisfying level of confidence in security mechanisms.

In model-based development of embedded real-time systems, statecharts are widely used for formal specification of their behavior, providing a sound basis for test generation and for detection of faults early in the development process. The paper presents a variety of new mutations for adequacy evaluation of tests used in validation of statecharts-based models. These mutations focus on key features of statecharts used in modeling of embedded systems: hierarchy, orthogonality and time expressions. We distinguish between two levels of tests adequacy. In the first, test results are expected to strictly follow the statecharts semantics. The second one takes into account possible deviations from this semantics based on typical implementation oriented decisions, e.g. mapping of orthogonal components into separate tasks executed concurrently. The considered mutations address both types of adequacy. In particular, we consider interleaving enforcing mutations, for testing of alternatives to the canonical "maximum parallelism" execution of statecharts.

This paper is devoted to advanced mutation operators for C# source code. They deal with object-oriented (OO mutations) and other complex features of the code. They require structural information about a code, unlike the standard mutations. Applicability of OO operators in C# is compared with those for other OO languages. Operators for specific features of C# language are also proposed. The detailed specification of operators can be provided in terms of pre- and post-conditions of a program transformation. Based on the operators’ specification, the generation of mutated C# programs can be automated.

Available soon...

Mutation is the practice of automatically generating possibly faulty variants of a program, for the purpose of assessing the adequacy of a test suite or comparing testing techniques. The cost of mutation often makes its application infeasible. The cost of mutation is usually assessed in terms of the number of mutants, and consequently the number of "mutation operators" that produce them. We address this problem by finding a smaller subset of mutation operators, called "sufficient", that can model the behaviour of the full set. To do this, we provide an experimental procedure and adapt statistical techniques proposed for variable reduction, model selection and nonlinear regression. Our preliminary results reveal interesting information about mutation operators.

Available soon...

Mutation testing traditionally involves mutating a program in order to produce a set of mutants and using these mutants in order to either estimate the effectiveness of a test suite or to drive test generation. Recently, however, this approach has been applied to specifications such as those written as finite state machines. This paper extends mutation testing to finite state machine models in which transitions have associated probabilities. The paper describes several ways of mutating a probabilistic finite state machine (PFSM) and shows how test sequences that distinguish between a PFSM and its mutants can be generated. Testing then involves applying each test sequence multiple times, observing the resultant output sequences and using results from statistical sampling theory in order to compare the observed frequency of each output sequence with that expected.

A set of mutation operators for SQL queries that retrieve information from a database is developed and tested against a set of queries drawn from the NIST SQL Conformance Test Suite. The mutation operators cover a wide spectrum of SQL features, including the handling of null values. Additional experiments are performed to explore whether the cost of executing mutants can be reduced using selective mutation or the test suite size can be reduced by using an appropriate ordering of the mutants. The SQL mutation approach can be helpful in assessing the adequacy of database test cases and their development, and as a tool for systematically injecting faults in order to compare different database testing techniques.

Available soon...

Software simulation remains the most used method for VHDL RTL functional verification. The functional verification process essentially consists of two parts. The first one is the functional qualification; the second one is the qualification- driven stimuli generation. Currently, the qualification and the generation tasks are iterative processes based on VHDL simulation which is dramatically time consuming. The simulation time increases with the circuits’ size and the required level of quality. In our previous works, we have proposed some approaches based on the mutation testing technique to evaluate and to improve functional validation quality. Now, to reduce this simulation time, we propose in this paper a new approach based on FPGA emulation. So, an hardware-software platform called “Meta-Mutant Testbench” is used to emulate mutants. Experimental results for some ITC’99 benchmark circuits show that our mutation emulator is about 20 times faster than classical software simulators; this speedup increases with the circuits’ size.