Available soon...

Available soon...

Available soon...

Available soon...

Recently parameterized unit testing has emerged as a promising and effective methodology to allow the separation of (1) specifying external, black-box behavior (e.g., assumptions and assertions) by developers and (2) generating and selecting internal, white-box test inputs (i.e., high-code-covering test inputs) by tools. A parameterized unit test (PUT) is simply a test method that takes parameters, specifies assumptions on the parameters, calls the code under test, and specifies assertions. The test effectiveness of PUTs highly depends on the way that they are written by developers. For example, if stronger assumptions are specified, only a smaller scope of test inputs than intended are generated by tools, leading to false negatives in terms of fault detection. If weaker assertions are specified, erroneous states induced by the test execution do not necessarily cause assertion violations, leading to false negatives. Detecting these false negatives is challenging since the insufficiently written PUTs would just pass. In this paper, we propose a novel mutation analysis approach for analyzing PUTs written by developers and identifying likely locations in PUTs for improvement. The proposed approach is a first step towards helping developers write better PUTs in practice.

Available soon...

Available soon...

Aspect-Oriented Programming (AOP) provides new modularization of software systems by encapsulating cross-cutting concerns. AspectJ, an AOP language, uses abstractions such as pointcuts, advice, and aspects to achieve AOP’s primary functionality. Faults in pointcuts can cause aspects to fail to satisfy their requirements. Hence, testing pointcuts is necessary in order to ensure correctness of aspects. In mutation testing of pointcuts (a type of fault-based pointcut testing), the number of mutants (i.e., variations) for pointcuts is usually large due to the usage of wildcards. It is tedious to manually identify effective mutants that are of appropriate strength and resemble closely the original pointcut expression, reflecting the kind of mistakes that developers may make. To reduce developers’ effort in this process, we have developed a new framework that automatically identifies the strength of each pointcut and generates pointcut mutants with different strengths. Developers can inspect the pointcut mutants and their join points for pointcut correctness or choose the mutants for conducting mutation testing. We conducted an empirical study on applying our framework on pointcuts from existing AspectJ programs. The results show that our framework can provide valuable assistance in generating effective mutants that are close to the original pointcuts and are of appropriate strength.

Firewalls are the mainstay of enterprise security and the most widely adopted technology for protecting private networks. As the quality of protection provided by a firewall directly depends on the quality of its policy (i.e., configuration), ensuring the correctness of security policies is important and yet difficult.To help ensure the correctness of a firewall policy, we propose a systematic structural testing approach for firewall policies. We define structural coverage (based on coverage criteria of rules, predicates, and clauses) on the policy under test. Considering achieving higher structural coverage effectively, we develop three automated packet generation techniques: the random packet generation, the one based on local constraint solving (considering individual rules locally in a policy), and the most sophisticated one based on global constraint solving (considering multiple rules globally in a policy).We have conducted an experiment on a set of real policies and a set of faulty policies to detect faults with generated packet sets. Generally, our experimental results show that a packet set with higher structural coverage has higher fault detection capability (i.e., detecting more injected faults). Our experimental results show that a reduced packet set (maintaining the same level of structural coverage with the corresponding original packet set) maintains similar fault detection capability with the original set.

Available soon...

Regression testing, which plays an important role in software maintenance, usually relies on test adequacy criteria to select and prioritize test cases. However, with the wide use and reuse of black-box components, such as reusable class libraries and COTS components, it is challenging to establish test adequacy criteria for testing software systems built on components whose source code is not available. Without source code or detailed documents, the misunderstanding between the system integrators and component providers has become a main factor of causing faults in component-based software. In this paper, we apply mutation on interface contracts, which can describe the rights and obligations between component users and providers, to simulate the faults that may occur in this way of software development. The mutation adequacy score for killing the mutants of interface contracts can serve as a test adequacy criterion. We performed an experimental study on three subject systems to evaluate the proposed approach together with four other existing criteria. The experimental results show that our adequacy criterion is helpful for both selecting good-quality test cases and scheduling test cases in an order of exposing faults quickly in regression testing of component-based software.

To increase confidence in the correctness of specified policies, policy developers can conduct policy testing by supplying typical test inputs (requests) and subsequently checking test outputs (responses) against expected ones. Unfortunately, manual testing is tedious and few tools exist for automated testing of access control policies. We present a fault model for access control policies and a framework to explore it. The framework includes mutation operators used to implement the fault model, mutant generation, equivalent-mutant detection, and mutant-killing determination. This framework allows us to investigate our fault model, evaluate coverage criteria for test generation and selection, and determine a relationship between structural coverage and fault-detection effectiveness. We have implemented the framework and applied it to various policies written in XACML. Our experimental results offer valuable insights into choosing mutation operators in mutation testing and choosing coverage criteria in test generation and selection.

Fault-based testing is an approach where the designed test data is used to demonstrate the absence of a set of prespecified faults, typically being frequently occurring faults. Mutation testing is a fault-based testing technique used to inject faults into an existing program, i.e., a variation of the original program and see if the test suite is sensitive enough to detect common faults. Aspect-Oriented Programming (AOP) provides new modularization of software systems by encapsulating crosscutting concerns. AspectJ, a language designed to support AOP uses abstractions like pointcuts, advice, and aspects to achieve AOP's primary functionality. Developers tend to write pointcut expressions with incorrect strength, thereby selecting additional events than intended to or leaving out necessary events. This incorrect strength causes aspects, the set of crosscutting concerns, to fail. Hence there is a need to test the pointcuts for their strength. Mutation testing of pointcuts includes two steps: creating effective mutants (variations) of a pointcut expression and testing these mutants using the designed test data. The number of mutants for a pointcut expression is usually large due to the usage of wildcards. It is tedious to manually identify effective mutants that are of appropriate strength and resemble closely the original pointcut expression. Our framework automatically generates mutants for a pointcut expression and identifies mutants that resemble closely the original expression. Then the developers could use the test data for the woven classes against these mutants to perform mutation testing.