Available soon...

Available soon...

Recently parameterized unit testing has emerged as a promising and effective methodology to allow the separation of (1) specifying external, black-box behavior (e.g., assumptions and assertions) by developers and (2) generating and selecting internal, white-box test inputs (i.e., high-code-covering test inputs) by tools. A parameterized unit test (PUT) is simply a test method that takes parameters, specifies assumptions on the parameters, calls the code under test, and specifies assertions. The test effectiveness of PUTs highly depends on the way that they are written by developers. For example, if stronger assumptions are specified, only a smaller scope of test inputs than intended are generated by tools, leading to false negatives in terms of fault detection. If weaker assertions are specified, erroneous states induced by the test execution do not necessarily cause assertion violations, leading to false negatives. Detecting these false negatives is challenging since the insufficiently written PUTs would just pass. In this paper, we propose a novel mutation analysis approach for analyzing PUTs written by developers and identifying likely locations in PUTs for improvement. The proposed approach is a first step towards helping developers write better PUTs in practice.

Conformance testing procedures for generating tests from the finite state model representation of Role-Based Access Control (RBAC) policies are proposed and evaluated. A test suite generated using one of these procedures has excellent fault detection ability but is astronomically large. Two approaches to reduce the size of the generated test suite were investigated. One is based on a set of six heuristics and the other directly generates a test suite from the finite state model using random selection of paths in the policy model. Empirical studies revealed that the second approach to test suite generation, combined with one or more heuristics, is most effective in the detection of both first-order mutation and malicious faults and generates a significantly smaller test suite than the one generated directly from the finite state models.

We present a model-based approach to testing access control requirements. By using combinatorial testing, we first automatically generate test cases from and without access control policies—i.e., the model—and assess the effectiveness of the test suites by means of mutation testing. We also compare them to purely random tests. For some of the investigated strategies, non-random tests kill considerably more mutants thanthe same number of random tests. Since we rely on policies only, no information on the application is required at this stage. As a consequence, our methodology applies to arbitrary implementations of the policy decision points.

Data from projects worldwide show that many software projects fail and most are completed late or over budget. Unit testing is a simple but effective technique to improve software in terms of quality, flexibility, and time-to-market. A key idea of unit testing is that each piece of code needs its own tests and the best person to design those tests is the developer who wrote the software. However, generating tests for each unit by hand is very expensive, possibly prohibitively so. Automatic test data generation is essential to support unit testing and as unit testing is achieving more attention, developers have a greater need for automated unit test data generation tools. However, developers have very little information about which tools are effective. This experiment compared three well-known public-accessible unit test data generation tools, JCrasher, TestGen4j, and JUB. We applied them to Java classes and evaluated them based on their mutation scores. As a comparison, we created two additional sets of tests for each class. One test set contained random values and the other contained values to satisfy edge coverage. Results showed that the automatic test data generation tools generated tests with almost the same mutation scores as the random tests.

Available soon...

Format string bugs (FSBs) make an implementation vulnerable to numerous types of malicious attacks. Testing an implementation against FSBs can avoid consequences due to exploits of FSBs such as denial of services, corruption of application states, etc. Obtaining an adequate test data set is essential for testing of FSBs. An adequate test data set contains effective test cases that can reveal FSBs. Unfortunately, traditional techniques do not address the issue of adequate testing of an application for FSB. Moreover, the application of source code mutation has not been applied for testing FSB. In this work, we apply the idea of mutation-based testing technique to generate an adequate test data set for testing FSBs. Our work addresses FSBs related to ANSI C libraries. We propose eight mutation operators to force the generation of adequate test dataset. A prototype mutation-based testing tool named MUFORMAT is developed to generate mutants automatically and perform mutation analysis. The proposed operators are validated by using four open source programs having FSBs. The results indicate that the proposed operators are effective for testing FSBs.

Buffer overflow (BOF) is one of the major vulnerabilities that leads to non-secure software.Testing an implementation for BOF vulnerabilities is challenging as the underlying reasons of buffer overflow vary widely. Moreover, the existing vulnerability testing approaches do not address the issue of generating adequate test data sets for testing BOF vulnerabilities. In this work, we apply the idea of mutation-based testing technique to generate adequate test data set for BOF vulnerabilities. Our work addresses those BOF vulnerabilities, which are related to an implementation language and its associated libraries. We apply the concept for ANSI Clanguage and its associated libraries. We propose 12 mutation operators to force the generation of adequate test data set for BOF vulnerabilities. The proposed operators are validated by using four open source programs. The results indicate that the proposed operators are effective for testing BOF vulnerabilities.

Mutation analysis is a widely-adopted strategy in software testing with two main purposes: measuring the quality of test suites, and identifying redundant code in programs. Similar approaches are applied in hardware verification and testing too, especially at RTL or gate level, where mutants are generally referred as faults, and mutation analysis is performed by means of fault modeling and fault simulation. However, in modern embedded systems there is a close integration between HW and SW parts, and verification strategies should be applied early in the design flow. This requires the definition of new mutation analysis-based strategies that work at system level, where HW and SW functionalities are not partitioned yet. In this context, the paper proposes a mutation model for perturbing transaction level modeling (TLM) SystemC descriptions. In particular, the main constructs provided by the SystemC TLM 2.0 library have been analyzed, and a set of mutants is proposed to perturb the primitives related to the TLM communication interfaces.