Available soon...

Mutation testing has traditionally been used as a defect injection technique to assess the effectiveness of a test suite as represented by a "mutation score." Recently, mutation testing tools have become more efficient, and industrial usage of mutation analysis is experiencing growth. Mutation analysis entails adding or modifying test cases until the test suite is sufficient to detect as many mutants as possible and the mutation score is satisfactory. The augmented test suite resulting from mutation analysis may reveal latent faults and provides a stronger test suite to detect future errors which might be injected. Software engineers often look for guidance on how to augment their test suite using information provided by line and/or branch coverage tools. As the use of mutation analysis grows, software engineers will want to know how the emerging technique compares with and/or complements coverage analysis for guiding the augmentation of an automated test suite. Additionally, software engineers can benefit from an enhanced understanding of efficient mutation analysis techniques. To address these needs for additional information about mutation analysis, we conducted an empirical study of the use of mutation analysis on two open source projects. Our results indicate that a focused effort on increasing mutation score leads to a corresponding increase in line and branch coverage to the point that line coverage, branch coverage and mutation score reach a maximum but leave some types of code structures uncovered. Mutation analysis guides the creation of additional "common programmer error" tests beyond those written to increase line and branch coverage. We also found that 74% of our chosen set of mutation operators is useful, on average, for producing new tests. The remaining 26% of mutation operators did not produce new test cases because their mutants were immediately detected by the initial test suite, indirectly detected by test suites we added to detect other mutants, or were not able to be detected by any test.

Available soon...

Available soon...

Logic criteria are used in software testing to find inputs that guarantee detecting certain faults. Thus, satisfying a logic criterion guarantees killing certain mutants. Some logic criteria are composed of other criteria. Determining component criterion feasibility can be used as a means to reduce test set size without sacrificing fault detection. This paper introduces a new logic criterion based on component criterion feasibility. Given a predicate in minimal DNF, a determination is made of which component criteria are feasible for individual literals and terms. This in turn provides determination of which criteria are necessary to detect double faults and kill second-order mutants. A test set satisfying this new criterion guarantees detecting the same double faults as a larger test set satisfying another criterion. An empirical study using predicates in avionics software showed that tests sets satisfying the new criterion detected all but one double fault type. For this one double fault type, 99.91% of the double faults were detected and combining equivalent single faults nearly always yielded an equivalent double fault.

Database application programs are ubiquitous, so good techniques for testing them are needed. Recently, several research groups have proposed new approaches to generating tests for database applications and for assessing test data adequacy. This paper describes a mutation testing tool, JDAMA (Java Database Application Mutation Analyzer), for Java programs that interact with a database via the JDBC interface. Our approach extends the mutation testing approach for SQL by Tuya et al, by integrating it with analysis and instrumentation of the application bytecode. JDAMA's use is illustrated through a small study which uses mutation scores to compare two test generation techniques for database applications.

Mutation testing is a technique for generating high quality test data. However, logic mutation testing is currently inefficient for three reasons. One, the same mutant is generated more than once. Two, mutants are generated that are guaranteed to be killed by a test that kills some other generated mutant. Three, mutants that when killed are guaranteed to kill many other mutants are not generated as valuable mutation operators are missing. This paper improves logic mutation testing by 1) extending a logic fault hierarchy to include existing logic mutation operators, 2) introducing new logic mutation operators based on existing faults in the hierarchy, 3) introducing new logic mutation operators having no corresponding faults in the hierarchy and extending the hierarchy to include them, and 4) addressing the precise effects of equivalent mutants on the fault hierarchy. An empirical study using minimal DNF predicates in avionics software showed that a new logic mutation testing approach generates fewer mutants, detects more faults, and outperforms an existing logic criterion.

Agile development methods have gained momentum inthe last few years and, as a consequence, test-driven developmenthas become more prevalent in practice. However, test cases arenot sufficient for producing dependable software and we ratheradvocate approaches that emphasize the use of assertions orcontracts over that of test cases. Yet, writing self-checks in codehas been shown to be difficult and is itself prone to errors. Astandard technique to specify runtime properties is design-bycontract(DbC). But how can one test if the contracts themselvesare sensible and sufficient? We propose a measure to quantifythe goodness of contracts (or assertions in a broader sense). Weintroduce meta-mutations at the source code level to simulatecommon programmer errors that the self-checks are supposedto detect. We then use random mutation testing to determinea lower and upper bound on the detectable mutations andcompare these bounds with the number of mutants detected bythe contracts. Contracts are considered “good” if they detect acertain percentage of the detectable mutations.We have evaluatedour tools on Java classes with contracts specified using theJava Modeling Language (JML). We have additionally tested thecontract quality of 19 implementations, written independently bystudents, based on the same specification.

Available soon...

Available soon...