Vulnerabilities in applications and their widespread exploitation through successful attacks are common these days. Testing applications for preventing vulnerabilities is an important step to address this issue. In recent years, a number of security testing approaches have been proposed. However, there is no comparative study of these work that might help security practitioners select an appropriate approach for their needs. Moreover, there is no comparison with respect to automation capabilities of these approaches. In this work, we identify seven criteria to analyze program security testing work. These are vulnerability coverage, source of test cases, test generation method, level of testing, granularity of test cases, testing automation, and target applications. We compare and contrast prominent security testing approaches available in the literature based on these criteria. In particular, we focus on work that address four most common but dangerous vulnerabilities namely buffer overflow, SQL injection, format string bug, and cross site scripting. Moreover, we investigate automation features available in these work across a security testing process. We believe that our findings will provide practical information for security practitioners in choosing the most appropriate tools.

Buffer overflow (BOF) is one of the major vulnerabilities that leads to non-secure software.Testing an implementation for BOF vulnerabilities is challenging as the underlying reasons of buffer overflow vary widely. Moreover, the existing vulnerability testing approaches do not address the issue of generating adequate test data sets for testing BOF vulnerabilities. In this work, we apply the idea of mutation-based testing technique to generate adequate test data set for BOF vulnerabilities. Our work addresses those BOF vulnerabilities, which are related to an implementation language and its associated libraries. We apply the concept for ANSI Clanguage and its associated libraries. We propose 12 mutation operators to force the generation of adequate test data set for BOF vulnerabilities. The proposed operators are validated by using four open source programs. The results indicate that the proposed operators are effective for testing BOF vulnerabilities.

SQL injection is one of the most prominent vulnerabilities for web-based applications. Exploitation of SQL injection vulnerabilities (SQLIV) through successful attacks might result in severe consequences such as authentication bypassing, leaking of private information etc. Therefore, testing an application for SQLIV is an important step for ensuring its quality. However, it is challenging as the sources of SQLIV vary widely, which include the lack of effective input filters in applications, insecure coding by programmers, inappropriate usage of APIs for manipulating databases etc. Moreover, existing testing approaches do not address the issue of generating adequate test data sets that can detect SQLIV. In this work, we present a mutation-based testing approach for SQLIV testing. We propose nine mutation operators that inject SQLIV in application source code. The operators result in mutants, which can be killed only with test data containing SQL injection attacks. By this approach, we force the generation of an adequate test data set containing effective test cases capable of revealing SQLIV. We implement a MUtation-based SQL Injection vulnerabilities Checking (testing) tool (MUSIC) that automatically generates mutants for the applications written in Java Server Pages (JSP) and performs mutation analysis. We validate the proposed operators with five open source web-based applications written in JSP. We show that the proposed operators are effective for testing SQLIV.

Format string bugs (FSBs) make an implementation vulnerable to numerous types of malicious attacks. Testing an implementation against FSBs can avoid consequences due to exploits of FSBs such as denial of services, corruption of application states, etc. Obtaining an adequate test data set is essential for testing of FSBs. An adequate test data set contains effective test cases that can reveal FSBs. Unfortunately, traditional techniques do not address the issue of adequate testing of an application for FSB. Moreover, the application of source code mutation has not been applied for testing FSB. In this work, we apply the idea of mutation-based testing technique to generate an adequate test data set for testing FSBs. Our work addresses FSBs related to ANSI C libraries. We propose eight mutation operators to force the generation of adequate test dataset. A prototype mutation-based testing tool named MUFORMAT is developed to generate mutants automatically and perform mutation analysis. The proposed operators are validated by using four open source programs having FSBs. The results indicate that the proposed operators are effective for testing FSBs.