Title: A Synergistic Analysis Method for Explaining Failed Regression Tests

Speaker: Prof. Zijiang Yang (Western Michigan University, USA)

         https://www.cs.wmich.edu/~zijiang/ 

Time: 8nd May 2015, 10:00

Venue: Seminar Room (334), Level 3, Building 5, Institute of Software, CAS

Abstract:

Experience has shown that software updates often introduce new bugs. Therefore, it is good practice to conduct regression testing during software development, which determines whether new bugs have been introduced into the code with previously working functionality. However, detecting these failures is only the first step. The more challenging task is to identify the relevant code changes and explain why these changes lead to the failure.

We propose a new automated debugging method for regression testing based on a synergistic application of both dynamic and semantic analysis.  Our method takes a failure-inducing test input, a buggy program, and an earlier correct version of the same program, and computes a minimal set of code changes responsible for the failure, as well as explaining how the code changes lead to the failure. Although this problem has been the subject of intensive research in recent years, existing methods are rarely adopted by developers in practice since they do not produce sufficiently accurate fault explanations for real applications. Our new method is significantly faster and more accurate than existing methods for explaining failed regression tests in real applications, due to its synergistic analysis framework that iteratively applies both dynamic analysis and a constraint solver based semantic analysis to leverage their complementary strengths. We have implemented our new method in a software tool based on the LLVM compiler and the KLEE symbolic virtual machine.  Our experiments on Linux applications show that the new method is both efficient and effective in practice. A paper based on this work has been accepted at ICSE’15. This project is in collaboration with Qiuping Yi, Dr. Jian Liu and Dr. Chen Zhao at ISCAS.