A human-centric approach for adopting bug inducing commit detection using machine learning models
Date
2022-01-31
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
ORCID
Type
Thesis
Degree Level
Masters
Abstract
When developing new software, testing can take up half of the resources. Although a considerable amount
of work has been done to automate software testing, fixing bugs after adding them to the source repository is
still a costly task from both management and financial perspectives. In recent times, the research community
has proposed various methodologies to detect bugs just-in-time at the commit level. Unfortunately, this
work, including state-of-the-art techniques, do not provide real-time solutions for the problem. Such a
limitation restricts developers from utilizing them in their day-to-day programming tasks. Our study focuses
on providing solutions that deliver real-time support to the developers by warning them about potential
bug-inducing commits. Such support can help developers by preventing them from adding a bug-inducing
commit to the source repository. Keeping this goal in mind, we conducted a developer survey to understand
the expectations of developers for bug-inducing commit detection tools. Motivated by their responses, we
built a GUI-based plug-in that warns the developers when they attempt to perform a potential buggy commit.
We accomplished this by training machine learning models on relevant features. We also built a command-line
tool for the developers who prefer to use a command-line interface. Our proposed solution has been designed
to work with various machine learning models (e.g. random forest, decision tree, and logistic regression) and
IDEs (e.g. Visual Studio, PyCharm, and WebStorm). It enables developers to work with a familiar interface
without leaving the IDE. As a proof of concept, we implemented a VSCode plug-in and an accompanying
command-line tool. Developers can customize these tools by choosing among various machine learning models
and features. Such customizability empowers the developers to understand the toolchain better and lets them
fit it into their specific use cases. Our user study shows that the toolchain offers satisfactory performance
in detecting bug-inducing commits and provides a sound user experience. The decision tree model achieved
the best performance with a 79% accuracy and an f1-score of 0.70 among the tested models. In addition,
we performed a user study with developers working in the software industries to validate the usability of
our toolchain. We found that the users can detect whether a commit is bug-inducing or not within a short
period of time. Furthermore, they prefer our tool over the state-of-the-art to detect potential bugs before
the commit operation. Alongside contributing a new multi-UI toolchain, our work enriches the research
community’s knowledge regarding developer usability of real-time bug detection tools.
Description
Keywords
Bug inducing commits, User interaction
Citation
Degree
Master of Science (M.Sc.)
Department
Computer Science
Program
Computer Science