Using acceptance tests to predict merge conflict risk

Abstract:

The occurrence of merge conflicts might impair software development. Developers might reduce conflicts by avoiding to perform tasks that change the same files in parallel. As predicting the set of files changed by a task is hard, we use the TAITI tool to compute task interfaces (an approximation of the changed files set). Such a tool takes as input a set of automated acceptance tests that validate the expected behavior of each task, assuming a BDD (Behaviour-Driven Development) context, in which tests guide code development. We measure the conflict risk between tasks as the intersection among its interfaces. To assess the predictions of conflict risk, we measure precision and recall of 6,360 task pairs from 19 Rails projects on GitHub. Our results confirm that Cucumber tests might help to predict the risk of merge conflicts, given the intersection among interfaces denotes a higher probability that the tasks change some file in common. A minimal intersection predicts conflict risk with 0.59 of precision and 0.98 of recall. Also, the higher the intersection size, the higher the number of files changed by both tasks. This way, developers might use the intersection size between TestI as a degree of conflict risk between tasks, prioritizing the selection of a task to work on whose TestI has the lowest intersection with others. Finally, a predictor of conflict risk based on TestI outperforms a predictor based on similar past tasks.  Regardless of promising results, sometimes the intersection among interfaces does not guess the conflicting files.