Defectors: A Large Scale Python Dataset for Defect Prediction

Defect prediction has been a major research problem in the software engineering domain for the last five decades.<br> In recent years, large deep-learning models have shifted the performance of software engineering tasks to new limits and are gaining usage in defect prediction.<br> However, these defect prediction models are often limited by the quality of their datasets, which are not large or diverse enough.<br> In this paper, we present Defectors, a large dataset for both line-level and just-in-time defect prediction.<br> Defectors consist of $\approx$ 213K source code files ($\approx$ 93K defective and $\approx$ 120K defect-free files) from 25 popular python projects from various domains and organizations.<br> These projects come from a diverse set of domains including machine learning, automation, and internet-of-things.<br> Such a scale and diversity make Defectors a suitable dataset for deep learning models, especially transformer models that require large and diverse datasets to effectively generalize defect-inducing patterns to predict future defects. Dataset Description File Name Description defectors.zip The original Dataset. Find its description in Section II of the paper. bug_inducing_commits.zip Each yaml file contains a map of bugfix commits to bug-inducing commits. filtered_bug_inducing_commits.yaml A map in structure {repo_name: {bug_inducing_commits: [list of python files in the commit]}}. This file only contains the bug-inducing commits that match the filtering criteria from Section III.D. repo_links.yaml Links to the repositories we used to construct the dataset.