SUM4SOFT: A Process Model to Build and Update Software Samples

The validity of research findings critically depends on the selection of software samples. However, Mining Software Repositories studies often rely on outdated datasets and lack systematic, transparent procedures for repository selection and dataset construction, which hinder their replicability and undermine their findings. In this article, we propose SUM4SOFT, a process model to provide a structured, transparent, and replicable approach for building and maintaining software samples in Mining Software Repositories studies. We designed SUM4SOFT following a four-step methodology based on established sampling protocols from the literature. We validated the model in two empirical studies: (i) a Mining Software Repositories study and (ii) a replication case study conducted with external researchers to test both its effectiveness and clarity. The mining study showed that datasets collected and updated with the model produced consistent distributions across key code metrics. Furthermore, the case study confirmed these findings and provided feedback that led to refinements in the model’s roles, tasks, and work products. Our validation studies demonstrated the effectiveness and practical utility of SUM4SOFT, although further studies in other contexts are needed to generalize its applicability.