Una propuesta para reportar replicaciones de experimentos en Ingeniería del Software

Abstract

In this dissertation, we report on our contribution to increase the benefit obtained through the replication of experiments, developing a proposal that facilitates the systematic specification of replication changes and their documentation in a homogeneous way. The importance of specifying replication changes is twofold; on the one hand, so that the author has a guide on how to report the replication. Some of the proposed changes will mitigate some limitations identified in the base experiment or are a consequence of adapting the experiment to the new environment and it is necessary to analyze their influence on the validity of the experiment. On the other hand, an external experimenter who carries out a replication needs to know the evolution of the original experiment within a family, as well as the type and reason for the changes. This facilitates the design of experiments without repeating failures already identified and adapting the experiment to the new environment successfully. Based on the information identified in the change description, a metamodel has been proposed which is displayed using a template that shows the data in a fixed form and serves as a guide to avoid the loss of relevant information. The template is completed with linguistic patterns that facilitate the writing of changes. In order to facilitate the definition of changes for experimenters, the CÆSAR tool based on the proposed template is presented. The changes are part of the information repository providing an overview of the family of experiments. To validate the template, it has been instantiated by means of a multi-case study that covers the areas of Software Engineering, Science and Automatic Experiments. The comparative use of the template has allowed us to establish differences between the areas due to the different terminology and concepts used when carrying out an experiment.