It's time! 10 + 1 reasons we should start replicating simulation studies

Abstract

The quantitative analysis of research data is a core element of empirical research. The performance of statistical methods that are used for analyzing empirical data can be evaluated and compared with computer simulations. A single simulation study can determine the analyses of thousands of empirical studies to follow. With great power comes great responsibility. Here, we argue that this responsibility includes replication of simulation studies to ensure a sound foundation for data analytical decisions. Furthermore, being designed, run and reported by humans, simulation studies face challenges similar to other experimental empirical research and hence should not be exempt from replication attempts. We highlight that the potential replicability of simulation studies is an opportunity and a luxury that quantitative methodology as a field should proudly embrace and use as a chance to lead by example.

Authors 3

1. Oscar Lorenzo Olvera Astivia

   University of Washington

   Affiliation as printed

   College of Education, University of Washington

2. Tim Peter Morris

   University College London · MRC Clinical Trials Unit at UCL

   Affiliation as printed

   MRC Clinical Trials Unit at UCL, Institute of Clinical Trials and Methodology, UCL

3. Rolf H. H. Groenwold