Consistency tests for comparing astrophysical models and observations

Monthly Notices of the Royal Astronomical Society, vol. 524, pp. 1061–1074

Abstract

ABSTRACT In astronomy, there is an opportunity to enhance the practice of validating models through statistical techniques, specifically to account for measurement error uncertainties. While models are commonly used to describe observations, there are instances where there is a lack of agreement between the two. This can occur when models are derived from incomplete theories, when a better-fitting model is not available or when measurement uncertainties are not correctly considered. However, with the application of specific tests that assess the consistency between observations and astrophysical models in a model-independent way, it is possible to address this issue. The consistency tests (ConTESTs) developed in this paper use a combination of non-parametric methods and distance measures to obtain a test statistic that evaluates the closeness of the astrophysical model to the observations. To draw conclusions on the consistency hypothesis, a simulation-based methodology is performed. In particular, we built two tests for density models and two for regression models to be used depending on the case at hand and the power of the test needed. We used ConTEST to examine synthetic examples in order to determine the effectiveness of the tests and provide guidance on using them while building a model. We also applied ConTEST to various astronomy cases, identifying which models were consistent and, if not, identifying the probable causes of rejection.

Authors 3

1. Fiorenzo Stoppa corresponding

   Radboud University Nijmegen

   Affiliation as printed

   Department of Astrophysics/IMAPP, Radboud University , P.O. Box 9010, NL-6500 GL, Nijmegen, the Netherlands

   Department of Mathematics/IMAPP, Radboud University , P.O. Box 9010, Nijmegen, NL-6500 GNijmegen,L, the Netherlands

2. Eric Cator

   Radboud University Nijmegen

   Affiliation as printed

   Department of Mathematics/IMAPP, Radboud University , P.O. Box 9010, Nijmegen, NL-6500 GNijmegen,L, the Netherlands

3. Gijs Nelemans Aachen Netherlands Institute for Space Research

   KU Leuven · Radboud University Nijmegen · Space Research Organisation Netherlands

   Affiliation as printed

   Department of Astrophysics/IMAPP, Radboud University , P.O. Box 9010, NL-6500 GL, Nijmegen, the Netherlands

   Institute of Astronomy , KU Leuven , Celestijnenlaan 200D, B-3001 Leuven, Belgium

   SRON, Netherlands Institute for Space Research , Sorbonnelaan 2 , Utrecht, NL-3584 CA, the Netherlands