On selection and conditioning in multiple testing and selective inference

Biometrika, vol. 111, pp. 393–416

Abstract

Summary We investigate a class of methods for selective inference that condition on a selection event. Such methods follow a two-stage process. First, a data-driven collection of hypotheses is chosen from some large universe of hypotheses. Subsequently, inference takes place within this data-driven collection, conditioned on the information that was used for the selection. Examples of such methods include basic data splitting as well as modern data-carving methods and post-selection inference methods for lasso coefficients based on the polyhedral lemma. In this article, we take a holistic view of such methods, considering the selection, conditioning and final error control steps together as a single method. From this perspective, we demonstrate that multiple testing methods defined directly on the full universe of hypotheses are always at least as powerful as selective inference methods based on selection and conditioning. This result holds true even when the universe is potentially infinite and only implicitly defined, such as in the case of data splitting. We provide general theory and intuition before investigating in detail several case studies where a shift to a nonselective or unconditional perspective can yield a power gain.

Authors 2

1. Jelle J. Goeman corresponding Aachen Department of Biomedical Data Sciences

   Leiden University Medical Center

   Affiliation as printed

   Department of Biomedical Data Sciences, Leiden University Medical Center , Einthovenweg 20 , 2333 ZC Leiden, The Netherlands

2. Aldo Solari

   University of Milano-Bicocca · Ca' Foscari University of Venice

   Affiliation as printed

   Department of Economics, Management and Statistics, University of Milano-Bicocca , Piazza dell’Ateneo Nuovo 1 , 20126 Milano, Italy

   Ca' Foscari University of Venice, Department of Economics