Large‐scale simultaneous inference under dependence

Scandinavian Journal of Statistics, vol. 50, pp. 750–796

Abstract

Abstract Simultaneous inference allows for the exploration of data while deciding on criteria for proclaiming discoveries. It was recently proved that all admissible post hoc inference methods for the true discoveries must employ closed testing. In this paper, we investigate efficient closed testing with local tests of a special form: thresholding a function of sums of test scores for the individual hypotheses. Under this special design, we propose a new statistic that quantifies the cost of multiplicity adjustments, and we develop fast (mostly linear‐time) algorithms for post hoc inference. Paired with recent advances in global null tests based on generalized means, our work instantiates a series of simultaneous inference methods that can handle many dependence structures and signal compositions. We provide guidance on the method choices via theoretical investigation of the conservativeness and sensitivity for different local tests, as well as simulations that find analogous behavior for local tests and full closed testing.

Authors 5

1. Jinjin Tian corresponding

   Carnegie Mellon University

   Affiliation as printed

   Department of Statistics and Data Science, Department of Machine Learning Carnegie Mellon University Pittsburgh Pennsylvania USA

   Department of Statistics and Data Science, Department of Machine Learning, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA

2. Xu Chen Aachen Department of Biomedical Data Sciences Leiden University Medical Center Leiden Netherlands Department of Biomedical Data Sciences

   Leiden University Medical Center

   Affiliation as printed

   Department of Biomedical Data Sciences Leiden University Medical Center Leiden Netherlands

   Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, Netherlands

3. Eugene Katsevich

   University of Pennsylvania

   Affiliation as printed

   Department of Statistics and Data Science University of Pennsylvania Philadelphia Pennsylvania USA

   Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, Pennsylvania, USA

4. Jelle J. Goeman Aachen Department of Biomedical Data Sciences Leiden University Medical Center Leiden Netherlands Department of Biomedical Data Sciences

   Leiden University Medical Center

   Affiliation as printed

   Department of Biomedical Data Sciences Leiden University Medical Center Leiden Netherlands

   Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, Netherlands

5. Aaditya Ramdas

   Carnegie Mellon University

   Affiliation as printed

   Department of Statistics and Data Science, Department of Machine Learning Carnegie Mellon University Pittsburgh Pennsylvania USA

   Department of Statistics and Data Science, Department of Machine Learning, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA