Kernel conditional tests from learning-theoretic bounds

neural information processing systems, pp. 55699–55751

Abstract

We propose a framework for hypothesis testing on conditional probability distributions, which we then use to construct statistical tests of functionals of conditional distributions. These tests identify the inputs where the functionals differ with high probability, and include tests of conditional moments or two-sample tests. Our key idea is to transform confidence bounds of a learning method into a test of conditional expectations. We instantiate this principle for kernel ridge regression (KRR) with subgaussian noise. An intermediate data embedding then enables more general tests -- including conditional two-sample tests -- via kernel mean embeddings of distributions. To have guarantees in this setting, we generalize existing pointwise-in-time or time-uniform confidence bounds for KRR to previously-inaccessible yet essential cases such as infinite-dimensional outputs with non-trace-class kernels. These bounds also circumvent the need for independent data, allowing for instance online sampling. To make our tests readily applicable in practice, we introduce bootstrapping schemes leveraging the parametric form of testing thresholds identified in theory to avoid tuning inaccessible parameters. We illustrate the tests on examples, including one in process monitoring and comparison of dynamical systems. Overall, our results establish a comprehensive foundation for conditional testing on functionals, from theoretical guarantees to an algorithmic implementation, and advance the state of the art on confidence bounds for vector-valued least squares estimation.

Authors 5

1. Pierre-François Massiani Aachen DSME -

   RWTH Aachen University

   Affiliation as printed

   DSME - RWTH Aachen

2. Christian Fiedler

   Munich Center for Machine Learning · Technical University of Munich

   Affiliation as printed

   Technical University of Munich (TUM) and Munich Center for Machine Learning (MCML)

3. Lukas Haverbeck Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen

4. Friedrich Solowjow Aachen

   RWTH Aachen University

   Affiliation as printed

   Rheinisch Westfälische Technische Hochschule Aachen

5. Sebastian Trimpe Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University