Adaptation using spatially distributed Gaussian Processes

arXiv (Cornell University)

Abstract

We consider the accuracy of an approximate posterior distribution in nonparametric regression problems by combining posterior distributions computed on subsets of the data defined by the locations of the independent variables. We show that this approximate posterior retains the rate of recovery of the full data posterior distribution, where the rate of recovery adapts to the smoothness of the true regression function. As particular examples we consider Gaussian process priors based on integrated Brownian motion and the Matérn kernel augmented with a prior on the length scale. Besides theoretical guarantees we present a numerical study of the methods both on synthetic and real world data. We also propose a new aggregation technique, which numerically outperforms previous approaches. Finally, we demonstrate empirically that spatially distributed methods can adapt to local regularities, potentially outperforming the original Gaussian process.

Authors 3

1. Botond Szabó

   Bocconi University

   Affiliation as printed

   Department of Decision Sciences and BIDSA , Bocconi University , Italy and

2. Amine Hadji Aachen Mathematical Institute

   Leiden University

   Affiliation as printed

   Mathematical Institute , Leiden University , The Netherlands

3. Aad van der Vaart

   Delft University of Technology

   Affiliation as printed

   Delft Institute of Applied Mathematics , DIAM , Delft University of Technology , The Netherlands