Statistical Learning Theory for Distributional Classification

Proceedings of the AAAI Conference on Artificial Intelligence, vol. 40, pp. 21120–21127

Abstract

In supervised learning with distributional inputs in the two-stage sampling setup, relevant to applications like learning-based medical screening or causal learning, the inputs (which are probability distributions) are not accessible in the learning phase, but only samples thereof. This problem is particularly amenable to kernel-based learning methods, where the distributions or samples are first embedded into a Hilbert space, often using kernel mean embeddings (KMEs), and then a standard kernel method like Support Vector Machines (SVMs) is applied, using a kernel defined on the embedding Hilbert space. In this work, we contribute to the theoretical analysis of this latter approach, with a particular focus on classification with distributional inputs using SVMs. We establish a new oracle inequality and derive consistency and learning rate results. Furthermore, for SVMs using the hinge loss and Gaussian kernels, we formulate a novel variant of an established noise assumption from the binary classification literature, under which we can establish learning rates. Finally, some of our technical tools like a new feature space for Gaussian kernels on Hilbert spaces are of independent interest.

Authors 1

1. Christian Fiedler corresponding Aachen

   RWTH Aachen University · Technical University of Munich · Munich Center for Machine Learning

   Affiliation as printed

   Technical University of Munich (TUM) Munich Center for Machine Learning (MCML) Institute for Data Science in Mechanical Engineering (DSME), RWTH Aachen University