Classification Error Bound for Low Bayes Error Conditions in Machine Learning

IEEE International Conference on Acoustics Speech and Signal Processing, pp. 1–5

Abstract

In statistical classification and machine learning, classification error is an important performance measure, which is minimized by the Bayes decision rule. In practice, the unknown true distribution is usually replaced with a model distribution estimated from the training data in the Bayes decision rule. This substitution introduces a mismatch between the Bayes error and the model-based classification error. In this work, we apply classification error bounds to study the relationship between the error mismatch and the Kullback-Leibler divergence in machine learning. Motivated by recent observations of low model-based classification errors in many machine learning tasks, bounding the Bayes error to be lower, we propose a linear approximation of the classification error bound for low Bayes error conditions. Then, the bound for class priors are discussed. Moreover, we extend the classification error bound for sequences. Using automatic speech recognition as a representative example of machine learning applications, this work analytically discusses the correlations among different performance measures with extended bounds, including cross-entropy loss, language model perplexity, and word error rate.

Authors 4

1. Zijian Győző Yang Aachen Machine Learning and Human Language Technology Group Lehrstuhl Informatik 6 Computer Science Department

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Machine Learning and Human Language Technology Group, Lehrstuhl Informatik 6,Computer Science Department,Germany

2. Vahe Eminyan Aachen Machine Learning and Human Language Technology Group Lehrstuhl Informatik 6 Computer Science Department

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Machine Learning and Human Language Technology Group, Lehrstuhl Informatik 6,Computer Science Department,Germany

3. Ralf Schlüter Aachen Machine Learning and Human Language Technology Group Lehrstuhl Informatik 6 Computer Science Department

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Machine Learning and Human Language Technology Group, Lehrstuhl Informatik 6,Computer Science Department,Germany

4. Hermann Ney Aachen Machine Learning and Human Language Technology Group Lehrstuhl Informatik 6 Computer Science Department

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Machine Learning and Human Language Technology Group, Lehrstuhl Informatik 6,Computer Science Department,Germany