Reverse Information Projections and Optimal E-Statistics

IEEE Transactions on Information Theory, vol. 70, pp. 7616–7631

Abstract

Information projections have found important applications in probability theory, statistics, and related areas. In the field of hypothesis testing in particular, the reverse information projection (RIPr) has recently been shown to lead to growth-rate optimal (GRO) e-statistics for testing simple alternatives against composite null hypotheses. However, the RIPr as well as the GRO criterion are undefined whenever the infimum information divergence between the null and alternative is infinite. We show that in such scenarios, under some assumptions, there still exists a measure in the null that is closest to the alternative in a specific sense. Whenever the information divergence is finite, this measure coincides with the usual RIPr. It therefore gives a natural extension of the RIPr to certain cases where the latter was previously not defined. This extended notion of the RIPr is shown to lead to optimal e-statistics in a sense that is a novel, but natural, extension of the GRO criterion. We also give conditions under which the (extension of the) RIPr is a strict sub-probability measure, as well as conditions under which an approximation of the RIPr leads to approximate e-statistics. For this case we provide tight relations between the corresponding approximation rates.

Authors 3

1. Tyron Lardy Aachen

   Leiden University · Centrum Wiskunde & Informatica

   Affiliation as printed

   CWI, Amsterdam, XG, The Netherlands

   CWI, Amsterdam, and Leiden University, The Netherlands

2. Peter Grünwald Aachen

   Leiden University · Centrum Wiskunde & Informatica

   Affiliation as printed

   CWI, Amsterdam, XG, The Netherlands

   CWI, Amsterdam, and Leiden University, The Netherlands

3. Peter Harremoës

   Centrum Wiskunde & Informatica

   Affiliation as printed

   CWI, Amsterdam, XG, The Netherlands

   Copenhagen Business College, Copenhagen, Denmark