RUS  ENG
Full version
JOURNALS // Problemy Peredachi Informatsii // Archive

Probl. Peredachi Inf., 2020 Volume 56, Issue 1, Pages 3–14 (Mi ppi2307)

This article is cited in 5 papers

Information Theory

On the maximum values of $f$-divergence and Rényi divergence under a given variational distance

V. V. Prelov

Kharkevich Institute for Information Transmission Problems, Russian Academy of Sciences, Moscow, Russia

Abstract: We consider the problem of finding maximum values of $f$-divergences $D_f(P \parallel Q)$ of discrete probability distributions $P$ and $Q$ with values on a finite set under the condition that the variation distance $V(P, Q)$ between them and one of the distributions $P$ or $Q$ are given. We obtain exact expressions for such maxima of $f$-divergences, which in a number of cases allow to obtain both explicit formulas and upper bounds for them. As a consequence, we obtain explicit expressions for the maxima of $f$-divergences $D_f(P \parallel Q)$ given that, besides $V(P, Q)$, we only know the value of the maximum component of either $P$ or $Q$. Analogous results are also obtained for the Rényi divergence.

Keywords: $f$-divergence, Rényi divergence, variation distance, discrete probability distributions.

UDC: 621.391.1 : 519.72

Received: 28.01.2020
Revised: 28.01.2020
Accepted: 05.02.2020

DOI: 10.31857/S0555292320010015


 English version:
Problems of Information Transmission, 2020, 56:1, 1–12

Bibliographic databases:


© Steklov Math. Inst. of RAS, 2024