Abstract:
We consider the problem of finding maximum values of $f$-divergences $D_f(P \parallel Q)$ of discrete probability distributions $P$ and $Q$ with values on a finite set under the condition that the variation distance $V(P, Q)$ between them and one of the distributions $P$ or $Q$ are given. We obtain exact expressions for such maxima of $f$-divergences, which in a number of cases allow to obtain both explicit formulas and upper bounds for them. As a consequence, we obtain explicit expressions for the maxima of $f$-divergences $D_f(P \parallel Q)$ given that, besides $V(P, Q)$, we only know the value of the maximum component of either $P$ or $Q$. Analogous results are also obtained for the Rényi divergence.
Keywords:$f$-divergence, Rényi divergence, variation distance, discrete probability distributions.