We study conditions on $f$ under which an $f$-divergence $D_f$ will satisfy $D_f \geq c_f V^2$ or $D_f \geq c_{ 2,f} V^2 + c_{ 4,f} V^4$, where $V$ denotes variational distance and the coefficients $c_f$, $c_{ 2,f}$ and $c_{ 4,f}$ are { \em best possible}. As a consequence, we obtain lower bounds in terms of $V$ for many well known distance and divergence measures. For instance, let $D_{ (\alpha)} (P,Q) = [\alpha (\alpha-1)]^{ -1} [\int q^{ \alpha} p^{ 1-\alpha} d \mu -1]$ and ${ \cal I}_\alpha (P,Q) = (\alpha -1)^{ -1} \log [\int p^\alpha q^{ 1-\alpha} d \mu]$ be respectively the { \em relative information of type} ($1-\alpha$) and { \em R\'{ e}nyi's information gain of order} $\alpha$. We show that $D_{ (\alpha)} \geq { 1/2} V^2 + { 1/72} (\alpha+1)(2-\alpha) V^4$ whenever $-1 \leq \alpha \leq 2$, $\alpha \not= 0,1$ and that ${ \cal I}_{ \alpha} = \frac{ \alpha}{ 2} V^2 + { 1/36} \alpha (1 + 5 \alpha - 5 \alpha^2) V^4$ for $0 < \alpha < 1$. Pinsker's inequality $D \geq { 1/2} V^2$ and its extension $D \geq { 1/2} V^2 + { 1/36} V^4$ are special cases of each one of these.
Categories: cs.IT math.IT