Consider a channel ${ \bf Y}={ \bf X}+ { \bf N}$ where ${ \bf X}$ is an $n$-dimensional random vector, and ${ \bf N}$ is a Gaussian vector with a covariance matrix ${ \bf \mathsf{ K}}_{ \bf N}$. The object under consideration in this paper is the conditional mean of ${ \bf X}$ given ${ \bf Y}={ \bf y}$, that is ${ \bf y} \to E[{ \bf X}|{ \bf Y}={ \bf y}]$. Several identities in the literature connect $E[{ \bf X}|{ \bf Y}={ \bf y}]$ to other quantities such as the conditional variance, score functions, and higher-order conditional moments. The objective of this paper is to provide a unifying view of these identities. In the first part of the paper, a general derivative identity for the conditional mean is derived. Specifically, for the Markov chain ${ \bf U} \leftrightarrow { \bf X} \leftrightarrow { \bf Y}$, it is shown that the Jacobian of $E[{ \bf U}|{ \bf Y}={ \bf y}]$ is given by ${ \bf \mathsf{ K}}_{ { \bf N}}^{ -1} { \bf Cov} ( { \bf X}, { \bf U} | { \bf Y}={ \bf y})$. In the second part of the paper, via various choices of ${ \bf U}$, the new identity is used to generalize many of the known identities and derive some new ones. First, a simple proof of the Hatsel and Nolte identity for the conditional variance is shown. Second, a simple proof of the recursive identity due to Jaffer is provided. Third, a new connection between the conditional cumulants and the conditional expectation is shown. In particular, it is shown that the $k$-th derivative of $E[X|Y=y]$ is the $(k+1)$-th conditional cumulant. The third part of the paper considers some applications. In a first application, the power series and the compositional inverse of $E[X|Y=y]$ are derived. In a second application, the distribution of the estimator error $(X-E[X|Y])$ is derived. In a third application, we construct consistent estimators (empirical Bayes estimators) of the conditional cumulants from an i.i.d. sequence $Y_1,...,Y_n$.
Categories: cs.IT math.IT math.ST stat.ML stat.TH