Jensen's inequality

From Wikimization

Jump to: navigation, search

By definition \,\phi\, is convex if and only if

\phi(ta + (1-t)b) \leq t \phi(a) + (1-t) \phi(b)

whenever \,0 \leq t \leq 1\, and \,a\,, b\, are in the domain of \,\phi\,.

It follows by induction on \,n\, that if \,t_j \geq 0\, for \,j = 1, 2\ldots n\, then


\phi(\sum t_j a_j) \leq \sum t_j \phi(a_j)           (1)


Jensen's inequality says this:
If \,\mu\, is a probability measure on \,X\, , 
\,f\, is a real-valued function on \,X\, , 
\,f\, is integrable, and
\,\phi\, is convex on the range of \,f\, then


\phi(\int f d \mu) \leq \int \phi \circ f d \mu\qquad          (2)


Proof 1: By some limiting argument we can assume that \,f\, is simple. (This limiting argument is a missing detail to this proof...)
That is, \,X\, is the disjoint union of \,X_1 \,\ldots\, X_n\, and \,f\, is constant on each \,X_j\, .

Say \,t_j=\mu(X_j)\, and \,a_j\, is the value of \,f\, on \,X_j\, . 

Then (1) and (2) say exactly the same thing. QED.


Proof 2:

Lemma. If \,a < b,\, \,a' < b',\, \,a \leq a'\, and \,b \leq b'\, then

\,(f(a) - f(b)) / (a - b) \leq (f(a') - f(b')) / (a' - b')\quad\diamond

The lemma shows:

  • \,\phi\, has a right-hand derivative at every point, and
  • the graph of \,\phi\, lies above the "tangent" line through any point on the graph with slope equal to the right derivative.

Say \,a = \int f d \mu\,

Let \,m\, be the right derivative of \,\phi\, at \,a\, ,  and let

\,L(t) = \phi(a) + m(t-a)\,

The bullets above say \,\phi(t)\geq L(t)\, for all \,t\, in the domain of  \,\phi\, .  So

\begin{array}{rl}\int \phi \circ f &\geq \int L \circ f\\ &= \int (\phi(a) + m(f - a))\\ &= \phi(a) + (m \int f) - ma\\ &= \phi(a)\\ &= \phi(\int f)\end{array}

\,-\,David C. Ullrich

Retrieved from "http://www.convexoptimization.com/wikimization/index.php/Jensen%27s_inequality"
Personal tools