Moreau's decomposition theorem

From Wikimization

(Difference between revisions)
Jump to: navigation, search
m (Protected "Moreau's decomposition theorem" [edit=autoconfirmed:move=autoconfirmed])
Line 3: Line 3:
=== Projection mapping ===
=== Projection mapping ===
-
Let <math>(\mathcal H,\langle\cdot,\cdot\rangle)</math> be a Hilbert space and <math>\mathcal C</math> a closed convex set in <math>\mathcal H.</math> The '''projection mapping''' <math>P_{\mathcal C}</math> onto <math>\mathcal C</math> is the mapping <math>P_{\mathcal C}:\mathcal H\to\mathcal H</math> defined by <math>P_{\mathcal C}(x)\in\mathcal C</math> and
+
Let <math>(\mathbb H,\langle\cdot,\cdot\rangle)</math> be a Hilbert space and <math>\mathcal C</math> a closed convex set in <math>\mathbb H.</math> The '''projection mapping''' <math>P_{\mathcal C}</math> onto <math>\mathcal C</math> is the mapping <math>P_{\mathcal C}:\mathbb H\to\mathbb H</math> defined by <math>P_{\mathcal C}(x)\in\mathcal C</math> and
<center>
<center>
Line 13: Line 13:
-
Let <math>(\mathcal H,\langle\cdot,\cdot\rangle)</math> be a Hilbert space, <math>\mathcal C</math> a closed convex set in <math>\mathcal H,\,u\in\mathcal H</math> and <math>v\in\mathcal C.</math> Then, <math>v=P_{\mathcal C}(u)</math> if and only if <math>\langle u-v,w-v\rangle\leq0</math> for all <math>w\in\mathcal C.</math>
+
Let <math>(\mathbb H,\langle\cdot,\cdot\rangle)</math> be a Hilbert space, <math>\mathcal C</math> a closed convex set in <math>\mathbb H,\,u\in\mathbb H</math> and <math>v\in\mathcal C.</math> Then, <math>v=P_{\mathcal C}(u)</math> if and only if <math>\langle u-v,w-v\rangle\leq0</math> for all <math>w\in\mathcal C.</math>
=== Proof ===
=== Proof ===
Line 48: Line 48:
under the addition of vectors and multiplication of vectors by positive scalars (see more at [http://en.wikipedia.org/wiki/Convex_cone Convex cone, Wikipedia] or for finite dimension at [[Convex cones | Convex cones, Wikimization]]).
under the addition of vectors and multiplication of vectors by positive scalars (see more at [http://en.wikipedia.org/wiki/Convex_cone Convex cone, Wikipedia] or for finite dimension at [[Convex cones | Convex cones, Wikimization]]).
-
'''Theorem (Moreau)''' Let <math>\mathcal K</math> be a closed convex cone in the Hilbert space <math>(\mathcal H,\langle\cdot,\cdot\rangle)</math> and <math>\mathcal K^\circ</math> its '''polar cone'''; that is, the closed convex cone defined by <math>K^\circ=\{a\in\mathcal H\mid\langle a,b\rangle\leq0,\,\forall b\in\mathcal K\}</math> (for finite dimension see more at [http://en.wikipedia.org/wiki/Dual_cone_and_polar_cone Dual cone and polar cone]; see also [[Farkas%27_lemma#Extended_Farkas.27_lemma | Extended Farkas' lemma]]). For <math>x,y,z\in\mathcal H</math> the following statements are equivalent:
+
'''Theorem (Moreau)''' Let <math>\mathcal K</math> be a closed convex cone in the Hilbert space <math>(\mathbb H,\langle\cdot,\cdot\rangle)</math> and <math>\mathcal K^\circ</math> its '''polar cone'''; that is, the closed convex cone defined by <math>K^\circ=\{a\in\mathbb H\mid\langle a,b\rangle\leq0,\,\forall b\in\mathcal K\}</math> (for finite dimension see more at [http://en.wikipedia.org/wiki/Dual_cone_and_polar_cone Dual cone and polar cone]; see also [[Farkas%27_lemma#Extended_Farkas.27_lemma | Extended Farkas' lemma]]). For <math>x,y,z\in\mathbb H</math> the following statements are equivalent:
<ol>
<ol>
Line 116: Line 116:
=== Nonlinear complementarity problems ===
=== Nonlinear complementarity problems ===
-
Let <math>\mathcal K</math> be a closed convex cone in the Hilbert space <math>(\mathcal H,\langle\cdot,\cdot\rangle)</math> and <math>f:\mathcal H\to\mathcal H.</math> Recall that the dual cone of <math>\mathcal K</math> is the closed convex cone <math>\mathcal K^*=-\mathcal K^\circ,</math> where <math>\mathcal K^\circ</math> is the [[Moreau's_decomposition_theorem#Moreau.27s_theorem |polar]] of <math>\mathcal K.</math> The '''nonlinear complementarity problem''' defined by <math>\mathcal K</math> and <math>f\,</math> is the problem
+
Let <math>\mathcal K</math> be a closed convex cone in the Hilbert space <math>(\mathbb H,\langle\cdot,\cdot\rangle)</math> and <math>f:\mathbb H\to\mathbb H.</math> Recall that the dual cone of <math>\mathcal K</math> is the closed convex cone <math>\mathcal K^*=-\mathcal K^\circ,</math> where <math>\mathcal K^\circ</math> is the [[Moreau's_decomposition_theorem#Moreau.27s_theorem |polar]] of <math>\mathcal K.</math> The '''nonlinear complementarity problem''' defined by <math>\mathcal K</math> and <math>f\,</math> is the problem
<center>
<center>
Line 131: Line 131:
=== Every nonlinear complementarity problem is equivalent to a fixed point problem ===
=== Every nonlinear complementarity problem is equivalent to a fixed point problem ===
-
Let <math>\mathcal K</math> be a closed convex cone in the Hilbert space <math>(\mathcal H,\langle\cdot,\cdot\rangle)</math> and <math>f:\mathcal H\to\mathcal H.</math> Then, the nonlinear complementarity problem <math>NCP(f,\mathcal K)</math> is equivalent to the fixed point problem
+
Let <math>\mathcal K</math> be a closed convex cone in the Hilbert space <math>(\mathbb H,\langle\cdot,\cdot\rangle)</math> and <math>f:\mathbb H\to\mathbb H.</math> Then, the nonlinear complementarity problem <math>NCP(f,\mathcal K)</math> is equivalent to the fixed point problem
-
<math>Fix(P_{\mathcal K}\circ(I-f)),</math> where <math>I:\mathcal H\to\mathcal H</math> is the identity mapping defined by <math>I(x)=x.\,</math>
+
<math>Fix(P_{\mathcal K}\circ(I-f)),</math> where <math>I:\mathbb H\to\mathbb H</math> is the identity mapping defined by <math>I(x)=x.\,</math>
=== Proof ===
=== Proof ===
-
For all <math>x\in\mathcal H</math> denote <math>z=x-f(x)\,</math> and <math>y=-f(x).\,</math> Then, <math>z=x+y.\,</math>
+
For all <math>x\in\mathbb H</math> denote <math>z=x-f(x)\,</math> and <math>y=-f(x).\,</math> Then, <math>z=x+y.\,</math>
<br>
<br>
<br>
<br>
Line 157: Line 157:
==== Variational inequalities ====
==== Variational inequalities ====
-
Let <math>\mathcal C</math> be a closed convex set in the Hilbert space <math>(\mathcal H,\langle\cdot,\cdot\rangle)</math> and <math>f:\mathcal H\to\mathcal H.</math> The '''variational inequality''' defined by <math>\mathcal C</math> and <math>f\,</math> is the problem
+
Let <math>\mathcal C</math> be a closed convex set in the Hilbert space <math>(\mathbb H,\langle\cdot,\cdot\rangle)</math> and <math>f:\mathbb H\to\mathbb H.</math> The '''variational inequality''' defined by <math>\mathcal C</math> and <math>f\,</math> is the problem
<center>
<center>
Line 172: Line 172:
==== Every variational inequality is equivalent to a fixed point problem ====
==== Every variational inequality is equivalent to a fixed point problem ====
-
Let <math>\mathcal C</math> be a closed convex set in the Hilbert space <math>(\mathcal H,\langle\cdot,\cdot\rangle)</math> and <math>f:\mathcal H\to\mathcal H.</math> Then the variational inequality <math>VI(f,\mathcal C)</math> is equivalent to the fixed point problem <math>Fix(P_{\mathcal C}\circ(I-f)).</math>
+
Let <math>\mathcal C</math> be a closed convex set in the Hilbert space <math>(\mathbb H,\langle\cdot,\cdot\rangle)</math> and <math>f:\mathbb H\to\mathbb H.</math> Then the variational inequality <math>VI(f,\mathcal C)</math> is equivalent to the fixed point problem <math>Fix(P_{\mathcal C}\circ(I-f)).</math>
==== Proof ====
==== Proof ====
Line 192: Line 192:
==== Every variational inequality defined on a closed convex cone is equivalent to a complementarity problem ====
==== Every variational inequality defined on a closed convex cone is equivalent to a complementarity problem ====
-
Let <math>\mathcal K</math> be a closed convex cone in the Hilbert space <math>(\mathcal H,\langle\cdot,\cdot\rangle)</math> and <math>f:\mathcal H\to\mathcal H.</math> Then, the nonlinear complementarity problem <math>NCP(f,\mathcal K)</math> is equivalent to the variational inequality <math>VI(f,\mathcal K).</math>
+
Let <math>\mathcal K</math> be a closed convex cone in the Hilbert space <math>(\mathbb H,\langle\cdot,\cdot\rangle)</math> and <math>f:\mathbb H\to\mathbb H.</math> Then, the nonlinear complementarity problem <math>NCP(f,\mathcal K)</math> is equivalent to the variational inequality <math>VI(f,\mathcal K).</math>
==== Proof ====
==== Proof ====

Revision as of 15:00, 13 July 2009

Contents

Projection on closed convex sets

Projection mapping

Let LaTeX: (\mathbb H,\langle\cdot,\cdot\rangle) be a Hilbert space and LaTeX: \mathcal C a closed convex set in LaTeX: \mathbb H. The projection mapping LaTeX: P_{\mathcal C} onto LaTeX: \mathcal C is the mapping LaTeX: P_{\mathcal C}:\mathbb H\to\mathbb H defined by LaTeX: P_{\mathcal C}(x)\in\mathcal C and

LaTeX: \|x-P_{\mathcal C}(x)\|=\min\{\|x-y\|\mid y\in\mathcal C\}.

Characterization of the projection

Let LaTeX: (\mathbb H,\langle\cdot,\cdot\rangle) be a Hilbert space, LaTeX: \mathcal C a closed convex set in LaTeX: \mathbb H,\,u\in\mathbb H and LaTeX: v\in\mathcal C. Then, LaTeX: v=P_{\mathcal C}(u) if and only if LaTeX: \langle u-v,w-v\rangle\leq0 for all LaTeX: w\in\mathcal C.

Proof

Suppose that LaTeX: v=P_{\mathcal C}u. Let LaTeX: w\in\mathcal C and LaTeX: t\in (0,1) be arbitrary. By using the convexity of LaTeX: \mathcal C, it follows that LaTeX: (1-t)v+tw\in\mathcal C. Then, by using the definition of the projection, we have

LaTeX: 
\|u-v\|^2\leq\|u-((1-t)v+tw)\|^2=\|u-v-t(w-v)\|^2=\|u-v\|^2-2t\langle u-v,w-v\rangle+t^2\|w-v\|^2,

Hence,

LaTeX: \langle u-v,w-v\rangle\leq\frac t2\|w-v\|^2.

By tending with LaTeX: t\, to LaTeX: 0,\, we get LaTeX: \langle u-v,w-v\rangle\leq0.

Conversely, suppose that LaTeX: \langle u-v,w-v\rangle\leq0, for all LaTeX: w\in\mathcal C. Then,

LaTeX: \|u-w\|^2=\|u-v-(w-v)\|^2=\|u-v\|^2-2\langle u-v,w-v\rangle+\|w-v\|^2\geq \|u-v\|^2,

for all LaTeX: w\in\mathcal C. Hence, by using the definition of the projection, we get LaTeX: v=P_{\mathcal C}u.

Moreau's theorem

Moreau's theorem is a fundamental result characterizing projections onto closed convex cones in Hilbert spaces. Recall that a convex cone in a vector space is a set which is invariant under the addition of vectors and multiplication of vectors by positive scalars (see more at Convex cone, Wikipedia or for finite dimension at Convex cones, Wikimization).

Theorem (Moreau) Let LaTeX: \mathcal K be a closed convex cone in the Hilbert space LaTeX: (\mathbb H,\langle\cdot,\cdot\rangle) and LaTeX: \mathcal K^\circ its polar cone; that is, the closed convex cone defined by LaTeX: K^\circ=\{a\in\mathbb H\mid\langle a,b\rangle\leq0,\,\forall b\in\mathcal K\} (for finite dimension see more at Dual cone and polar cone; see also Extended Farkas' lemma). For LaTeX: x,y,z\in\mathbb H the following statements are equivalent:

  1. LaTeX: z=x+y,\,x\in\mathcal K,\,y\in\mathcal K^\circ and LaTeX: \langle x,y\rangle=0,
  2. LaTeX: x=P_{\mathcal K}z and LaTeX: y=P_{\mathcal K^\circ}z.

Proof of Moreau's theorem

  • 1LaTeX: \Rightarrow2: For all LaTeX: p\in K we have

    LaTeX: \langle z-x,p-x\rangle=\langle y,p-x\rangle=\langle y,p\rangle\leq0.

    Then, by the characterization of the projection, it follows that LaTeX: x=P_{\mathcal K}z. Similarly, for all LaTeX: q\in K^\circ we have

    LaTeX: \langle z-y,q-y\rangle=\langle x,q-y\rangle=\langle x,q\rangle\leq0

    and thus LaTeX: y=P_{\mathcal K^\circ}z.
  • 2LaTeX: \Rightarrow1: By using the characterization of the projection, we have LaTeX: \langle z-x,p-x\rangle\leq0, for all LaTeX: p\in\mathcal K. In particular, if LaTeX: p=0,\, then LaTeX: \langle z-x,x\rangle\geq0 and if LaTeX: p=2x,\, then LaTeX: \langle z-x,x\rangle\leq0. Thus, LaTeX: \langle z-x,x\rangle=0. Denote LaTeX: u=z-x.\, Then, LaTeX: \langle x,u\rangle=0. It remained to show that LaTeX: u=y.\, First, we prove that LaTeX: u\in\mathcal K^\circ. For this we have to show that LaTeX: \langle u,p\rangle\leq0, for all LaTeX: p\in\mathcal K. By using the characterization of the projection, we have

    LaTeX: 
\langle u,p\rangle=\langle u,p-x\rangle=\langle z-x,p-x\rangle\leq0,

    for all LaTeX: p\in\mathcal K. Thus, LaTeX: u\in\mathcal K^\circ. We also have

    LaTeX: 
\langle z-u,q-u\rangle=\langle x,q-u\rangle=\langle x,q\rangle\leq0,

    for all LaTeX: q\in K^\circ, because LaTeX: x\in K. By using again the characterization of the projection, it follows that LaTeX: u=y.\,

References

  • J. J. Moreau, Décomposition orthogonale d'un espace hilbertien selon deux cones mutuellement polaires, C. R. Acad. Sci., volume 255, pages 238–240, 1962.

An application to nonlinear complementarity problems

Fixed point problems

Let LaTeX: \mathcal A be a set and LaTeX: F:\mathcal A\to\mathcal A. The fixed point problem defined by LaTeX: F\, is the problem

LaTeX: 
Fix(F):\left\{
\begin{array}{l}
Find\,\,\,x\in\mathcal A\,\,\,such\,\,\,that\\
F(x)=x.
\end{array}
\right.

Nonlinear complementarity problems

Let LaTeX: \mathcal K be a closed convex cone in the Hilbert space LaTeX: (\mathbb H,\langle\cdot,\cdot\rangle) and LaTeX: f:\mathbb H\to\mathbb H. Recall that the dual cone of LaTeX: \mathcal K is the closed convex cone LaTeX: \mathcal K^*=-\mathcal K^\circ, where LaTeX: \mathcal K^\circ is the polar of LaTeX: \mathcal K. The nonlinear complementarity problem defined by LaTeX: \mathcal K and LaTeX: f\, is the problem

LaTeX: 
NCP(f,\mathcal K):\left\{
\begin{array}{l} 
Find\,\,\,x\in\mathcal K\,\,\,such\,\,\,that\\ 
f(x)\in\mathcal K^*\,\,\,and\,\,\,\langle x,f(x)\rangle=0.
\end{array}
\right.

Every nonlinear complementarity problem is equivalent to a fixed point problem

Let LaTeX: \mathcal K be a closed convex cone in the Hilbert space LaTeX: (\mathbb H,\langle\cdot,\cdot\rangle) and LaTeX: f:\mathbb H\to\mathbb H. Then, the nonlinear complementarity problem LaTeX: NCP(f,\mathcal K) is equivalent to the fixed point problem LaTeX: Fix(P_{\mathcal K}\circ(I-f)), where LaTeX: I:\mathbb H\to\mathbb H is the identity mapping defined by LaTeX: I(x)=x.\,

Proof

For all LaTeX: x\in\mathbb H denote LaTeX: z=x-f(x)\, and LaTeX: y=-f(x).\, Then, LaTeX: z=x+y.\,

Suppose that LaTeX: x\, is a solution of LaTeX: NCP(f,\mathcal K). Then, LaTeX: z=x+y,\, with LaTeX: x\in\mathcal K, LaTeX: y\in\mathcal K^\circ and LaTeX: \langle x,y\rangle=0. Hence, by using Moreau's theorem, we get LaTeX: x=P_{\mathcal K}z. Therefore, LaTeX: x\, is a solution of LaTeX: Fix(P_{\mathcal K}\circ(I-f)).

Conversely, suppose that LaTeX: x\, is a solution of LaTeX: Fix(P_{\mathcal K}\circ(I-f)). Then, LaTeX: x\in\mathcal K and by using Moreau's theorem

LaTeX: z=P_{\mathcal K}(z)+P_{\mathcal K^\circ}(z)=x+P_{\mathcal K^\circ}(z).

Hence, LaTeX: P_{\mathcal K^\circ}(z)=z-x=y,. Thus, LaTeX: y\in\mathcal K^\circ. Moreau's theorem also implies that LaTeX: \langle x,y\rangle=0. In conclusion, LaTeX: x\in\mathcal K, LaTeX: f(x)=-y\in\mathcal K^* and LaTeX: \langle x,f(x)\rangle=0. Therefore, LaTeX: x\, is a solution of LaTeX: NCP(f,\mathcal K).

An alternative proof without Moreau's theorem

Variational inequalities

Let LaTeX: \mathcal C be a closed convex set in the Hilbert space LaTeX: (\mathbb H,\langle\cdot,\cdot\rangle) and LaTeX: f:\mathbb H\to\mathbb H. The variational inequality defined by LaTeX: \mathcal C and LaTeX: f\, is the problem

LaTeX: 
VI(f,\mathcal C):\left\{
\begin{array}{l} 
Find\,\,\,x\in\mathcal C\,\,\,such\,\,\,that\\ 
\langle y-x,f(x)\rangle\geq 0,\,\,\,for\,\,\,all\,\,\,y\in\mathcal C.
\end{array}
\right.

Every variational inequality is equivalent to a fixed point problem

Let LaTeX: \mathcal C be a closed convex set in the Hilbert space LaTeX: (\mathbb H,\langle\cdot,\cdot\rangle) and LaTeX: f:\mathbb H\to\mathbb H. Then the variational inequality LaTeX: VI(f,\mathcal C) is equivalent to the fixed point problem LaTeX: Fix(P_{\mathcal C}\circ(I-f)).

Proof

LaTeX: x\, is a solution of LaTeX: Fix(P_{\mathcal C}\circ(I-f)) if and only if LaTeX: x=P_{\mathcal C}(x-f(x)). By using the characterization of the projection the latter equation is equivalent to

LaTeX: \langle x-f(x)-x,y-x\rangle\leq0,

for all LaTeX: y\in\mathcal C. But this holds if and only if LaTeX: x\, is a solution of LaTeX: VI(f,\mathcal C).

Remark

The next section shows that the equivalence of variational inequalities and fixed point problems is much stronger than the equivalence of nonlinear complementarity problems and fixed point problems, because each nonlinear complementarity problem is a variational inequality defined on a closed convex cone.

Every variational inequality defined on a closed convex cone is equivalent to a complementarity problem

Let LaTeX: \mathcal K be a closed convex cone in the Hilbert space LaTeX: (\mathbb H,\langle\cdot,\cdot\rangle) and LaTeX: f:\mathbb H\to\mathbb H. Then, the nonlinear complementarity problem LaTeX: NCP(f,\mathcal K) is equivalent to the variational inequality LaTeX: VI(f,\mathcal K).

Proof

Suppose that LaTeX: x\, is a solution of LaTeX: NCP(f,\mathcal K). Then, LaTeX: x\in\mathcal K, LaTeX: f(x)\in\mathcal K^* and LaTeX: \langle x,f(x)\rangle=0. Hence,

LaTeX: \langle y-x,f(x)\rangle\geq 0,

for all LaTeX: y\in\mathcal K. Therefore, LaTeX: x\, is a solution of LaTeX: VI(f,\mathcal K).

Conversely, suppose that LaTeX: x\, is a solution of LaTeX: VI(f,\mathcal K). Then, LaTeX: x\in\mathcal K and

LaTeX: \langle y-x,f(x)\rangle\geq 0,

for all LaTeX: y\in\mathcal K. Particularly, taking LaTeX: y=0\, and LaTeX: y=2x\,, respectively, we get LaTeX: \langle x,f(x)\rangle=0. Thus, LaTeX: \langle y,f(x)\rangle\geq 0, for all LaTeX: y\in\mathcal K, or equivalently LaTeX: f(x)\in\mathcal K^*. In conclusion, LaTeX: x\in\mathcal K, LaTeX: f(x)\in\mathcal K^* and LaTeX: \langle x,f(x)\rangle=0. Therefore, LaTeX: x\, is a solution of LaTeX: NCP(f,\mathcal K).

Concluding the alternative proof

Since LaTeX: \mathcal K is a closed convex cone, the nonlinear complementarity problem LaTeX: NCP(f,\mathcal K) is equivalent to the variational inequality LaTeX: VI(f,\mathcal K), which is equivalent to the fixed point problem LaTeX: Fix(P_{\mathcal K}\circ(I-f)).

Personal tools