Moreau's decomposition theorem

From Wikimization

(Difference between revisions)
Jump to: navigation, search
(→Proof of extended Farkas' lemma)
Current revision (05:07, 28 June 2019) (edit) (undo)
m (→Projection mapping)
 
(106 intermediate revisions not shown.)
Line 1: Line 1:
-
'''Moreau's theorem''' is a fundamental result characterizing projections onto closed convex cones in Hilbert spaces.
+
[http://web.mat.bham.ac.uk/S.Z.Nemeth/ <math>-</math> Sándor Zoltán Németh]
-
Let <math>\mathcal K</math> be a closed convex cone in the Hilbert space <math>(\mathcal H,\langle\cdot,\cdot\rangle)</math> and <math>\mathcal K^\circ</math> its polar. For an arbitrary closed convex set <math>\mathcal C</math> in <math>\mathcal H</math>, denote by <math>P_{\mathcal C}</math> the projection onto <math>\mathcal C</math>. For <math>x,y,z\in\mathcal H</math> the following two statements are equivalent:
+
(In particular, we can have <math>\mathbb{H}=\mathbb{R}^n</math> everywhere in this page.)
-
<ol>
+
== Projection on closed convex sets ==
-
<li><math>z=x+y</math>, <math>x\in\mathcal K, y\in\mathcal K^\circ</math> and <math>\langle x,y\rangle=0</math></li>
+
-
<li><math>x=P_{\mathcal K}z</math> and <math>y=P_{\mathcal K^\circ}z</math>
+
-
</li>
+
-
</ol>
+
-
== Proof of Moreau's theorem ==
+
=== Projection mapping ===
-
 
+
Let <math>(\mathbb{H},\langle\cdot,\cdot\rangle)</math> be a Hilbert space and <math>\mathcal{C}</math> a closed convex set in <math>\mathbb{H}.</math> The '''projection mapping''' <math>P_{\mathcal{C}}</math> onto <math>\mathcal{C}</math> is the mapping <math>P_{\mathcal{C}}:\mathbb{H}\to\mathbb{H}</math> defined by <math>P_{\mathcal{C}}(x)\in\mathcal{C}</math> and
-
Let <math>\mathcal C</math> be an arbitrary closed convex set in <math>\mathcal H</math>, <math>u\in\mathcal H</math> and <math>v\in\mathcal C</math>. Then, it is well known that <math>v=P_{\mathcal C}u</math> if and only if <math>\langle u-v,w-v\rangle\leq0</math> for all <math>w\in\mathcal C</math>. We will call this result the '''''characterization of the projection'''''.
+
-
 
+
-
<ul>
+
-
<li>1<math>\Rightarrow</math>2: For all <math>p\in K</math> we have
+
<center>
<center>
-
<math>\langle z-x,p-x\rangle=\langle y,p-x\rangle=\langle y,p\rangle\leq0</math>.
+
<math>\parallel x-P_{\mathcal{C}}(x)\parallel=\min\{\parallel x-y\parallel\,:y\in\mathcal{C}\}.</math>
</center>
</center>
-
Then, by the characterization of the projection, it follows that <math>x=P_{\mathcal K}z</math>. Similarly, for all <math>q\in K^\circ</math> we have
+
=== Characterization of the projection ===
 +
Let <math>(\mathbb{H},\langle\cdot,\cdot\rangle)</math> be a Hilbert space, <math>\mathcal{C}</math> a closed convex set in <math>\mathbb{H},\,u\in\mathbb{H}</math> and <math>v\in\mathcal{C}.</math> Then <math>v=P_{\mathcal{C}}(u)</math> if and only if <math>\langle u-v,w-v\rangle\leq0</math> for all <math>w\in\mathcal{C}.</math>
 +
 
 +
=== Proof ===
 +
Suppose that <math>v=P_{\mathcal{C}}u.</math> Let <math>w\in\mathcal{C}</math> and <math>t\in (0,1)</math> be arbitrary. By using the convexity of <math>\mathcal{C},</math> it follows that <math>(1-t)v+tw\in\mathcal{C}.</math> Then, by using the definition of the projection, we have
<center>
<center>
-
<math>\langle z-y,q-y\rangle=\langle x,q-y\rangle=\langle x,q\rangle\leq0</math>
+
<math>
 +
\parallel u-v\parallel^2\leq\parallel u-((1-t)v+tw)\parallel^2=\parallel u-v-t(w-v)\parallel^2=\parallel u-v\parallel^2-2t\langle u-v,w-v\rangle+t^2\parallel w-v\parallel^2,
 +
</math>
</center>
</center>
-
and thus <math>y=P_{\mathcal K^\circ}z</math>.</li>
+
Hence,
-
<li>2<math>\Rightarrow</math>1: Let <math>x=P_{\mathcal K}z</math>. By the characterization of the projection we have <math>\langle z-x,p-x\rangle\leq0,</math> for all <math>p\in\mathcal K</math>. In particular, if <math>p=0</math>, then <math>\langle z-x,x\rangle\geq0</math> and if <math>p=2x</math>, then <math>\langle z-x,x\rangle\leq0</math>. Thus, <math>\langle z-x,x\rangle=0</math>. Denote <math>y=z-x</math>. Then, <math>\langle x,y\rangle=0</math>. It remained to show that <math>y=P_{\mathcal K^\circ}z</math>. First, we prove that <math>y\in\mathcal K^\circ</math>. For this we have to show that <math>\langle y,p\rangle\leq0</math>, for
+
-
all <math>p\in\mathcal K</math>. By using the characterization of the projection, we have
+
<center>
<center>
-
<math>
+
<math>\langle u-v,w-v\rangle\leq\frac t2\parallel w-v\parallel^2.</math>
-
\langle y,p\rangle=\langle y,p-x\rangle=\langle z-x,p-x\rangle\leq0,
+
-
</math>
+
</center>
</center>
-
for all <math>p\in\mathcal K</math>. Thus, <math>y\in\mathcal K^\circ</math>. We also have
+
By tending with <math>t\,</math> to <math>0,\,</math> we get <math>\langle u-v,w-v\rangle\leq0.</math>
 +
<br>
 +
<br>
 +
 
 +
Conversely, suppose that <math>\langle u-v,w-v\rangle\leq0,</math> for all <math>w\in\mathcal C.</math> Then
<center>
<center>
-
<math>
+
<math>\parallel u-w\parallel^2=\parallel u-v-(w-v)\parallel^2=\parallel u-v\parallel^2-2\langle u-v,w-v\rangle+\parallel w-v\parallel^2\geq \parallel u-v\parallel^2,</math>
-
\langle z-y,q-y\rangle=\langle x,q-y\rangle=\langle x,q\rangle\leq0,
+
-
</math>
+
</center>
</center>
-
for all <math>q\in K^\circ</math>, because <math>x\in K</math>. By using again the characterization of the projection, it follows that <math>y=P_{\mathcal K^\circ}z</math>.
+
for all <math>w\in\mathcal C.</math> Hence, by using the definition of the projection, we get <math>v=P_{\mathcal C}u.</math>
-
</ul>
+
== Moreau's theorem ==
 +
Moreau's theorem is a fundamental result characterizing projections onto closed convex cones in Hilbert spaces.
-
== References ==
+
Recall that a '''convex cone''' in a vector space is a set which is invariant
 +
under the addition of vectors and multiplication of vectors by positive scalars.
-
* J. J. Moreau, D&eacute;composition orthogonale d'un espace hilbertien selon deux cones mutuellement polaires, C. R. Acad. Sci., volume 255, pages 238–240, 1962.
+
'''Theorem (Moreau).''' Let <math>\mathcal{K}</math> be a closed convex cone in the Hilbert space <math>(\mathbb{H},\langle\cdot,\cdot\rangle)</math> and <math>\mathcal{K}^\circ</math> its '''polar cone'''; that is, the closed convex cone defined by <math>\mathcal{K}^\circ=\{a\in\mathbb{H}\,\mid\,\langle a,b\rangle\leq0,\,\forall b\in\mathcal{K}\}.</math>
-
== The extended Farkas' lemma ==
+
For <math>x,y,z\in\mathbb{H}</math> the following statements are equivalent:
-
For any closed convex cone <math>\mathcal J</math> in the Hilbert space <math>(\mathcal H,\langle\cdot,\cdot\rangle)</math>, denote by <math>\mathcal J^\circ</math> the polar cone of <math>\mathcal J</math>. Let <math>\mathcal K</math> be an arbitrary closed convex cone in <math>\mathcal H</math>. Then, the extended Farkas' lemma asserts that <math>\mathcal K^{\circ\circ}=\mathcal K.</math> Hence, denoting <math>\mathcal L=\mathcal K^\circ,</math> it follows that <math>\mathcal L^\circ=\mathcal K</math>. Therefore, the cones <math>\mathcal K</math> and <math>\mathcal L</math> are called ''mutually polar pair of cones''.
+
<ol>
 +
<li><math>z=x+y,\,x\in\mathcal{K},\,y\in\mathcal{K}^\circ</math> and <math>\langle x,y\rangle=0,</math></li>
 +
<li><math>x=P_{\mathcal{K}}z</math> and <math>y=P_{\mathcal{K}^\circ}z.</math>
 +
</li>
 +
</ol>
-
== Proof of extended Farkas' lemma ==
+
=== Proof of Moreau's theorem ===
 +
<ul>
 +
<li>1<math>\Rightarrow</math>2: For all <math>p\in\mathcal{K}</math> we have
-
Let <math>z\in\mathcal H</math> be arbitrary. Then, by Moreau's theorem we have
+
<center>
 +
<math>\langle z-x,p-x\rangle=\langle y,p-x\rangle=\langle y,p\rangle\leq0.</math>
 +
</center>
 +
 
 +
Then, by the characterization of the projection, it follows that <math>x=P_{\mathcal{K}}z.</math> Similarly, for all <math>q\in\mathcal{K}^\circ</math> we have
<center>
<center>
-
<math>
+
<math>\langle z-y,q-y\rangle=\langle x,q-y\rangle=\langle x,q\rangle\leq0</math>
-
z=P_{\mathcal K}z+P_{\mathcal K^\circ}z
+
-
</math>
+
</center>
</center>
-
and
+
and thus <math>y=P_{\mathcal{K}^\circ}z.</math></li>
 +
<li>2<math>\Rightarrow</math>1: By using the characterization of the projection, we have <math>\langle z-x,p-x\rangle\leq0,</math> for all <math>p\in\mathcal K.</math> In particular, if <math>p=0,\,</math> then <math>\langle z-x,x\rangle\geq0</math> and if <math>p=2x,\,</math> then <math>\langle z-x,x\rangle\leq0.</math> Thus, <math>\langle z-x,x\rangle=0.</math> Denote <math>u=z-x.\,</math> Then <math>\langle x,u\rangle=0.</math> It remains to show that <math>u=y.\,</math> First, we prove that <math>u\in\mathcal{K}^\circ.</math> For this we have to show that <math>\langle u,p\rangle\leq0,</math> for
 +
all <math>p\in\mathcal{K}.</math> By using the characterization of the projection, we have
<center>
<center>
<math>
<math>
-
z=P_{\mathcal K^\circ}z+P_{\mathcal K^{\circ\circ}}z.
+
\langle u,p\rangle=\langle u,p-x\rangle=\langle z-x,p-x\rangle\leq0,
</math>
</math>
</center>
</center>
-
Therefore,
+
for all <math>p\in\mathcal{K}.</math> Thus, <math>u\in\mathcal{K}^\circ.</math> We also have
<center>
<center>
<math>
<math>
-
P_{\mathcal K}z=P_{\mathcal K^{\circ\circ}}z=z-P_{\mathcal K^\circ}z.
+
\langle z-u,q-u\rangle=\langle x,q-u\rangle=\langle x,q\rangle\leq0,
</math>
</math>
</center>
</center>
 +
 +
for all <math>q\in\mathcal{K}^\circ,</math> because <math>x\in\mathcal{K}.</math> By using again the characterization of the projection, it follows that <math>u=y.\,</math>
 +
</ul>
 +
 +
=== notes ===
 +
For definition of ''convex cone'' in finite dimension see [[Convex cones | Convex cones, Wikimization]].
 +
 +
For definition of ''polar cone'' in finite dimension, see [http://meboo.convexoptimization.com/Meboo.html Convex Optimization & Euclidean Distance Geometry].
 +
 +
<math>\mathcal{K}^{\circ\circ}=\mathcal{K}</math> see [[Farkas%27_lemma#Extended_Farkas.27_lemma|Extended Farkas' lemma]].
 +
 +
== Applications ==
 +
For applications see [[Complementarity_problem#Every_nonlinear_complementarity_problem_is_equivalent_to_a_fixed_point_problem|Every nonlinear complementarity problem is equivalent to a fixed point problem]], [[Complementarity_problem#Every_implicit_complementarity_problem_is_equivalent_to_a_fixed_point_problem|Every implicit complementarity problem is equivalent to a fixed point problem]],
 +
and [[Projection_on_Polyhedral_Cone#Projection_on_isotone_projection_cones|Projection on isotone projection cone]].
 +
 +
=== References ===
 +
* J. J. Moreau, D&eacute;composition orthogonale d'un espace hilbertien selon deux cones mutuellement polaires, C. R. Acad. Sci., volume 255, pages 238–240, 1962.

Current revision

LaTeX: - Sándor Zoltán Németh

(In particular, we can have LaTeX: \mathbb{H}=\mathbb{R}^n everywhere in this page.)

Contents

Projection on closed convex sets

Projection mapping

Let LaTeX: (\mathbb{H},\langle\cdot,\cdot\rangle) be a Hilbert space and LaTeX: \mathcal{C} a closed convex set in LaTeX: \mathbb{H}. The projection mapping LaTeX: P_{\mathcal{C}} onto LaTeX: \mathcal{C} is the mapping LaTeX: P_{\mathcal{C}}:\mathbb{H}\to\mathbb{H} defined by LaTeX: P_{\mathcal{C}}(x)\in\mathcal{C} and

LaTeX: \parallel x-P_{\mathcal{C}}(x)\parallel=\min\{\parallel x-y\parallel\,:y\in\mathcal{C}\}.

Characterization of the projection

Let LaTeX: (\mathbb{H},\langle\cdot,\cdot\rangle) be a Hilbert space, LaTeX: \mathcal{C} a closed convex set in LaTeX: \mathbb{H},\,u\in\mathbb{H} and LaTeX: v\in\mathcal{C}. Then LaTeX: v=P_{\mathcal{C}}(u) if and only if LaTeX: \langle u-v,w-v\rangle\leq0 for all LaTeX: w\in\mathcal{C}.

Proof

Suppose that LaTeX: v=P_{\mathcal{C}}u. Let LaTeX: w\in\mathcal{C} and LaTeX: t\in (0,1) be arbitrary. By using the convexity of LaTeX: \mathcal{C}, it follows that LaTeX: (1-t)v+tw\in\mathcal{C}. Then, by using the definition of the projection, we have

LaTeX: \parallel u-v\parallel^2\leq\parallel u-((1-t)v+tw)\parallel^2=\parallel u-v-t(w-v)\parallel^2=\parallel u-v\parallel^2-2t\langle u-v,w-v\rangle+t^2\parallel w-v\parallel^2,

Hence,

LaTeX: \langle u-v,w-v\rangle\leq\frac t2\parallel w-v\parallel^2.

By tending with LaTeX: t\, to LaTeX: 0,\, we get LaTeX: \langle u-v,w-v\rangle\leq0.

Conversely, suppose that LaTeX: \langle u-v,w-v\rangle\leq0, for all LaTeX: w\in\mathcal C. Then

LaTeX: \parallel u-w\parallel^2=\parallel u-v-(w-v)\parallel^2=\parallel u-v\parallel^2-2\langle u-v,w-v\rangle+\parallel w-v\parallel^2\geq \parallel u-v\parallel^2,

for all LaTeX: w\in\mathcal C. Hence, by using the definition of the projection, we get LaTeX: v=P_{\mathcal C}u.

Moreau's theorem

Moreau's theorem is a fundamental result characterizing projections onto closed convex cones in Hilbert spaces.

Recall that a convex cone in a vector space is a set which is invariant under the addition of vectors and multiplication of vectors by positive scalars.

Theorem (Moreau). Let LaTeX: \mathcal{K} be a closed convex cone in the Hilbert space LaTeX: (\mathbb{H},\langle\cdot,\cdot\rangle) and LaTeX: \mathcal{K}^\circ its polar cone; that is, the closed convex cone defined by LaTeX: \mathcal{K}^\circ=\{a\in\mathbb{H}\,\mid\,\langle a,b\rangle\leq0,\,\forall b\in\mathcal{K}\}.

For LaTeX: x,y,z\in\mathbb{H} the following statements are equivalent:

  1. LaTeX: z=x+y,\,x\in\mathcal{K},\,y\in\mathcal{K}^\circ and LaTeX: \langle x,y\rangle=0,
  2. LaTeX: x=P_{\mathcal{K}}z and LaTeX: y=P_{\mathcal{K}^\circ}z.

Proof of Moreau's theorem

  • 1LaTeX: \Rightarrow2: For all LaTeX: p\in\mathcal{K} we have

    LaTeX: \langle z-x,p-x\rangle=\langle y,p-x\rangle=\langle y,p\rangle\leq0.

    Then, by the characterization of the projection, it follows that LaTeX: x=P_{\mathcal{K}}z. Similarly, for all LaTeX: q\in\mathcal{K}^\circ we have

    LaTeX: \langle z-y,q-y\rangle=\langle x,q-y\rangle=\langle x,q\rangle\leq0

    and thus LaTeX: y=P_{\mathcal{K}^\circ}z.
  • 2LaTeX: \Rightarrow1: By using the characterization of the projection, we have LaTeX: \langle z-x,p-x\rangle\leq0, for all LaTeX: p\in\mathcal K. In particular, if LaTeX: p=0,\, then LaTeX: \langle z-x,x\rangle\geq0 and if LaTeX: p=2x,\, then LaTeX: \langle z-x,x\rangle\leq0. Thus, LaTeX: \langle z-x,x\rangle=0. Denote LaTeX: u=z-x.\, Then LaTeX: \langle x,u\rangle=0. It remains to show that LaTeX: u=y.\, First, we prove that LaTeX: u\in\mathcal{K}^\circ. For this we have to show that LaTeX: \langle u,p\rangle\leq0, for all LaTeX: p\in\mathcal{K}. By using the characterization of the projection, we have

    LaTeX: \langle u,p\rangle=\langle u,p-x\rangle=\langle z-x,p-x\rangle\leq0,

    for all LaTeX: p\in\mathcal{K}. Thus, LaTeX: u\in\mathcal{K}^\circ. We also have

    LaTeX: \langle z-u,q-u\rangle=\langle x,q-u\rangle=\langle x,q\rangle\leq0,

    for all LaTeX: q\in\mathcal{K}^\circ, because LaTeX: x\in\mathcal{K}. By using again the characterization of the projection, it follows that LaTeX: u=y.\,

notes

For definition of convex cone in finite dimension see Convex cones, Wikimization.

For definition of polar cone in finite dimension, see Convex Optimization & Euclidean Distance Geometry.

LaTeX: \mathcal{K}^{\circ\circ}=\mathcal{K} see Extended Farkas' lemma.

Applications

For applications see Every nonlinear complementarity problem is equivalent to a fixed point problem, Every implicit complementarity problem is equivalent to a fixed point problem, and Projection on isotone projection cone.

References

  • J. J. Moreau, Décomposition orthogonale d'un espace hilbertien selon deux cones mutuellement polaires, C. R. Acad. Sci., volume 255, pages 238–240, 1962.
Retrieved from "http://www.convexoptimization.com/wikimization/index.php/Moreau%27s_decomposition_theorem"
Personal tools