Orthogonality

Orthogonality generalises perpendicularity to an inner product space.

Mathematical definition

Let vector space $V$ be an inner product space.

Two vectors $u,v\in V$ are orthogonal if $\langle u, v \rangle=0$.

$u\bot v$ denotes this orthogonality.

Properties

$\vec 0$ is orthogonal to every vector

$$ \forall v \in V, \vec 0 \bot v $$

Proof:

$$ Recall the properties of a vector space by definition. By additive identity, $\vec 0 +\vec 0 = \vec 0$

Therefore,

$$ \langle \vec 0, u \rangle = \langle \vec 0+\vec 0, u\rangle $$

Recall the properties of the inner product by definition, by linearity in the 1st slot,

$$ \langle \vec 0+\vec 0, u\rangle = \langle\vec 0, u \rangle+\langle \vec 0, u\rangle $$

Subtracting $\langle \vec 0, u\rangle$ from both sides gives

$$ \langle \vec 0, u \rangle=0. $$

Generalised Pythagorean theorem

If $u,v\in V; u\bot v$, then $\| \cdot \|$ defined by $\| v \|:=\sqrt{\langle v, v \rangle}$ obeys

$$\| u+v \|^2 = \| u \|^2 + \| v \|^2.$$

Proof::

By definition,

$$ \| u+v \|^2 = \langle u+v, u+v \rangle $$

Recall the properties of the inner product by definition:

By linearity in the 1st slot,

$$ \langle u+v, u+v \rangle = \langle u, u+v \rangle + \langle v, u+v \rangle $$

By conjugate symmetry,

$$ \langle u, u+v \rangle + \langle u, u+v \rangle = \overline{\langle u+v, u \rangle} + \overline{\langle u+v, v \rangle} $$

Again by linearity in the first slot,

$$ \overline{\langle u+v, u \rangle} + \overline{\langle u+v, v \rangle}=\overline{\langle u, u \rangle} + \overline{\langle v, u \rangle} + \overline{\langle u, v \rangle} + \overline{\langle v, v \rangle} $$

Again by conjugate symmetry,

$$ \overline{\langle u, u \rangle} + \overline{\langle v, u \rangle} + \overline{\langle u, v \rangle} + \overline{\langle v, v \rangle} = \langle u, u \rangle + \langle u, v \rangle + \langle v, u \rangle + \langle v, v \rangle $$

since $u,v\in V; u\bot v$:

$$ \langle u, v \rangle=0;\langle v, u \rangle=0 $$

By definition

$$ \langle u, u \rangle=\left\| u \right\|^2;\langle v, v \rangle=\left\| v \right\|^2 $$

Hence

$$\| u+v \|^2 = \| u \|^2 + \| v \|^2.$$

Cauchy-Schwarz inequality

$$\forall u,v\in V,\; |\langle u,v\rangle|\le\|u\|\|v\|.$$

Furthermore,

$$|\langle u,v\rangle|=\|u\|\|v\|\iff \exists \lambda\in\mathbb{F},\; u=\lambda v$$

Proof:

If $v=0$,

$$\begin{gathered} |\langle u,v\rangle| = 0, \|u\|\|v\| = 0 \\ \therefore |\langle u,v\rangle| = \|u\|\|v\| \end{gathered}$$

If $v\neq 0$,

Consider the orthogonal decomposition

$$u = \frac{\langle u, v \rangle}{\| v \|^2} v + u_2$$

where $u_2\bot v$.

By the Pythagorean theorem,

$$\| u \|^2 = \left\| \frac{\langle u, v \rangle}{\| v \|^2}v \right\|^2 + \| u_2 \|^2 = \frac{|\langle u, v \rangle|^2}{\| v \|^2} + \| u_2 \|^2 $$

Since $v\neq 0$:

$$\begin{gathered} \frac{|\langle u, v \rangle|^2}{\| v \|^2} + \| u_2 \|^2 \ge \frac{|\langle u, v \rangle|^2}{\| v \|^2} \\ \Rightarrow \| u \|^2 \ge \frac{|\langle u, v \rangle|^2}{\| v \|^2} \\ \Rightarrow |\langle u,v\rangle|\le\|u\|\|v\|. \end{gathered}$$

Triangle inequality

$$\forall u,v\in V,\; \left\| u+v \right\| \le \left\| u \right\| + \left\| v \right\|.$$

Proof:

$$ \begin{aligned} \left\| u+v \right\|^2 &=\langle u+v,u+v\rangle =\langle u,u\rangle+\langle v,v\rangle+\langle u,v\rangle+\langle v,u\rangle\\ &=\langle u,u\rangle+\langle v,v\rangle+\langle u,v\rangle+\overline{\langle u,v\rangle} =\left\| u \right\|^2+\left\| v \right\|^2+2\operatorname{Re}\langle u,v\rangle,\\ \text{Note }\operatorname{Re}\langle u,v\rangle &\le|\langle u,v\rangle| \stackrel{\text{Cauchy-Schwarz}}{\le}\left\| u \right\|\left\| v \right\|,\\ \left\| u \right\|^2+\left\| v \right\|^2+2\operatorname{Re}\langle u,v\rangle &\le\left\| u \right\|^2+\left\| v \right\|^2+2\left\| u \right\|\left\| v \right\| =(\left\| u \right\|+\left\| v \right\|)^2,\\ \Rightarrow \left\| u+v \right\|^2 \leq (\left\| u \right\|+\left\| v \right\|)^2,\\ \because \left\| . \right\| \geq 0 \\ \therefore \left\| u+v \right\|&\le\left\| u \right\|+\left\| v \right\| \end{aligned} $$