Orthogonality

Orthogonality generalises perpendicularity to an inner product space.

Mathematical definition

Let vector space $V$ be an inner product space.

Two vectors $u,v\in V$ are orthogonal if $\langle u, v \rangle=0$.

$u\bot v$ denotes this orthogonality.

Application

Orthogonal decomposition

Given vectors $u,v\in V$ and $v\neq 0$, $u$ can be uniquely decomposed into two pieces: one parallel to $v$ and one orthogonal to $v$. This is called an orthogonal decomposition.

In mathmetical terms:

$$u=u_1+u_2,$$

Where $u_1=a v, a \in \mathbb{F}; u_2\bot v$

Derivation:

$$\begin{gathered} \because u = u_1 +u_2 \\ \therefore u_2=u-u_1=u-av \\ \because u_2 \bot v \\ \therefore \langle u_2, v \rangle = \langle u- av, v \rangle = \langle u, v \rangle - a \left\| v \right\|^2 = \vec 0 \\ \text{Solve for }a,a = \frac{\langle u, v \rangle}{\left\| v \right\|^2} \\ \text{Substitute back into }u_1=av, u_2=u-av, \\ u=\frac{\langle u, v \rangle}{\| v \|^2} v + \left( u-\frac{\langle u, v \rangle}{\| v \|^2} v \right) \end{gathered}$$

Properties

$\vec 0$ is orthogonal to every vector

$$ \forall v \in V, \vec 0 \bot v $$

Proof:

$$ Recall the properties of a vector space by definition. By additive identity, $\vec 0 +\vec 0 = \vec 0$

Therefore,

$$ \langle \vec 0, u \rangle = \langle \vec 0+\vec 0, u\rangle $$

Recall the properties of the inner product by definition, by linearity in the 1st slot,

$$ \langle \vec 0+\vec 0, u\rangle = \langle\vec 0, u \rangle+\langle \vec 0, u\rangle $$

Subtracting $\langle \vec 0, u\rangle$ from both sides gives

$$ \langle \vec 0, u \rangle=0. $$

Generalised Pythagorean theorem

If $u,v\in V; u\bot v$, then $\| \cdot \|$ defined by $\| v \|:=\sqrt{\langle v, v \rangle}$ obeys

$$\| u+v \|^2 = \| u \|^2 + \| v \|^2.$$

Proof::

By definition,

$$ \| u+v \|^2 = \langle u+v, u+v \rangle $$

Recall the properties of the inner product by definition:

By linearity in the 1st slot,

$$ \langle u+v, u+v \rangle = \langle u, u+v \rangle + \langle v, u+v \rangle $$

By conjugate symmetry,

$$ \langle u, u+v \rangle + \langle u, u+v \rangle = \overline{\langle u+v, u \rangle} + \overline{\langle u+v, v \rangle} $$

Again by linearity in the first slot,

$$ \overline{\langle u+v, u \rangle} + \overline{\langle u+v, v \rangle}=\overline{\langle u, u \rangle} + \overline{\langle v, u \rangle} + \overline{\langle u, v \rangle} + \overline{\langle v, v \rangle} $$

Again by conjugate symmetry,

$$ \overline{\langle u, u \rangle} + \overline{\langle v, u \rangle} + \overline{\langle u, v \rangle} + \overline{\langle v, v \rangle} = \langle u, u \rangle + \langle u, v \rangle + \langle v, u \rangle + \langle v, v \rangle $$

since $u,v\in V; u\bot v$:

$$ \langle u, v \rangle=0;\langle v, u \rangle=0 $$

By definition

$$ \langle u, u \rangle=\left\| u \right\|^2;\langle v, v \rangle=\left\| v \right\|^2 $$

Hence

$$\| u+v \|^2 = \| u \|^2 + \| v \|^2.$$

Triangle inequality

$$\forall u,v\in V,\; \left\| u+v \right\| \le \left\| u \right\| + \left\| v \right\|.$$

Proof:

$$ \begin{aligned} \left\| u+v \right\|^2 &=\langle u+v,u+v\rangle =\langle u,u\rangle+\langle v,v\rangle+\langle u,v\rangle+\langle v,u\rangle\\ &=\langle u,u\rangle+\langle v,v\rangle+\langle u,v\rangle+\overline{\langle u,v\rangle} =\left\| u \right\|^2+\left\| v \right\|^2+2\operatorname{Re}\langle u,v\rangle,\\ \text{Note }\operatorname{Re}\langle u,v\rangle &\le|\langle u,v\rangle| \stackrel{\text{Cauchy-Schwarz}}{\le}\left\| u \right\|\left\| v \right\|,\\ \left\| u \right\|^2+\left\| v \right\|^2+2\operatorname{Re}\langle u,v\rangle &\le\left\| u \right\|^2+\left\| v \right\|^2+2\left\| u \right\|\left\| v \right\| =(\left\| u \right\|+\left\| v \right\|)^2,\\ \Rightarrow \left\| u+v \right\|^2 \leq (\left\| u \right\|+\left\| v \right\|)^2,\\ \because \left\| . \right\| \geq 0 \\ \therefore \left\| u+v \right\|&\le\left\| u \right\|+\left\| v \right\| \end{aligned} $$