Orthogonality

Orthogonality generalises perpendicularity to an inner product space.

Mathematical definition

Let vector space $V$ be an inner product space.

Two vectors $u,v\in V$ are orthogonal if $\langle u, v \rangle=0$.

$u\bot v$ denotes this orthogonality.

Properties

$\vec 0$ is orthogonal to every vector

$$ \forall v \in V, \vec 0 \bot v $$

Proof:

$$ Recall the properties of a vector space by definition. By additive identity, $\vec 0 +\vec 0 = \vec 0$

Therefore,

$$ \langle \vec 0, u \rangle = \langle \vec 0+\vec 0, u\rangle $$

Recall the properties of the inner product by definition, by linearity in the 1st slot,

$$ \langle \vec 0+\vec 0, u\rangle = \langle\vec 0, u \rangle+\langle \vec 0, u\rangle $$

Subtracting $\langle \vec 0, u\rangle$ from both sides gives

$$ \langle \vec 0, u \rangle=0. $$

Generalised Pythagorean theorem

If $u,v\in V; u\bot v$, then $\| \cdot \|$ defined by $\| v \|:=\sqrt{\langle v, v \rangle}$ obeys

$$\| u+v \|^2 = \| u \|^2 + \| v \|^2.$$

Proof::

By definition,

$$ \| u+v \|^2 = \langle u+v, u+v \rangle $$

Recall the properties of the inner product by definition:

By linearity in the 1st slot,

$$ \langle u+v, u+v \rangle = \langle u, u+v \rangle + \langle v, u+v \rangle $$

By conjugate symmetry,

$$ \langle u, u+v \rangle + \langle u, u+v \rangle = \overline{\langle u+v, u \rangle} + \overline{\langle u+v, v \rangle} $$

Again by linearity in the first slot,

$$ \overline{\langle u+v, u \rangle} + \overline{\langle u+v, v \rangle}=\overline{\langle u, u \rangle} + \overline{\langle v, u \rangle} + \overline{\langle u, v \rangle} + \overline{\langle v, v \rangle} $$

Again by conjugate symmetry,

$$ \overline{\langle u, u \rangle} + \overline{\langle v, u \rangle} + \overline{\langle u, v \rangle} + \overline{\langle v, v \rangle} = \langle u, u \rangle + \langle u, v \rangle + \langle v, u \rangle + \langle v, v \rangle $$

since $u,v\in V; u\bot v$:

$$ \langle u, v \rangle=0;\langle v, u \rangle=0 $$

By definition

$$ \langle u, u \rangle=\left\| u \right\|^2;\langle v, v \rangle=\left\| v \right\|^2 $$

Hence

$$\| u+v \|^2 = \| u \|^2 + \| v \|^2.$$

Cauchy-Schwarz inequality

$$\forall u,v\in V,\; |\langle u,v\rangle|\le\|u\|\|v\|.$$

Furthermore,

$$|\langle u,v\rangle|=\|u\|\|v\|\iff \exists \lambda\in\mathbb{F},\; u=\lambda v$$

Proof:

If $v=0$,

$$\begin{gathered} |\langle u,v\rangle| = 0, \|u\|\|v\| = 0 \\ \therefore |\langle u,v\rangle| = \|u\|\|v\| \end{gathered}$$

If $v\neq 0$,

Consider the orthogonal decomposition

$$u = \frac{\langle u, v \rangle}{\| v \|^2} v + u_2$$

where $u_2\bot v$.

By the Pythagorean theorem,

$$\| u \|^2 = \left\| \frac{\langle u, v \rangle}{\| v \|^2}v \right\|^2 + \| u_2 \|^2 = \frac{|\langle u, v \rangle|^2}{\| v \|^2} + \| u_2 \|^2 $$

Since $v\neq 0$:

$$\begin{gathered} \frac{|\langle u, v \rangle|^2}{\| v \|^2} + \| u_2 \|^2 \ge \frac{|\langle u, v \rangle|^2}{\| v \|^2} \\ \Rightarrow \| u \|^2 \ge \frac{|\langle u, v \rangle|^2}{\| v \|^2} \\ \Rightarrow |\langle u,v\rangle|\le\|u\|\|v\|. \end{gathered}$$