Orthogonality

Orthogonality generalises perpendicularity to an inner product space.

Mathematical definition

Let vector space $V$ be an inner product space.

Two vectors $u,v\in V$ are orthogonal if $\langle u, v \rangle=0$.

$u\bot v$ denotes this orthogonality.

Application

Orthogonal decomposition

Given vectors $u,v\in V$ and $v\neq 0$, $u$ can be uniquely decomposed into two pieces: one parallel to $v$ and one orthogonal to $v$. This is called an orthogonal decomposition.

In mathmetical terms:

$$u=u_1+u_2,$$

Where $u_1=a v, a \in \mathbb{F}; u_2\bot v$

Derivation:

$$\begin{gathered} \because u = u_1 +u_2 \\ \therefore u_2=u-u_1=u-av \\ \because u_2 \bot v \\ \therefore \langle u_2, v \rangle = \langle u- av, v \rangle = \langle u, v \rangle - a \left\| v \right\|^2 = \vec 0 \\ \text{Solve for }a,a = \frac{\langle u, v \rangle}{\left\| v \right\|^2} \\ \text{Substitute back into }u_1=av, u_2=u-av, \\ u=\frac{\langle u, v \rangle}{\| v \|^2} v + \left( u-\frac{\langle u, v \rangle}{\| v \|^2} v \right) \end{gathered}$$

Properties

$\vec 0$ is orthogonal to every vector

$$ \forall v \in V, \vec 0 \bot v $$

Proof:

Recall the properties of a vector space by definition. By additive identity,

$$ \vec 0 +\vec 0 = \vec 0 $$

Therefore,

$$ \langle \vec 0, u \rangle = \langle \vec 0+\vec 0, u\rangle $$

Recall the properties of the inner product by definition, by linearity in the 1st slot,

$$ \langle \vec 0+\vec 0, u\rangle = \langle\vec 0, u \rangle+\langle \vec 0, u\rangle $$

Subtracting $\langle \vec 0, u\rangle$ from both sides gives

$$ \langle \vec 0, u \rangle=0. $$

Generalised Pythagorean theorem

If $u,v\in V; u\bot v$, then $\| \cdot \|$ defined by $\| v \|:=\sqrt{\langle v, v \rangle}$ obeys

$$\| u+v \|^2 = \| u \|^2 + \| v \|^2.$$

Proof::

By definition,

$$ \| u+v \|^2 = \langle u+v, u+v \rangle $$

Recall the properties of the inner product by definition:

By linearity in the 1st slot,

$$ \langle u+v, u+v \rangle = \langle u, u+v \rangle + \langle v, u+v \rangle $$

By conjugate symmetry,

$$ \langle u, u+v \rangle + \langle u, u+v \rangle = \overline{\langle u+v, u \rangle} + \overline{\langle u+v, v \rangle} $$

Again by linearity in the first slot,

$$ \overline{\langle u+v, u \rangle} + \overline{\langle u+v, v \rangle}=\overline{\langle u, u \rangle} + \overline{\langle v, u \rangle} + \overline{\langle u, v \rangle} + \overline{\langle v, v \rangle} $$

Again by conjugate symmetry,

$$ \overline{\langle u, u \rangle} + \overline{\langle v, u \rangle} + \overline{\langle u, v \rangle} + \overline{\langle v, v \rangle} = \langle u, u \rangle + \langle u, v \rangle + \langle v, u \rangle + \langle v, v \rangle $$

since $u,v\in V; u\bot v$:

$$ \langle u, v \rangle=0;\langle v, u \rangle=0 $$

By definition

$$ \langle u, u \rangle=\left\| u \right\|^2;\langle v, v \rangle=\left\| v \right\|^2 $$

Hence

$$\| u+v \|^2 = \| u \|^2 + \| v \|^2.$$

Orthogonal vector list

A list of nonzero vectors $(e_1,\ldots,e_m) \in V$ is orthogonal if

$$ \langle e_i, e_j \rangle = 0 \Leftarrow 1\le i\neq j \le m $$