Let $\R^n$ be an inner product space with inner product $\langle \mathbf{x}, \mathbf{y}\rangle=\mathbf{x}^{\trans}\mathbf{y}$ for $\mathbf{x}, \mathbf{y}\in \R^n$.

A linear transformation $T:\R^n \to \R^n$ is called orthogonal transformation if for all $\mathbf{x}, \mathbf{y}\in \R^n$, it satisfies
\[\langle T(\mathbf{x}), T(\mathbf{y})\rangle=\langle\mathbf{x}, \mathbf{y} \rangle.\]

Prove that if $T:\R^n\to \R^n$ is an orthogonal transformation, then $T$ is an isomorphism.

 

We give two proofs.
The second one uses a fact about the injectivity of linear transformations.

Proof 1.

As $T$ is a linear transformation from $\R^n$ to itself, it suffices to show that $T$ is an injective linear transformation.

Suppose that $T(\mathbf{x})=T(\mathbf{y})$ for $\mathbf{x}, \mathbf{y}\in \R^n$.
We show that $\mathbf{x}=\mathbf{y}$.

We have
\begin{align*}
&\|\mathbf{x}-\mathbf{y}\|^2\\
&=(\mathbf{x}-\mathbf{y})^{\trans}(\mathbf{x}-\mathbf{y})\\
&=(\mathbf{x}^{\trans}-\mathbf{y}^{\trans})(\mathbf{x}-\mathbf{y})\\
&=\mathbf{x}^{\trans}\mathbf{x}-\mathbf{x}^{\trans}\mathbf{y}-\mathbf{y}^{\trans}\mathbf{x}+\mathbf{y}^{\trans}\mathbf{y}\\
&=\langle\mathbf{x}, \mathbf{x} \rangle-\langle\mathbf{x}, \mathbf{y} \rangle-\langle\mathbf{y}, \mathbf{x} \rangle+\langle\mathbf{y}, \mathbf{y} \rangle\\
&=\langle T(\mathbf{x}), T(\mathbf{x}) \rangle-\langle T(\mathbf{x}), T(\mathbf{y}) \rangle-\langle T(\mathbf{y}), T(\mathbf{x}) \rangle+\langle T(\mathbf{y}), T(\mathbf{y}) \rangle\\
&\text{(since $T$ is an orthogonal transformation)}\\
&=\langle T(\mathbf{x}), T(\mathbf{x}) \rangle-\langle T(\mathbf{x}), T(\mathbf{x}) \rangle-\langle T(\mathbf{x}), T(\mathbf{x}) \rangle+\langle T(\mathbf{x}), T(\mathbf{x}) \rangle\\
&\text{(since $T(\mathbf{x})=T(\mathbf{y})$)}\\
&=0.
\end{align*}
It follows that $\|\mathbf{x}-\mathbf{y}\|=0$ and hence $\mathbf{x}=\mathbf{y}$.
This proves that $T:\R^n\to \R^n$ is injective.

As $T$ is an injective linear transformation from the $n$-dimensional vector space $\R^n$ to itself, it is also surjective, and thus $T$ is an isomorphism.

Proof 2.

Recall that the linear transformation $T$ is injective if and only if the null space $\calN(T)=\{\mathbf{0}\}$, that is, $T(\mathbf{x})=\mathbf{0}$ implies that $\mathbf{x}=\mathbf{0}$.
(See the post “A Linear Transformation is Injective (One-To-One) if and only if the Nullity is Zero” for the proof of this fact.)

We use this fact to show that $T$ is injective.
Suppose that $T(\mathbf{x})=\mathbf{0}$.
Then we have
\begin{align*}
\|\mathbf{x}\|^2&=\langle \mathbf{x}, \mathbf{x}\rangle\\
&=\langle T(\mathbf{x}), T(\mathbf{x})\rangle &&\text{as $T$ is orthogonal}\\
&=\langle \mathbf{0}, \mathbf{0}\rangle=0 &&\text{as $T(\mathbf{x})=\mathbf{0}$}.
\end{align*}

It follows that the length $\|\mathbf{x}\|=0$, and hence $\mathbf{x}=\mathbf{0}$.
This proves that the null space $\calN(T)=\{\mathbf{0}\}$ and $T$ is injective.

As $T$ is an injective linear transformation from $\R^n$ to itself, it is also surjective, and hence $T$ is an isomorphism.

The post An Orthogonal Transformation from $\R^n$ to $\R^n$ is an Isomorphism first appeared on Problems in Mathematics.

Let $\mathbf{v}$ be a vector in an inner product space $V$ over $\R$.
Suppose that $\{\mathbf{u}_1, \dots, \mathbf{u}_n\}$ is an orthonormal basis of $V$.
Let $\theta_i$ be the angle between $\mathbf{v}$ and $\mathbf{u}_i$ for $i=1,\dots, n$.

Prove that
\[\cos ^2\theta_1+\cdots+\cos^2 \theta_n=1.\]

Definition (Angle between Vectors).

Let $\langle\mathbf{a}, \mathbf{b}\rangle$ denote the inner product of vectors $\mathbf{a}$ and $\mathbf{b}$ in $V$.

Recall that the angle $\theta$ between $\mathbf{a}$ and $\mathbf{b}$ is defined as the unique number $\theta$ between $0$ and $\pi$ satisfying
\[\cos \theta=\frac{\langle\mathbf{a}, \mathbf{b}\rangle}{\|\mathbf{a}\| \|\mathbf{b}\|}.\]

Proof.

Express the vector $\mathbf{v}$ as a linear combination of the basis vectors as
\[\mathbf{v}=a_1\mathbf{u}_1+\dots+a_n\mathbf{u}_n\] for some real numbers $a_1, \dots, a_n$.

The length of the vector $\mathbf{v}$ is given by
\[\|\mathbf{v}\|=\sqrt{a_1^2+\cdots+a_n^2}. \tag{*}\]

For each $i$, we have using the properties of the inner product
\begin{align*}
\langle \mathbf{v}, \mathbf{u}_i\rangle&=\langle a_1\mathbf{u}_1+\dots+a_n\mathbf{u}_n, \mathbf{u}_i\rangle\\
&=a_1\langle\mathbf{u}_1, \mathbf{u}_i\rangle+\cdots +a_n \langle\mathbf{u}_n, \mathbf{u}_i \rangle\\
&=a_i \tag{**}
\end{align*}
since $\langle\mathbf{u}_i, \mathbf{u}_i\rangle=1$ and $\langle\mathbf{u}_j, \mathbf{u}_i\rangle=0$ if $j\neq i$ as $\{\mathbf{u}_1, \dots, \mathbf{u}_n\}$ is orthonormal.

By definition of the angle, we have
\begin{align*}
\cos \theta_i&=\frac{\langle\mathbf{v}, \mathbf{u}_i\rangle}{\|\mathbf{v}\| \|\mathbf{u}_i\|}=\frac{\langle\mathbf{v}, \mathbf{u}_i\rangle}{\|\mathbf{v}\| } && \text{since $\|\mathbf{u}_i\|=1$.}
\end{align*}
It follows that
\begin{align*}
\cos ^2\theta_1+\cdots+\cos^2 \theta_n &=\frac{\langle\mathbf{v}, \mathbf{u}_1\rangle^2}{\|\mathbf{v}\|^2 }+\cdots+\frac{\langle\mathbf{v}, \mathbf{u}_n\rangle^2}{\|\mathbf{v}\|^2 }\\[6pt] &=\frac{1}{\|\mathbf{v}\|^2}(a_1^2+\cdots a_n^2) &&\text{by (**)}\\[6pt] &=\frac{1}{\|\mathbf{v}\|^2}\cdot \|\mathbf{v}\|^2 &&\text{by (*)}\\[6pt] &=1.
\end{align*}

Thus we obtain
\[\cos ^2\theta_1+\cdots+\cos^2 \theta_n=1\] as required.

The post The Sum of Cosine Squared in an Inner Product Space first appeared on Problems in Mathematics.

Consider the $2\times 2$ real matrix
\[A=\begin{bmatrix}
1 & 1\\
1& 3
\end{bmatrix}.\]

(a) Prove that the matrix $A$ is positive definite.

(b) Since $A$ is positive definite by part (a), the formula
\[\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans} A \mathbf{y}\] for $\mathbf{x}, \mathbf{y} \in \R^2$ defines an inner product on $\R^n$.
Consider $\R^2$ as an inner product space with this inner product.

Prove that the unit vectors
\[\mathbf{e}_1=\begin{bmatrix}
1 \\
0
\end{bmatrix} \text{ and } \mathbf{e}_2=\begin{bmatrix}
0 \\
1
\end{bmatrix}\] are not orthogonal in the inner product space $\R^2$.

(c) Find an orthogonal basis $\{\mathbf{v}_1, \mathbf{v}_2\}$ of $\R^2$ from the basis $\{\mathbf{e}_1, \mathbf{e}_2\}$ using the Gram-Schmidt orthogonalization process.

Proof.

(a) Prove that the matrix $A$ is positive definite.

We prove that for every nonzero vector $\mathbf{x}=\begin{bmatrix}
x \\
y
\end{bmatrix}\in \R^2$, we have $\mathbf{x}^{\trans} A \mathbf{x} > 0$.
We have
\begin{align*}
\mathbf{x}^{\trans} A \mathbf{x}&=\begin{bmatrix}
x & y
\end{bmatrix} \begin{bmatrix}
1 & 1\\
1& 3
\end{bmatrix}\begin{bmatrix}
x \\
y
\end{bmatrix}=\begin{bmatrix}
x & y
\end{bmatrix}\begin{bmatrix}
x+y \\
x+3y
\end{bmatrix}\\[6pt] &=x(x+y)+y(x+3y)=x^2+2xy+3y^2\\
&=x^2+2xy+y^2+2y^2=(x+y)^2+2y^2.
\end{align*}

Since $\mathbf{x}\neq \mathbf{0}$, at least one of $x, y$ is nonzero.
Thus the last expression is always positive.
Hence $A$ is a positive definite matrix.

(b) Prove that $\mathbf{e}_1, \mathbf{e}_2$ are not orthogonal in the inner product space $\R^2$.

Note that by post “A Symmetric Positive Definite Matrix and An Inner Product on a Vector Space“, the formula $\langle \mathbf{x}, \mathbf{y}\rangle$ defines an inner product on $\R^2$.

Two vectors $\mathbf{x}$ and $\mathbf{y}$ is said to be orthogonal if $\langle \mathbf{x}, \mathbf{y}\rangle=0$.

The vectors $\mathbf{e}_1, \mathbf{e}_2$ are not orthogonal with this inner product since
\begin{align*}
\langle \mathbf{e}_1, \mathbf{e}_2\rangle=\begin{bmatrix}
1 & 0
\end{bmatrix}\begin{bmatrix}
1 & 1\\
1& 3
\end{bmatrix}\begin{bmatrix}
0 \\
1
\end{bmatrix}=\begin{bmatrix}
1 & 0
\end{bmatrix}\begin{bmatrix}
1 \\
3
\end{bmatrix}=1\neq 0.
\end{align*}

(c) Find an orthogonal basis using the Gram-Schmidt orthogonalization process.

By the Gram-Schmidt orthogonalization process, we have
\begin{align*}
\mathbf{v}_1&=\mathbf{e}_1\\
\mathbf{v}_2&=\mathbf{e}_2-\frac{\langle \mathbf{v}_1, \mathbf{e}_2 \rangle}{\langle \mathbf{v}_1, \mathbf{v}_1 \rangle}\mathbf{v}_1
=\mathbf{e}_2-\frac{\langle \mathbf{e}_1, \mathbf{e}_2 \rangle}{\langle \mathbf{e}_1, \mathbf{e}_1 \rangle}\mathbf{e}_1.
\end{align*}

We compute
\begin{align*}
\langle \mathbf{e}_1, \mathbf{e}_1 \rangle=\begin{bmatrix}
1 & 0
\end{bmatrix}\begin{bmatrix}
1 & 1\\
1& 3
\end{bmatrix}\begin{bmatrix}
1 \\
0
\end{bmatrix}=\begin{bmatrix}
1 & 0
\end{bmatrix}\begin{bmatrix}
1 \\
1
\end{bmatrix}=1.
\end{align*}
We also have $\langle \mathbf{e}_1, \mathbf{e}_2\rangle=1$ from part (b).
Thus, we have
\begin{align*}
\mathbf{v}_2=\mathbf{e}_2-\mathbf{e}_1=\begin{bmatrix}
-1 \\
1
\end{bmatrix}.
\end{align*}
Thus, the Gram-Schmidt orthogonalization process yields the orthogonal basis
\[\mathbf{v}_1=\begin{bmatrix}
1 \\
0
\end{bmatrix}, \mathbf{v}_2=\begin{bmatrix}
-1 \\
1
\end{bmatrix}.\]

Double Check

Let us verify that $\mathbf{v}_1, \mathbf{v}_2$ are orthogonal by computing their inner product directly as follows.
We have
\begin{align*}
\langle \mathbf{v}_1, \mathbf{v}_2\rangle=\mathbf{v}_1^{\trans} A\mathbf{v}_2=\begin{bmatrix}
1 & 0
\end{bmatrix}\begin{bmatrix}
1 & 1\\
1& 3
\end{bmatrix}\begin{bmatrix}
-1 \\
1
\end{bmatrix}=\begin{bmatrix}
1 & 0
\end{bmatrix}\begin{bmatrix}
0 \\
2
\end{bmatrix}=0.
\end{align*}

The post The Inner Product on $\R^2$ induced by a Positive Definite Matrix and Gram-Schmidt Orthogonalization first appeared on Problems in Mathematics.