Using Gram-Schmidt orthogonalization, find an orthogonal basis for the span of the vectors $\mathbf{w}_{1},\mathbf{w}_{2}\in\R^{3}$ if
\[
\mathbf{w}_{1}
=
\begin{bmatrix}
1 \\ 0 \\ 3
\end{bmatrix}
,\quad
\mathbf{w}_{2}
=
\begin{bmatrix}
2 \\ -1 \\ 0
\end{bmatrix}
.
\]

Solution.

We apply Gram-Schmidt orthogonalization as follows. The first step is to define $\mathbf{u}_{1}=\mathbf{w}_{1}$. Before defining $\mathbf{u}_{2}$, we must compute
\begin{align*}
\mathbf{u}_{1}^{T}\mathbf{w}_{2}
&=
\mathbf{w}_{1}^{T}\mathbf{w}_{2}
=
\begin{bmatrix}
1 & 0 & 3
\end{bmatrix}
\begin{bmatrix}
2 \\ -1 \\ 0
\end{bmatrix}
=2+0+0=2,
\\
\mathbf{u}_{1}^{T}\mathbf{u}_{1}
&=
\mathbf{w}_{1}^{T}\mathbf{w}_{1}
=
\begin{bmatrix}
1 & 0 & 3
\end{bmatrix}
\begin{bmatrix}
1 \\ 0 \\ 3
\end{bmatrix}
=1+0+9=10.
\end{align*}

Next, we define
\[
\mathbf{u}_{2}
=
\mathbf{w}_{2}
-\dfrac{\mathbf{u}_{1}^{T}\mathbf{w}_{2}}
{\mathbf{u}_{1}^{T}\mathbf{u}_{1}}
\mathbf{u}_{1}
=
\begin{bmatrix}
2 \\ -1 \\ 0
\end{bmatrix}
-\dfrac{2}{10}
\begin{bmatrix}
1 \\ 0 \\ 3
\end{bmatrix}
=
\begin{bmatrix}
10/5 \\ -1 \\ 0
\end{bmatrix}
–
\begin{bmatrix}
1/5 \\ 0 \\ 3/5
\end{bmatrix}
=
\begin{bmatrix}
9/5 \\ -1 \\ -3/5
\end{bmatrix}
.
\] By Gram-Schmidt orthogonalization, $\{\mathbf{u}_{1},\mathbf{u}_{2}\}$ is an orthogonal basis for the span of the vectors $\mathbf{w}_{1}$ and $\mathbf{w}_{2}$.

Remark

Note that since scalar multiplication by a nonzero number does not change the orthogonality of vectors and the new vectors still form a basis, we could have used $5\mathbf{u}_2$, instead of $\mathbf{u}_2$ to avoid a fraction in our computation.
We have
\[5\mathbf{u}_2=\begin{bmatrix}
10 \\
-5 \\
0
\end{bmatrix}-\begin{bmatrix}
1 \\
0 \\
3
\end{bmatrix}=\begin{bmatrix}
9 \\
-5 \\
-3
\end{bmatrix},\] and $\{\mathbf{u}_1, 5\mathbf{u}_2\}$ is an orthogonal basis for the span.

The post Using Gram-Schmidt Orthogonalization, Find an Orthogonal Basis for the Span first appeared on Problems in Mathematics.

Let $W$ be a subspace of $\R^4$ with a basis
\[\left\{\, \begin{bmatrix}
1 \\
0 \\
1 \\
1
\end{bmatrix}, \begin{bmatrix}
0 \\
1 \\
1 \\
1
\end{bmatrix} \,\right\}.\]

Find an orthonormal basis of $W$.

(The Ohio State University, Linear Algebra Midterm)
 

Solution.

Let
\[\mathbf{v}_1= \begin{bmatrix}
1 \\
0 \\
1 \\
1
\end{bmatrix}, \mathbf{v}_2=\begin{bmatrix}
0 \\
1 \\
1 \\
1
\end{bmatrix} .\] Note that they are not orthogonal as the dot product is
\[\mathbf{v}_1\cdot \mathbf{v}_2= \begin{bmatrix}
1 \\
0 \\
1 \\
1
\end{bmatrix}\cdot \begin{bmatrix}
0 \\
1 \\
1 \\
1
\end{bmatrix}=1\cdot0+ 0\cdot 1+ 1\cdot 1+1\cdot 1=2\neq 0.\]

Let us first find an orthogonal basis for $W$ by the Gram-Schmidt orthogonalization process.

Let $\mathbf{w}_1:=\mathbf{v}_1$.
Next, let $\mathbf{w}_2:=\mathbf{v}_2+a\mathbf{v}_1$, where $a$ is a scalar to be determined so that $\mathbf{w}_1\cdot \mathbf{w}_2=0$.
(You may also use the formula of the Gram-Schmidt orthogonalization.)

As $\mathbf{w}_1$ and $\mathbf{w}_2$ is orthogonal, we have
\begin{align*}
0&=\mathbf{w}_1\cdot \mathbf{w}_2=\mathbf{v}_1\cdot(\mathbf{v}_2+a\mathbf{v}_1)\\
&=\mathbf{v}_1\cdot\mathbf{v}_2+a\mathbf{v}_1\cdot\mathbf{v}_1\\
&=2+3a.
\end{align*}
It follows that $a=-2/3$ and
\[\mathbf{w}_2=\mathbf{v}_2-\frac{2}{3}\mathbf{v}_1=\begin{bmatrix}
0 \\
1 \\
1 \\
1
\end{bmatrix}-\frac{2}{3}\begin{bmatrix}
1 \\
0 \\
1 \\
1
\end{bmatrix}.\]

Now, to avoid fractions in our computation, let us consider $3\mathbf{w}_2$, instead of $\mathbf{w}_2$. Note that the scaling does not change the orthogonality.
We have
\[3\mathbf{w}_2=3\begin{bmatrix}
0 \\
1 \\
1 \\
1
\end{bmatrix}-2\begin{bmatrix}
1 \\
0 \\
1 \\
1
\end{bmatrix}=\begin{bmatrix}
-2\\ 3\\ 1 \\1
\end{bmatrix}.\]

Thus the set $\{\mathbf{w}_1, 3\mathbf{w}_2\}$ is an orthogonal basis for $W$.
However, the length of these vectors are not $1$ as we see
\begin{align*}
\|\mathbf{w}_1\|&=\sqrt{1^2+0^2+1^2+1^2}=\sqrt{3}\\
\|3\mathbf{w}_2\|&=\sqrt{(-2)^2+3^2+1^2+1^2}=\sqrt{15}.
\end{align*}

Now it suffices to normalize the vectors $\mathbf{w}_1, 3\mathbf{w}_2$ to obtain an orthonormal basis.
Therefore, the set
\[\left\{\, \frac{1}{\sqrt{3}} \begin{bmatrix}
1 \\
0 \\
1 \\
1
\end{bmatrix}, \, \frac{1}{\sqrt{15}}\begin{bmatrix}
-2\\ 3\\ 1 \\1
\end{bmatrix}\,\right\}.\] is an orthonormal basis for $W$.

Comment.

This is one of the midterm 2 exam problems for Linear Algebra (Math 2568) in Autumn 2017.

One common mistake is just to normalize the vectors by dividing them by their length $\sqrt{3}$.
The resulting vectors have length $1$, but they are not orthogonal.

Another mistake is that you just changed the numbers in the vectors so that they are orthogonal.
The issue here is that if you change the numbers randomly, then the new vectors might no longer belong to the subspace $W$.

The point of the Gram-Schmidt orthogonalization is that the process converts any basis for $W$ to an orthogonal basis for $W$.
The above solution didn’t use the full formula of the Gram-Schmidt orthogonalization. Of course, you may use the formula in the exam but you must remember it correctly.

List of Midterm 2 Problems for Linear Algebra (Math 2568) in Autumn 2017

1. Vector Space of 2 by 2 Traceless Matrices
2. Find an Orthonormal Basis of the Given Two Dimensional Vector Space ←The current problem
3. Are the Trigonometric Functions $\sin^2(x)$ and $\cos^2(x)$ Linearly Independent?
4. Find Bases for the Null Space, Range, and the Row Space of a $5\times 4$ Matrix
5. Matrix Representation, Rank, and Nullity of a Linear Transformation $T:\R^2\to \R^3$
6. Determine the Dimension of a Mysterious Vector Space From Coordinate Vectors
7. Find a Basis of the Subspace Spanned by Four Polynomials of Degree 3 or Less

The post Find an Orthonormal Basis of the Given Two Dimensional Vector Space first appeared on Problems in Mathematics.

Let $\mathbf{v}_1=\begin{bmatrix}
2/3 \\ 2/3 \\ 1/3
\end{bmatrix}$ be a vector in $\R^3$.

Find an orthonormal basis for $\R^3$ containing the vector $\mathbf{v}_1$.

 

The first solution uses the Gram-Schumidt orthogonalization process.
On the other hand, the second solution uses the cross product.

Solution 1 (The Gram-Schumidt Orthogonalization)

First of all, note that the length of the vector $\mathbf{v}_1$ is $1$ as
\[\|\mathbf{v}_1\|=\sqrt{\left(\frac{2}{3}\right)^2+\left(\frac{2}{3}\right)^2+\left(\frac{1}{3}\right)^2}=1.\]

We want to find two vectors $\mathbf{v}_2, \mathbf{v}_3$ such that $\{\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3\}$ is an orthonormal basis for $\R^3$.
The vectors $\mathbf{v}_2, \mathbf{v}_3$ must lie on the plane that is perpendicular to the vector $\mathbf{v}_1$.
So consider the subspace
\[W=\left \{\,\begin{bmatrix} x\\y\\z \end{bmatrix} \in \R^3 \quad \middle | \quad \begin{bmatrix} x\\y\\z \end{bmatrix} \cdot \begin{bmatrix}
2/3 \\ 2/3 \\ 1/3
\end{bmatrix}=0 \,\right\}\]

Note that $W$ consists of all vectors that are perpendicular to $\mathbf{v}_1$, hence $W$ is a plane that is perpendicular to $\mathbf{v}_1$.

The relation
\[ \begin{bmatrix} x\\y\\z \end{bmatrix} \cdot \begin{bmatrix}
2/3 \\ 2/3 \\ 1/3
\end{bmatrix}=0\] can be written as
\[\frac{2}{3}x+\frac{2}{3}y+\frac{1}{3}z=0,\] or equivalently
\[z=-2x-2y.\] Hence the vectors in $W$ can be written as
\[\begin{bmatrix} x\\y\\z \end{bmatrix}=\begin{bmatrix} x\\y\\-2x-2y \end{bmatrix}=x\begin{bmatrix}1\\0\\-2\end{bmatrix}+y\begin{bmatrix}0\\1\\-2\end{bmatrix}.\]

It follows that
\[\left\{\,\begin{bmatrix}1\\0\\-2\end{bmatrix}, \begin{bmatrix}0\\1\\-2\end{bmatrix} \,\right\}\] is a basis for the subspace $W$. Let us call these vectors $\mathbf{u}_1, \mathbf{u}_2$, respectively.

We apply the Gram-Schmidt orthogonalization to this basis $\{\mathbf{u}_1, \mathbf{u}_2\}$ and obtain an orthogonal basis as follows.
We do not change the first vector: let $\mathbf{w}_1=\mathbf{u}_1$.

Next, we set
\[\mathbf{w}_2=\mathbf{u}_2+a\mathbf{u}_1\] for some scalar $a$.
To determine $a$, we compute
\begin{align*}
0&=\mathbf{w}_1\cdot \mathbf{w}_2=\mathbf{u}_1\cdot(\mathbf{u}_2+a\mathbf{u}_1) \\
&=\mathbf{u}_1\cdot\mathbf{u}_2+a\mathbf{u}_1\cdot\mathbf{u}_1\\
&=(1\cdot 0+0\cdot 1+(-2)\cdot(-2))+a(1\cdot 1+ 0\cdot 0 +(-2)\cdot (-2)\\
&=4+5a.
\end{align*}
Hence, $a=-4/5$ and we obtain
\begin{align*}
\mathbf{w}_2&=\mathbf{u}_2-\frac{4}{5}\mathbf{u}_1\\[6pt] &=\begin{bmatrix}0\\1\\-2\end{bmatrix} -\frac{4}{5}\begin{bmatrix}1\\0\\-2\end{bmatrix}.
\end{align*}
(Note that you may use the Gram-Schumidt orthogonalization formula, instead of the above method.)

As scaling does not change the orthogonality, consider $5\mathbf{w}_2$, instead of $\mathbf{w}_2$ (to avoids fractions).
We have
\[5\mathbf{w}_2=\begin{bmatrix}0\\5\\-10\end{bmatrix} -4\begin{bmatrix}1\\0\\-2\end{bmatrix}=\begin{bmatrix}
-4\\ 5\\-2
\end{bmatrix}.\]

Therefore, $\{\mathbf{w}_1, 5\mathbf{w}_2\}$ is an orthogonal basis of $W$.

We obtain an orthonormal basis of $W$ by normalizing the length of these basis vectors.
As
\[\|\mathbf{w}_1\|=\sqrt{1^2+0^2+(-2)^2}=\sqrt{5}\] and
\[\|5\mathbf{w}_2\|=\sqrt{(-4)^2+5^2+(-2)^2}=\sqrt{45}=3\sqrt{5},\] the vectors
\[\mathbf{v}_2:=\frac{\mathbf{w}_1}{\|\mathbf{w}_1\|}=\frac{1}{\sqrt{5}}\begin{bmatrix}1\\0\\-2\end{bmatrix}\] and
\[\mathbf{v}_3:=\frac{5\mathbf{w}_2}{\|5\mathbf{w}_2\|}=\frac{1}{3\sqrt{5}}\begin{bmatrix}-4\\5\\-2\end{bmatrix}\] form an orthonormal basis of $W$.
Note that as the vectors $\mathbf{v}_2, \mathbf{v}_3$ lie in $W$, they are still perpendicular to the vector $\mathbf{v}_1$.

It follows that $\{\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3\}$ is an orthonomal set in $\R^3$, thus it is an orthonormal basis for $\R^3$.

Solution 2 (Cross Product)

Next, we solve the problem using the cross product.

Let $\mathbf{v}_1=\frac{1}{3}\mathbf{u}_1$, where $\mathbf{u}_1=\begin{bmatrix}
2 \\
2 \\
1
\end{bmatrix}$.
Our first goal is to find the vectors $\mathbf{u}_2$ and $\mathbf{u}_3$ such that $\{\mathbf{u}_1,\mathbf{u}_2, \mathbf{u}_3\}$ is an orthogonal basis for $\R^3$.

Let $\mathbf{x}=\begin{bmatrix}
x \\
y \\
z
\end{bmatrix}$ be a vector that is perpendicular to $\mathbf{u}_1$.
Then we have $\mathbf{x}\cdot \mathbf{u}_1=0$, and hence we have the relation
\[2x+2y+z=0.\] For example, the vector $\mathbf{u}_2:=\begin{bmatrix}
1 \\
0 \\
-2
\end{bmatrix}$ satisfies the relation, and hence $\mathbf{u}_2\cdot \mathbf{u}_1=0$.
(So far, it is not so different from Solution 1.)

Now, let us define the third vector $\mathbf{u}_3$ to be the cross product of $\mathbf{u}_1$ and $\mathbf{u}_2$:
\[\mathbf{u}_3:=\mathbf{u}_1\times \mathbf{u}_2=\begin{bmatrix}
2 \\
2 \\
1
\end{bmatrix}\times \begin{bmatrix}
1 \\
0 \\
-2
\end{bmatrix}=\begin{bmatrix}
-4 \\
5 \\
-2
\end{bmatrix}.\] By the property of the cross product, the vector $\mathbf{u}_3$ is perpendicular to both $\mathbf{u}_1, \mathbf{u}_2$.

Therefore, the set
\[\{\mathbf{u}_1, \mathbf{u}_2, \mathbf{u}_3\}=\left\{\,\begin{bmatrix}
2 \\
2 \\
1
\end{bmatrix}, \begin{bmatrix}
1 \\
0 \\
-2
\end{bmatrix}, \begin{bmatrix}
-4 \\
5 \\
-2
\end{bmatrix} \,\right\}\] is an orthogonal basis for $\R^3$ as it consists of three nonzero orthogonal vectors.

Finally, to obtain an orthonormal basis for $\R^2$, we just need to normalize the lengths of these vectors.
The lengths are
\begin{align*}
\|\mathbf{u}_1\|&=\sqrt{2^2+2^2+1^2}=\sqrt{9}=3 \\
\|\mathbf{u}_2\|&=\sqrt{1^2+0^2+(-2)^2}=\sqrt{5}\\
\|\mathbf{u}_3\|&=\sqrt{(-4)^2+5^2+(-2)^2}=\sqrt{45}=3\sqrt{5}.
\end{align*}
Then we have $\mathbf{v}_1=\frac{\mathbf{u}_1}{\|\mathbf{u}_1\|}$.
Let $\mathbf{v}_2:=\frac{\mathbf{u}_2}{\|\mathbf{u}_2\|}$ and $\mathbf{v}_3:=\frac{\mathbf{u}_3}{\|\mathbf{u}_3\|}$.

It follows that the set
\[\{\mathbf{v}_1\mathbf{v}_2\mathbf{v}_3\}=\left\{\, \frac{1}{3}\begin{bmatrix}
2 \\
2 \\
1
\end{bmatrix}, \, \frac{1}{\sqrt{5}}\begin{bmatrix}1\\0\\-2\end{bmatrix}, \, \frac{1}{3\sqrt{5}}\begin{bmatrix}-4\\5\\-2\end{bmatrix} \,\right\}\] is an orthonormal basis for $\R^3$.

The post Find an Orthonormal Basis of $\R^3$ Containing a Given Vector first appeared on Problems in Mathematics.

Consider the $2\times 2$ real matrix
\[A=\begin{bmatrix}
1 & 1\\
1& 3
\end{bmatrix}.\]

(a) Prove that the matrix $A$ is positive definite.

(b) Since $A$ is positive definite by part (a), the formula
\[\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans} A \mathbf{y}\] for $\mathbf{x}, \mathbf{y} \in \R^2$ defines an inner product on $\R^n$.
Consider $\R^2$ as an inner product space with this inner product.

Prove that the unit vectors
\[\mathbf{e}_1=\begin{bmatrix}
1 \\
0
\end{bmatrix} \text{ and } \mathbf{e}_2=\begin{bmatrix}
0 \\
1
\end{bmatrix}\] are not orthogonal in the inner product space $\R^2$.

(c) Find an orthogonal basis $\{\mathbf{v}_1, \mathbf{v}_2\}$ of $\R^2$ from the basis $\{\mathbf{e}_1, \mathbf{e}_2\}$ using the Gram-Schmidt orthogonalization process.

Proof.

(a) Prove that the matrix $A$ is positive definite.

We prove that for every nonzero vector $\mathbf{x}=\begin{bmatrix}
x \\
y
\end{bmatrix}\in \R^2$, we have $\mathbf{x}^{\trans} A \mathbf{x} > 0$.
We have
\begin{align*}
\mathbf{x}^{\trans} A \mathbf{x}&=\begin{bmatrix}
x & y
\end{bmatrix} \begin{bmatrix}
1 & 1\\
1& 3
\end{bmatrix}\begin{bmatrix}
x \\
y
\end{bmatrix}=\begin{bmatrix}
x & y
\end{bmatrix}\begin{bmatrix}
x+y \\
x+3y
\end{bmatrix}\\[6pt] &=x(x+y)+y(x+3y)=x^2+2xy+3y^2\\
&=x^2+2xy+y^2+2y^2=(x+y)^2+2y^2.
\end{align*}

Since $\mathbf{x}\neq \mathbf{0}$, at least one of $x, y$ is nonzero.
Thus the last expression is always positive.
Hence $A$ is a positive definite matrix.

(b) Prove that $\mathbf{e}_1, \mathbf{e}_2$ are not orthogonal in the inner product space $\R^2$.

Note that by post “A Symmetric Positive Definite Matrix and An Inner Product on a Vector Space“, the formula $\langle \mathbf{x}, \mathbf{y}\rangle$ defines an inner product on $\R^2$.

Two vectors $\mathbf{x}$ and $\mathbf{y}$ is said to be orthogonal if $\langle \mathbf{x}, \mathbf{y}\rangle=0$.

The vectors $\mathbf{e}_1, \mathbf{e}_2$ are not orthogonal with this inner product since
\begin{align*}
\langle \mathbf{e}_1, \mathbf{e}_2\rangle=\begin{bmatrix}
1 & 0
\end{bmatrix}\begin{bmatrix}
1 & 1\\
1& 3
\end{bmatrix}\begin{bmatrix}
0 \\
1
\end{bmatrix}=\begin{bmatrix}
1 & 0
\end{bmatrix}\begin{bmatrix}
1 \\
3
\end{bmatrix}=1\neq 0.
\end{align*}

(c) Find an orthogonal basis using the Gram-Schmidt orthogonalization process.

By the Gram-Schmidt orthogonalization process, we have
\begin{align*}
\mathbf{v}_1&=\mathbf{e}_1\\
\mathbf{v}_2&=\mathbf{e}_2-\frac{\langle \mathbf{v}_1, \mathbf{e}_2 \rangle}{\langle \mathbf{v}_1, \mathbf{v}_1 \rangle}\mathbf{v}_1
=\mathbf{e}_2-\frac{\langle \mathbf{e}_1, \mathbf{e}_2 \rangle}{\langle \mathbf{e}_1, \mathbf{e}_1 \rangle}\mathbf{e}_1.
\end{align*}

We compute
\begin{align*}
\langle \mathbf{e}_1, \mathbf{e}_1 \rangle=\begin{bmatrix}
1 & 0
\end{bmatrix}\begin{bmatrix}
1 & 1\\
1& 3
\end{bmatrix}\begin{bmatrix}
1 \\
0
\end{bmatrix}=\begin{bmatrix}
1 & 0
\end{bmatrix}\begin{bmatrix}
1 \\
1
\end{bmatrix}=1.
\end{align*}
We also have $\langle \mathbf{e}_1, \mathbf{e}_2\rangle=1$ from part (b).
Thus, we have
\begin{align*}
\mathbf{v}_2=\mathbf{e}_2-\mathbf{e}_1=\begin{bmatrix}
-1 \\
1
\end{bmatrix}.
\end{align*}
Thus, the Gram-Schmidt orthogonalization process yields the orthogonal basis
\[\mathbf{v}_1=\begin{bmatrix}
1 \\
0
\end{bmatrix}, \mathbf{v}_2=\begin{bmatrix}
-1 \\
1
\end{bmatrix}.\]

Double Check

Let us verify that $\mathbf{v}_1, \mathbf{v}_2$ are orthogonal by computing their inner product directly as follows.
We have
\begin{align*}
\langle \mathbf{v}_1, \mathbf{v}_2\rangle=\mathbf{v}_1^{\trans} A\mathbf{v}_2=\begin{bmatrix}
1 & 0
\end{bmatrix}\begin{bmatrix}
1 & 1\\
1& 3
\end{bmatrix}\begin{bmatrix}
-1 \\
1
\end{bmatrix}=\begin{bmatrix}
1 & 0
\end{bmatrix}\begin{bmatrix}
0 \\
2
\end{bmatrix}=0.
\end{align*}

The post The Inner Product on $\R^2$ induced by a Positive Definite Matrix and Gram-Schmidt Orthogonalization first appeared on Problems in Mathematics.