A Symmetric Positive Definite Matrix and An Inner Product on a Vector Space
Problem 538
(a) Suppose that $A$ is an $n\times n$ real symmetric positive definite matrix.
Prove that
\[\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans}A\mathbf{y}\]
defines an inner product on the vector space $\R^n$.
(b) Let $A$ be an $n\times n$ real matrix. Suppose that
\[\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans}A\mathbf{y}\]
defines an inner product on the vector space $\R^n$.
Prove that $A$ is symmetric and positive definite.
Sponsored Links
Definitions.
Inner Product on a Real Vector Space
Let $V$ be a real vector space. An inner product on $V$ is a function that assigns a real number $\langle \mathbf{u}, \mathbf{v}\rangle$ to each pair of vectors $\mathbf{u}$ and $\mathbf{v}$ in $V$ satisfying the following properties.
For any vectors $\mathbf{u}, \mathbf{v}, \mathbf{w}$ and a real number $r\in \R$,
- Symmetry \[\langle \mathbf{u}, \mathbf{v}\rangle=\langle \mathbf{v}, \mathbf{u}\rangle\]
- Linearity in the first argument
\begin{align*}
\langle r\mathbf{u}, \mathbf{v}\rangle &=r\langle \mathbf{u}, \mathbf{v}\rangle\\
\langle \mathbf{u}+ \mathbf{v}, \mathbf{w}\rangle &=\langle \mathbf{u}, \mathbf{w}\rangle+ \langle \mathbf{v}, \mathbf{w}\rangle
\end{align*} - Positive-Definiteness
\begin{align*}
\langle \mathbf{u}, \mathbf{u}\rangle &\geq 0 \\
\langle \mathbf{u}, \mathbf{u}\rangle &=0 \text{ if and only if } \mathbf{u}=\mathbf{0}
\end{align*}
A Positive-Definite Matrix
A real symmetric $n\times n$ matrix $A$ is called positive definite if $\mathbf{x}^{\trans}A\mathbf{x} > 0$ for each nonzero vector $\mathbf{x}\in \R^n$.
Proof.
(a) If $A$ is positive definite, then $\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans}A\mathbf{y}$ defines an inner product
First of all, note that we can write
\[\langle \mathbf{x}, \mathbf{y}\rangle=\mathbf{x}^{\trans}A\mathbf{y}=\mathbf{x}\cdot (A\mathbf{y}),\]
where the “dot” is the dot product of $\R^n$.
Thus, $\langle \mathbf{x}, \mathbf{y}\rangle$ is a real number.
We verify the three properties of an inner product.
Since the dot product is commutative, we have
\begin{align*}
\langle \mathbf{x}, \mathbf{y}\rangle&=\mathbf{x}\cdot (A\mathbf{y})= (A\mathbf{y})\cdot \mathbf{x}\\
&=(A\mathbf{y})^{\trans}\mathbf{x}=\mathbf{y}^{\trans}A^{\trans}\mathbf{x}\\
&=\mathbf{y}^{\trans}A\mathbf{x} &&\text{since $A$ is symmetric}\\
&=\langle \mathbf{y}, \mathbf{x} \rangle.
\end{align*}
Thus, the function $\langle\,,\,\rangle$ is symmetric.
Next, for any vectors $\mathbf{x}, \mathbf{y}, \mathbf{z}$ and any real number $r$, we have
\begin{align*}
\langle r\mathbf{x}, \mathbf{y}\rangle &=(r\mathbf{x})^{\trans}A\mathbf{y}=r\mathbf{x}^{\trans}A\mathbf{y}=r\langle \mathbf{x}, \mathbf{y}\rangle
\end{align*}
and
\begin{align*}
\langle \mathbf{x}+\mathbf{y}, \mathbf{z}\rangle &=(\mathbf{x}+\mathbf{y})^{\trans}A\mathbf{z}=(\mathbf{x}^{\trans}+\mathbf{y}^{\trans})A\mathbf{z}\\
&=\mathbf{x}^{\trans}A\mathbf{z}+\mathbf{y}^{\trans}A\mathbf{z}=\langle \mathbf{x}, \mathbf{z}\rangle+\langle \mathbf{y}, \mathbf{z}\rangle.
\end{align*}
Thus, the linearity in the first argument is satisfied.
If $\mathbf{x}$ is a nonzero vector in $\R^n$, then we have
\begin{align*}
\langle \mathbf{x}, \mathbf{x}\rangle=\mathbf{x}^{\trans}A\mathbf{x} > 0
\end{align*}
since $A$ is positive definite.
We also have
\begin{align*}
\langle \mathbf{0}, \mathbf{0}\rangle=\mathbf{0}^{\trans}A\mathbf{0}=0.
\end{align*}
It follows that $\langle \mathbf{x}, \mathbf{x}\rangle \geq 0$ for any vector $\mathbf{x}\in \R^n$.
Suppose that $\langle \mathbf{x}, \mathbf{x}\rangle=0$.
Then we have
\begin{align*}
\langle \mathbf{x}, \mathbf{x}\rangle=\mathbf{x}^{\trans}A\mathbf{x}=0.
\end{align*}
Since $A$ is positive definite, this happens if and only if $\mathbf{x}=\mathbf{0}$.
Hence $\langle \mathbf{x}, \mathbf{x}\rangle=0$ if and only if $\mathbf{x}=0$.
This proves the positive-definiteness of the function $\langle\,,\,\rangle$.
This completes the verification of the three properties, and hence $\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans}A\mathbf{y}$ defines an inner product on $\R^n$.
(b) If $\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans}A\mathbf{y}$ defines an inner product, then $A$ is symmetric positive definite
Suppose that $\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans}A\mathbf{y}$ is an inner product on $\R^n$.
Let us write $A=(a_{ij})$.
We first prove that $A$ is symmetric.
Let $\mathbf{e}_i$ denote the $i$-th standard unit vectors, that is,
\[\mathbf{e}_i=\begin{bmatrix}
0 \\
\vdots \\
1 \\
\vdots \\
0
\end{bmatrix},\]
where $1$ is on the $i$-th columns and other entries are all zero.
We compute
\begin{align*}
\langle \mathbf{e}_i, \mathbf{e}_j\rangle &=\mathbf{e}^{\trans}_iA\mathbf{e}_j\\
&=\begin{bmatrix}
0 & \dots &1 & \dots 0
\end{bmatrix}
\begin{bmatrix}
a_{1 j} \\
a_{2 j} \\
\vdots \\
a_{n j}
\end{bmatrix}=a_{ij}.
\end{align*}
Similarly, $\langle \mathbf{e}_j, \mathbf{e}_i\rangle =a_{j i}$. By symmetry of the inner product, it yields that
\[a_{ij}=\langle \mathbf{e}_i, \mathbf{e}_j\rangle =\langle \mathbf{e}_j, \mathbf{e}_i\rangle =a_{j i}.\]
Therefore, the matrix $A$ is symmetric.
Next, we show that $A$ is positive definite.
Let $\mathbf{x}$ be a nonzero vector in $\R^n$.
Then we have
\[\mathbf{x}^{\trans}A\mathbf{x}=\langle \mathbf{x}, \mathbf{x}\rangle > 0\]
by positive-definiteness property of the inner product.
This proves that $A$ is a positive definite matrix.
Related Question.
A concrete example of a positive-definite matrix is given in the next problem.
Consider the $2\times 2$ real matrix
\[A=\begin{bmatrix}
1 & 1\\
1& 3
\end{bmatrix}.\]
(a) Prove that the matrix $A$ is positive definite.
(b) Since $A$ is positive definite by part (a), the formula
\[\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans} A \mathbf{y}\]
for $\mathbf{x}, \mathbf{y} \in \R^2$ defines an inner product on $\R^n$.
Consider $\R^2$ as an inner product space with this inner product.
Prove that the unit vectors
\[\mathbf{e}_1=\begin{bmatrix}
1 \\
0
\end{bmatrix} \text{ and } \mathbf{e}_2=\begin{bmatrix}
0 \\
1
\end{bmatrix}\]
are not orthogonal in the inner product space $\R^2$.
(c) Find an orthogonal basis $\{\mathbf{v}_1, \mathbf{v}_2\}$ of $\R^2$ from the basis $\{\mathbf{e}_1, \mathbf{e}_2\}$ using the Gram-Schmidt orthogonalization process.
See the post ↴
The Inner Product on $\R^2$ induced by a Positive Definite Matrix and Gram-Schmidt Orthogonalization
for proofs.
Add to solve later
Sponsored Links
1 Response
[…] that by post “A Symmetric Positive Definite Matrix and An Inner Product on a Vector Space“, the formula $langle mathbf{x}, mathbf{y}rangle$ defines an inner product on […]