A Symmetric Positive Definite Matrix and An Inner Product on a Vector Space

Linear Algebra Problems and Solutions

Problem 538

(a) Suppose that $A$ is an $n\times n$ real symmetric positive definite matrix.
Prove that
\[\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans}A\mathbf{y}\] defines an inner product on the vector space $\R^n$.

(b) Let $A$ be an $n\times n$ real matrix. Suppose that
\[\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans}A\mathbf{y}\] defines an inner product on the vector space $\R^n$.

Prove that $A$ is symmetric and positive definite.

 
LoadingAdd to solve later

Sponsored Links


Definitions.

Inner Product on a Real Vector Space

Let $V$ be a real vector space. An inner product on $V$ is a function that assigns a real number $\langle \mathbf{u}, \mathbf{v}\rangle$ to each pair of vectors $\mathbf{u}$ and $\mathbf{v}$ in $V$ satisfying the following properties.
For any vectors $\mathbf{u}, \mathbf{v}, \mathbf{w}$ and a real number $r\in \R$,

  • Symmetry \[\langle \mathbf{u}, \mathbf{v}\rangle=\langle \mathbf{v}, \mathbf{u}\rangle\]
  • Linearity in the first argument
    \begin{align*}
    \langle r\mathbf{u}, \mathbf{v}\rangle &=r\langle \mathbf{u}, \mathbf{v}\rangle\\
    \langle \mathbf{u}+ \mathbf{v}, \mathbf{w}\rangle &=\langle \mathbf{u}, \mathbf{w}\rangle+ \langle \mathbf{v}, \mathbf{w}\rangle
    \end{align*}
  • Positive-Definiteness
    \begin{align*}
    \langle \mathbf{u}, \mathbf{u}\rangle &\geq 0 \\
    \langle \mathbf{u}, \mathbf{u}\rangle &=0 \text{ if and only if } \mathbf{u}=\mathbf{0}
    \end{align*}

A Positive-Definite Matrix

A real symmetric $n\times n$ matrix $A$ is called positive definite if $\mathbf{x}^{\trans}A\mathbf{x} > 0$ for each nonzero vector $\mathbf{x}\in \R^n$.

Proof.

(a) If $A$ is positive definite, then $\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans}A\mathbf{y}$ defines an inner product

First of all, note that we can write
\[\langle \mathbf{x}, \mathbf{y}\rangle=\mathbf{x}^{\trans}A\mathbf{y}=\mathbf{x}\cdot (A\mathbf{y}),\] where the “dot” is the dot product of $\R^n$.
Thus, $\langle \mathbf{x}, \mathbf{y}\rangle$ is a real number.


We verify the three properties of an inner product.
Since the dot product is commutative, we have
\begin{align*}
\langle \mathbf{x}, \mathbf{y}\rangle&=\mathbf{x}\cdot (A\mathbf{y})= (A\mathbf{y})\cdot \mathbf{x}\\
&=(A\mathbf{y})^{\trans}\mathbf{x}=\mathbf{y}^{\trans}A^{\trans}\mathbf{x}\\
&=\mathbf{y}^{\trans}A\mathbf{x} &&\text{since $A$ is symmetric}\\
&=\langle \mathbf{y}, \mathbf{x} \rangle.
\end{align*}
Thus, the function $\langle\,,\,\rangle$ is symmetric.


Next, for any vectors $\mathbf{x}, \mathbf{y}, \mathbf{z}$ and any real number $r$, we have
\begin{align*}
\langle r\mathbf{x}, \mathbf{y}\rangle &=(r\mathbf{x})^{\trans}A\mathbf{y}=r\mathbf{x}^{\trans}A\mathbf{y}=r\langle \mathbf{x}, \mathbf{y}\rangle
\end{align*}
and
\begin{align*}
\langle \mathbf{x}+\mathbf{y}, \mathbf{z}\rangle &=(\mathbf{x}+\mathbf{y})^{\trans}A\mathbf{z}=(\mathbf{x}^{\trans}+\mathbf{y}^{\trans})A\mathbf{z}\\
&=\mathbf{x}^{\trans}A\mathbf{z}+\mathbf{y}^{\trans}A\mathbf{z}=\langle \mathbf{x}, \mathbf{z}\rangle+\langle \mathbf{y}, \mathbf{z}\rangle.
\end{align*}
Thus, the linearity in the first argument is satisfied.


If $\mathbf{x}$ is a nonzero vector in $\R^n$, then we have
\begin{align*}
\langle \mathbf{x}, \mathbf{x}\rangle=\mathbf{x}^{\trans}A\mathbf{x} > 0
\end{align*}
since $A$ is positive definite.
We also have
\begin{align*}
\langle \mathbf{0}, \mathbf{0}\rangle=\mathbf{0}^{\trans}A\mathbf{0}=0.
\end{align*}
It follows that $\langle \mathbf{x}, \mathbf{x}\rangle \geq 0$ for any vector $\mathbf{x}\in \R^n$.


Suppose that $\langle \mathbf{x}, \mathbf{x}\rangle=0$.
Then we have
\begin{align*}
\langle \mathbf{x}, \mathbf{x}\rangle=\mathbf{x}^{\trans}A\mathbf{x}=0.
\end{align*}
Since $A$ is positive definite, this happens if and only if $\mathbf{x}=\mathbf{0}$.
Hence $\langle \mathbf{x}, \mathbf{x}\rangle=0$ if and only if $\mathbf{x}=0$.
This proves the positive-definiteness of the function $\langle\,,\,\rangle$.


This completes the verification of the three properties, and hence $\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans}A\mathbf{y}$ defines an inner product on $\R^n$.

(b) If $\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans}A\mathbf{y}$ defines an inner product, then $A$ is symmetric positive definite

Suppose that $\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans}A\mathbf{y}$ is an inner product on $\R^n$.
Let us write $A=(a_{ij})$.


We first prove that $A$ is symmetric.
Let $\mathbf{e}_i$ denote the $i$-th standard unit vectors, that is,
\[\mathbf{e}_i=\begin{bmatrix}
0 \\
\vdots \\
1 \\
\vdots \\
0
\end{bmatrix},\] where $1$ is on the $i$-th columns and other entries are all zero.


We compute
\begin{align*}
\langle \mathbf{e}_i, \mathbf{e}_j\rangle &=\mathbf{e}^{\trans}_iA\mathbf{e}_j\\
&=\begin{bmatrix}
0 & \dots &1 & \dots 0
\end{bmatrix}
\begin{bmatrix}
a_{1 j} \\
a_{2 j} \\
\vdots \\
a_{n j}
\end{bmatrix}=a_{ij}.
\end{align*}
Similarly, $\langle \mathbf{e}_j, \mathbf{e}_i\rangle =a_{j i}$. By symmetry of the inner product, it yields that
\[a_{ij}=\langle \mathbf{e}_i, \mathbf{e}_j\rangle =\langle \mathbf{e}_j, \mathbf{e}_i\rangle =a_{j i}.\] Therefore, the matrix $A$ is symmetric.


Next, we show that $A$ is positive definite.
Let $\mathbf{x}$ be a nonzero vector in $\R^n$.
Then we have
\[\mathbf{x}^{\trans}A\mathbf{x}=\langle \mathbf{x}, \mathbf{x}\rangle > 0\] by positive-definiteness property of the inner product.
This proves that $A$ is a positive definite matrix.

Related Question.

A concrete example of a positive-definite matrix is given in the next problem.

Problem.
Consider the $2\times 2$ real matrix
\[A=\begin{bmatrix}
1 & 1\\
1& 3
\end{bmatrix}.\]

(a) Prove that the matrix $A$ is positive definite.

(b) Since $A$ is positive definite by part (a), the formula
\[\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans} A \mathbf{y}\] for $\mathbf{x}, \mathbf{y} \in \R^2$ defines an inner product on $\R^n$.
Consider $\R^2$ as an inner product space with this inner product.

Prove that the unit vectors
\[\mathbf{e}_1=\begin{bmatrix}
1 \\
0
\end{bmatrix} \text{ and } \mathbf{e}_2=\begin{bmatrix}
0 \\
1
\end{bmatrix}\] are not orthogonal in the inner product space $\R^2$.

(c) Find an orthogonal basis $\{\mathbf{v}_1, \mathbf{v}_2\}$ of $\R^2$ from the basis $\{\mathbf{e}_1, \mathbf{e}_2\}$ using the Gram-Schmidt orthogonalization process.

See the post ↴
The Inner Product on $\R^2$ induced by a Positive Definite Matrix and Gram-Schmidt Orthogonalization
for proofs.


LoadingAdd to solve later

Sponsored Links

More from my site

  • Transpose of a Matrix and Eigenvalues and Related QuestionsTranspose of a Matrix and Eigenvalues and Related Questions Let $A$ be an $n \times n$ real matrix. Prove the followings. (a) The matrix $AA^{\trans}$ is a symmetric matrix. (b) The set of eigenvalues of $A$ and the set of eigenvalues of $A^{\trans}$ are equal. (c) The matrix $AA^{\trans}$ is non-negative definite. (An $n\times n$ […]
  • Positive definite Real Symmetric Matrix and its EigenvaluesPositive definite Real Symmetric Matrix and its Eigenvalues A real symmetric $n \times n$ matrix $A$ is called positive definite if \[\mathbf{x}^{\trans}A\mathbf{x}>0\] for all nonzero vectors $\mathbf{x}$ in $\R^n$. (a) Prove that the eigenvalues of a real symmetric positive-definite matrix $A$ are all positive. (b) Prove that if […]
  • Inverse Matrix of Positive-Definite Symmetric Matrix is Positive-DefiniteInverse Matrix of Positive-Definite Symmetric Matrix is Positive-Definite Suppose $A$ is a positive definite symmetric $n\times n$ matrix. (a) Prove that $A$ is invertible. (b) Prove that $A^{-1}$ is symmetric. (c) Prove that $A^{-1}$ is positive-definite. (MIT, Linear Algebra Exam Problem)   Proof. (a) Prove that $A$ is […]
  • Questions About the Trace of a MatrixQuestions About the Trace of a Matrix Let $A=(a_{i j})$ and $B=(b_{i j})$ be $n\times n$ real matrices for some $n \in \N$. Then answer the following questions about the trace of a matrix. (a) Express $\tr(AB^{\trans})$ in terms of the entries of the matrices $A$ and $B$. Here $B^{\trans}$ is the transpose matrix of […]
  • The set of $2\times 2$ Symmetric Matrices is a SubspaceThe set of $2\times 2$ Symmetric Matrices is a Subspace Let $V$ be the vector space over $\R$ of all real $2\times 2$ matrices. Let $W$ be the subset of $V$ consisting of all symmetric matrices. (a) Prove that $W$ is a subspace of $V$. (b) Find a basis of $W$. (c) Determine the dimension of $W$.   Proof. […]
  • The Intersection of Two Subspaces is also a SubspaceThe Intersection of Two Subspaces is also a Subspace Let $U$ and $V$ be subspaces of the $n$-dimensional vector space $\R^n$. Prove that the intersection $U\cap V$ is also a subspace of $\R^n$.   Definition (Intersection). Recall that the intersection $U\cap V$ is the set of elements that are both elements of $U$ […]
  • Sequences Satisfying Linear Recurrence Relation Form a SubspaceSequences Satisfying Linear Recurrence Relation Form a Subspace Let $V$ be a real vector space of all real sequences \[(a_i)_{i=1}^{\infty}=(a_1, a_2, \cdots).\] Let $U$ be the subset of $V$ defined by \[U=\{ (a_i)_{i=1}^{\infty} \in V \mid a_{k+2}-5a_{k+1}+3a_{k}=0, k=1, 2, \dots \}.\] Prove that $U$ is a subspace of […]
  • Every $n$-Dimensional Vector Space is Isomorphic to the Vector Space $\R^n$Every $n$-Dimensional Vector Space is Isomorphic to the Vector Space $\R^n$ Let $V$ be a vector space over the field of real numbers $\R$. Prove that if the dimension of $V$ is $n$, then $V$ is isomorphic to $\R^n$.   Proof. Since $V$ is an $n$-dimensional vector space, it has a basis \[B=\{\mathbf{v}_1, \dots, […]

You may also like...

1 Response

  1. 08/16/2017

    […] that by post “A Symmetric Positive Definite Matrix and An Inner Product on a Vector Space“, the formula $langle mathbf{x}, mathbf{y}rangle$ defines an inner product on […]

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

More in Linear Algebra
Linear algebra problems and solutions
True or False: If $A, B$ are 2 by 2 Matrices such that $(AB)^2=O$, then $(BA)^2=O$

Let $A$ and $B$ be $2\times 2$ matrices such that $(AB)^2=O$, where $O$ is the $2\times 2$ zero matrix. Determine...

Close