Linear Algebra

Positive definite Real Symmetric Matrix and its Eigenvalues

Problem 396

A real symmetric $n \times n$ matrix $A$ is called positive definite if
\[\mathbf{x}^{\trans}A\mathbf{x}>0\] for all nonzero vectors $\mathbf{x}$ in $\R^n$.

(a) Prove that the eigenvalues of a real symmetric positive-definite matrix $A$ are all positive.

(b) Prove that if eigenvalues of a real symmetric matrix $A$ are all positive, then $A$ is positive-definite.

Add to solve later

Proof.

(a) Prove that the eigenvalues of a real symmetric positive-definite matrix $A$ are all positive.

Recall that the eigenvalues of a real symmetric matrix are real.
(See the corollary in the post “Eigenvalues of a Hermitian matrix are real numbers“.)

Let $\lambda$ be a (real) eigenvalue of $A$ and let $\mathbf{x}$ be a corresponding real eigenvector. That is, we have
\[A\mathbf{x}=\lambda \mathbf{x}.\] Then we multiply by $\mathbf{x}^{\trans}$ on left and obtain
\begin{align*}
\mathbf{x}^{\trans}A\mathbf{x}&=\lambda \mathbf{x}^{\trans} \mathbf{x}\\
&=\lambda ||\mathbf{x}||^2.
\end{align*}

The left hand side is positive as $A$ is positive definite and $\mathbf{x}$ is a nonzero vector as it is an eigenvector.
Since the length $||\mathbf{x}||^2$ is positive, we must have $\lambda$ is positive.
It follows that every eigenvalue $\lambda$ of $A$ is real.

(b) Prove that if eigenvalues of a real symmetric matrix $A$ are all positive, then $A$ is positive-definite.

Note that a real symmetric matrix is diagonalizable by an orthogonal matrix.

So there exists an on orthogonal matrix $Q$ such that $Q^{\trans}AQ=D$, where $D$ is the diagonal matrix
\[D=\begin{bmatrix}
\lambda_1 & 0 & 0 & 0 \\
0 &\lambda_2 & 0 & 0 \\
\vdots & \cdots & \ddots & \vdots \\
0 & 0 & \cdots & \lambda_n
\end{bmatrix}\] whose diagonal entries $\lambda_i$ are eigenvalues of $A$, which are positive by assumption.

Let $\mathbf{x}$ be an arbitrary nonzero vector in $\R^n$.
Since $A=QDQ^{\trans}$ (remark that $Q^{-1}=Q^{\trans}$), we have
\[\mathbf{x}^{\trans} A\mathbf{x}=\mathbf{x}^{\trans}QDQ^{\trans}\mathbf{x}.\] Putting $\mathbf{y}=Q^{\trans}\mathbf{x}$, we can rewrite the above equation as
\[\mathbf{x}^{\trans} A\mathbf{x}=\mathbf{y}^{\trans}D\mathbf{y}.\]

Let
\[\mathbf{y}=\begin{bmatrix}
y_1 \\
y_2 \\
\vdots \\
y_n
\end{bmatrix}.\] Then we have
\begin{align*}
&\mathbf{x}^{\trans} A\mathbf{x}=\mathbf{y}^{\trans}D\mathbf{y}\\
&=\begin{bmatrix}
y_1 & y_2 & \cdots & y_n
\end{bmatrix}
\begin{bmatrix}
\lambda_1 & 0 & 0 & 0 \\
0 &\lambda_2 & 0 & 0 \\
\vdots & \cdots & \ddots & \vdots \\
0 & 0 & \cdots & \lambda_n
\end{bmatrix}
\begin{bmatrix}
y_1 \\
y_2 \\
\vdots \\
y_n
\end{bmatrix}\\[6pt] &=\lambda_1y_1^2+\lambda_2 y_2^2+\cdots +\lambda_n y_n^2.
\end{align*}

By assumption eigenvalues $\lambda_i$ are positive.
Also, since $\mathbf{x}$ is a nonzero vector and $Q$ is invertible, $\mathbf{y}=Q^{\trans}\mathbf{x}$ is not a zero vector.
Thus the sum expression above is positive, hence $\mathbf{x}^{\trans} A\mathbf{x}$ is positive for any nonzero vector $\mathbf{x}$.
Therefore, the matrix $A$ is positive-definite.

Related Questions.

Problem. Let $A$ be an $n \times n$ real matrix. Prove the followings.

(a) The matrix $AA^{\trans}$ is a symmetric matrix.

(b) The set of eigenvalues of $A$ and the set of eigenvalues of $A^{\trans}$ are equal.

(c) The matrix $AA^{\trans}$ is non-negative definite.
(An $n\times n$ matrix $B$ is called non-negative definite if for any $n$ dimensional vector $\mathbf{x}$, we have $\mathbf{x}^{\trans}B \mathbf{x} \geq 0$.)

(d) All the eigenvalues of $AA^{\trans}$ is non-negative.

For a solution, see the post “Transpose of a matrix and eigenvalues and related questions.“.

Problem. Suppose $A$ is a positive definite symmetric $n\times n$ matrix.

(a) Prove that $A$ is invertible.

(b) Prove that $A^{-1}$ is symmetric.

(c) Prove that $A^{-1}$ is positive-definite.

For proofs, see the post “Inverse matrix of positive-definite symmetric matrix is positive-definite“.

Problem.
Prove that a positive definite matrix has a unique positive definite square root.

For a solution of this problem, see the post
A Positive Definite Matrix Has a Unique Positive Definite Square Root

Add to solve later

More from my site

Inverse Matrix of Positive-Definite Symmetric Matrix is Positive-Definite Suppose $A$ is a positive definite symmetric $n\times n$ matrix. (a) Prove that $A$ is invertible. (b) Prove that $A^{-1}$ is symmetric. (c) Prove that $A^{-1}$ is positive-definite. (MIT, Linear Algebra Exam Problem) Proof. (a) Prove that $A$ is […]
Transpose of a Matrix and Eigenvalues and Related Questions Let $A$ be an $n \times n$ real matrix. Prove the followings. (a) The matrix $AA^{\trans}$ is a symmetric matrix. (b) The set of eigenvalues of $A$ and the set of eigenvalues of $A^{\trans}$ are equal. (c) The matrix $AA^{\trans}$ is non-negative definite. (An $n\times n$ […]
Eigenvalues of a Hermitian Matrix are Real Numbers Show that eigenvalues of a Hermitian matrix $A$ are real numbers. (The Ohio State University Linear Algebra Exam Problem) We give two proofs. These two proofs are essentially the same. The second proof is a bit simpler and concise compared to the first one. […]
$Eigenvalues of $2\times 2$ Symmetric Matrices are Real by Considering Characteristic Polynomials$ Eigenvalues of $2\times 2$ Symmetric Matrices are Real by Considering Characteristic Polynomials Let $A$ be a $2\times 2$ real symmetric matrix. Prove that all the eigenvalues of $A$ are real numbers by considering the characteristic polynomial of $A$. Proof. Let $A=\begin{bmatrix} a& b \\ c& d \end{bmatrix}$. Then […]
Questions About the Trace of a Matrix Let $A=(a_{i j})$ and $B=(b_{i j})$ be $n\times n$ real matrices for some $n \in \N$. Then answer the following questions about the trace of a matrix. (a) Express $\tr(AB^{\trans})$ in terms of the entries of the matrices $A$ and $B$. Here $B^{\trans}$ is the transpose matrix of […]
Sequence Converges to the Largest Eigenvalue of a Matrix Let $A$ be an $n\times n$ matrix. Suppose that $A$ has real eigenvalues $\lambda_1, \lambda_2, \dots, \lambda_n$ with corresponding eigenvectors $\mathbf{u}_1, \mathbf{u}_2, \dots, \mathbf{u}_n$. Furthermore, suppose that \[|\lambda_1| > |\lambda_2| \geq \cdots \geq […]
There is at Least One Real Eigenvalue of an Odd Real Matrix Let $n$ be an odd integer and let $A$ be an $n\times n$ real matrix. Prove that the matrix $A$ has at least one real eigenvalue. We give two proofs. Proof 1. Let $p(t)=\det(A-tI)$ be the characteristic polynomial of the matrix $A$. It is a degree $n$ […]
A Symmetric Positive Definite Matrix and An Inner Product on a Vector Space (a) Suppose that $A$ is an $n\times n$ real symmetric positive definite matrix. Prove that \[\langle \mathbf{x}, \mathbf{y}\rangle:=\mathbf{x}^{\trans}A\mathbf{y}\] defines an inner product on the vector space $\R^n$. (b) Let $A$ be an $n\times n$ real matrix. Suppose […]