A Matrix Equation of a Symmetric Matrix and the Limit of its Solution

Problem 457

Let $A$ be a real symmetric $n\times n$ matrix with $0$ as a simple eigenvalue (that is, the algebraic multiplicity of the eigenvalue $0$ is $1$), and let us fix a vector $\mathbf{v}\in \R^n$.

(a) Prove that for sufficiently small positive real $\epsilon$, the equation
\[A\mathbf{x}+\epsilon\mathbf{x}=\mathbf{v}\] has a unique solution $\mathbf{x}=\mathbf{x}(\epsilon) \in \R^n$.

(b) Evaluate
\[\lim_{\epsilon \to 0^+} \epsilon \mathbf{x}(\epsilon)\] in terms of $\mathbf{v}$, the eigenvectors of $A$, and the inner product $\langle\, ,\,\rangle$ on $\R^n$.

 
(University of California, Berkeley, Linear Algebra Qualifying Exam)

Add to solve later

Sponsored Links

Proof.

(a) Prove that $A\mathbf{x}+\epsilon\mathbf{x}=\mathbf{v}$has a unique solution $\mathbf{x}=\mathbf{x}(\epsilon) \in \R^n$.

Recall that the eigenvalues of a real symmetric matrices are all real numbers and it is diagonalizable by an orthogonal matrix.

Note that the equation $A\mathbf{x}+\epsilon\mathbf{x}=\mathbf{v}$ can be written as
\[(A+\epsilon I)\mathbf{x}=\mathbf{v}, \tag{*}\] where $I$ is the $n\times n$ identity matrix. Thus to show that the equation (*) has a unique solution, it suffices to show that the matrix $A+\epsilon I$ is invertible.

Since $A$ is diagonalizable, there exists an invertible matrix $S$ such that
\[S^{-1}AS=\begin{bmatrix}
\lambda_1 & 0 & \cdots & 0 \\
0 & \lambda_2 & \cdots & 0\\
\vdots & \vdots & \ddots & \vdots \\
0 & 0 & \cdots & \lambda_n
\end{bmatrix},\] where $\lambda_i$ are eigenvalues of $A$.
Since the algebraic multiplicity of $0$ is $1$, without loss of generality, we may assume that $\lambda_1=0$ and $\lambda_i, i > 1$ are nonzero.

Then we have
\begin{align*}
S^{-1}(A+\epsilon I)S&=S^{-1}AS+\epsilon I=\begin{bmatrix}
\epsilon & 0 & \cdots & 0 \\
0 & \epsilon+\lambda_2 & \cdots & 0\\
\vdots & \vdots & \ddots & \vdots \\
0 & 0 & \cdots & \epsilon+\lambda_n
\end{bmatrix}.
\end{align*}

---

If $\epsilon > 0$ is smaller than the lengths of $|\lambda_i|, i > 1$, then none of the diagonal entries $\epsilon+ \lambda_i$ are zero.

Hence we have
\begin{align*}
\det(A+\epsilon I)&=\det(S)^{-1}\det(A+\epsilon I)\det(S)\\
&=\det\left(\, S^{-1}(A+\epsilon I) S \,\right)\\
&=\epsilon(\epsilon+\lambda_2)\cdots (\epsilon+\lambda_n)\neq 0.
\end{align*}
Since $\det(A+\epsilon I)\neq 0$, it yields that $A$ is invertible, hence the equation (*) has a unique solution
\[\mathbf{x}(\epsilon)=(A+\epsilon I)^{-1}\mathbf{v}.\]

Remark

This result is in general true for any square matrix.
Instead of using the diagonalization, we can use the triangulation of a matrix.

(b) Evaluate $\lim_{\epsilon \to 0^+} \epsilon \mathbf{x}(\epsilon)$

As noted earlier that a real symmetric matrix can be diagonalizable by an orthogonal matrix.
This means that there is an eigenvector $\mathbf{v}_i$ corresponding to the eigenvalue $\lambda_i$ for each $i$ such that the eigenvectors $\mathbf{v}_i$ form an orthonormal basis of $\R^n$.
That is,
\begin{align*}
A\mathbf{v}_i=\lambda_i \mathbf{v}_i \\
\langle \mathbf{v}_i,\mathbf{v}_j \rangle=\delta_{i,j},
\end{align*}
where $\delta_{i,j}$ is the Kronecker delta symbol, where $\delta_{i,i}=1, \delta_{i,j}=0$ if $i\neq j$.
From this, we deduce that
\begin{align*}
(A+\epsilon I)\mathbf{v}_i=(\lambda_i+\epsilon)\mathbf{v}_i\\
(A+\epsilon I)^{-1}\mathbf{v}_i=\frac{1}{\lambda_i+\epsilon}\mathbf{v}_i. \tag{**}
\end{align*}
Using the basis $\{\mathbf{v}_i\}$, we write
\[\mathbf{v}=\sum_{i=1}^nc_i \mathbf{v}_i\] for some $c_i\in \R$.

---

Then we compute
\begin{align*}
A\mathbf{x}(\epsilon)&=A(A+\epsilon I)^{-1}\mathbf{v} && \text{by part (a)}\\
&=A(A+\epsilon I)^{-1}\left(\, \sum_{i=1}^nc_i \mathbf{v}_i \,\right)\\
&=\sum_{i=1}^n c_iA(A+\epsilon I)^{-1}\mathbf{v}_i\\
&=\sum _{i=1}^n c_iA\left(\, \frac{1}{\lambda_i+\epsilon}\mathbf{v}_i \,\right) && \text{by (**)}\\
&=\sum_{i=1}^n c_i\frac{\lambda_i}{\lambda_i+\epsilon}\mathbf{v}_i && \text{since $A\mathbf{v}_i=\lambda_i\mathbf{v}_i$}\\
&=\sum_{i=2}^n c_i\frac{\lambda_i}{\lambda_i+\epsilon}\mathbf{v}_i && \text{since $\lambda_1=0$}.
\end{align*}

---

Therefore we have
\begin{align*}
\lim_{\epsilon \to 0^+} \epsilon \mathbf{x}(\epsilon)&=\lim_{\epsilon \to 0^+}\left(\, \mathbf{v}-A\mathbf{x}(\epsilon) \,\right)\\
&=\mathbf{v}-\lim_{\epsilon \to 0^+}\left(\, A\mathbf{x}(\epsilon) \,\right)\\
&= \sum_{i=1}^nc_i\mathbf{v}_i-\lim_{\epsilon \to 0^+}\left(\, \sum_{i=2}^n c_i\frac{\lambda_i}{\lambda_i+\epsilon}\mathbf{v}_i \,\right)\\
&=\sum_{i=1}c_i \mathbf{v}_i-\sum_{i=2}^n c_i \mathbf{v}_i\\
&=c_1\mathbf{v}_1.
\end{align*}
Using the orthonormality of the basis $\{\mathbf{v}_i\}$, we have
\[\langle\mathbf{v}, \mathbf{v}_1 \rangle=\sum_{i=1}^n \langle c_i\mathbf{v}_i, \mathbf{v}_1 \rangle=c_1.\]

Hence the required expression is
\[\lim_{\epsilon \to 0^+} \epsilon \mathbf{x}(\epsilon)=\langle\mathbf{v}, \mathbf{v}_1 \rangle\mathbf{v}_1,\] where $\mathbf{v}_1$ is the unit eigenvector corresponding to the eigenvalue $0$.

Click here if solved 15
Add to solve later

Sponsored Links