A sequence of events $\{E_n\}_{n \geq 1}$ is said to be increasing if it satisfies the ascending condition
\[E_1 \subset E_2 \subset \cdots \subset E_n \subset \cdots.\] Also, a sequence $\{E_n\}_{n \geq 1}$ is called decreasing if it satisfies the descending condition
\[E_1 \supset E_2 \supset \cdots \supset E_n \supset \cdots.\]

When $\{E_n\}_{n \geq 1}$ is an increasing sequence, we define a new event denoted by $\lim_{n \to \infty} E_n$ by
\[\lim_{n \to \infty} E_n := \bigcup_{n=1}^{\infty} E_n.\]

Also, when $\{E_n\}_{n \geq 1}$ is a decreasing sequence, we define a new event denoted by $\lim_{n \to \infty} E_n$ by
\[\lim_{n \to \infty} E_n := \bigcap_{n=1}^{\infty} E_n.\]

(1) Suppose that $\{E_n\}_{n \geq 1}$ is an increasing sequence of events. Then prove the equality of probabilities
\[\lim_{n \to \infty} P(E_n) = P\left(\lim_{n \to \infty} E_n \right).\] Hence, the limit and the probability are interchangeable.

(2) Suppose that $\{E_n\}_{n \geq 1}$ is a decreasing sequence of events. Then prove the equality of probabilities
\[\lim_{n \to \infty} P(E_n) = P\left(\lim_{n \to \infty} E_n \right). \]

Proof.

Proof of (1)

Let $\{E_n\}_{n \geq 1}$ be an increasing sequence of events. Then we define new events $F_n$ as follows.
\begin{align*}
F_1 &= E_1\\
F_n & = E_n \setminus E_{n-1} && \text{ for any } n \geq 2
\end{align*}
The event $F_n$ is depicted as a yellow region in the figure below.

From the figure, we can see that the events $\{F_n\}_{n \geq 1}$ are mutually exclusive, that is, $F_iF_j = \emptyset$ for any $i \neq j$.
Furthermore, note that
\[\bigcup_{i=1}^n F_i = \bigcup_{i=1}^n E_i \tag{*}\] for any $n$, and
\[\bigcup_{i=1}^{\infty} F_i = \bigcup_{i=1}^{\infty} E_i. \tag{**}\]

Now we proceed to the main part and prove the equality
\[\lim_{n \to \infty} P(E_n) = P \left(\lim_{n \to \infty} E_n \right).\] We start with the right-hand-side and show that it is equal to the left-hand-side.
\begin{align*}
P\left(\lim_{n \to \infty} E_i \right) &= P \left(\bigcup_{i=1}^{\infty} E_n \right) && \text{ definition of } \lim_{n \to \infty} E_n\\[6pt] &= P \left(\bigcup_{i=1}^{\infty} F_i \right) && \text{ by } (**) \\[6pt] &= \sum_{i=1}^{\infty} P(F_i) && \text{ by one of the axioms of probability} \\
& && \text{ with mutually exclusive events } \{F_n\}\\[6pt] &= \lim_{n \to \infty} \sum_{i=1}^n P(F_i) \\[6pt] &= \lim_{n \to \infty} P\left( \bigcup_{i=1}^n F_i \right) && \text{ by additivity with mutually exclusive events } \{F_n\}\\[6pt] &= \lim_{n \to \infty} P\left(\bigcup_{i=1}^n E_i \right) && \text{ by } (*)\\[6pt] &= \lim_{n \to \infty} P(E_n) && \text{as $\{E_n\}$ is increasing}.
\end{align*}
This proves the desired equality.

Proof of (2)

Now, we consider a decreasing sequence of events $\{E_n\}_{n\geq 1}$.
As $\{E_n\}_{n\geq 1}$ is decreasing, its complement $\{E_n^c\}_{n \geq 1}$ is increasing. Thus, by the result of part (1), we see that
\[\lim_{n \to \infty} P(E_n^c) = P \left(\lim_{n \to \infty} E_n^c \right). \tag{***}\] By definition, we have
\[\lim_{n \to \infty} E_n^c = \bigcup_{n=1}^{\infty} E_n^c = \left(\bigcap_{n=1}^{\infty} E_n \right)^c.\]

It follows that the right-hand-side of the equality (***) becomes
\begin{align*}
P \left(\lim_{n \to \infty} E_n^c \right) &= P\left(\left(\bigcap_{n=1}^{\infty} E_n \right)^c \right)\\[6pt] &= 1 – P\left(\bigcap_{n=1}^{\infty} E_n \right)\\[6pt] &= 1 – P\left(\lim_{n \to \infty} E_n \right).
\end{align*}
On the other hand, the left-hand-side of the equality (***) becomes
\begin{align*}
\lim_{n \to \infty} P(E_n^c) &= \lim_{n \to \infty} \left( 1 – P(E_n) \right)\\[6pt] &= 1 – \lim_{n \to \infty} P(E_n).
\end{align*}

Combining these results yields
\[1 – \lim_{n \to \infty} P(E_n) = 1 – P\left(\lim_{n \to \infty} E_n \right).\] Equivalently, we have
\[\lim_{n \to \infty} P(F_n) = P\left(\lim_{n \to \infty} E_n \right),\] which is the desired equality. This completes the proof.

The post Interchangeability of Limits and Probability of Increasing or Decreasing Sequence of Events first appeared on Problems in Mathematics.

Let $A$ be an $n\times n$ matrix. Suppose that $A$ has real eigenvalues $\lambda_1, \lambda_2, \dots, \lambda_n$ with corresponding eigenvectors $\mathbf{u}_1, \mathbf{u}_2, \dots, \mathbf{u}_n$.
Furthermore, suppose that
\[|\lambda_1| > |\lambda_2| \geq \cdots \geq |\lambda_n|.\] Let
\[\mathbf{x}_0=c_1\mathbf{u}_1+c_2\mathbf{u}_2+\cdots+c_n\mathbf{u}_n\] for some real numbers $c_1, c_2, \dots, c_n$ and $c_1\neq 0$.

Define
\[\mathbf{x}_{k+1}=A\mathbf{x}_k \text{ for } k=0, 1, 2,\dots\] and let
\[\beta_k=\frac{\mathbf{x}_k\cdot \mathbf{x}_{k+1}}{\mathbf{x}_k \cdot \mathbf{x}_k}=\frac{\mathbf{x}_k^{\trans} \mathbf{x}_{k+1}}{\mathbf{x}_k^{\trans} \mathbf{x}_k}.\]

Prove that
\[\lim_{k\to \infty} \beta_k=\lambda_1.\]

Proof.

We have
\begin{align*}
x_k=A\mathbf{x}_{k-1}=A^2\mathbf{x}_{k-2}=\cdots=A^k\mathbf{x}_0
\end{align*}
by applying the definition of $\mathbf{x}_k$ successively.
Then we have
\begin{align*}
\mathbf{x}_k&=A^k\mathbf{x}_0\\
&=A^k(c_1\mathbf{u}_1+c_2\mathbf{u}_2+\cdots+c_n\mathbf{u}_n)\\
&=c_1A^k\mathbf{u}_1+c_2A^k\mathbf{u}_2+\cdots+c_nA^k\mathbf{u}_n\\
&=c_1\lambda_1^k\mathbf{u}_1+c_2\lambda_2^k\mathbf{u}_2+\cdots+c_n\lambda_n^k\mathbf{u}_n,
\end{align*}
where the last step follows from the equalities $A^k\mathbf{u}_i=\lambda_i^k \mathbf{u}_i$, which in turn follows from the definitions of eigenvalues and eigenvectors $A\mathbf{u}_i=\lambda_i\mathbf{u}_i$.
Using the sum notation, we simply write it as
\[\mathbf{x}_k=\sum_{i=1}^nc_i\lambda_i^k\mathbf{u}_i.\] This formula is true for any nonnegative integer $k$.

Let us next compute the dot product $\mathbf{x}_k\cdot \mathbf{x}_k$. We have
\begin{align*}
\mathbf{x}_k\cdot \mathbf{x}_k&=\left(\, \sum_{i=1}^nc_i\lambda_i^k\mathbf{u}_i \,\right)\cdot \left(\, \sum_{j=1}^nc_j\lambda_j^k\mathbf{u}_j \,\right)\\
&=\sum_{i=1}^n\sum_{j=1}^nc_ic_j\lambda_i^k\lambda_j^k\mathbf{u}_i \cdot \mathbf{u}_j\\
&=\lambda_1^{2k}\sum_{i=1}^n\sum_{j=1}^nc_ic_j \left(\, \frac{\lambda_i}{\lambda_1} \,\right)^k\left(\, \frac{\lambda_j}{\lambda_1} \,\right)^k\mathbf{u}_i \cdot \mathbf{u}_j.
\end{align*}
Since $\lambda_1$ is strictly larger than any other eigenvalues, we have
\begin{align*}
\lim_{k \to \infty} \left(\, \frac{\lambda_i}{\lambda_1} \,\right)^k
=\begin{cases} 1 & \text{ if } i=1\\
0 & \text{ if } i=2, 3,\dots, n.
\end{cases}
\end{align*}
It follows that
\begin{align*}
\lim_{k \to \infty}\mathbf{x}_k\cdot \mathbf{x}_k=\lambda_1^{2k}c_1^2 \mathbf{u}_1\cdot \mathbf{u}_1.
\end{align*}

Similarly, we compute
\begin{align*}
\mathbf{x}_k\cdot \mathbf{x}_{k+1}&=\left(\, \sum_{i=1}^nc_i\lambda_i^k\mathbf{u}_i \,\right)\cdot \left(\, \sum_{j=1}^nc_j\lambda_j^{k+1}\mathbf{u}_j \,\right)\\
&=\sum_{i=1}^n\sum_{j=1}^nc_ic_j\lambda_i^k\lambda_j^{k+1}\mathbf{u}_i \cdot \mathbf{u}_j\\
&=\lambda_1^{2k+1}\sum_{i=1}^n\sum_{j=1}^nc_ic_j \left(\, \frac{\lambda_i}{\lambda_1} \,\right)^k\left(\, \frac{\lambda_j}{\lambda_1} \,\right)^{k+1}\mathbf{u}_i \cdot \mathbf{u}_j.
\end{align*}

Thus we obtain
\begin{align*}
\lim_{k\to \infty} \mathbf{x}_k\cdot \mathbf{x}_{k+1}=\lambda_1^{2k+1}c_1^2\mathbf{u}_1\cdot \mathbf{u}_1.
\end{align*}

Combining these computations, we have
\begin{align*}
\lim_{k\to \infty}\beta_k&=\frac{\lim_{k\to \infty}\mathbf{x}_k\cdot \mathbf{x}_{k+1}}{\lim_{k\to \infty}\mathbf{x}_k \cdot \mathbf{x}_k}\\[6pt] &=\frac{\lambda_1^{2k+1}c_1^2 \mathbf{u}_1\cdot \mathbf{u}_1}{\lambda_1^{2k}c_1^2\mathbf{u}_1\cdot \mathbf{u}_1}\\[6pt] &=\lambda_1
\end{align*}
as required. This completes the proof.

The post Sequence Converges to the Largest Eigenvalue of a Matrix first appeared on Problems in Mathematics.

Let $A$ and $B$ be $n \times n$ matrices.

Prove that the characteristic polynomials for the matrices $AB$ and $BA$ are the same.

Hint.

1. Consider the case when the matrix $A$ is invertible.
2. Even if $A$ is not invertible, note that $A-\epsilon I$ is invertible matrix for sufficiently small $\epsilon$.
   (See Problem Perturbation of a singular matrix is nonsingular for a proof of this fact.)
3. Take the limit $\epsilon \to 0$.

Proof.

We want to show that $|AB-\lambda I|=|BA-\lambda I|$, where $\lambda$ is an unknown and $I$ is the $n \times n$ identity matrix.

First let us consider the case when the matrix $A$ is invertible.
So suppose that $A$ is invertible.
Then we have
\begin{align*}
|AB-\lambda I| &=|A^{-1}(AB-\lambda I)A| =|BA-\lambda I|.
\end{align*}
Here we use the fact that $|A||A^{-1}|=|AA^{-1}|=|I|=1$.
Thus when $A$ is invertible, the claim is proved.

Now we consider the general case.
Whether or not $A$ is invertible, the matrix $A-\epsilon I$ is invertible for sufficiently small $\epsilon$.

(More precisely, if $\epsilon$ is smaller than absolute values of all nonzero eigenvalues of $A$. then $A-\epsilon I$ is invertible. See Problem Perturbation of a singular matrix is nonsingular for a proof.)

Then by the previous case, we have
\[ |(A-\epsilon I) B-\lambda I|=|B(A-\epsilon I)-\lambda I|.\] Taking the limit $\epsilon \to 0$, we obtain
\[|AB-\lambda I|=|BA-\lambda I|.\]

Comment.

When you study hard linear algebra, you might have forgotten about calculus.
This problem suggests that for some problems in linear algebra, techniques from calculus (like limits) might help to solve them.

The post Characteristic Polynomials of $AB$ and $BA$ are the Same first appeared on Problems in Mathematics.

Let
\[A=\begin{bmatrix}
\frac{1}{7} & \frac{3}{7} & \frac{3}{7} \\
\frac{3}{7} &\frac{1}{7} &\frac{3}{7} \\
\frac{3}{7} & \frac{3}{7} & \frac{1}{7}
\end{bmatrix}\] be $3 \times 3$ matrix. Find

\[\lim_{n \to \infty} A^n.\]

(Nagoya University Linear Algebra Exam)

Hint.

1. The matrix $A$ is symmetric, hence diagonalizable.
2. Diagonalize $A$.
3. You may want to find eigenvalues
   without computing the characteristic polynomial for $A$.

Solution.

Note that the matrix $A$ is symmetric, hence it is diagonalizable.
We observe several things to simplify the computation.

First, note that the sum of the entries in each row is $1$.
(Such a matrix is call a stochastic matrix. See the comment below.)
Thus the matrix $A$ has eigenvalue $1$ and $\begin{bmatrix}
1 \\
1 \\
1
\end{bmatrix}$ is an eigenvector.

Also note that if we add $2/7(=-\lambda)$ to diagonal entries, then every entry becomes $3/7$.
That is,
\[A+\frac{2}{7}I=\begin{bmatrix}
\frac{3}{7} & \frac{3}{7} & \frac{3}{7} \\
\frac{3}{7} &\frac{3}{7} &\frac{3}{7} \\
\frac{3}{7} & \frac{3}{7} & \frac{3}{7}
\end{bmatrix},\] which is clearly singular.

From this observation, we notice that $-2/7$ is an eigenvalue of $A$ and eigenvectors are nonzero solution of $x_1+x_2+x_3=0$. Therefore $\begin{bmatrix}
1 \\
-1 \\
0
\end{bmatrix}$ and $\begin{bmatrix}
0 \\
1 \\
-1
\end{bmatrix}$ are linearly independent eigenvectors for the eigenvalue $-2/7$.

Hence the geometric (and hence algebraic) multiplicity of the eigenvalue $-2/7$ is $2$.
(Note that we found all eigenvalues of $A$ without actually computing the characteristic polynomial.)

Now the invertible matix $P=\begin{bmatrix}
1 & 1 & 0 \\
1 &-1 &1 \\
1 & 0 & -1
\end{bmatrix}$ diagonalize $A$. Namely we have
\[P^{-1}AP=\begin{bmatrix}
1 & 0 & 0 \\
0 &-2/7 &0 \\
0 & 0 & -2/7
\end{bmatrix}.\] Hence
\[A=P\begin{bmatrix}
1 & 0 & 0 \\
0 &-2/7 &0 \\
0 & 0 & -2/7
\end{bmatrix}P^{-1}.\] Therefore we have
\[ A^n=P\begin{bmatrix}
1^n & 0 & 0 \\
0 & (-2/7)^n &0 \\
0 & 0 & (-2/7)^n
\end{bmatrix}P^{-1}
\text{ and } \lim_{n \to \infty} A^n=P\begin{bmatrix}
1 & 0 & 0 \\
0 & 0 &0 \\
0 & 0 & 0
\end{bmatrix}P^{-1}. \] Find the inverse $P^{-1}$ by your favorite method
\[P^{-1}=\begin{bmatrix}
\frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\[6pt] \frac{2}{3} &\frac{-1}{3} &\frac{-1}{3} \\[6pt] \frac{1}{3} & \frac{1}{3} & \frac{-2}{3}
\end{bmatrix}.\] Then the answer is
\[\lim_{n \to \infty} A^n=\begin{bmatrix}
\frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\[6pt] \frac{1}{3} &\frac{1}{3} &\frac{1}{3} \\[6pt] \frac{1}{3} & \frac{1}{3} & \frac{1}{3}
\end{bmatrix}.\]

Comment.

You might come up with the idea to use the diagonalization as it is an exam problem in linear algebra.
But you soon realize that computing the characteristic polynomial for $A$ is a bit messy.
So sometimes it is important to find eigenvalues without finding the roots of the characteristic polynomial.

We observed that

1. the sum of row entries of $A$ is $1$
2. if we add $(2/7)I$ to $A$ then we obtain a matrix whose entries are all identical.

These observation yields the eigenvalues of $A$ without using the characteristic polynomial.

If the sum of entries of each row of a matrix is $1$, then the matrix is called called a stochastic matrix (or Markov matrix, probability matrix).
The matrix $A$ in the problem is an example of a stochastic matrix.

Stochastic matrices have always $1$ as an eigenvalue.
see the post
 Stochastic matrix (Markov matrix) and its eigenvalues and eigenvectors
for an example of a $2\times 2$ stochastic matrix.

The post Find the Limit of a Matrix first appeared on Problems in Mathematics.