For example, let $A$ be the following $3\times 3$ matrix.
\[A=\begin{bmatrix}
0 & 1 & 0 \\
0 &0 &1 \\
0 & 0 & 0
\end{bmatrix}.\]
Then $A$ is a nonzero matrix and we have
\[A^2=\begin{bmatrix}
0 & 1 & 0 \\
0 &0 &1 \\
0 & 0 & 0
\end{bmatrix}\begin{bmatrix}
0 & 1 & 0 \\
0 &0 &1 \\
0 & 0 & 0
\end{bmatrix}
=\begin{bmatrix}
0 & 0 & 1 \\
0 &0 &0 \\
0 & 0 & 0
\end{bmatrix}\neq O.\]

The third power of $A$ is
\[A^3=A^2A=\begin{bmatrix}
0 & 0 & 1 \\
0 &0 &0 \\
0 & 0 & 0
\end{bmatrix}\begin{bmatrix}
0 & 1 & 0 \\
0 &0 &1 \\
0 & 0 & 0
\end{bmatrix}=
\begin{bmatrix}
0 & 0 & 0 \\
0 &0 &0 \\
0 & 0 & 0
\end{bmatrix}=O.\]
Thus, the nonzero matrix $A$ satisfies the required conditions $A^2\neq O, A^3=O$.

Comment.

A square matrix $A$ is called nilpotent if there is a non-negative integer $k$ such that $A^k$ is the zero matrix.
The smallest such an integer $k$ is called degree or index of $A$.

The matrix $A$ in the solution above gives an example of a $3\times 3$ nilpotent matrix of degree $3$.

If Every Trace of a Power of a Matrix is Zero, then the Matrix is Nilpotent
Let $A$ be an $n \times n$ matrix such that $\tr(A^n)=0$ for all $n \in \N$.
Then prove that $A$ is a nilpotent matrix. Namely there exist a positive integer $m$ such that $A^m$ is the zero matrix.
Steps.
Use the Jordan canonical form of the matrix $A$.
We want […]

If the Matrix Product $AB=0$, then is $BA=0$ as Well?
Let $A$ and $B$ be $n\times n$ matrices. Suppose that the matrix product $AB=O$, where $O$ is the $n\times n$ zero matrix.
Is it true that the matrix product with opposite order $BA$ is also the zero matrix?
If so, give a proof. If not, give a […]

Sum of Squares of Hermitian Matrices is Zero, then Hermitian Matrices Are All Zero
Let $A_1, A_2, \dots, A_m$ be $n\times n$ Hermitian matrices. Show that if
\[A_1^2+A_2^2+\cdots+A_m^2=\calO,\]
where $\calO$ is the $n \times n$ zero matrix, then we have $A_i=\calO$ for each $i=1,2, \dots, m$.
Hint.
Recall that a complex matrix $A$ is Hermitian if […]

10 True or False Problems about Basic Matrix Operations
Test your understanding of basic properties of matrix operations.
There are 10 True or False Quiz Problems.
These 10 problems are very common and essential.
So make sure to understand these and don't lose a point if any of these is your exam problems.
(These are actual exam […]

Is a Set of All Nilpotent Matrix a Vector Space?
Let $V$ denote the vector space of all real $n\times n$ matrices, where $n$ is a positive integer.
Determine whether the set $U$ of all $n\times n$ nilpotent matrices is a subspace of the vector space $V$ or not.
Definition.
An matrix $A$ is a nilpotent matrix if […]

Every Complex Matrix Can Be Written as $A=B+iC$, where $B, C$ are Hermitian Matrices
(a) Prove that each complex $n\times n$ matrix $A$ can be written as
\[A=B+iC,\]
where $B$ and $C$ are Hermitian matrices.
(b) Write the complex matrix
\[A=\begin{bmatrix}
i & 6\\
2-i& 1+i
\end{bmatrix}\]
as a sum $A=B+iC$, where $B$ and $C$ are Hermitian […]