You may use the augmented matrix method to find the inverse matrix.
Here we give an alternative way to find the inverse matrix by noting that $A$ is an orthogonal matrix.

Recall that a matrix $B$ is orthogonal if $B^{\trans}B=B^{\trans}B=I$.
Thus, once we know $B$ is an orthogonal matrix, then the inverse matrix $B^{-1}$ is just the transpose matrix $B^{\trans}$.

Also, recall that a matrix $B$ is orthogonal if and only if the column vectors of $B$ form an orthonormal set.

Solution.

We first show that $A$ is an orthogonal matrix.
To do this, it suffices to that the column vectors form an orthonormal set.

Let
\[ \mathbf{v}_1= \begin{bmatrix}
\frac{2}{7} \\[6 pt]
\frac{6}{7} \\[6pt]
-\frac{3}{7}
\end{bmatrix},
\mathbf{v}_2=\begin{bmatrix}
\frac{3}{7} \\[6 pt]
\frac{2}{7} \\[6pt]
\frac{6}{7} \end{bmatrix},
\mathbf{v}_3=\begin{bmatrix}
\frac{6}{7} \\[6 pt]
-\frac{3}{7} \\[6pt]
-\frac{2}{7}
\end{bmatrix}\]
be the column vectors of $A$.

Then the length of the vector $\mathbf{v}_1$ is
\[||\mathbf{v}_1||=\sqrt{(2/7)^2+(6/7)^2+(-3/7)^2}=1.\]
Similarly, we have $||\mathbf{v}_2||=||\mathbf{v}_3||=1$.
Thus, column vectors are unit vectors.

The dot (inner) product of the vectors $\mathbf{v}_1$ and $\mathbf{v}_2$ is
\[\mathbf{v}_1\cdot \mathbf{v}_2=\frac{2}{7}\cdot \frac{3}{7}+\frac{6}{7}\cdot \frac{2}{7}+\left( -\frac{3}{7}\right) \cdot \frac{6}{7}=0.\]
Similarly, we have
\[\mathbf{v}_1\cdot \mathbf{v}_3=0, \quad \mathbf{v}_2\cdot \mathbf{v}_3=0.\]

Therefore, the column vectors are orthogonal.
Hence the column vectors of $A$ are orthonormal, and this implies that $A$ is an orthogonal matrix. Namely, $A^{\trans}=A^{-1}$.
Thus the inverse matrix of $A$ is
\[A^{-1}=\begin{bmatrix}
\frac{2}{7} & \frac{6}{7} & -\frac{3}{7} \\[6 pt]
\frac{3}{7} &\frac{2}{7} &\frac{6}{7} \\[6pt]
\frac{6}{7} & -\frac{3}{7} & -\frac{2}{7}
\end{bmatrix}.\]

Rotation Matrix in Space and its Determinant and Eigenvalues
For a real number $0\leq \theta \leq \pi$, we define the real $3\times 3$ matrix $A$ by
\[A=\begin{bmatrix}
\cos\theta & -\sin\theta & 0 \\
\sin\theta &\cos\theta &0 \\
0 & 0 & 1
\end{bmatrix}.\]
(a) Find the determinant of the matrix $A$.
(b) Show that $A$ is an […]

Find the Distance Between Two Vectors if the Lengths and the Dot Product are Given
Let $\mathbf{a}$ and $\mathbf{b}$ be vectors in $\R^n$ such that their length are
\[||\mathbf{a}||=||\mathbf{b}||=1\]
and the inner product
\[\mathbf{a}\cdot \mathbf{b}=\mathbf{a}^{\trans}\mathbf{b}=-\frac{1}{2}.\]
Then determine the length $||\mathbf{a}-\mathbf{b}||$.
(Note […]

Inner Product, Norm, and Orthogonal Vectors
Let $\mathbf{u}_1, \mathbf{u}_2, \mathbf{u}_3$ are vectors in $\R^n$. Suppose that vectors $\mathbf{u}_1$, $\mathbf{u}_2$ are orthogonal and the norm of $\mathbf{u}_2$ is $4$ and $\mathbf{u}_2^{\trans}\mathbf{u}_3=7$. Find the value of the real number $a$ in […]

Diagonalizable by an Orthogonal Matrix Implies a Symmetric Matrix
Let $A$ be an $n\times n$ matrix with real number entries.
Show that if $A$ is diagonalizable by an orthogonal matrix, then $A$ is a symmetric matrix.
Proof.
Suppose that the matrix $A$ is diagonalizable by an orthogonal matrix $Q$.
The orthogonality of the […]

Unit Vectors and Idempotent Matrices
A square matrix $A$ is called idempotent if $A^2=A$.
(a) Let $\mathbf{u}$ be a vector in $\R^n$ with length $1$.
Define the matrix $P$ to be $P=\mathbf{u}\mathbf{u}^{\trans}$.
Prove that $P$ is an idempotent matrix.
(b) Suppose that $\mathbf{u}$ and $\mathbf{v}$ be […]

Prove the Cauchy-Schwarz Inequality
Let $\mathbf{a}, \mathbf{b}$ be vectors in $\R^n$.
Prove the Cauchy-Schwarz inequality:
\[|\mathbf{a}\cdot \mathbf{b}|\leq ||\mathbf{a}||\,||\mathbf{b}||.\]
Here $\mathbf{a}\cdot \mathbf{b}$ is the dot (inner) product of $\mathbf{a}$ and $\mathbf{b}$, and $||\mathbf{a}||$ is […]

Sherman-Woodbery Formula for the Inverse Matrix
Let $\mathbf{u}$ and $\mathbf{v}$ be vectors in $\R^n$, and let $I$ be the $n \times n$ identity matrix. Suppose that the inner product of $\mathbf{u}$ and $\mathbf{v}$ satisfies
\[\mathbf{v}^{\trans}\mathbf{u}\neq -1.\]
Define the matrix […]