Let $e$ be the identity element of $G$.
We compute
\begin{align*}
&xy^2x\stackrel{(1)}{=} y^3x^2=y^2\cdot yx^2\stackrel{(2)}{=}y^2\cdot x^3y=y\cdot yx^2 \cdot y \stackrel{(2)}{=}y\cdot x^3y \cdot y\\[6pt]
&=yx^2 \cdot xy^2 \stackrel{(2)}{=}x^3y\cdot xy^2\stackrel{(1)}{=}x^3y\cdot y^3x=x^3y^4x.
\end{align*}

Canceling the leftmost and the rightmost $x$, we obtain
\[y^2=x^2y^4.\]

Canceling $y^2$, we get
\[e=x^2y^2.\]
Hence we have
\[y^2=x^{-2}.\]
Substituting this into the relation (1) yields that
\[x^{-1}=yx^{-1},\]
and thus $y=e$.
Then it follows from the relation (2) that $x^2=x^3$, and hence $x=e$.

Therefore the generators $x, y$ must be the identity element.
Thus, $G$ is the trivial group $\{e\}$.

A Simple Abelian Group if and only if the Order is a Prime Number
Let $G$ be a group. (Do not assume that $G$ is a finite group.)
Prove that $G$ is a simple abelian group if and only if the order of $G$ is a prime number.
Definition.
A group $G$ is called simple if $G$ is a nontrivial group and the only normal subgroups of $G$ is […]

Dihedral Group and Rotation of the Plane
Let $n$ be a positive integer. Let $D_{2n}$ be the dihedral group of order $2n$. Using the generators and the relations, the dihedral group $D_{2n}$ is given by
\[D_{2n}=\langle r,s \mid r^n=s^2=1, sr=r^{-1}s\rangle.\]
Put $\theta=2 \pi/n$.
(a) Prove that the matrix […]

Centralizer, Normalizer, and Center of the Dihedral Group $D_{8}$
Let $D_8$ be the dihedral group of order $8$.
Using the generators and relations, we have
\[D_{8}=\langle r,s \mid r^4=s^2=1, sr=r^{-1}s\rangle.\]
(a) Let $A$ be the subgroup of $D_8$ generated by $r$, that is, $A=\{1,r,r^2,r^3\}$.
Prove that the centralizer […]

Non-Abelian Simple Group is Equal to its Commutator Subgroup
Let $G$ be a non-abelian simple group. Let $D(G)=[G,G]$ be the commutator subgroup of $G$. Show that $G=D(G)$.
Definitions/Hint.
We first recall relevant definitions.
A group is called simple if its normal subgroups are either the trivial subgroup or the group […]

Commutator Subgroup and Abelian Quotient Group
Let $G$ be a group and let $D(G)=[G,G]$ be the commutator subgroup of $G$.
Let $N$ be a subgroup of $G$.
Prove that the subgroup $N$ is normal in $G$ and $G/N$ is an abelian group if and only if $N \supset D(G)$.
Definitions.
Recall that for any $a, b \in G$, the […]

Group Generated by Commutators of Two Normal Subgroups is a Normal Subgroup
Let $G$ be a group and $H$ and $K$ be subgroups of $G$.
For $h \in H$, and $k \in K$, we define the commutator $[h, k]:=hkh^{-1}k^{-1}$.
Let $[H,K]$ be a subgroup of $G$ generated by all such commutators.
Show that if $H$ and $K$ are normal subgroups of $G$, then the subgroup […]

A Condition that a Commutator Group is a Normal Subgroup
Let $H$ be a normal subgroup of a group $G$.
Then show that $N:=[H, G]$ is a subgroup of $H$ and $N \triangleleft G$.
Here $[H, G]$ is a subgroup of $G$ generated by commutators $[h,k]:=hkh^{-1}k^{-1}$.
In particular, the commutator subgroup $[G, G]$ is a normal subgroup of […]