Find all affine transformations that map the ellipse…

Find all affine transformations that map the ellipse $\dfrac{x^2}{a^2}+\dfrac{y^2}{b^2}=1$ onto itself.

Step 1: Reduction to the circle

An affine map sends the centre (the centre of symmetry) to the centre, so it fixes $O$ and is linear. With $X=\tfrac xa$, $Y=\tfrac yb$ the ellipse becomes the unit circle $X^2+Y^2=1$. A linear map preserving the unit circle preserves length, so it is orthogonal:

\[ \begin{pmatrix}\cos t&-\sin t\\ \sin t&\cos t\end{pmatrix}\quad\text{or}\quad\begin{pmatrix}\cos t&\sin t\\ \sin t&-\cos t\end{pmatrix}. \]

Step 2: Back to $x,y$

Conjugating with $\operatorname{diag}(a,b)$: $\text{I: }\ x=x'\cos t-\tfrac ab\,y'\sin t,\quad y=\tfrac ba\,x'\sin t+y'\cos t$.

\[\text{II: }\ x=x'\cos t+\tfrac ab\,y'\sin t,\quad y=\tfrac ba\,x'\sin t-y'\cos t\]

Family I maps $P(a\cos s,b\sin s)$ to $(a\cos(s+t),b\sin(s+t))$, i.e. it slides points along the ellipse. Family II maps it to $(a\cos(t-s),b\sin(t-s))$. For $t=0$ this is the reflection in the $x$-axis, and for $t=\pi$ the reflection in the $y$-axis. The identity (I, $t=0$) and the central symmetry (I, $t=\pi$) are included. For $a=b$ we get exactly the rotations and reflections of the circle.