First PrinciplesStart anywhere. Prove it, then move on.

Rung 36

Eigenvalues and eigenvectors

The directions a transformation leaves alone.

Best attempted after 31. Basis and change of basis. Nothing stops you trying this now — the gate will tell you if you were right.

The gate

Find the eigenvectors and eigenvalues of a 2×2 matrix by hand, then verify numerically that Av equals λv for each. Then take a rotation matrix, show it has no real eigenvectors, and explain that geometrically rather than algebraically.

Nobody checks this but you. Do it honestly and the rungs above hold; do it loosely and they will not, somewhere further up where the cause is much harder to find.

Apply a transformation to a vector and it usually changes direction as well as length. Usually — but some vectors come out pointing exactly the way they went in, merely stretched or squashed. Those are the eigenvectors, and the stretch factor is the eigenvalue.

Av = λv. A matrix on the left, a plain number on the right. For those special directions, the whole transformation collapses to multiplication.

Why this is on the ladder

Because it is the cleanest example of rung 31's lesson: pick the right basis and a complicated object becomes simple. In the eigenvector basis, a transformation is just "scale this direction by this much" for each direction independently.

That is the foundation of rung 37's decomposition, of principal component analysis, and of understanding what repeated application of a transformation does — the largest eigenvalue dominating over time is why some systems blow up and others settle.

Do this

Take [[3, 1], [0, 2]]. Solve det(A − λI) = 0 for the eigenvalues. Because the matrix is triangular you should find them sitting on the diagonal, which is worth noticing. For each, solve (A − λI)v = 0 to get the direction.

Verify numerically: multiply the matrix by each eigenvector and confirm the result is that eigenvector times its eigenvalue. Componentwise, to several decimals.

Then the geometry. Use the Setosa page to drag a vector around and watch its image move; the eigenvectors are the positions where input and image line up. Seeing that happen is worth more than the determinant calculation.

Now take a 90-degree rotation. Try to find a real eigenvector and fail. Then say why in one sentence about the picture: a rotation moves every direction, so no direction survives pointing the same way. Rung 12 predicted this — the eigenvalues are complex, and complex numbers are rotation.

Where people get stuck

Losing the meaning inside the characteristic polynomial. The algebra is a means of finding the directions; the directions are the point. If the arithmetic is going badly, go back to the picture and ask which arrows could possibly stay put.

The other snag is expecting eigenvectors always to exist as real vectors. Rotations have none, some matrices have repeated eigenvalues with fewer independent directions than dimensions, and those exceptions are real rather than edge cases to be waved away.

Reading