First PrinciplesStart anywhere. Prove it, then move on.

Rung 31

Basis and change of basis

The same vector, described from a different point of view.

Best attempted after 29. Matrices as transformations. Nothing stops you trying this now — the gate will tell you if you were right.

The gate

Take one vector and write its coordinates in two different bases, by hand. Build the change-of-basis matrix and its inverse, and show that converting there and back returns exactly the original coordinates. Then say what stayed the same throughout, and what only ever depended on the description.

Nobody checks this but you. Do it honestly and the rungs above hold; do it loosely and they will not, somewhere further up where the cause is much harder to find.

Coordinates are not a property of a vector. They are a property of a vector and the axes you chose. The arrow does not move when you change basis; only the numbers describing it do.

This is a small idea that reorganises everything after it.

Why this is on the ladder

Because a great deal of what follows is choosing a better description of the same data. Rung 36 finds the directions a transformation treats simply, and rung 37 throws away the directions that carry little information. Both are changes of basis. Neither makes sense if coordinates feel absolute.

It also explains what the hidden layers of a network are doing. Each layer re-describes the input in terms it finds more useful. That is a change of basis with a bend added.

Do this

Take the vector that is (3, 2) in the usual basis. Now use a different pair of axes — say (1, 1) and (-1, 1). Ask the defining question: how much of each new axis do I need to reach the same arrow? Solve for the two amounts. Those are its coordinates in the new basis, and they are different numbers describing the identical arrow.

Build the change-of-basis matrix whose columns are the new basis vectors written in the old coordinates. Confirm it converts new coordinates back to old. Compute its inverse and confirm it goes the other way. Round-trip a vector: out and back, landing on exactly what you started with, allowing for floating point.

Then check the invariants. The arrow's length did not change. Its angle with another arrow did not change. Only the numbers did. Say which of the quantities you have computed on this ladder are properties of the vector and which are properties of the description.

Where people get stuck

Getting the direction backwards — using the matrix where the inverse is needed. The round-trip test in the gate exists because it catches this immediately, and because reasoning about which way it goes tends to produce confident errors.

The other difficulty is believing the standard basis is special. It is not. It is a default, and the moment a different one describes your data better, the default is just a habit.

Reading