First PrinciplesStart anywhere. Prove it, then move on.

Rung 20

The derivative

The slope of a curve at a single point.

Best attempted after 19. Limits. Nothing stops you trying this now — the gate will tell you if you were right.

The gate

Derive the derivative of x² from the limit definition by hand, showing every line. Then compute it numerically in Python for a small step h and demonstrate that the error shrinks roughly in proportion to h as h shrinks — and find the h where it stops shrinking and starts growing, and explain that.

Nobody checks this but you. Do it honestly and the rungs above hold; do it loosely and they will not, somewhere further up where the cause is much harder to find.

Rung 6 gave you the slope of a line: pick two points, divide the rise by the run. A curve has no single slope — it changes everywhere. So pick two points on the curve, take that slope, and then slide them together. The limit of that ratio, as the gap closes, is the slope at a point.

That is the derivative. It is rung 19's machinery pointed at rung 6's question.

Why this is on the ladder

Because training a model is nothing but this. At rung 41 you will have a number measuring how wrong the model is, and you will need to know which way to nudge each of its parameters to make that number smaller. "Which way, and how steeply" is a derivative. There is no version of this ladder that routes around rung 20.

Do this

Derive it. For f(x) = x², the slope between x and x + h is

((x + h)² - x²) / h  =  (2xh + h²) / h  =  2x + h

Every step by hand. Then let h go to zero: the slope is 2x. Notice you could only cancel the h because h was not zero — you are taking a limit, not substituting. That is why rung 19 came first.

Now compute it numerically. Pick x = 3, so the true answer is 6. Evaluate (f(3+h) - f(3))/h for h = 0.1, 0.01, 0.001, … and print the error each time. It falls roughly tenfold each step, as 2x + h predicts.

Keep going. Somewhere around h = 1e-8 the error stops falling and starts rising, and by 1e-16 the answer is garbage. That is not a bug in your code — it is subtraction of two nearly equal floating-point numbers destroying the precision you were relying on. Finding that floor yourself is the more valuable half of this gate.

Where people get stuck

Setting h = 0 instead of taking the limit, getting 0/0, and concluding the whole thing is nonsense. The algebra has to happen while h is still alive.

The numerical floor surprises people badly, and it should. It is the first time on this ladder that the machine's arithmetic is not the arithmetic you learned, and it will not be the last — at rung 45 the same effect shows up as a gradient that mysteriously stops improving.

Reading