First PrinciplesStart anywhere. Prove it, then move on.

Rung 40

Bayes' theorem

Updating a belief when the evidence arrives.

Best attempted after 39. Expectation, variance, covariance. Nothing stops you trying this now — the gate will tell you if you were right.

The gate

Work the classic screening problem by hand: a test with high sensitivity and specificity applied to a rare condition. Compute the probability that a positive result is correct. Then verify it by simulating a large population and counting. State how the answer changes if the condition is ten times more common.

Nobody checks this but you. Do it honestly and the rungs above hold; do it loosely and they will not, somewhere further up where the cause is much harder to find.

Bayes' theorem is a rule for changing your mind. You believed something with some confidence; evidence arrived; here is exactly what you should believe now.

The formula is short and the arithmetic is easy. What is hard — and worth the rung — is that the answer routinely contradicts a strong intuition, and the intuition is wrong in a consistent, predictable direction.

Why this is on the ladder

Because a classifier's output is P(class | data), and what training gives you most naturally is P(data | class). Those two are not the same, and Bayes is the bridge. Confusing them is the single most consequential error in applied probability.

The rung also inoculates against a specific failure that will matter at rung 45: a model with excellent accuracy on a rare event can still be wrong nearly every time it fires, and no amount of staring at the accuracy number reveals that.

Do this

A condition affects 1 in 1000. The test catches 99% of cases and has a 5% false positive rate. Someone tests positive. What is the chance they have it?

Guess first, and write the guess down. Most people say something near 95%.

Now count, using a population of 100,000. One hundred have it, and 99 of those test positive. Of the 99,900 who do not, 5% — 4,995 people — also test positive. So 5,094 positives, of which 99 are real: about 1.9%.

Not 95%. Not close. The false positives from an enormous healthy group swamp the true positives from a tiny sick one, and no property of the test causes this — the base rate does.

Verify by simulation: generate the population, apply the test with those rates, count. The simulation should land near 1.9%.

Then raise the prevalence to 1 in 100 and redo it. The answer climbs to about 17% with an unchanged test. That the same test yields wildly different answers on different populations is the rung's real content.

Where people get stuck

Reading "99% accurate" as "99% of positives are correct". Those are different statements about different denominators, and the gap between them is where the entire lesson lives.

The other snag is skipping the natural-frequency count and going straight to the formula. The formula is easy to apply and easy to apply backwards. Counting a concrete population makes the answer feel inevitable instead of arbitrary, and it is also how you explain the result to someone else.

Reading