Probability & Statistics · Reasoning under uncertainty

Probability & Statistics

Probability reasons from a model to the data; statistics works back from data to the model. Together they decide what to believe, and how firmly. Wear both on button-ups and socks, with a mathematics tie to match.

Behind this page a Galton board of 12 rows of pegs fills as you scroll. Each peg is a fair coin flip, not a collision simulation (seeded, so every visit sees the same balls), and between pegs each ball falls under gravity. 4,096 balls, 2 to the 12th, fall in doubling batches. The first ball goes left right right right right left left left right left right left and lands in bin 6. For Bayes’ theorem, after 16 balls, the posterior for the chance of bouncing right is 0.4845 plus or minus 0.0358; it narrows with every ball, to 0.4966 plus or minus 0.0023 at 4,096. For the central limit theorem, after 1,024 balls, the pile follows the binomial counts and the normal curve with mean 6 and variance 3. For the law of large numbers, the running mean bin after all 4,096 balls is 5.960, against 6 expected. For regression to the mean, the 146 balls that finished four or more bins out are dropped again from where they passed row 6, halfway down: they finished 4.12 bins out, and their re-runs land 2.23 bins out, about half as far, as Galton’s two-stage board shows. For the p-value, a chi-squared test of the 4,096 against a fair board gives 8.34 on 6 degrees of freedom, p = 0.214; tested after every ball from 128 on, p wanders, and is below 0.05 at 8.6 percent of the looks, though the board is fair. For Simpson’s paradox, two more boards (a model): one leans left, one right, and yellow balls drop one peg to the right of blue ones, so on each board yellow finishes about one bin further right; but most yellow balls go to the left-leaning board, so pooled, yellow finishes 0.49 bins further left. At the end the pegs show Pascal’s triangle, the number of paths to each peg: a Galton board is Pascal’s triangle with gravity, and with 4,096 balls each bin expects exactly its binomial coefficient, 924 in the middle bin, which counted 939.

Why it matters

For anyone who asks what the sample size was

Probability began at the gaming table. In 1654 Blaise Pascal and Pierre de Fermat exchanged letters on how to divide the stakes of a game stopped before its end, and laid the foundations of expected value. Statistics grew up in less likely places. In 1908 William Sealy Gosset, a chemist at the Guinness brewery in Dublin, published the t-test under the pen name Student, because the company preferred its staff not to publish under their own names. It was built for small samples, which is what a brewery mostly has.

Good practice means plotting the data before modeling it, because Francis Anscombe showed in 1973 that four datasets with near-identical summary statistics can look nothing alike. It means asking about confounders, missing values and how the sample was drawn before anyone mentions a p-value. It means R or Python in one window and the study design in another, a long-running and mostly friendly argument between Bayesians and frequentists, and simulation whenever the algebra runs out. Statisticians are often asked to rescue an experiment afterwards; the lucky ones are invited before it starts.

SAMPLE SIZE

Eight cups, two pots, 787 guesses, fifteen pairs.

Each is a classic for its n: Fisher’s eight cups, Student’s two pots of barley, Galton’s 787 tickets and Darwin’s fifteen pairs of maize, from the smallest sample that can carry a test to a crowd.

  • The Lady Tasting TeaEight identical plain white porcelain cups of milky tea on saucers, evenly spaced in a ring on a round walnut tray, every handle pointing right.
  • Student’s tOne ripe ear of two-row barley, golden, about 24 grains in a herringbone, long awns to the upper right.
  • Vox PopuliA prize fat ox, a red-and-white beef steer of a traditional English type, in exact side view.
  • Darwin’s Crossed MaizeFour terracotta pots of young maize from Darwin’s experiment, crossed plants on one side of each divider and self-fertilized plants on the other.
Tell me when it’s ready

Leave your email and we’ll write the day it’s ready.

INVERSE PROBABILITY

From what you saw back to what must be true.

Inverse probability was the nineteenth-century name for reasoning from effects to causes, the name Fisher used when he attacked it. Bayes’s posterior, the urn that updates like a Bayesian, Bertrand’s lesson in conditioning and the likeliest population behind a recapture all run the same direction, from data back to the unknown.

  • Bayes’ Billiard TableA square table of pale maple in walnut rails, no pockets: five red balls left of the brass rod, two right of it, and the ivory ball on its line.
  • The Pólya UrnA clear hand-blown glass urn of classical shape, with 7 ruby and 5 milk-white glass balls heaped at the bottom.
  • Bertrand’s BoxThree identical walnut caskets, one top center and two below, each with two drawers pulled out and one plain medal in each: gold-gold, silver-silver, gold-silver.
  • Capture–RecaptureA European plaice, eyed side up, orange-spotted, one small blank bone disk wired on near the base of the dorsal fin.
Tell me when it’s ready

Leave your email and we’ll write the day it’s ready.

MEMORYLESS

Where it goes next depends only on where it is.

Four random processes, each forgetting its past: a lattice walk that may never come home, Perrin’s jittering grain, Markov’s vowels and consonants, and Bortkiewicz’s horse kicks. Memorylessness is the property every stochastic-processes student learns to check first.

  • Pólya’s Random WalkA clear optical-glass cube holding a walk of 36 equal brass rods at right angles, from a gold bead at the center to a ruby bead near a corner.
  • Brownian MotionA dark-field multiple exposure: 40 bright points, one grain’s positions every 30 s, joined by a faint, jagged thread of light.
  • The Markov ChainA strand of 25 beads lying in an open loop, carnelian for vowels and bone for consonants in the order of the first line (C V C C V C V C V C V C C V C C C V C C C V C V C).
  • The Law of Small NumbersA plain hand-forged iron horseshoe lying flat, toe at the top, 8 nail holes in the fuller, 4 a side.
Tell me when it’s ready

Leave your email and we’ll write the day it’s ready.

Socks mix and match across every field: three pairs for $69, and two ship free in the U.S.

Crew socks · One size

Crew socks

A field of your own, from cuff to toe.

  • The pattern
  • The previewA studio mockup; photos of the finished pair are on the way.
Shop now · Regular price $26 Sale price $23.40

Leave your email and we’ll write the day this pair is ready.

Necktie · 3.5 in wide

Subject ties

A formal introduction to your field.

  • The color
  • The previewStudio render of the tie. Zoom shows the print up close.
Shop now · Regular price $36 Sale price $32.40

Leave your email and we’ll write the day this tie is ready.

The science on the shirt

What you’re wearing

Every object on the shirts, and the science behind it.

The Lady Tasting Tea

SAMPLE SIZE

The Lady Tasting Tea

8 cups, 4 milk first: 1 in 70

The null hypothesis, randomization and an exact p-value in one cup of tea; every statistics course tells it.

Student’s t

SAMPLE SIZE

Student’s t

n = 2: ±12.71 s.e., not ±1.96

Every t-test ever run comes from this paper, and the pseudonym (his employer did not let staff publish under their own names) is the field’s favorite piece of trivia.

Vox Populi

SAMPLE SIZE

Vox Populi

787 guesses: median within 1 %

The founding story of the wisdom of crowds, and a case for the median that every robust statistician enjoys.

Darwin’s Crossed Maize

SAMPLE SIZE

Darwin’s Crossed Maize

13 of 15 crossed plants taller

Darwin grew fifteen pairs of young maize, one cross-fertilized and one self-fertilized plant in each, four pots of them, and published the heights in 1876. The crossed plant was the taller in 13 of the 15 pairs; Fisher reanalyzed the numbers in The Design of Experiments (1935), and they remain the classic small paired sample.

Bayes’ Billiard Table

INVERSE PROBABILITY

Bayes’ Billiard Table

5 left, 2 right: Beta(6, 3)

Bayesian updating with nothing hidden, the prior being physical; it is the picture Bayesians draw when someone calls priors arbitrary.

The Pólya Urn

INVERSE PROBABILITY

The Pólya Urn

every final mix equally likely

The simplest rich-get-richer process and the cleanest bridge from urns to Bayes; nonparametric Bayesians meet its big sibling in the Dirichlet process.

Bertrand’s Box

INVERSE PROBABILITY

Bertrand’s Box

the other medal is gold: 2/3

It teaches conditioning on how the evidence arose; anyone who has argued about the three doors has argued about this box.

Capture–Recapture

INVERSE PROBABILITY

Capture–Recapture

N ≈ Mn/m, the likeliest N

Maximum likelihood you can do in a field notebook; ecologists and epidemiologists run it every season.

Pólya’s Random Walk

MEMORYLESS

Pólya’s Random Walk

in 3-D, only 34.05 % return

Recurrence against transience in one line, and the probabilist’s favorite dinner-party theorem.

Brownian Motion

MEMORYLESS

Brownian Motion

positions every 30 s, 1909

The experiment that made atoms real, and the original random-walk figure every stochastic-processes course reprints.

The Markov Chain

MEMORYLESS

The Markov Chain

vowel after vowel: 12.8 %

The first Markov chain fitted to data, and the ancestor of every n-gram language model.

The Law of Small Numbers

MEMORYLESS

The Law of Small Numbers

horse kicks: 0.61 a corps-year

The most quoted dataset for the Poisson distribution, and a grim joke that survives every introductory course.

The big ideas

What people in probability and statistics carry around

Bayes’ Theorem

P(A|B) = P(B|A)P(A)/P(B): how evidence should update belief. Published in 1763, two years after Bayes died.

The Central Limit Theorem

Sums of many independent variables with finite variance tend toward a normal distribution, whatever their own shape. Hence bell curves everywhere.

The Law of Large Numbers

Averages of repeated trials settle toward the expected value. Jacob Bernoulli proved the first version, published in 1713.

Regression to the Mean

Extreme results tend to be followed by less extreme ones. Francis Galton noticed it in the heights of parents and their children.

The p-Value

The probability of data at least this extreme if the null hypothesis were true. Not the probability that the hypothesis is true.

Simpson’s Paradox

A trend in every subgroup can reverse when the groups are combined. Berkeley’s 1973 graduate admissions are the classic case.

Shop Probability & Statistics: Wear it to work

Made to order for you · free U.S. shipping on $46+ (shirts, ties and socks) · 10% off through 30 Nov

More Probability & Statistics designs are on the way.

Leave your email and we’ll write the day they’re ready.