故事 · Neyman-Pearson 与 Cohen 的功效 Origin Story · Neyman-Pearson and Cohen on Power
1933 年 Neyman 与 Pearson 把假设检验框架化,引入了两类错误:α(弃真,假阳性)和 β(取伪,漏检)。 长期以来人们只盯着 α,直到心理学家 Jacob Cohen 在 1960 年代大声疾呼:太多研究功效低得可怜, 根本抓不到真实效应,却把"不显著"当成"没差异"。他提出效应量与功效分析,主张实验前就算好需要多大样本 —— 这成了现代实验设计的铁律:先定功效,再定 n。 In 1933, Neyman and Pearson formalized hypothesis testing and named two kinds of errors: α (rejecting a true null — a false positive) and β (missing a real effect). For decades the field stared only at α. Then in the 1960s the psychologist Jacob Cohen sounded the alarm: too many studies had laughably low power, no chance of catching real effects, yet their authors treated "not significant" as "no difference." Cohen introduced effect size and power analysis, and argued that you should calculate the required sample size before you collect a single data point. That became the iron rule of modern study design: pick the power first, then back out n.

1 两条分布的博弈:α 假阳 vs β 漏检 Two Distributions in Tension: α False Alarm vs β Miss

功效 —Power —

左边灰钟是 H₀(无效应),右边蓝钟是 H₁(真有效应)。 紫色竖线是判定门槛(由 α 决定)。门槛右侧灰区是 α 假阳性,门槛左侧蓝区是 β 漏检,蓝区右侧才是功效。 The grey bell on the left is H₀ (no effect); the blue bell on the right is H₁ (a real effect). The purple vertical line is the decision threshold (set by α). The grey area to its right is the α false-positive zone; the blue area to its left is the β miss zone. Everything in blue to the right of the threshold is power.

2 样本量越大,功效曲线越快爬到顶 Larger n Drives the Power Curve to the Ceiling Faster

横轴是样本量 n,纵轴是功效。绿线是当前 d、α 下功效随 n 的变化,红点是你现在的位置,虚线是 80% 功效的常用目标 —— 样本不足时它远在曲线左下方。 x-axis is sample size n, y-axis is power. The green curve shows how power grows with n at the current d and α, the red dot is your current position, and the dashed line is the conventional 80% power target — when you're underpowered, you sit well below and to the left of it.

3 现实里的功效与样本量 Power and Sample Size in the Real World

实验设计:临床试验、工厂改善前都做功效分析,先算好 n,否则白烧预算。 Study design: clinical trials and process-improvement projects run a power analysis up front to lock in n — otherwise the budget burns for nothing.
避免假阴性:功效不足时"不显著"≠"没效果",可能只是样本太小没抓住。 Beware false negatives: when power is low, "not significant" does not equal "no effect" — the sample may simply be too small to detect what's real.
效应量:报告差异要附 Cohen's d 或 η²,光看 p 值不知道差异到底大不大。 Report the effect size: pair every difference with Cohen's d or η². A p-value alone never tells you how big the effect actually is.
A/B 测试样本量:想检出 1% 转化率提升,需要的曝光量可能是百万级,先算再上线。 A/B test sizing: to detect a 1% lift in conversion you may need millions of impressions. Run the calculation before shipping the experiment, not after.
一句话In One Line
功效 = 1 − β,本质是"真效应存在时,检验能把它揪出来的概率"。它由三件事决定: 效应量越大(两钟越分开)、样本量越大(钟越窄)、α 越宽(门槛越靠左),功效就越高。 这四个量锁死成一个方程,定了三个,第四个就唯一确定。先决定要多大功效(常用 80%),再倒推 n —— 这才是负责任的实验设计。 样本不足,等于花了钱却没有能力看见真相。 Power = 1 − β is, at heart, "the probability the test catches the effect when the effect is real." Three levers drive it: a larger effect size (the bells pull apart), a larger sample size (the bells get tighter), and a more permissive α (the threshold slides left). Any of these pushes power up. The four quantities are locked in one equation — fix any three and the fourth is determined. Pick the power you want first (80% is the convention), then back-solve for n. That's responsible study design. An underpowered study spends real money to remain unable to see the truth.
常见误用Common Mistakes
"p 不显著就是没差异"。功效不足时漏检常见,要先确认 n 足够才能下结论。 "A non-significant p means no difference". Underpowered studies miss real effects routinely. Confirm n is sufficient before drawing any conclusion.
实验后才补做功效分析。功效分析必须在收数据前做,事后功效几乎没意义。 Running power analysis after the experiment. Power analysis is a pre-data exercise. Post-hoc power is statistically near-useless.
只关心 α、不管 β。两类错误要一起权衡,漏检的代价有时远大于误报。 Watching α, ignoring β. Both error types deserve a seat at the table. The cost of a miss often dwarfs the cost of a false alarm.

样本量与功效