故事 · 高斯、勒让德与「最小二乘」
Origin Story · Gauss, Legendre and "Least Squares"
1805 年 Legendre 正式发表了「最小二乘法」,但 Gauss 声称自己早在 1795 年就用它预测过小行星谷神星的轨道。
核心洞见极其朴素:观测总有误差,与其纠结哪条线「看着顺眼」,不如定一个客观标准 —— 让误差的平方和最小。
为什么是平方而不是绝对值?因为平方把大偏差狠狠放大(逼着线去照顾离群点),更重要的是它能解出唯一、漂亮的闭式公式。
后来 Galton 在研究父子身高时发现子代会「回归」到平均值,给这条线起了名字:regression(回归),一直沿用至今。
Legendre formally published the method of least squares in 1805, but Gauss claimed he had already used it in 1795 to predict the orbit of the asteroid Ceres.
The core idea is disarmingly simple: every observation carries error, so instead of arguing about which line "looks right", commit to one objective rule — minimize the sum of squared errors.
Why squared and not absolute? Squaring blows up large deviations (forcing the line to respect outliers), and — crucially — it admits a unique, closed-form solution.
Later, Galton studying father-son heights noticed that sons "regressed" toward the mean. The name regression stuck, and we've used it ever since.
1 拖点看线 · 残差平方和最小的那条 Drag a Point · The Line With Minimum Squared Residuals
R²=—2 现实里的线性回归 Linear Regression in the Real World
趋势预测:广告投入 vs 销量、温度 vs 用电量 —— 拟合一条线就能给出「投入多少、预期产出多少」的估计。
Trend forecasting: ad spend vs sales, temperature vs power draw — one fitted line gives a defensible "spend X, expect Y" estimate.
最小二乘:β₁、β₀ 不是凭感觉画的,而是让残差平方和最小的唯一解,每次重算结果一致、可复现。
Least squares: β₁ and β₀ aren't eyeballed — they're the unique minimizers of SS_res, so the answer is reproducible and audit-friendly.
R² 解释力:R²=0.85 表示 x 解释了 85% 的 y 变异,剩下 15% 是噪声或别的因素 —— 它是模型有用程度的量尺。
R² explanatory power: R² = 0.85 means x accounts for 85% of y's variation; the remaining 15% is noise or omitted drivers — a clean yardstick for how useful the model is.
外推风险:在 20-80 区间拟合的线,硬套到 x=200 上几乎必错 —— 直线关系在远处往往崩塌。
Extrapolation risk: a line fitted on x ∈ [20, 80] is almost guaranteed to fail at x = 200 — linearity tends to break down far from the training range.
一句话In One Line
回归线的本质是一句承诺:给我一个 x,我吐给你一个最优的 ŷ。「最优」的定义就是
残差平方和 SS_res 最小 —— 把每个点到线的竖直误差平方后相加,再让它降到不能再低。
R² 的逻辑是对比:如果你啥都不知道,只能拿 ȳ(一条水平线)去猜,误差是 SS_tot;
有了回归线,误差降到 SS_res。R² = 1 − SS_res/SS_tot 就是「这条线帮你削掉了多大比例的无知」。
但回归只刻画关联趋势,从不证明谁导致谁;而且它只在数据覆盖的区间内可信,外推等于赌博。
A regression line is really a promise: give me an x, I'll hand back the best possible ŷ. "Best" has a precise meaning —
minimum sum of squared residuals (SS_res): square every vertical error and shrink the total as far as it will go.
R² is a comparison: with no model you'd just guess ȳ (a flat line), incurring SS_tot of error;
with the line, error drops to SS_res. R² = 1 − SS_res/SS_tot is literally "the fraction of ignorance the line erased".
But regression only describes association, never causation; and it is only trustworthy inside the data range — extrapolation is gambling.
常见误用Common Mistakes
把回归斜率当因果证据。回归只测相关趋势;要证因果需控制实验或排除混杂变量。
Treating the slope as proof of causation. Regression only measures association — establishing causation needs a controlled experiment or careful confounder adjustment.
把线外推到数据范围之外。拟合区间外的预测不可信,直线关系在远端常常失效。
Extrapolating beyond the fitted range. Predictions outside the training range are unreliable — linear relationships rarely hold at the extremes.
只看 R² 不看残差图。R² 高也可能掩盖弯曲趋势或离群点,务必看残差是否随机散布。
Trusting R² without inspecting residuals. A high R² can still hide curvature or leverage points — always check that residuals are randomly scattered.