Softmax · Squeeze Scores into Probabilities

01 · 先想一块固定大小的蛋糕 01 · Picture one fixed-size cake

三个人的胃口分数想写多大都行，怎样把它们变成恰好分完一整块蛋糕的份额？ Three appetite scores can be any size. How do you turn them into shares that divide exactly one whole cake?

02 归一化压舱：分数柱原地变成概率柱 Normalization Press: Scores Become Probability Bars

分数 → 概率 · 2/3 score → probability · 2/3

当前视图 Current view

logits

柱子总和 Bar total

6.5

任意 arbitrary

头部份额 Top share

—

先执行 softmax apply softmax first

03 一次 softmax 的 4 个动作 Four Moves Inside Softmax

Σp_i = 1

PART 01

减去最大值 Subtract the maximum

数值更稳定，同时保留所有分数差。 Keeps the calculation stable without changing score gaps.

PART 02

Exponentiate

用 e^z 把任意实数变成正数。 Uses e^z to turn every real score positive.

PART 03

求总量 Add the total

把所有正数份额加成共同分母。 Adds all positive weights into one shared denominator.

PART 04

按总量分杯 Divide by the total

每项除以总量，最后严格加到 1。 Divides each item by the total so the result sums exactly to one.

04 最小公式：每一项都在同一个分母里竞争 Minimal Formula: Every Item Competes in One Denominator

softmax(z)

p_i = e^z_i / Σ_je^z_j

上方调高一个 z_i，分子变大，同时总分母也变大，所以其他 p_j 会被挤小。 Raise one z_i above: its numerator grows, and so does the shared denominator, squeezing every other p_j.

一项 +1 One score +1 e^z 放大 e^z grows

共同分母变大 Shared total grows 全体重新分配 all shares reallocate

总和仍为 1 Total stays at one 竞争不会溢出 competition cannot overflow

05 边界实验：把最高分拉到 12 Boundary Test: Push the Top Score to 12

主动制造失败 Create a failure

显示成 0.0% 不等于数学上为零 Displaying 0.0% does not mean mathematical zero

制造极端分数差，尾部概率会小到显示精度以下，但 softmax 仍给每项正概率。 Create an extreme score gap. Tail probabilities fall below display precision, but softmax still assigns every item a positive probability.

尚未执行。注意 0.0% 只是四舍五入。 Not run yet. Remember that 0.0% may only be rounding.

✗ softmax 只把每项除以 logits 总和。 Softmax simply divides by the sum of logits. → ✓ 先取指数，再除以指数和。 It exponentiates first, then divides by the exponential total.

✗ 调高一项只影响它自己。 Raising one item affects only itself. → ✓ 共同分母让所有候选一起变化。 The shared denominator moves every candidate.

✗ 显示 100% 就是绝对确定。 A displayed 100% means absolute certainty. → ✓ 先检查显示精度和尾部非零值。 Check display precision and nonzero tail values first.

06 · 一句话带走 06 · One line to keep

Softmax 把任意 logits 挤成总和为 1 的概率：调高一根柱会让它抢到更多份额，也会同时挤小所有其他候选。 Softmax squeezes arbitrary logits into probabilities summing to one: raise one bar and it gains share while every competing candidate shrinks.