Logits · Raw Scores Before Probability

01 · 先看裁判打分 01 · Start with judges' scores

歌唱比赛的原始分数可以有负数、也不必加起来等于 1；模型选字前的分数也一样吗？ Contest scores may be negative and never need to add up to one. Do a model's pre-selection scores work the same way?

02 原始打分板：直接拖柱顶 Raw Scoreboard: Drag a Bar Top

分数 → 概率 · 1/3 score → probability · 1/3

Logit 总和 Logit sum

0.0

无需等于 1 need not equal 1

最高分候选 Top-scored token

0.0

—

概率预览总和 Preview probability sum

1.000

softmax preview

03 把一根柱拆成 4 个零件 Break One Bar into Four Parts

同一组候选 Same candidates

PART 01

Context

已有 token 提供上下文，例如“天空是”。 Existing tokens provide context, such as “the sky is”.

PART 02

Candidate token

每根柱对应一个可能的下一 token。 Each bar stands for one possible next token.

PART 03

Logit z_i

可正可负、无固定上下界的相对分数。 A relative score that may be positive or negative, with no fixed range.

PART 04

Rank

分数越高，softmax 后概率排序也越高。 A higher score keeps a higher rank after softmax.

04 最小机制：只看分数差 Minimal Mechanism: Only Gaps Matter

z_i − z_j

z = [z₁, z₂, z₃]

给全部 logits 同加一个常数，概率不变；真正起作用的是候选之间的差距。 Adding the same constant to every logit leaves probabilities unchanged. The gaps between candidates are what matter.

拖高一项 Raise one score 分数差扩大 its gap widens

排序上升 Rank rises 候选靠前 candidate moves ahead

概率份额变大 Probability share grows 但 logit 仍非概率 but a logit is still not probability

05 边界实验：让 logits 恰好加起来等于 1 Boundary Test: Make the Logits Sum to One

主动制造失败 Create a failure

“总和为 1”仍不等于概率 A sum of one still does not make probabilities

把三项设成 0.6、0.3、0.1。它们看起来像概率，但 softmax 后会得到另一组数。 Set the scores to 0.6, 0.3, and 0.1. They look like probabilities, but softmax produces a different set.

尚未执行。观察主画布右侧的概率预览。 Not run yet. Watch the probability preview on the right of the chart.

✗ logit = 0 表示不可能。 A zero logit means impossible. → ✓ 它只是相对分数，softmax 后概率仍大于 0。 It is a relative score; softmax still gives it positive probability.

✗ 负 logit 就是负概率。 A negative logit is a negative probability. → ✓ 概率不会为负；负数只表示相对更低。 Probabilities cannot be negative; the score is merely lower.

✗ 把 logits 凑成 1 就完成归一化。 Making logits sum to one normalizes them. → ✓ 要经 softmax 才得到概率分布。 Softmax is what turns them into a probability distribution.

06 · 一句话带走 06 · One line to keep

Logits 是候选 token 的未归一化相对分数：拖动柱子会改变排序与概率预览，但分数本身不必在 0–1，也不必总和为 1。 Logits are unnormalized relative scores for candidate tokens: dragging a bar changes ranking and probability preview, but the scores need not sit in 0–1 or sum to one.