S4-07 · 基础S4-07 · Foundation

温度 · 候选概率的调音旋钮 Temperature · The Probability Tuning Knob

fixed logits ÷ temperature → softmax → sample

温度缩放 logits 的差距,控制采样分布是尖锐还是平坦。它不改原始 logits,只改变候选之间的相对概率。 Temperature scales the gaps between logits, making the sampling distribution sharper or flatter. It leaves the original logits unchanged.

01 · 先从选择困难开始01 · Start with a choice

同一组候选词,怎样让回答更稳定,又怎样让它偶尔冒出新花样? With the same candidate words, how do you make answers steadier—or let fresh options surface?

02 概率风洞:只动温度Probability Wind Tunnel: Change Temperature Only

滑块分布型Distribution slider
最高概率Top probability
0%
分布熵Entropy
0.00
越高越分散higher = flatter
Logit range
3.4
始终不变always fixed

03 把滑块变化拆成 4 个零件Break the Slider Effect into Four Parts

同一组候选Same candidates
PART 01

Logits

候选 token 的未归一化分数。上方五个值固定,不随温度改变。 Unnormalized candidate scores. The five values above stay fixed as temperature moves.

PART 02

温度缩放Temperature scaling

每个 logit 都除以同一个 T。T 小会放大差距,T 大会压缩差距。 Every logit is divided by the same T. Low T widens gaps; high T compresses them.

PART 03

Softmax

把缩放后的分数转成总和为 1 的概率分布。 Converts the scaled scores into probabilities that sum to one.

PART 04

采样Sampling

按新概率抽取下一 token。温度不是“创造力开关”,只是改变抽样权重。 Draws the next token from the new probabilities. Temperature changes weights; it is not a creativity switch.

04 最小公式:差距怎样被放大或压平Minimal Formula: Widen or Compress the Gaps

Σpi = 1
pi(T) = ezi/T / Σjezj/T
z 是不变的 logit;T 是上方唯一变量;p 是画布中的彩色柱。 z is the fixed logit, T is the only variable above, and p is each colored bar.
降低 TLower T z/T 差距放大z/T gaps widen
概率变尖Distribution sharpens 头部候选更集中Top candidates dominate
抽样更稳定Samples stabilize 重复词更常出现The same token appears more often

05 边界实验:把分布调得太平Boundary Test: Flatten the Distribution Too Far

主动制造失败Create a failure

让低分候选过度抬头Overpromote weak candidates

点击把 T 拉到 2.0。观察最高概率下降、熵上升,原本很弱的候选也更常进入样本。 Set T to 2.0. Watch the top probability fall, entropy rise, and weak candidates enter the sample more often.

尚未执行。高温不保证“更有创意”,只会让分布更平。 Not run yet. High temperature does not guarantee creativity; it only flattens the distribution.
温度会修改模型参数或原始 logits。Temperature changes model weights or raw logits. → ✓ 它只在采样前缩放固定 logits。It only scales fixed logits before sampling.
温度越高,答案质量一定越高。Higher temperature always improves quality. → ✓ 高温增加多样性,也会提高弱候选被抽中的机会。High T adds variety and also raises the odds of weak candidates.
低温等于完全确定。Low temperature is fully deterministic. → ✓ 只要还在采样,非头部候选通常仍有非零概率。While sampling remains enabled, non-top candidates usually retain nonzero probability.
06 · 一句话带走06 · One line to keep

温度通过缩放 logits 的差距,让采样概率变尖或变平;拖回低温,最高候选会再次集中,原始 logits 始终不动。 Temperature sharpens or flattens sampling probabilities by scaling logit gaps. Drag it back down and the top candidate concentrates again while raw logits stay fixed.