温度 · 候选概率的调音旋钮Temperature · The Probability Tuning Knob
fixed logits ÷ temperature → softmax → sample
温度缩放 logits 的差距,控制采样分布是尖锐还是平坦。它不改原始 logits,只改变候选之间的相对概率。Temperature scales the gaps between logits, making the sampling distribution sharper or flatter. It leaves the original logits unchanged.
01 · 先从选择困难开始01 · Start with a choice
同一组候选词,怎样让回答更稳定,又怎样让它偶尔冒出新花样?With the same candidate words, how do you make answers steadier—or let fresh options surface?
T°
02概率风洞:只动温度Probability Wind Tunnel: Change Temperature Only
滑块分布型Distribution slider
最高概率Top probability
0%
—
分布熵Entropy
0.00
越高越分散higher = flatter
Logit range
3.4
始终不变always fixed
03把滑块变化拆成 4 个零件Break the Slider Effect into Four Parts
同一组候选Same candidates
PART 01
Logits
候选 token 的未归一化分数。上方五个值固定,不随温度改变。Unnormalized candidate scores. The five values above stay fixed as temperature moves.
PART 02
温度缩放Temperature scaling
每个 logit 都除以同一个 T。T 小会放大差距,T 大会压缩差距。Every logit is divided by the same T. Low T widens gaps; high T compresses them.
PART 03
Softmax
把缩放后的分数转成总和为 1 的概率分布。Converts the scaled scores into probabilities that sum to one.
PART 04
采样Sampling
按新概率抽取下一 token。温度不是“创造力开关”,只是改变抽样权重。Draws the next token from the new probabilities. Temperature changes weights; it is not a creativity switch.
04最小公式:差距怎样被放大或压平Minimal Formula: Widen or Compress the Gaps
Σpi = 1
pi(T) = ezi/T / Σjezj/T
z 是不变的 logit;T 是上方唯一变量;p 是画布中的彩色柱。z is the fixed logit, T is the only variable above, and p is each colored bar.
抽样更稳定Samples stabilize重复词更常出现The same token appears more often
05边界实验:把分布调得太平Boundary Test: Flatten the Distribution Too Far
主动制造失败Create a failure
让低分候选过度抬头Overpromote weak candidates
点击把 T 拉到 2.0。观察最高概率下降、熵上升,原本很弱的候选也更常进入样本。Set T to 2.0. Watch the top probability fall, entropy rise, and weak candidates enter the sample more often.
尚未执行。高温不保证“更有创意”,只会让分布更平。Not run yet. High temperature does not guarantee creativity; it only flattens the distribution.
✗ 温度会修改模型参数或原始 logits。Temperature changes model weights or raw logits. → ✓ 它只在采样前缩放固定 logits。It only scales fixed logits before sampling.
✗ 温度越高,答案质量一定越高。Higher temperature always improves quality. → ✓ 高温增加多样性,也会提高弱候选被抽中的机会。High T adds variety and also raises the odds of weak candidates.
✗ 低温等于完全确定。Low temperature is fully deterministic. → ✓ 只要还在采样,非头部候选通常仍有非零概率。While sampling remains enabled, non-top candidates usually retain nonzero probability.
06 · 一句话带走06 · One line to keep
温度通过缩放 logits 的差距,让采样概率变尖或变平;拖回低温,最高候选会再次集中,原始 logits 始终不动。Temperature sharpens or flattens sampling probabilities by scaling logit gaps. Drag it back down and the top candidate concentrates again while raw logits stay fixed.