S4-09 · 进阶 S4-09 · Advanced

Top-p 采样 · 让候选数跟着分布自适应 Top-p Sampling · Let Candidate Count Adapt to the Distribution

sort probabilities → smallest cumulative set ≥ p → renormalize → sample

Top-p 采样保留累计概率达到 p 的最小候选集合,再从中采样。候选数会随概率分布的尖锐或平坦自动改变。 Top-p Sampling keeps the smallest candidate set whose cumulative probability reaches p, then samples from it. Candidate count adapts to how sharp or flat the distribution is.

01 · 装满八成水就停 01 · Stop once the cup reaches eighty percent

同样要求装到 80%,为什么大水杯可能只倒两勺,小水杯却要倒五勺? For the same 80% target, why can one distribution need two scoops while another needs five?

02 拖 p 并切换分布,追踪累计截点 Move p, Switch Distributions, and Track the Cutoff

自适应核 Adaptive nucleus
状态 A Metric A
状态 B Metric B
状态 C Metric C

概率与输出均为固定种子的教学模拟,不代表真实模型。拖回主变量即可复核结论。 Probabilities and outputs are fixed-seed teaching simulations, not real model output. Move the main variable back to verify the conclusion.

03 把变化拆成 4 个零件 Break the Change into Four Parts

Top-p Sampling
PART 01

概率排序 Probability order

候选仍先按概率从高到低排列。 Candidates are still sorted from highest to lowest probability.

PART 02

累计曲线 Cumulative curve

从头部逐项相加,曲线单调向 1 上升。 Probabilities accumulate from the top, producing a curve that rises toward one.

PART 03

最小集合 Smallest set

刚好让累计质量达到或超过 p 时停止收集。 Collection stops as soon as cumulative mass reaches or exceeds p.

PART 04

自适应数量 Adaptive count

尖锐分布用少量候选,平坦分布需要更多候选。 Sharp distributions need fewer candidates; flat ones need more.

04 最小机制:找第一个累计概率越过 p 的位置 Minimal Mechanism: Find the First Cumulative Point Crossing p

Top-p Sampling
k* = min{k : Σi≤k pi ≥ p}
p 是概率质量阈值,k* 是由当前分布算出的候选数量,不是手工固定值。 p is a probability-mass threshold. k* is derived from the current distribution, not fixed by hand.
拖动 p Move p 累计目标改变 cumulative target changes
切换分布 Switch shape 曲线陡峭度改变 curve steepness changes
移动截点 Move cutoff 保留数量自适应 retained count adapts

05 边界实验:平坦分布配高 p Boundary Test: Pair a Flat Distribution with High p

主动制造失败 Create a failure

采样核几乎吞下整个词表尾部 The nucleus nearly swallows the whole tail

切到平坦分布并设 p = 0.95。累计曲线很慢才越线,弱候选也会进入保留集合。 Use the flat distribution and p = 0.95. The cumulative curve crosses late, admitting weak candidates into the retained set.

尚未执行。先在主交互中观察正常机制。 Not run yet. Observe the normal mechanism in the main interaction first.
Top-p 永远保留固定数量。 Top-p always keeps a fixed count. → ✓ 它固定概率质量,数量随分布改变。 It fixes probability mass; count changes with the distribution.
p = 0.8 就恰好保留 80%。 p = 0.8 keeps exactly 80%. → ✓ 最后一个完整候选可能让累计值略高于 p。 The final whole candidate may push cumulative mass above p.
Top-p 会修改模型 logits。 Top-p modifies model logits. → ✓ 它在概率生成后裁剪并重归一化。 It prunes and renormalizes after probabilities are produced.
06 · 一句话带走 06 · One line to keep

Top-p 保留累计概率达到 p 的最小集合;切换尖锐/平坦分布,就能看到截点和候选数自动移动。 Top-p keeps the smallest set reaching cumulative mass p. Switch between sharp and flat distributions to watch the cutoff and candidate count move automatically.