Sampling · Draw the Next Token by Probability

01 · 先想一个不均匀转盘 01 · Picture an uneven wheel

红色占 62% 的转盘，第一次会不会偏偏落到只有 11% 的蓝色？转 100 次又会怎样？ On a wheel that is 62% red, could the first spin still land on an 11% blue slice? What happens over 100 spins?

02 采样台：左边抽一次，右边累积成直方图 Sampling Bench: One Wheel Draw, One Growing Histogram

转盘 + 100 次 wheel + 100 draws

累计抽样 Total draws

/ 100

最近抽中 Latest token

—

尚未抽样 not drawn yet

分布偏差 Distribution gap

—

越低越接近 lower is closer

03 一次采样的 4 个零件 Four Parts of One Sample

同一分布 same distribution

PART 01

Distribution

62%、27%、11% 决定三个扇区的面积。 62%, 27%, and 11% set the three slice areas.

PART 02

Seed

固定种子生成可重放的伪随机数序列。 A fixed seed produces a replayable pseudorandom sequence.

PART 03

Draw

一个 0–1 随机数落入某个累计概率区间。 One random number from 0–1 lands in a cumulative probability interval.

PART 04

Frequency

每次结果计数，形成经验频率直方图。 Each result increments a count, building an empirical histogram.

04 最小机制：单次可以意外，多次才显露分布 Minimal Mechanism: One Draw Can Surprise; Many Reveal the Distribution

f_i / n → p_i

p̂_i = count_i / n

右侧实心柱是经验频率 p̂；虚线目标是原概率 p。n 增大时通常更接近，但不保证每一步都更近。 Solid bars on the right are empirical frequencies p̂; dashed targets are original probabilities p. They usually get closer as n grows, but not on every single draw.

概率划分区间 Probabilities form intervals 大概率占更宽 larger p gets more space

随机点落入 Random point lands 一次只出一个 token one token per draw

频率逐渐稳定 Frequencies stabilize 直方图逼近目标 histogram approaches target

05 边界实验：只抽一次就宣布分布 Boundary Test: Declare the Distribution after One Draw

主动制造失败 Create a failure

一次结果不等于概率分布 One outcome is not a probability distribution

清空后只抽一次。直方图会显示某一项 100%、其余 0%，与 62/27/11 的目标相差很远。 Clear and draw once. The histogram shows one item at 100% and the rest at 0%, far from the 62/27/11 target.

尚未执行。单次样本不能反推出完整分布。 Not run yet. One sample cannot recover the full distribution.

✗ 采样总会选最高概率项。 Sampling always picks the highest-probability item. → ✓ 低概率项也占有非零扇区。 Low-probability items still own nonzero slices.

✗ 62% 表示每 100 次必定出现 62 次。 62% means exactly 62 appearances in every 100 draws. → ✓ 62 次是期望附近，不是硬配额。 Sixty-two is a long-run expectation, not a quota.

✗ 随机就无法复现。 Random means unreproducible. → ✓ 固定伪随机种子可以重放同一抽样序列。 A fixed pseudorandom seed replays the same draw sequence.

06 · 一句话带走 06 · One line to keep

采样按候选概率划分转盘并抽一个 token：单次可能落到小扇区，重复到 100 次时，直方图才逐渐显出原分布。 Sampling divides a wheel by candidate probabilities and draws one token: a single draw may hit a small slice, while a 100-draw histogram gradually reveals the original distribution.