Repetition Penalty

01 · 把回声旋钮拧高01 · Turn up the anti-echo dial

开会时有人不断重复同一句话；如果规定“说过的词都别再说”，重点词还能正常复述吗？Someone keeps repeating the same line in a meeting. If every spoken word is suppressed, can an important term still be repeated when needed?

02拖惩罚值，看已出现词候选柱被压低Move the Penalty and Watch Seen-token Bars Fall

唯一变量：惩罚值one variable: penalty

状态 AMetric A

—

状态 BMetric B

—

状态 CMetric C

—

所有概率、得分与输出均为固定种子的教学模拟，不代表真实模型。拖回主变量即可复核结论。All probabilities, scores, and outputs are fixed-seed teaching simulations, not real model output. Move the main variable back to verify the conclusion.

03把刚才的变化拆成 4 个零件Break the Change into Four Parts

Repetition Penalty

SEEN

出现记录Seen-token record

保存当前输出里已经出现过的 token。Tracks tokens already present in the current output.

SCORE

原候选分数Base candidate score

同一组固定候选先给出未惩罚分数。Starts from one fixed set of unpenalized candidate scores.

PENALTY

分数压低Score suppression

已出现 token 的分数除以惩罚值。Divides scores of seen tokens by the penalty.

TRADE-OFF

循环与复述Loops versus repetition

适量减少循环；过量会误伤必要复述。A moderate value reduces loops; an extreme value harms useful repetition.

04最小机制：主交互能逐项验证Minimal Mechanism: Verify Each Link in the Interaction

MECHANISM

adjusted_score(t) = seen(t) ? score(t) / penalty : score(t)

主交互里的虚线是原柱高；拖动 penalty 后，只有已出现词的实柱下移。Dashed markers show original heights. Moving the penalty lowers only bars for seen tokens.

标记已出现Mark seen tokens 模型、会model, can

压低分数Suppress scores 只作用于已出现项seen items only

重新归一化Renormalize 候选排序改变ranking changes

05把惩罚推到 2.50×，误伤必要复述Push Penalty to 2.50× and Harm Useful Repetition

主动制造失败Create a failure

边界实验Boundary experiment

一句解释必须再次说“模型”。把惩罚拉满，观察循环变少的同时，必要复述也从候选前列消失。An explanation must say “model” again. Max out the penalty and watch useful repetition disappear along with loops.

尚未执行。先在主交互中观察正常机制。Not run yet. Observe the normal mechanism in the main interaction first.

✗ 惩罚越高，回答一定越好。Higher penalty always improves an answer. → ✓ 过高会压掉术语、主语等必要复述。Too much suppresses required terms and subjects.

✗ 重复减少说明模型理解更深。Less repetition means deeper understanding. → ✓ 这是解码分数后处理，不是理解能力升级。This is score post-processing, not improved understanding.

✗ 所有出现过的词都该同样禁止。Every seen word should be banned equally. → ✓ 惩罚是倾向调整，不应当被误当成绝对禁词表。A penalty adjusts preference; it is not an absolute ban list.

06 · 一句话带走06 · One line to keep

重复惩罚只压低已出现 token 的再选倾向；拖得过高时，循环少了，必要复述也会一起消失。Repetition penalty suppresses reselection of seen tokens. Push it too far and useful repetition vanishes along with loops.