MASK
遮罩位置Mask positions
选择少量 token 替换为 [MASK]。Replace a small set of tokens with [MASK].
01 · 破损句子两边都有线索01 · A damaged sentence has clues on both sides
“小猫坐在［空白］垫子上”——补这个词时，为什么不只看左边？“The cat sat on the [blank] mat.” Why ignore the clue on the right when restoring the missing word?
02点选遮罩位置，用双向上下文恢复并对照因果目标Choose Mask Positions, Recover Bidirectionally, and Compare with Causal Training
点击 Canvas tokenclick Canvas tokens
状态 AMetric A
—
—
状态 BMetric B
—
—
状态 CMetric C
—
—
所有概率、得分与输出均为固定种子的教学模拟，不代表真实模型。拖回主变量即可复核结论。All probabilities, scores, and outputs are fixed-seed teaching simulations, not real model output. Move the main variable back to verify the conclusion.
03把刚才的变化拆成 4 个零件Break the Change into Four Parts
Masked Language ModelingLEFT
左侧上下文Left context
遮罩左边提供语义与句法线索。Tokens to the left provide semantic and grammatical clues.
RIGHT
右侧上下文Right context
遮罩右边也同时参与恢复。Tokens to the right also participate in recovery.
RECOVER
恢复 lossRecovery loss
只在被遮位置比较候选与原 token。Compares candidates with original tokens only at masked positions.
04最小机制：主交互能逐项验证Minimal Mechanism: Verify Each Link in the Interaction
MECHANISM
lossmask = −log p(original_token | left_context,
right_context)
点击 token 增减
MASK；每多遮一处，可用上下文减少，候选集中度的玩具值随之下降。Click tokens to add or remove MASK positions. More masks leave
less context, so the toy candidate concentration falls.
选择遮罩Choose masks
隐藏少量原 tokenhide selected originals
汇合两侧Combine both sides
左文 + 右文left + right context
恢复原词Recover originals
在 MASK 位置算 lossloss at MASK positions
05遮住 4/5 个 token，让双向文脉也无从下手Mask 4/5 Tokens and Starve Both Directions of Context
主动制造失败Create a failure边界实验Boundary experiment
遮罩不是越多越难越好。大部分 token 消失后，左右两侧都缺少线索，恢复候选会变散。More masking is not automatically better. Once most tokens disappear, both sides lose clues and recovery candidates spread out.
尚未执行。先在主交互中观察正常机制。Not run yet. Observe the normal mechanism in the main interaction
first.
✗ 所有语言模型都只会逐词续写。Every language model is trained only by next-token
continuation.
→ ✓ MLM 用双向上下文恢复被遮 token。MLM restores masked tokens using both directions.
✗ 遮罩越多，训练信号越强。More masking always means a stronger signal.
→ ✓ 遮太多会破坏可用上下文，让目标含糊。Too much masking destroys useful context and makes the target
ambiguous.
✗ MLM 推理时必须永远保留 [MASK]。MLM inference must always contain [MASK].
→ ✓ [MASK]
是训练任务的一部分，不等于所有下游输入形式。[MASK] is part of the training objective, not every downstream
input format.
06 · 一句话带走06 · One line to keep