S3-09 · 进阶 · 未来信息闸门S3-09 · Advanced · Future-information gate

因果掩码 · 答案写在右边,也不准看Causal Mask · The Answer May Be There, but It Stays Hidden

position i may attend only to positions j ≤ i

因果掩码(Causal Mask)屏蔽当前位置之后的信息,保证 decoder 预测下一个 token 时不能偷看未来。A causal mask blocks information after the current position, so a decoder cannot peek at future tokens while predicting the next one.

01 · 先盖住答案01 · Cover the answer first

做完形填空时,如果下一格答案一直露着,测出来的还是真本事吗?If the next answer stays visible during a fill-in-the-blank test, are you measuring real ability?

02 开关三角遮罩,逐行检查谁能看谁Toggle the Triangle and Inspect Who Can See Whom

线宽 = 权重Width = weight
当前查询Current query
喜欢
第 3 个位置position 3
可见位置Visible positions
3 / 5
含当前位置including itself
未来泄漏Future leakage
0
条违规连线illegal links

沿用 S3 注意力线宽语法:lineWidth = 1.5 + weight × 10。灰格不是小权重,而是根本不允许参与。S3 line grammar continues: lineWidth = 1.5 + weight × 10. A gray cell is not a small weight; it is forbidden from participating.

03 三角遮罩的 4 个零件Four Parts of a Causal Mask

past ✓ · future ×
PART 01

查询行Query row

每一行代表一个正在补信息的位置。Each row is one position gathering information.

PART 02

过去与当前Past and present

主对角线及左侧位置保持可见。The diagonal and positions to its left stay visible.

PART 03

未来三角Future triangle

右上区域被置为不可参与,而非降权。The upper-right region is excluded, not down-weighted.

PART 04

重新归一化Renormalization

剩余允许位置重新分完 100% 权重。Allowed positions redistribute the full 100% weight.

04 最小机制:未来分数先变成 −∞Minimal Mechanism: Future Scores Become −∞ First

j > i ⇒ blocked
scoreij + maskij
maskij = 0 if j ≤ i
maskij = −∞ if j > i
softmax 后,被遮位置权重变为 0;拖查询位置即可验证三角边界随行号移动。After softmax, masked positions receive zero weight. Move the query to verify that the triangular boundary follows the row.
定位当前行Set current row确定时间边界Place the time boundary
封住右侧Block the right side未来分数不参与Future scores cannot enter
重分权重Redistribute weight只汇总已知内容Gather known content only

05 边界实验:关掉遮罩,直接偷看答案Boundary Test: Disable the Mask and Peek at the Answer

主动制造失败Create a failure

训练看似更准,却泄漏了未来Training looks easier because the future leaked

让“喜欢”直接连到后面的“苹果”。这相当于做题时把答案印在旁边,训练分数失去意义。Let “likes” connect directly to the later word “apples.” That is the answer printed beside the test, making the training score meaningless.

尚未执行。未来区域保持封闭。Not run yet. The future region remains closed.
灰格只是权重比较小。Gray cells merely have less weight. → ✓ 灰格被规则排除,权重为 0。Gray cells are excluded and receive zero weight.
遮罩让模型忘掉过去。The mask makes the model forget the past. → ✓ 它只封住当前位置之后的未来。It blocks only positions after the current one.
训练时可以看答案,生成时再关掉。Training may peek if generation stops peeking. → ✓ 训练和生成必须遵守同一信息边界。Training and generation must share the same information boundary.
06 · 一句话带走06 · One line to keep

因果掩码把未来 token 从注意力候选中彻底排除;开关三角遮罩,就能看见“不能偷看”如何变成矩阵规则。A causal mask completely removes future tokens from attention. Toggle the triangle to see “no peeking” become a matrix rule.