Encoder · AI.WIKI

01 · 做完形填空时 01 · When solving a fill-in-the-blank

为什么先读完整句话，再补中间空格，往往比只看左半句更可靠？ Why is reading the entire sentence before filling the middle blank usually more reliable than seeing only the left half?

02 整句一次点亮，遮一词看双向补足 Light the Whole Sentence, Then Mask One Word

与 decoder 成对 Paired with decoder

状态 A Metric A

—

状态 B Metric B

—

状态 C Metric C

—

直接操作主变量，再拖回去验证一句话结论。所有数值均为固定种子的教学模拟。 Operate the main variable, then move it back to verify the one-line conclusion. All values are fixed-seed teaching simulations.

03 把变化拆成 4 个零件 Break the Change into Four Parts

Encoder

PART 01

完整输入 Complete input

句子中的所有 token 在编码开始前已经同时存在。 Every token in the sentence exists before encoding starts.

PART 02

双向上下文 Bidirectional context

一个位置可同时利用左侧和右侧信息。 A position can use information from both left and right.

PART 03

上下文化表示 Contextual representation

同一个词在不同句子里会得到不同的上下文表示。 The same word gets a different contextual representation in another sentence.

PART 04

一次点亮 One-pass lighting

页面把所有位置同时点亮，与 decoder 逐格追加形成对照。 The page lights all positions together, contrasting with decoder's slot-by-slot append.

04 最小机制：每个位置汇总完整输入序列 Minimal Mechanism: Every Position Gathers from the Full Input

Encoder

H = Encoder(x₁, …, x_n)

输出 H 仍有 n 个位置，但每个 hᵢ 已经结合整句上下文。 Output H still has n positions, but every hᵢ now incorporates full-sentence context.

整句进入 Load full sentence 所有位置可见 all positions visible

并行注意 Attend in parallel 左右文共同补足 both sides contribute

输出 H Produce H 每格变成上下文化表示 each slot becomes contextual

05 边界实验：错误地给 encoder 套因果限制 Boundary Test: Wrongly Force a Causal Limit onto the Encoder

主动制造失败 Create a failure

遮住中间词，却禁止它看右侧 Mask the middle word but block its right context

制造左侧可见、右侧不可见的错误限制。被遮词失去“红球”等右侧线索，双向编码能力被破坏。 Force a wrong left-only rule. The masked word loses right-side clues such as “red ball,” breaking bidirectional encoding.

尚未执行。先在主交互中观察正常机制。 Not run yet. Observe the normal mechanism in the main interaction first.

✗ encoder 也是逐 token 写答案。 An encoder writes answers one token at a time. → ✓ 它一次处理完整输入，输出每个位置的表示。 It processes the full input and outputs a representation per position.

✗ 完整输入等于没有掩码训练。 Full input means no masking task exists. → ✓ 被遮位置仍可利用其余左右上下文恢复。 A masked position can still use the remaining context on both sides.

✗ encoder 与 decoder 只是同名组件。 Encoder and decoder are interchangeable names. → ✓ 全句并行理解与因果逐步生成的可见范围不同。 Full-sequence encoding and causal generation have different visibility.

06 · 一句话带走 06 · One line to keep

encoder 把完整输入一次变成上下文化表示；遮住一个词时，左右文能共同补足，而 decoder 对照仍是逐步生成。 An encoder turns the complete input into contextual representations in one pass. Mask a word and both sides can help, while the decoder contrast still generates step by step.