Self-Attention · AI.WIKI

01 · 同一排同学互相传纸条 01 · One row of students passing notes

点一句话里的不同词，为什么“谁从谁取信息”的整张关系网会重画？ Why does the whole “who gathers from whom” network redraw when you select a different word?

02 输入短句，逐词重画关系网 Enter a Sentence and Redraw Word by Word

来源边界对照 Source-boundary comparison

当前模式 Current mode

SELF

同一序列来源 same-sequence source

查询词 Query token

皮球

点击任一词切换 select any token

最粗来源 Thickest source

—

复用 S3 线宽语法：lineWidth = 1.5 + weight × 10。切换模式只改变来源边界，不改变粗细含义。 S3 line grammar reused: lineWidth = 1.5 + weight × 10. Switching modes changes the source boundary, not what thickness means.

03 自注意力关系网的 4 个零件 Four Parts of a Self-Attention Network

self = 同源 self = same source

PART 01

同一序列 One sequence

同一排 token 同时提供 Query、Key 与 Value。 One token row supplies Query, Key, and Value.

PART 02

逐位置查询 Per-position query

每点一个词，就把它作为当前查询位置。 Selecting a word makes it the current query position.

PART 03

全行候选 Whole-row candidates

该行所有位置都可成为来源，包括自己。 Every position in that row can be a source, including itself.

PART 04

上下文化结果 Contextual result

同一个词因周围来源不同而得到新表示。 The same word gets a new representation from surrounding sources.

04 最小机制：先看 Q、K、V 来自哪一排 Minimal Mechanism: First Check Which Row Supplies Q, K, and V

Self: X → Q, K, V

Q = XW_Q
K = XW_K
V = XW_V

Self 模式里三者都来自 X。Cross 模式里 Q 来自 X，而 K、V 来自另一来源 Y。 In self mode, all three come from X. In cross mode, Q comes from X while K and V come from another source Y.

点击查询词 Select query token 确定当前 Q 位置 Set current Q position

同排匹配 Match within the row 所有 K 竞争权重 All K candidates compete

同排汇总 Gather from the row Value 形成新表示 Values form new context

05 边界实验：把跨序列来源错叫成“self” Boundary Test: Mislabel a Cross-Sequence Source as “Self”

主动制造失败 Create a failure

名称看来源边界，不看模型是否“在想自己” The name follows source boundaries, not introspection

让 Q 留在句子行，却把 K、V 放到“图像标签”行，同时仍标作 SELF。两排来源不同，这个命名立即自相矛盾。 Keep Q in the sentence row, move K and V to the image-label row, and still call it SELF. The source rows differ, so the label contradicts the mechanism.

尚未执行。Self 模式的 Q、K、V 来自同一排。 Not run yet. In self mode, Q, K, and V come from the same row.

✗ “self”表示模型在自我反思。 “Self” means the model reflects on itself. → ✓ 它表示查询与来源属于同一序列。 It means query and sources belong to one sequence.

✗ 一个词只能看自己。 A token can only look at itself. → ✓ 它可从同一序列所有允许位置取信息。 It can gather from all allowed positions in that sequence.

✗ 来源换到另一排仍叫 self。 A different source row is still self. → ✓ Q 与 K/V 分属两排时是跨序列注意力。 Q and K/V from different rows make cross-attention.

06 · 一句话带走 06 · One line to keep

自注意力让同一序列内各位置彼此查询并汇总信息；点击不同词并切换跨序列模式，就能检验“self”说的是来源边界。 Self-attention lets positions query and gather within the same sequence. Select different tokens and compare cross-attention to verify that “self” describes the source boundary.