Positional Encoding

01 · 同一桌三张词卡 01 · The same three cards on a table

“狗、咬、人”只交换首尾两张卡，为什么意思会完全反过来？ Why does swapping only the first and last cards in “dog, bites, person” reverse the meaning?

02 交换词卡，再开关位置波纹 Swap Word Cards, Then Toggle Position Waves

语序开关 Order switch

状态 A Metric A

—

状态 B Metric B

—

状态 C Metric C

—

直接操作主变量，再拖回去验证一句话结论。所有数值均为固定种子的教学模拟。 Operate the main variable, then move it back to verify the one-line conclusion. All values are fixed-seed teaching simulations.

03 把变化拆成 4 个零件 Break the Change into Four Parts

Positional Encoding

PART 01

Token 表示 Token representation

词卡说明“是什么”，但单独看不含自己排在第几位。 A token says what it is, but not where it sits by itself.

PART 02

位置信号 Position signal

第 1、2、3 位叠加不同波纹，让相同词在不同位置可区分。 Positions 1, 2, and 3 add distinct waves so the same word differs by location.

PART 03

排列 Arrangement

交换“狗/人”不改变词集合，却改变每个词绑定的位置。 Swapping dog and person keeps the token set but changes which position each token owns.

PART 04

结果表示 Result

打开位置后两种语序得到 R₁ / R₂；关闭时近似折叠。 With positions on, the orders yield R₁ / R₂; with them off, the results nearly collapse.

04 最小机制：内容向量加上位置向量 Minimal Mechanism: Add a Position Vector to Content

Positional Encoding

x_i = token_i + position_i

位置不是另一个词，而是与第 i 个 token 表示相加的顺序信号。 Position is not another word. It is an order signal added to the token representation at slot i.

交换词卡 Swap cards 内容集合不变 token set stays fixed

绑定新位置 Bind new slots 位置波纹换位 position waves move

注意力汇总 Attend and gather R₁ 与 R₂ 分开 R₁ and R₂ separate

05 边界实验：关掉位置后再交换主客体 Boundary Test: Remove Positions and Swap Subject/Object

主动制造失败 Create a failure

模型只看见一袋无序词卡 The model sees only a bag of unordered cards

关闭位置编码并切到“人咬狗”。如果汇总仍与“狗咬人”近似相同，说明顺序信息已经丢失。 Turn positions off and switch to “person bites dog.” If the summary still matches “dog bites person,” order information has disappeared.

尚未执行。先在主交互中观察正常机制。 Not run yet. Observe the normal mechanism in the main interaction first.

✗ 注意力天然知道词序。 Attention naturally knows word order. → ✓ 没有位置线索，排列可被当作同一集合。 Without positional cues, arrangements can look like the same set.

✗ 位置编码会改掉 token 本身。 Position encoding replaces the token. → ✓ 它把顺序信号加到 token 表示上。 It adds an order signal to the token representation.

✗ 位置越大，词就越重要。 A larger position means a more important word. → ✓ 位置编号表达次序，不等于重要性。 A position index expresses order, not importance.

06 · 一句话带走 06 · One line to keep

位置编码把顺序加入 token 表示；关掉它再交换“狗/人”，就能看见两种语序为什么会被误当成近似同一件事。 Positional encoding adds order to token representations. Turn it off and swap dog/person to see why two meanings can collapse into nearly the same thing.