S1-14 · 基础 · 表示链收官S1-14 · Foundation · Representation finale

嵌入 · 让离散对象学会靠近 Embedding · Let Discrete Objects Learn Proximity

discrete item → dense vector → learnable position

独热编码只能区分身份;embedding 把对象映射为可学习的稠密向量,让训练反馈把相关对象拉近、无关对象推远。 One-hot vectors separate identity. Embeddings map objects to learnable dense vectors so feedback can pull related items together and push unrelated ones apart.

01 · 从安排座位开始01 · Start with a seating plan

如果“猫”和“小猫”永远坐得比“猫”和“发票”近,机器能否把这种关系变成可计算的位置? If “cat” and “kitten” consistently sit closer than “cat” and “invoice,” can a machine turn that relationship into computable positions?

02拖词卡,给“相近 / 不相近”反馈 Drag Word Cards into Similar / Dissimilar Feedback

唯一变量:反馈关系One variable: feedback relation
锚点Anchor
cat
embedding [0.00, 0.00]
已选词 Selected
小猫 kitten
distance 2.65
反馈轮数Feedback rounds
0
位置逐轮更新 positions update by round
已选“小猫”。把它拖到“相近”,观察两个词点聚拢。 “Kitten” is selected. Drop it on “Similar” and watch the points converge.

03一次反馈更新的 4 个零件 Four Parts of One Feedback Update

identity → vector → relation → update
ITEM

离散对象Discrete item

词卡先只是一个可识别的类别。A word card begins as an identifiable category.

VECTOR

稠密向量 Dense vector

两个连续坐标把词放进可移动平面。 Two continuous coordinates place the word in a movable plane.

FEEDBACK

关系反馈 Relational feedback

“相近 / 不相近”给出移动方向。 “Similar / dissimilar” supplies an update direction.

TRACE

更新痕迹 Update trace

箭头记录刚才是哪条反馈造成移动。 An arrow records which feedback caused the latest move.

04最小机制:反馈沿距离方向更新坐标 Minimal Mechanism: Feedback Updates Along the Distance Direction

e ← e + Δe
similar: e ← e + η(a − e)
dissimilar: e ← e + η(e − a)
a 是“猫”的锚点,η=0.35 是教学步长。反复拖同一词卡,距离会逐轮缩短或拉长。 a is the “cat” anchor and η=0.35 is a teaching step size. Repeated drops shrink or expand distance round by round.
拖入反馈区 Drop into a zone确定拉近或推远 choose pull or push
计算 ΔeCompute Δe 沿锚点连线更新update along anchor line
词点移动Point moves 关系变成可计算距离relation becomes distance

05边界实验:错误反馈也会被“学会” Boundary Test: Wrong Feedback Can Also Be Learned

主动制造失败Create a failure

强行把“发票”标成与“猫”相近 Force “invoice” to be similar to “cat”

坐标会忠实地靠近,但错误标签不会因为变成向量就变正确。训练数据决定表示能学到什么偏差。 The coordinate will obediently move closer, but a wrong label does not become correct by becoming a vector. Training data determines which biases the representation learns.

尚未执行。先用正确反馈移动“小猫”。 Not run yet. Move “kitten” with correct feedback first.
embedding 是词的唯一真实含义。An embedding is a word's one true meaning. → ✓ 它是由目标和数据学出的任务表示。 It is a task representation learned from objectives and data.
点靠近就证明事实关系成立。 Nearby points prove a factual relationship. → ✓ 靠近只反映训练出的表示关系。Proximity reflects a learned representational relationship.
稠密向量天然无偏。 Dense vectors are naturally unbiased. → ✓ 错误或偏斜反馈会进入坐标。 Wrong or skewed feedback enters the coordinates.
06 · 一句话带走 · S1 表示链收官 06 · One line to keep · representation finale

Embedding 把离散对象映射成可学习的稠密向量,让反馈把相关对象拉近、无关对象推远;拖词卡即可检验每次移动来自哪条反馈。 An embedding maps a discrete object to a learnable dense vector, letting feedback pull related items together and push unrelated ones apart. Drag a card to verify which feedback caused each move.