训练语料(Training
Corpus)是用来学习模型参数的数据集合,其组成、质量和重复会塑造模型行为。相关偏向不等于模型逐字记住了每篇原文。A training corpus is the data collection used to learn model
parameters. Its composition, quality, and duplication shape behavior,
but a learned tendency is not proof of verbatim memorization.
01 · 先从日常问题开始01 · Start with an everyday question
一个人一个月只看菜谱,再让他补完“今天适合……”——你猜他会先想到什么?Someone reads only recipes for a month, then finishes “Today is
good for…” What do you expect first?
▦
02调整专业语料占比,看续写偏向改变Change Specialist Share and Watch Completion Bias Move
拖动语料桶占比drag corpus mix
维修主题占比Repair topic share
—
of 5 toy samples
重复率Duplicate rate
—
exact duplicate share
维修续写偏向Repair completion bias
—
toy probability
03语料地图只需看四件事A Corpus Map Needs Four Things
把刚才的变化拆开Map the change to parts
01 · MIX
组成Composition
不同主题各占多少,决定模型反复看见哪些模式。Topic proportions determine which patterns the model sees
repeatedly.
02 · QUALITY
质量Quality
错误、过期和低质量样本也会参与参数更新。Incorrect, stale, and low-quality samples still influence
updates.
03 · REPEAT
重复Duplication
同一文本重复出现,会让局部模式被过度强调。Repeated text can overemphasize local patterns.
04 · BOUNDARY
证据边界Evidence boundary
观察到偏向只说明相关,不能据此断言记住某篇原文。Observed bias shows association; it does not prove memorization
of a specific source.
这是 5
样本概念地图,不是真实训练。固定种子只负责稳定展示顺序;比例变化用于演示因果方向。This is a five-sample concept map, not real training. The fixed
seed stabilizes display order; the ratio illustrates causal
direction only.
续写偏向移动Completion bias moves不是原文记忆证明not proof of memorization
05边界实验:把重复当成更多知识Boundary Test: Mistake Duplicates for More Knowledge
主动制造失败Create a failure
同一维修样本复制三次Copy the same repair sample three times
打开重复模式并把维修占比拉到
80%。主题偏向会变强,但语料多样性反而下降。Enable duplicates and raise repair share to 80%. Topic bias grows
while corpus diversity falls.
尚未执行。先在上方改变主变量。Not run yet. Change the main variable above first.
✗ 语料越多,质量一定越高。More corpus always means better quality.
→ ✓ 组成、质量与重复必须分开检查。Inspect composition, quality, and duplication
separately.
✗ 模型偏向维修,说明记住了某篇文章。A repair bias proves the model memorized one
article.
→ ✓ 行为相关不等于逐字记忆证据。Behavioral correlation is not proof of verbatim
memory.
✗ 复制同一文本能增加知识多样性。Copying one text increases knowledge diversity.
→ ✓ 重复只增加曝光,不增加独立信息。Duplication increases exposure, not independent
information.
06 · 一句话带走06 · One line to keep
训练语料的组成、质量和重复会塑造模型行为;调语料桶能看到偏向变化,但“相关”不能被说成“记住了原文”。Corpus composition, quality, and duplication shape model behavior.
Changing the bucket reveals bias, but association must not be
claimed as memorization.