Train/Validation/Test Split · Three Locked Boxes

01 · 考前练习与期末考01 · Practice versus final exam

如果学生考前背过期末原题，满分还能证明他会解决没见过的问题吗？If a student memorized the final questions beforehand, does a perfect score prove they can solve unseen problems?

02三只上锁箱：拖动样本，守住用途Three Locked Boxes: Move Samples, Preserve Roles

唯一变量：样本去向One variable: sample destination

训练箱Train box

学参数learn parameters

验证箱Validation box

选方案choose setup

最终验收Final acceptance

干净CLEAN

独立证据independent evidence

训练箱TRAIN🔒 70%

验证箱VALIDATION🔒 15%

测试箱TEST🔒 15%

03不是把数据随便切成三堆This Is Not Three Arbitrary Piles

learn · choose · accept · lock

TRAIN

训练集Training set

参与梯度更新，直接改变参数。Participates in gradients and changes parameters.

VALIDATION

验证集Validation set

比较方案、调超参数、决定何时停止。Compares setups, tunes hyperparameters, and stops training.

TEST

测试集Test set

开发期保持不可见，只做最终独立检验。Stays unseen during development and performs final independent testing.

LINEAGE

数据血缘Data lineage

记录题目是否进入训练、提示或调参。Records whether questions entered training, prompts, or tuning.

04最小机制：最终证据必须来自未参与选择的数据Minimal Mechanism: Final Evidence Must Not Choose the Model

fit(train) → choose(validation) → accept(test)

θ ← train
方案 ← validation
最终证据 ← test

测试集若影响 θ 或方案选择，就不再独立。分数可以计算，但必须标成“已污染”，不能冒充干净证据。If test data influences θ or setup choice, it is no longer independent. A score can still be computed, but it must be marked contaminated.

训练学参数Train learns parameters允许反复看repeated access allowed

验证选方案Validation chooses setup开发证据development evidence

测试只验收Test only accepts保持留出与密封held out and sealed

05边界实验：测试题回到训练箱Boundary Test: Return a Test Question to Training

主动制造失败Create a failure

从“未知题”变成“背过的题”Turn an unseen question into a memorized one

上方拖动 T-17 回训练箱。分数或许更高，但它只证明系统适应了这道已知题——这就是“自证预言”。Drag T-17 into training above. The score may rise, but it only proves adaptation to a known question: a self-fulfilling evaluation.

尚未执行。Not run yet.

✗ 数据切三份后就永远干净。Splitting once keeps data clean forever. → ✓ 任何后续回流都会污染血缘。Any later feedback contaminates lineage.

✗ 测试分高就证明泛化。A high test score proves generalization. → ✓ 先证明测试集未参与开发。First prove the test set stayed out of development.

✗ 看了测试答案仍可叫 held-out。A viewed test answer is still held out. → ✓ 密封一破，报告必须区分干净与污染分。Once unsealed, reporting must separate clean and contaminated scores.

06 · 一句话带走06 · One line to keep

训练集学参数，验证集选方案，测试集做独立验收；测试题一旦回流，高分就可能只是自证预言。Training learns parameters, validation chooses the setup, and test independently accepts it; test feedback can turn a high score into self-fulfilling evidence.