Held-out Test Set

01 · 考前看过答案，再考满分还能证明不会做假题吗01 · If you saw the answers before the exam, what does a perfect score prove?

团队说“只偷看一题来排错”，随后又根据答案改了提示。最终报告还能把所有题都算成独立验收吗？The team peeks at one answer for debugging, then changes the prompt around it. Can the final report still call every case independent validation?

02逐次揭开隐藏答案，观察封条、污染计数与干净分/污染分分流Reveal Hidden Answers and Watch the Seal, Contamination Count, and Split Scores Change

评测证据血缘 · 蓝/黄/红eval lineage · blue/yellow/red

状态 AMetric A

—

状态 BMetric B

—

状态 CMetric C

—

所有概率、分数、成本、token 与输出均为固定种子的教学模拟，不代表真实模型输出。把主变量拖回即可复核结论。All probabilities, scores, costs, tokens, and outputs are fixed-seed teaching simulations, not real model output. Move the main variable back to verify the conclusion.

03把刚才的变化拆成 4 个零件Break the Change into Four Parts

Held-out Test Set

HIDE

答案隐藏Hidden answers

开发与调参阶段只看任务，不看最终真值。Development and tuning can see tasks but not final truths.

SEAL

封条记录Seal record

每次揭示都留不可抹除的计数与样本 ID。Every reveal leaves an append-only count and case ID.

SPLIT

分数分流Score split

未揭示题形成干净分；已揭示题只能进入污染分。Unseen cases form the clean score; revealed cases belong only to the contaminated score.

FINAL

最终检验Final validation

只在方案冻结后运行，不能边看答案边继续调参。Run only after the candidate is frozen, never while tuning against visible answers.

04最小机制：主交互能逐项验证Minimal Mechanism: Verify Each Link in the Interaction

MECHANISM

score_clean = pass(unseen) / n_unseen

污染题不会从历史中消失；报告把 score_clean 与 score_contaminated 并列，绝不混成一个漂亮总分。Contaminated cases never disappear from history. The report shows clean and contaminated scores separately instead of blending a flattering total.

封存答案Seal answers开发不可见hidden during development

记录偷看Record peeks封条破损seal is broken

分流报告Split report干净 / 污染clean / contaminated

05连续偷看三条隐藏答案并继续调参Peek at Three Hidden Answers and Continue Tuning

主动制造失败Create a failure

边界实验Boundary experiment

封条破损不是道德标签，而是证据状态变化。被看过的题不能再回到干净分母。A broken seal is not a moral label; it is an evidence-state change. Revealed cases cannot return to the clean denominator.

尚未执行。先在主交互中观察正常机制。Not run yet. Observe the normal mechanism in the main interaction first.

✗ 只看一题不算污染。One peek does not count as contamination.→ ✓ 任何隐藏答案暴露都要逐样本记录。Every hidden-answer reveal must be recorded by case.

✗ 删掉浏览记录就能恢复干净。Deleting browser history restores cleanliness.→ ✓ 证据血缘不可逆；只能换一批真正未见题。Evidence lineage is irreversible; replace it with genuinely unseen cases.

✗ 干净分和污染分可以加权成总分。Clean and contaminated scores can be blended.→ ✓ 必须分栏报告，否则读者无法判断独立性。Report them separately so readers can judge independence.

06 · 一句话带走06 · One line to keep

留出测试集靠答案不可见来保留独立证据；每次偷看都必须破封计数，最终只把未见题计入干净分。A held-out set preserves independent evidence by hiding answers. Every peek breaks the seal, and only unseen cases belong in the clean score.