01 · DATA
固定数据 Fixed data
相同代表性样本让版本差异可归因，而不是题目变化。 The same representative samples attribute differences to versions, not changing questions.
01 · 先从日常问题开始
01 · Start with an everyday question
两个新版本总分都写 86 分，一个把安全题全做错，另一个只是慢了些——你会把它们当成一样吗？ Two versions both score 86. One fails every safety case; the other is merely slower. Are they equivalent?
02 选择版本和样本覆盖，查看分项与失败案例 Choose a Version and Coverage, Then Inspect Components and Failures
切换版本，拖样本数 switch version + samples
事实性
Factuality
—
passed / run
格式合规
Format compliance
—
passed / run
安全案例
Safety cases
—
passed / run
03 离线评测要固定四件事 Offline Evaluation Fixes Four Things
把刚才的变化拆开 Map the change to parts02 · ENV
固定环境 Fixed environment
锁定提示、检索库、工具模拟和运行条件。 Lock prompts, retrieval corpus, tool simulations, and runtime conditions.
03 · METRIC
分项指标 Component metrics
事实、格式、安全分别判定，禁止只看加权总分。 Judge factuality, format, and safety separately; never rely only on a weighted total.
04 · FAIL
失败样例 Failure examples
分数必须能下钻到具体失败，才能定位与修复。 Scores must drill into concrete failures so teams can locate and fix regressions.
04 最小比较：同一量尺下看候选减基线 Minimal Comparison: Candidate Minus Baseline on One Yardstick
Δmetric = v2 − v1
Δmetric = scorev2 − scorev1
每一项都使用相同固定样本和判定规则。总分相近也不能抵消某个硬门的回退。
Every component uses the same fixed samples and rubric. Similar
totals cannot cancel a regression on a hard gate.
锁定数据环境
Lock data and environment
版本比较有共同分母
versions share a denominator
运行分项判定
Run component checks
事实、格式、安全分轴
facts, format, safety
打开失败样例
Inspect failures
定位回退而非猜测
locate regressions
05 边界实验：只看平均总分 Boundary Test: Look Only at the Average
主动制造失败 Create a failure候选总分看似接近，安全硬门却回退 Candidate total looks close while its safety gate regresses
运行全部 5 条候选样本。候选格式更好，但一条提示注入安全样本失败，不能被平均掉。 Run all five candidate samples. Format improves, but one prompt-injection safety case fails and cannot be averaged away.
尚未执行。先在上方改变主变量。
Not run yet. Change the main variable above first.
✗ 离线高分等于线上一定成功。
A high offline score guarantees live success.
→ ✓ 离线证据只覆盖固定数据与环境。
Offline evidence covers only fixed data and
environment.
✗ 只看一个总分就够。
One total score is enough.
→ ✓ 分项、失败样例和基线差异必须同屏。
Show components, failures, and baseline deltas
together.
✗ 评测样本越少，结论同样稳定。
Fewer samples yield equally stable conclusions.
→ ✓ 覆盖不足必须明确显示，不外推全量表现。
Expose insufficient coverage and do not
extrapolate.
06 · 一句话带走
06 · One line to keep