Retrieval Evaluation

01 · 一个低分，修哪一站 01 · One low score, which station do you fix?

答案只得 60 分：是证据没召回、关键条款排太后，还是回答根本没引用？ The answer scores 60. Was evidence never recalled, ranked too low, or simply not cited by the answer?

02 故障漏斗：注入一次，逐站定位 Failure Funnel: Inject Once, Diagnose by Stage

阶段诊断 stage diagnosis

Recall@3

100%

相关证据进候选 relevant evidence recalled

排序命中 Ranking hit

1.00

MRR

引用覆盖 Citation coverage

100%

受支持主张 supported claims

03 三个指标回答三个不同问题 Three Metrics Answer Three Different Questions

召回 · 排序 · 使用 recall · rank · use

RECALL@K

该找的进来了吗 Was required evidence found?

相关标注证据是否出现在前 k 个候选中。 Check whether labeled relevant evidence appears in the first k candidates.

RANK

关键证据够靠前吗 Was decisive evidence high enough?

MRR 或 nDCG 衡量正确证据在顺序中的位置质量。 MRR or nDCG measures where correct evidence lands in the order.

CITATION

回答真的使用了吗 Did the answer actually use it?

最终主张是否被召回证据支持并正确引用。 Check whether final claims are supported and correctly cited by retrieved evidence.

BREAKPOINT

最早坏点决定修法 Earliest failure guides the fix

先坏召回就修索引/查询；召回好再看排序与引用。 If recall fails first, fix indexing or query; if recall passes, inspect ranking and citation.

04 最小机制：按最早失败站定位根因范围 Minimal Mechanism: Diagnose the Earliest Failing Stage

first failing gate

diagnosis = first(metric_stage < threshold)

Recall@k 未过线时，后面的排序与引用低分可能只是连锁后果；必须从最早故障站开始修。 When Recall@k misses threshold, later ranking and citation drops may be downstream effects. Repair starts at the earliest failing station.

固定查询与标注 Freeze queries and labels 同一评测集 same eval set

逐站计算指标 Score each stage Recall@k → MRR → cite

定位最早断点 Locate first breakpoint 映射到 S6-11 站点 map to S6-11 station

05 边界实验：只留一个总分 Boundary Test: Keep Only One Aggregate Score

主动制造失败 Create a failure

把三站压成“总分 61” Collapse three stages into “total 61”

总分告诉你系统差，却不告诉你修检索、重排还是引用。恢复漏斗后，同一个总分会露出最早断点。 The aggregate says the system is weak but not whether to fix retrieval, reranking, or citation. Restoring the funnel reveals the earliest breakpoint.

尚未执行。当前三站指标分别可见。 Not run yet. All three stage metrics are visible.

✗ 答案差就是生成模型差。 A bad answer means a bad generator. → ✓ 先检查证据是否召回、排好并被使用。 First check whether evidence was recalled, ranked, and used.

✗ Recall@k 高就说明 RAG 好。 High Recall@k means RAG is good. → ✓ 还要看顺序、答案使用与引用质量。 Order, answer use, and citation quality still matter.

✗ 一个总分最便于比较。 One total score is best for comparison. → ✓ 总分可汇报，阶段指标才能诊断。 Aggregates can report; stage metrics diagnose.

06 · 一句话带走 06 · One line to keep

检索评测要把 Recall@k、排序质量和引用覆盖分站测量，并从最早失败站定位 RAG 断点；一个总分只能报警，不能告诉你修哪里。 Retrieval evaluation measures Recall@k, ranking quality, and citation coverage by stage and locates the earliest RAG breakpoint; one aggregate can alert but cannot direct the repair.