Sparse Retrieval · How Exact Terms Retrieve

01 · 一个编号胜过一段近义话 01 · One identifier can beat a paragraph

查“ZX-42”，你希望系统理解设备语义，还是先把含 ZX-42 的原文精准捞出来？ For “ZX-42,” should the system infer device meaning first, or precisely retrieve the text containing ZX-42?

02 倒排表探照灯：换查询，看两路排名 Inverted-Index Spotlight: Change the Query, Compare Two Rankings

查询词项 query terms

命中词项 Matched terms

倒排表 postings

稀疏第一 Sparse #1

BM25-style

稠密第一 Dense #1

语义近邻 semantic neighbor

稀疏排名 Sparse rank

稠密对照 Dense comparison

03 稀疏检索的四个零件 Four Parts of Sparse Retrieval

词项 · 倒排 · 权重 terms · postings · weights

TERMS

查询拆成词项 Tokenize query terms

ZX-42 与“故障码”是两个可匹配词项。 ZX-42 and “fault code” become two matchable terms.

POSTINGS

倒排表找文档 Postings find documents

每个词项指向包含它的文档编号，而非扫描全库。 Each term points to document IDs containing it instead of scanning the corpus.

IDF

稀有词权重更高 Rare terms weigh more

只在一篇出现的 ZX-42 比普遍词“设备”更有区分力。 ZX-42 appearing once is more discriminating than a common term like “device.”

LENGTH

文档长度要校正 Normalize document length

长文不能只因出现词更多就永远得高分。 Long documents should not win merely because they contain more words.

04 最小机制：匹配词频乘以区分力 Minimal Mechanism: Match Frequency Weighted by Discrimination

BM25-style

score(d,q) = Σ IDF(t) × TF_norm(t,d)

本页用简化 BM25 式分数演示：没有共享词项时，分数就是 0；精确且稀有的词项会强烈抬高对应文档。 This lesson uses a simplified BM25-style score. With no shared term, the score is zero; a rare exact term strongly lifts its document.

解析查询词 Parse query terms ZX-42 + 故障码 fault code

读取倒排项 Read postings 命中文档集合 matching document set

加权并排序 Weight and rank 稀有精确词优先 rare exact term leads

05 边界实验：同义改写没有共同词 Boundary Test: A Paraphrase Shares No Terms

主动制造失败 Create a failure

用“机器闹脾气”代替“故障排查” Replace “troubleshooting” with “machine acting up”

意思接近，但没有共享词项。稀疏分数全部归零，稠密检索仍可能把故障排查排在前面。 The meaning is close but no term overlaps. Sparse scores fall to zero while dense retrieval can still rank troubleshooting first.

尚未执行。当前精确编号能命中倒排表。 Not run yet. The exact identifier currently hits the postings.

✗ 稀疏表示就是信息少。 Sparse representation means little information. → ✓ 它表示大词表中只有少量非零词项。 It means only a few vocabulary dimensions are nonzero.

✗ 关键词检索只会数出现次数。 Keyword retrieval only counts occurrences. → ✓ BM25 式方法还考虑稀有度与长度。 BM25-style methods also consider rarity and length.

✗ 精确词项能覆盖所有语义。 Exact terms cover every meaning. → ✓ 无共同词的近义表达仍会漏检。 Paraphrases without shared terms can still be missed.

06 · 一句话带走 06 · One line to keep

稀疏检索用倒排表和词项权重精准召回编号、专名与术语；共享词项为零时，它也会把近义表达完全漏掉。 Sparse retrieval uses postings and term weights to precisely recall identifiers, names, and terminology; with zero shared terms, it can miss a paraphrase entirely.