预测目标 · 模型究竟要回答什么Prediction Target · What Exactly Must Be Predicted?
same examples + different target → different winner
预测目标是模型被要求输出并接受评价的那个量或类别。目标设错时,模型可以在数字上很准,却对真正的业务问题毫无用处。A prediction target is the quantity or class a model must output and be evaluated against. With the wrong target, a
model can score well while failing the actual business question.
01 · 先问对问题01 · Ask the right question
客服真正要找高危投诉时,把“回复写多长”预测得很准,有用吗?If support needs to find high-risk complaints, does accurately predicting reply length help?
?
02同一数据,两种目标Same Data, Two Targets
排名翻转型Ranking reversal
当前目标Current target
高危Risk
被评价的答案answer being scored
当前冠军Current winner
A
只对当前目标for this target only
样本Examples
6
同一批工单same six tickets
柱高表示针对当前目标的教学评分;换目标时数据和模型都没有改变。Bar height is a teaching score for the current target. The data and models do not change when the target
changes.
03把“做得好”拆成 4 件事Four Ingredients of “Doing Well”
任务定义Task definition
PART 01
输入Input
两种任务都读取同一批客服工单特征。Both tasks read the same support-ticket features.
PART 02
目标Target
一个目标是类别,一个目标是字符数量。One target is a class; the other is a character count.
PART 03
评价尺Scoring rule
高危看分对与否,长度看预测误差。Risk uses class correctness; length uses numeric error.
PART 04
业务用途Business use
只有目标对应决策,评分才有实际意义。A score matters only when its target supports the decision.
04最小机制:评价只看指定目标Minimal Mechanism: Evaluation Follows the Chosen Target
score(ŷ, ytarget)
x → ŷtarget → score(ŷ, y)
切换上方按钮就是替换 ytarget。目标一换,什么叫“预测正确”也跟着换。The buttons above replace ytarget. Once the target changes, the meaning of a correct prediction
changes with it.
选择目标Choose target规定要回答什么Define the answer
选择评价尺Choose score规定什么叫好Define what good means
排名模型Rank models冠军只对该目标成立The winner is target-specific
05边界实验:用长度冠军处理高危Boundary Test: Use the Length Winner for Risk
主动制造失败Create a failure
错用评价结果Use the wrong evaluation
让系统因为“回复长度更准”而选模型 B
去识别高危工单,看看数字准确怎样变成业务失误。Select Model B for risk handling merely because it predicts reply length well, and see numerical accuracy turn
into a business error.
尚未执行。当前冠军与当前目标一致。Not run yet. The current winner matches the current target.
✗ 准确率高就说明模型有用。High accuracy proves a model is useful.
→ ✓ 先确认准确的是不是业务需要的目标。First verify that the accurate target matches the business need.
✗ 同一数据集只有一种正确任务。A dataset has only one valid task.
→ ✓ 同一输入可以对应不同可评价目标。The same inputs can support different scorable targets.
✗ 模型冠军与评价目标无关。The best model is independent of the target.
→ ✓ 冠军只在特定目标与评价尺下成立。A winner exists only under a specific target and scoring rule.
06 · 一句话带走06 · One line to keep
预测目标规定模型输出什么、按什么答案受评价;切换高危与回复长度,就能直接检验同一模型排名为何会反转。A prediction target defines what the model outputs and which answer scores it. Switch between risk and reply
length to verify why the same models reverse rank.