Supervised Fine-tuning

01 · 示范会留下习惯01 · Demonstrations become habits

新员工连续照着五条客服示范回答；其中一条乱承诺，他只会学到好示范吗？A new support agent practices five sample replies. If one makes a reckless promise, will only the good examples be learned?

02编辑输入—答案对入批次，观察更新与坏答案偏差Add an Input-answer Pair and Watch Updates—including Bad-answer Bias

最多 5 对玩具样本up to 5 toy pairs

状态 AMetric A

—

状态 BMetric B

—

状态 CMetric C

—

所有概率、得分与输出均为固定种子的教学模拟，不代表真实模型。拖回主变量即可复核结论。All probabilities, scores, and outputs are fixed-seed teaching simulations, not real model output. Move the main variable back to verify the conclusion.

03把刚才的变化拆成 4 个零件Break the Change into Four Parts

Supervised Fine-tuning

INPUT

输入Input

任务、问题或上下文组成样本左侧。A task, question, or context forms the left side.

ANSWER

期望答案Expected answer

人工或流程给出希望模型学习的输出。A person or process supplies the output to learn.

BATCH

训练批次Training batch

多对样本共同形成一次玩具更新信号。Several pairs form one toy update signal.

BIAS

质量偏差Quality bias

错误答案也会把行为推向错误方向。Incorrect answers push behavior in the wrong direction too.

04最小机制：主交互能逐项验证Minimal Mechanism: Verify Each Link in the Interaction

MECHANISM

loss_SFT = Σ loss(model(input_i), expected_answer_i)

加入样本会改变批次构成；好、坏答案都进入求和，因此都能影响更新方向。Adding a pair changes the batch. Both good and bad answers enter the sum and can influence the update direction.

配对Pair 输入 ↔ 期望答案input ↔ expected answer

入批Batch 固定最多 5 对fixed maximum of five

更新Update 行为靠近期望输出behavior moves toward examples

05注入一条过度承诺的坏答案Inject One Overpromising Bad Answer

主动制造失败Create a failure

边界实验Boundary experiment

把“保证今天到账”当成期望答案加入。玩具质量条下降，更新后输出开始复制这种偏差。Add “promise it arrives today” as the expected answer. The toy quality bar drops and post-update output begins copying the bias.

尚未执行。先在主交互中观察正常机制。Not run yet. Observe the normal mechanism in the main interaction first.

✗ SFT 只是把示例放进 prompt。SFT merely puts examples in the prompt. → ✓ SFT 用配对样本计算 loss 并更新参数。SFT computes loss on paired examples and updates parameters.

✗ 一条坏答案会被模型自动忽略。The model automatically ignores one bad answer. → ✓ 没有质量机制时，坏答案也是训练目标。Without a quality mechanism, a bad answer is still a training target.

✗ 答案写得越长，监督越强。Longer answers provide stronger supervision. → ✓ 关键是答案是否正确、一致并匹配任务。What matters is correctness, consistency, and task fit.

06 · 一句话带走06 · One line to keep

监督微调用输入—期望答案对更新参数；样本质量直接进入学习信号，所以坏答案也会留下偏差。Supervised fine-tuning updates parameters from input-answer pairs. Data quality enters the learning signal directly, so bad answers also leave bias.