PEFT · Freeze the Base and Train a Small Delta

01 · 改整栋楼，还是换一组可拆隔板01 · Rebuild the building or swap modular partitions?

同一个基座要适配八个部门，必须保存八份完整模型，还是能冻结主体、每个部门只换一小组参数？One base model must serve eight departments. Do we need eight full copies, or can we freeze the base and swap a small parameter set for each team?

02切换全量与 PEFT，看可训练参数格、显存条和适配器数量Switch Full Tuning and PEFT to Compare Parameter Cells, Memory, and Adapters

全量 vs PEFTfull tuning vs PEFT

状态 AMetric A

—

状态 BMetric B

—

状态 CMetric C

—

所有概率、得分与输出均为固定种子的教学模拟，不代表真实模型。拖回主变量即可复核结论。All probabilities, scores, and outputs are fixed-seed teaching simulations, not real model output. Move the main variable back to verify the conclusion.

03把刚才的变化拆成 4 个零件Break the Change into Four Parts

Parameter-Efficient Fine-tuning

BASE

冻结基座Frozen base

大部分预训练权重保持不变。Most pretrained weights remain unchanged.

DELTA

可训练增量Trainable delta

只更新新增或挑选出的少量参数。Only added or selected parameters receive updates.

MEMORY

训练状态Training state

少量可训练参数减少梯度与优化器状态。Fewer trainable parameters reduce gradient and optimizer state.

ADAPTER

可切换适配器Swappable adapters

同一基座可挂接不同任务增量。One base can host different task-specific deltas.

04最小机制：主交互能逐项验证Minimal Mechanism: Verify Each Link in the Interaction

MECHANISM

trainable_ratio = trainable_parameters / total_parameters ≪ 1

切换路线时，同一 12×6 参数格只改变红色可训练单元；显存与适配器指标由这个比例直接驱动。Switching routes changes only the red trainable cells in the same 12×6 grid. Memory and adapter indicators follow that ratio directly.

冻结主体Freeze base保留通用能力retain general capability

训练小增量Train delta少量参数适配任务adapt with few parameters

组合部署Compose deployment按任务切换适配器swap adapters by task

05在 32% 显存预算下误选全量微调Choose Full Tuning Under a 32% Memory Budget

主动制造失败Create a failure

边界实验Boundary experiment

全量路线让整张参数格变红，玩具显存达 94%；还没比较效果，资源约束已经失败。Full tuning turns the whole grid red and reaches 94% toy memory. The resource constraint fails before effectiveness can even be compared.

尚未执行。先在主交互中观察正常机制。Not run yet. Observe the normal mechanism in the main interaction first.

✗ PEFT 就是 LoRA。PEFT is just LoRA.→ ✓ LoRA 是 PEFT 家族中的一种低秩方法。LoRA is one low-rank method in the PEFT family.

✗ 可训练参数少，推理模型也必然更小。Fewer trainable parameters always mean a smaller inference model.→ ✓ 基座仍要加载；节省主要发生在训练与多适配器存储。The base still loads; savings center on training and multi-adapter storage.

✗ PEFT 与全量微调效果必然相同。PEFT always matches full tuning.→ ✓ 表达容量与任务复杂度仍需实测。Capacity and task complexity still require evaluation.

06 · 一句话带走06 · One line to keep

PEFT 冻结大部分基座，只训练小增量；它降低训练资源并支持多适配器，但不能把少参数误写成零代价或等价效果。PEFT freezes most of the base and trains a small delta. It reduces training resources and supports many adapters, but fewer parameters do not mean zero cost or identical quality.