参数量 · 把“多少 B”拆成三笔账
Parameter Count · Break “How Many B” into Three Bills
vocabulary × width + layers × width² + output head → total parameters
参数量(Parameter Count)是模型中可学习参数的总数,是容量与资源需求的一项尺度;它不等同于智能、上下文长度或运行成本。
Parameter Count is the total number of learnable parameters in a model. It is one measure of capacity and resource demand—not intelligence, context length, or runtime cost.
01 · 买仓库不能只问面积
01 · Warehouse size is not the whole business
两个仓库体积相差十倍,为什么出货能力未必也自动相差十倍?
If one warehouse is ten times larger, why might its shipping capability not be ten times better?
#B
02
调层数、宽度、词表,逐项数参数
Move Depth, Width, and Vocabulary and Count Each Part
三滑块分项
Three-part count
状态 A
Metric A
—
—
状态 B
Metric B
—
—
状态 C
Metric C
—
—
直接操作主变量,再拖回去验证一句话结论。所有数值均为固定种子的教学模拟。
Operate the main variable, then move it back to verify the one-line conclusion. All values are fixed-seed teaching simulations.
03
把变化拆成 4 个零件
Break the Change into Four Parts
Parameter Count
PART 01
词表 × 宽度
Vocabulary × width
每个词表项需要一行表示,词表越大或宽度越大,嵌入参数越多。
Each vocabulary item needs a representation row; larger vocabulary or width means more embedding parameters.
PART 02
层数 × 宽度²
Layers × width²
Transformer 块中的矩阵常随宽度平方增长,再按层数重复。
Matrices in Transformer blocks often grow with width squared and repeat across layers.
PART 03
输出头
Output head
输出映射也连接宽度与候选词表,是另一笔参数账。
The output projection also connects model width to candidate vocabulary.
PART 04
能力提示
Capability hint
页面只给非线性教学提示,不把参数倍数冒充能力承诺。
The page gives a nonlinear teaching hint without treating parameter multiples as capability promises.
04
最小机制:把三部分相加,而不是只看层数
Minimal Mechanism: Add Three Parts Instead of Counting Layers Alone
Parameter Count
P ≈ Vd + 12Ld2 + 0.25dV
这是教学近似,用来观察增长阶数,不代表某个真实模型的精确结构。
This teaching approximation exposes growth rates; it is not the exact architecture of a real model.
增加词表 V
Grow vocabulary V
嵌入与输出头增长
embeddings and head grow
增加宽度 d
Grow width d
块参数平方增长
block parameters grow quadratically
增加层数 L
Add layers L
重复更多块
more blocks repeat
05
边界实验:三项拉满并把参数当能力
Boundary Test: Max Every Slider and Equate Parameters with Capability
主动制造失败
Create a failure
资源暴涨不等于能力等比增长
Resource growth is not proportional capability growth
把层数、宽度、词表全拉满。参数与资源条会暴涨,但能力提示刻意饱和,提醒你不能据此承诺效果。
Max depth, width, and vocabulary. Parameters and resources surge, while the capability hint deliberately saturates to block a proportional claim.
尚未执行。先在主交互中观察正常机制。
Not run yet. Observe the normal mechanism in the main interaction first.
✗
参数量就是智能分数。
Parameter count is an intelligence score.
→ ✓
它是容量尺度之一,能力还取决于数据、训练与架构。
It is one capacity measure; capability also depends on data, training, and architecture.
✗
模型翻倍,运行成本一定精确翻倍。
Double parameters means exactly double runtime cost.
→ ✓
量化、稀疏激活、硬件与批次都会改变成本。
Quantization, sparse activation, hardware, and batching change cost.
✗
词表越大,上下文窗口越长。
A larger vocabulary means a longer context window.
→ ✓
词表大小与一次可接收的 token 数是不同尺度。
Vocabulary size and accepted token count are different scales.
06 · 一句话带走
06 · One line to keep
参数量是层数、宽度和词表等结构共同贡献的可学习参数总数;拖动三项能看到资源增长,但能力不会自动等比增长。
Parameter count is the total learnable capacity contributed by depth, width, vocabulary, and other structure. Move all three to see resources grow without proportional capability.