S2-13 · 进阶 · 3DS2-13 · Advanced · 3D

梯度下降 · 每一步都朝局部下坡Gradient Descent · Take Each Step Locally Downhill

parameters ← parameters − learning rate × gradient

梯度下降让参数沿梯度反方向小步移动,反复重算当前位置的方向,逐步减小损失。这是“模型怎样学”的第一幅完整动态图。Gradient descent moves parameters a small distance opposite the gradient, recomputes direction, and gradually lowers loss. This is the first complete moving picture of how a model learns.

01 · 大雾里下山01 · Descend through fog

看不见整条下山路线,只能每走一步重新摸脚下坡度,还能到谷底吗?If fog hides the full route and you can only recheck the slope after each step, can you still reach the valley?

02点“走一步”,让小球沿曲面下山Press “One Step” and Move Downhill

唯一变量:更新步数One variable: update steps
更新步Updates
0
steps
当前损失Current loss
L(θ)
梯度长度Gradient norm
‖∇L‖
正在加载曲面;若提示持续显示,则需要 WebGL / three.js。Loading surface; if this remains visible, WebGL or three.js is unavailable.
η = 0.62 · θ ← θ − η∇L
红线 = 已走路径 · 拖动旋转red = traveled path · drag to orbit

03一步更新里的 4 个动作Four Actions Inside One Update

measure · reverse · scale · move
MEASURE

测梯度Measure gradient

读取当前位置最陡上坡。Read steepest ascent here.

REVERSE

取反方向Reverse it

从 +∇L 变成 −∇L。Turn +∇L into −∇L.

SCALE

乘学习率Scale by rate

决定这一小步走多远。Set how far this step moves.

REPEAT

移动再重算Move and repeat

新位置需要新的梯度。The new position needs a new gradient.

04最小机制:不是一次跳到答案Minimal Mechanism: Do Not Jump Straight to the Answer

θt+1 = θt − η∇L(θt)
θt+1 = θt − η∇L(θt)
每一帧都用当前 θt 的梯度。走完以后,旧梯度失效,必须重新计算。Each frame uses the gradient at current θt. After moving, the old gradient expires and must be recomputed.
当前参数Current parameters定位曲面位置locate surface point
负梯度更新Negative-gradient update朝局部下坡移动move locally downhill
损失下降Loss decreases下一步重新测量measure again next step

05边界实验:把减号写成加号Boundary Test: Replace Minus with Plus

主动制造失败Create a failure

故意沿梯度上坡Deliberately follow the gradient uphill

只改更新方向,不改当前位置和曲面。损失立刻增大,验证公式中的减号不是装饰。Change only the update direction. Loss rises immediately, proving the minus sign is essential.

尚未执行。Not run yet.
梯度下降直接求出全局最低点。Gradient descent directly solves the global minimum. → ✓ 它只反复利用局部方向。It repeatedly uses local directions.
梯度算一次就能一直用。Compute the gradient once and reuse it forever. → ✓ 参数一移动就要重算。Recompute after parameters move.
沿 +∇L 也能下山。Following +∇L also descends. → ✓ 下降必须使用 −∇L。Descent requires −∇L.
06 · 一句话带走06 · One line to keep

梯度下降在每个新位置重算方向,并沿负梯度小步更新参数,让损失逐步降低。Gradient descent recomputes direction at every new position and takes small negative-gradient parameter steps to lower loss.