δ 是什麼？權重梯度怎麼來？Delta and the weight gradient

IDL115 · CH03 · KP06 · 13/44

δ 衡量「激活前輸入改一點，損失會改多少」。 / Delta measures loss sensitivity to pre-activation.

敏感度定義Sensitivity

δ 不是原始預測誤差，需包含激活與損失導數。 / Delta is not simply prediction error.

\delta_i:=\frac{\partial e}{\partial a_i}\quad\text{(12)}

一條邊的梯度 = 本層敏感度 × 邊的輸入。 / Multiply sensitivity by the incoming value.

\frac{\partial e}{\partial w_{i\ell}}=\frac{\partial e}{\partial a_i}\frac{\partial a_i}{\partial w_{i\ell}}=\delta_i z_\ell\quad\text{(11)}

偏置對 a 的導數是 1。 / The bias derivative is one.

\frac{\partial e}{\partial b_i}=\delta_i