隱藏層:沿所有下游路徑加總Hidden-layer delta sums downstream paths

IDL115 · CH03 · KP06 · 14/44
一個隱藏節點影響多個下游節點,貢獻必須加總。 / A hidden unit contributes through every downstream path.

下游依賴Downstream dependence

下游 aⱼ 對本層 aᵢ 的導數包含權重與激活導數。 / The local derivative includes both factors.

∂aj∂ai=wjiσi′(ai)(14)\frac{\partial a_j}{\partial a_i}=w_{ji}\sigma_i'(a_i)\quad\text{(14)}

δ 遞迴Delta recursion

從輸出層往回算,重用下游 δ。 / Reuse downstream deltas.

δi=σi′(ai)∑jwjiδj(13),(15)\delta_i=\sigma_i'(a_i)\sum_jw_{ji}\delta_j\quad\text{(13),(15)}

隱藏權重梯度Hidden weight gradient

回傳得到 δ 後,仍套用同一個 δ×輸入規則。 / The weight-gradient rule is unchanged.

∂e∂wiℓ=zℓσi′(ai)∑jwjiδj(16)\frac{\partial e}{\partial w_{i\ell}}=z_\ell\sigma_i'(a_i)\sum_jw_{ji}\delta_j\quad\text{(16)}