▶ 展示 · 回索引
index.html
<!doctype html><html lang="zh-Hant"><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>IDL115 Ch03｜KP1–KP16 雙語資訊圖表</title><style>body{font-family:'Microsoft JhengHei',Arial;background:#eff3f8;color:#14233c;margin:0;padding:40px 6%;line-height:1.7}h1{font-size:32px}h2{border-bottom:2px solid #a9bfdc}main{max-width:1100px;margin:auto}.item{display:block;background:white;padding:18px 24px;margin:12px 0;text-decoration:none;color:#153d72;border-left:4px solid #2871c9}.item span{display:block;color:#64758d}nav{display:flex;flex-wrap:wrap;gap:12px}nav a{color:#2259a2}p{max-width:900px}</style><main><h1>反向傳播與優化<br>Backpropagation & Optimization</h1><p>IDL115 · 第三章 · KP1–KP16 · 44 張獨立 16:9 雙語 HTML5＋Canvas 資訊圖表。每張均可離線開啟、操作與嵌入 iframe。公式已預先排版，無需網路載入字型或套件。</p><p>建議使用 1280×720 或較大視窗。小視窗會將整張 16:9 頁面同比縮放；閱讀細節時請放大視窗。曲線若未標為實算，均為概念示意，非實驗結果。</p><nav><a href="#kp1">KP1</a><a href="#kp2">KP2</a><a href="#kp3">KP3</a><a href="#kp4">KP4</a><a href="#kp5">KP5</a><a href="#kp6">KP6</a><a href="#kp7">KP7</a><a href="#kp8">KP8</a><a href="#kp9">KP9</a><a href="#kp10">KP10</a><a href="#kp11">KP11</a><a href="#kp12">KP12</a><a href="#kp13">KP13</a><a href="#kp14">KP14</a><a href="#kp15">KP15</a><a href="#kp16">KP16</a></nav><section id="kp1"><h2>KP1</h2><a class="item" href="kp01-training-map.html"><strong>訓練的三個問題</strong><span>Three questions behind training</span></a><a class="item" href="kp01-first-second-order.html"><strong>一階與二階：資訊與成本</strong><span>First and second order: information and cost</span></a></section><section id="kp2"><h2>KP2</h2><a class="item" href="kp02-gd-update.html"><strong>梯度下降：方向乘上步長</strong><span>Gradient descent: direction times step size</span></a><a class="item" href="kp02-smoothness.html"><strong>L-smooth：步長為何重要</strong><span>L-smoothness explains safe steps</span></a><a class="item" href="kp02-convex-nonconvex.html"><strong>凸與非凸：能保證什麼？</strong><span>Convex and nonconvex guarantees</span></a></section><section id="kp3"><h2>KP3</h2><a class="item" href="kp03-stopping.html"><strong>什麼時候停止？四種訊號</strong><span>Four stopping signals</span></a><a class="item" href="kp03-saddle.html"><strong>梯度為零，為何不是谷底？</strong><span>Zero gradient does not imply a minimum</span></a></section><section id="kp4"><h2>KP4</h2><a class="item" href="kp04-backtracking.html"><strong>線搜索：太大就減半</strong><span>Backtracking: halve an oversized step</span></a><a class="item" href="kp04-armijo-exact.html"><strong>充分下降與精確線搜索</strong><span>Armijo and exact line search</span></a></section><section id="kp5"><h2>KP5</h2><a class="item" href="kp05-momentum.html"><strong>動量：把歷史方向帶進下一步</strong><span>Momentum carries past directions</span></a><a class="item" href="kp05-momentum-vectors.html"><strong>動量的向量合成</strong><span>Vector addition behind momentum</span></a></section><section id="kp6"><h2>KP6</h2><a class="item" href="kp06-forward.html"><strong>前向：先保存中間計算</strong><span>Forward pass: save intermediate values</span></a><a class="item" href="kp06-delta-gradient.html"><strong>δ 是什麼？權重梯度怎麼來？</strong><span>Delta and the weight gradient</span></a><a class="item" href="kp06-hidden-recursion.html"><strong>隱藏層：沿所有下游路徑加總</strong><span>Hidden-layer delta sums downstream paths</span></a><a class="item" href="kp06-backprop-example.html"><strong>完整回傳：算完，再更新</strong><span>Complete backprop: compute, then update</span></a></section><section id="kp7"><h2>KP7</h2><a class="item" href="kp07-sgd-unbiased.html"><strong>SGD：用一筆估計全體</strong><span>SGD estimates the full gradient</span></a><a class="item" href="kp07-schedule.html"><strong>學習率排程：大步到小步</strong><span>Learning-rate schedules</span></a></section><section id="kp8"><h2>KP8</h2><a class="item" href="kp08-batch-epoch.html"><strong>Batch 與 Epoch：一小批與一整輪</strong><span>Batch and epoch</span></a><a class="item" href="kp08-batch-update.html"><strong>Mini-batch：平均一小批梯度</strong><span>Mini-batch gradient averaging</span></a><a class="item" href="kp08-batch-variance.html"><strong>抽樣方差：為何大批次更穩？</strong><span>Sampling variance and batch size</span></a></section><section id="kp9"><h2>KP9</h2><a class="item" href="kp09-gd-rate.html"><strong>全梯度：凸平滑下的收斂界</strong><span>Full-gradient convergence under convexity</span></a><a class="item" href="kp09-sgd-rate.html"><strong>SGD：先說清楚衡量什麼</strong><span>SGD rates need a stated metric</span></a><a class="item" href="kp09-noise-floor.html"><strong>噪聲地板：上界不是下限</strong><span>Noise floor: an upper bound is not a lower bound</span></a><a class="item" href="kp09-weighted-bound.html"><strong>SGD 上界：從一步下降到加總</strong><span>Deriving an averaged stationarity bound</span></a></section><section id="kp10"><h2>KP10</h2><a class="item" href="kp10-adagrad.html"><strong>AdaGrad：每個維度一把尺</strong><span>AdaGrad uses a separate scale per coordinate</span></a><a class="item" href="kp10-adagrad-history.html"><strong>AdaGrad 的記憶：只加不減</strong><span>AdaGrad never forgets</span></a></section><section id="kp11"><h2>KP11</h2><a class="item" href="kp11-rmsprop.html"><strong>RMSProp：記住近期，忘掉遠期</strong><span>RMSProp emphasizes recent gradients</span></a><a class="item" href="kp11-adam.html"><strong>Adam：方向記憶加逐維步幅</strong><span>Adam combines direction and scale</span></a><a class="item" href="kp11-adam-bias.html"><strong>偏差修正：補回從零開始的低估</strong><span>Bias correction for zero initialization</span></a></section><section id="kp12"><h2>KP12</h2><a class="item" href="kp12-flat-sharp.html"><strong>同樣低的點，不同寬度的谷</strong><span>Equal minima, different neighborhood widths</span></a><a class="item" href="kp12-flat-generalization.html"><strong>平坦與泛化：動機不是保證</strong><span>Flatness motivates, but does not guarantee</span></a></section><section id="kp13"><h2>KP13</h2><a class="item" href="kp13-sam-objective.html"><strong>SAM：先找附近最壞，再降低它</strong><span>SAM minimizes the worst nearby loss</span></a><a class="item" href="kp13-sam-sharpness.html"><strong>SAM 損失 = 原損失 + 銳度</strong><span>SAM loss equals loss plus sharpness</span></a><a class="item" href="kp13-sam-perturbation.html"><strong>一階近似：往上坡推一小步</strong><span>First-order adversarial perturbation</span></a><a class="item" href="kp13-sam-dual-update.html"><strong>一般範數與兩次反傳</strong><span>Dual norms and the two-pass update</span></a></section><section id="kp14"><h2>KP14</h2><a class="item" href="kp14-esam-swp.html"><strong>ESAM：只擾動一部分權重</strong><span>ESAM stochastic weight perturbation</span></a><a class="item" href="kp14-esam-sds.html"><strong>SDS：挑損失上升較多的樣本</strong><span>Select sharpness-sensitive samples</span></a><a class="item" href="kp14-asam.html"><strong>ASAM：讓鄰域跟著權重尺度</strong><span>ASAM adapts the neighborhood scale</span></a></section><section id="kp15"><h2>KP15</h2><a class="item" href="kp15-overparameterization.html"><strong>過參數化：訓得動的理論</strong><span>Overparameterization and optimization</span></a><a class="item" href="kp15-landscape-theorem.html"><strong>命題 4：局部谷底與鞍點</strong><span>Proposition 4: minima and saddles</span></a><a class="item" href="kp15-deep-convergence.html"><strong>命題 5：深網收斂與條件</strong><span>Proposition 5: deep-network convergence</span></a><a class="item" href="kp15-deep-width-steps.html"><strong>深網定理：寬度、步長、高機率</strong><span>Width, step size and high probability</span></a></section><section id="kp16"><h2>KP16</h2><a class="item" href="kp16-research-map.html"><strong>延伸理論：三個研究方向</strong><span>Three directions beyond the core theorems</span></a><a class="item" href="kp16-chapter-summary.html"><strong>全章收網：算梯度、走一步、驗結果</strong><span>Compute gradients, update, then evaluate</span></a></section><p><a href="audit-notes.html">公式修正、前提與稽核紀錄 / Audit notes</a></p></main></html>