43 lines
2.0 KiB
TeX
43 lines
2.0 KiB
TeX
This appendix gives the full proof of \Cref{prop:gradient-balance}.
|
|
|
|
\begin{proof}
|
|
We treat each class separately via a uniform-shift argument.
|
|
|
|
\medskip\noindent\textit{Sum over $\mathcal{B}_0$.}
|
|
From \Cref{eq:roll-gradient}, for $y_i = 0$:
|
|
\[
|
|
\frac{\partial \mathcal{L}}{\partial f_\theta(\mathbf{x}_i)}
|
|
= \frac{\partial \hat{F}_1(\tau)}{\partial \tau}
|
|
\cdot \frac{\partial \tau}{\partial f_\theta(\mathbf{x}_i)}
|
|
\]
|
|
We claim $\sum_{\mathbf{x}_i \in \mathcal{B}_0}\frac{\partial \tau}{\partial f_\theta(\mathbf{x}_i)} = 1$.
|
|
Shift all negative scores uniformly: $f_\theta(\mathbf{x}_i) \to f_\theta(\mathbf{x}_i) + \delta$ for
|
|
$\mathbf{x}_i \in \mathcal{B}_0$. By translation invariance, $\hat{F}_0$ simply translates, so
|
|
$\tau = \hat{F}_0^{-1}(1-\alpha)$ shifts by exactly $\delta$. Differentiating with respect to $\delta$ at
|
|
$\delta = 0$:
|
|
\[
|
|
1 = \frac{\mathrm{d}\tau}{\mathrm{d}\delta}\bigg|_{\delta=0}
|
|
= \sum_{\mathbf{x}_i \in \mathcal{B}_0} \frac{\partial \tau}{\partial f_\theta(\mathbf{x}_i)}
|
|
\]
|
|
Summing the $y_i = 0$ gradients therefore gives:
|
|
\begin{equation}
|
|
\sum_{\mathbf{x}_i \in \mathcal{B}_0}
|
|
\frac{\partial \mathcal{L}}{\partial f_\theta(\mathbf{x}_i)}
|
|
= \frac{\partial \hat{F}_1(\tau)}{\partial \tau} \cdot 1
|
|
= \frac{\partial \hat{F}_1(\tau)}{\partial \tau}
|
|
\label{eq:grad-balance-b0}
|
|
\end{equation}
|
|
|
|
\medskip\noindent\textit{Sum over $\mathcal{B}_1$.}
|
|
For $y_i = 1$, $\frac{\partial \mathcal{L}}{\partial f_\theta(\mathbf{x}_i)} = \frac{\partial \hat{F}_1(\tau)}{\partial f_\theta(\mathbf{x}_i)}$.
|
|
Shifting all positive scores by $\delta$ and applying translation invariance of $\hat{F}_1$:
|
|
$\hat{F}_1(\tau\,;\,\mathbf{s}+\delta\mathbf{1}) = \hat{F}_1(\tau-\delta\,;\,\mathbf{s})$.
|
|
Differentiating at $\delta = 0$:
|
|
\[
|
|
\sum_{\mathbf{x}_i \in \mathcal{B}_1}
|
|
\frac{\partial \hat{F}_1(\tau)}{\partial f_\theta(\mathbf{x}_i)}
|
|
= -\frac{\partial \hat{F}_1(\tau)}{\partial \tau}
|
|
\]
|
|
Combining with \eqref{eq:grad-balance-b0} gives \eqref{eq:grad-balance}.
|
|
\end{proof}
|