69 lines
3.1 KiB
TeX
69 lines
3.1 KiB
TeX
We derive the gradient of \eqref{eq:roll-tpr-at-fpr-gaussian} with respect to
|
|
$f_\theta(\mathbf{x}_i)$, instantiating the general structure~\eqref{eq:roll-gradient}
|
|
for the Gaussian case. Throughout, $\tau = \hat{F}_0^{-1}(1-\alpha) =
|
|
\mu_0 + \sigma_0\sqrt{2}\,\operatorname{ierf}(1-2\alpha)$.
|
|
|
|
\paragraph{Case $y_i = 1$.}
|
|
|
|
The score $f_\theta(\mathbf{x}_i)$ enters through $\mu_1$ and $\sigma_1$.
|
|
Differentiating $\hat{F}_1(\tau) = \frac{1}{2}\bigl[1+\operatorname{erf}\!\bigl(\frac{\tau-\mu_1}{\sigma_1\sqrt{2}}\bigr)\bigr]$:
|
|
|
|
\[
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial \mu_1} =
|
|
-\frac{1}{\sigma_1\sqrt{2\pi}}\exp\!\left(-\frac{(\tau - \mu_1)^2}{2\sigma_1^2}\right)
|
|
\]
|
|
|
|
\[
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial \sigma_1} =
|
|
\frac{\mu_1 - \tau}{\sigma_1^2\sqrt{2\pi}}\exp\!\left(-\frac{(\tau - \mu_1)^2}{2\sigma_1^2}\right)
|
|
\]
|
|
|
|
Combined with $\frac{\partial \mu_1}{\partial f_\theta(\mathbf{x}_i)} = \frac{1}{|\mathcal{B}_1|}$
|
|
and $\frac{\partial \sigma_1}{\partial f_\theta(\mathbf{x}_i)} = \frac{f_\theta(\mathbf{x}_i)-\mu_1}{|\mathcal{B}_1|\sigma_1}$,
|
|
the chain rule gives:
|
|
|
|
\begin{equation}
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial f_{\theta}(\mathbf{x}_i)}
|
|
= -\frac{1}{|\mathcal{B}_1|\,\sigma_1\sqrt{2\pi}}
|
|
\exp\!\left(-\frac{(\tau - \mu_1)^2}{2\sigma_1^2}\right)
|
|
\left(1 + \frac{(\tau - \mu_1)(f_\theta(\mathbf{x}_i) - \mu_1)}{\sigma_1^2}\right)
|
|
\label{eq:gauss-grad-y1}
|
|
\end{equation}
|
|
|
|
\paragraph{Case $y_i = 0$.}
|
|
|
|
The score $f_\theta(\mathbf{x}_i)$ affects the loss only through $\tau$. By the chain rule:
|
|
|
|
\[
|
|
\frac{\partial \hat{F}_1(\tau)}{\partial f_\theta(\mathbf{x}_i)}
|
|
= \underbrace{\frac{\partial \hat{F}_1(\tau)}{\partial \tau}}_{\text{PDF of class 1 at }\tau}
|
|
\cdot \frac{\partial \tau}{\partial f_\theta(\mathbf{x}_i)}
|
|
\]
|
|
|
|
The first factor is the Gaussian PDF at $\tau$:
|
|
\[
|
|
\frac{\partial \hat{F}_1(\tau)}{\partial \tau}
|
|
= \frac{1}{\sigma_1\sqrt{2\pi}}\exp\!\left(-\frac{(\tau - \mu_1)^2}{2\sigma_1^2}\right)
|
|
\]
|
|
|
|
For the second factor, since $\tau = \mu_0 + \sigma_0\sqrt{2}\,\operatorname{ierf}(1-2\alpha)$:
|
|
$\frac{\partial\tau}{\partial\mu_0}=1$ and
|
|
$\frac{\partial\tau}{\partial\sigma_0}=\frac{\tau-\mu_0}{\sigma_0}$.
|
|
Combined with $\frac{\partial\mu_0}{\partial f_\theta(\mathbf{x}_i)}=\frac{1}{|\mathcal{B}_0|}$
|
|
and $\frac{\partial\sigma_0}{\partial f_\theta(\mathbf{x}_i)}=\frac{f_\theta(\mathbf{x}_i)-\mu_0}{|\mathcal{B}_0|\sigma_0}$:
|
|
\[
|
|
\frac{\partial\tau}{\partial f_\theta(\mathbf{x}_i)}
|
|
= \frac{1}{|\mathcal{B}_0|}\!\left(1+\frac{(\tau-\mu_0)(f_\theta(\mathbf{x}_i)-\mu_0)}{\sigma_0^2}\right)
|
|
\]
|
|
|
|
Therefore:
|
|
\begin{equation}
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial f_\theta(\mathbf{x}_i)}
|
|
= \frac{1}{|\mathcal{B}_0|\,\sigma_1\sqrt{2\pi}}
|
|
\exp\!\left(-\frac{(\tau - \mu_1)^2}{2\sigma_1^2}\right)
|
|
\left(1 + \frac{(\tau - \mu_0)(f_\theta(\mathbf{x}_i) - \mu_0)}{\sigma_0^2}\right)
|
|
\label{eq:gauss-grad-y0}
|
|
\end{equation}
|
|
|
|
Substituting \eqref{eq:gauss-grad-y1} and \eqref{eq:gauss-grad-y0} into \eqref{eq:roll-gradient} yields \eqref{eq:gauss-grad-combined}.
|