144 lines
6.7 KiB
TeX
144 lines
6.7 KiB
TeX
We instantiate the general ROLL framework (\Cref{sec:roll-formulation}) with
|
|
Gaussian-distributed scores. Each class's scores are modelled as
|
|
$\mathcal{N}(\mu_k, \sigma_k^2)$, fitted by maximum-likelihood
|
|
estimation.\footnote{If a class's scores are literally constant, $\sigma_k=0$ and the
|
|
CDF and gradient expressions throughout this appendix are undefined (division by
|
|
$\sigma_k$ or $\sigma_k^2$). This edge case is unsupported in the current
|
|
implementation; it does not arise with real, continuously-valued model outputs, but
|
|
supporting it in general would require an explicit floor on $\sigma_k$.}
|
|
|
|
\paragraph{Forward pass.}
|
|
|
|
The MLE parameters are:
|
|
\begin{equation}
|
|
\mu_k = \frac{1}{|\mathcal{B}_k|}\sum_{\mathbf{x}_j \in \mathcal{B}_k} f_\theta(\mathbf{x}_j),
|
|
\qquad
|
|
\sigma_k^2 = \frac{1}{|\mathcal{B}_k|}\sum_{\mathbf{x}_j \in \mathcal{B}_k}(f_\theta(\mathbf{x}_j)-\mu_k)^2
|
|
\end{equation}
|
|
|
|
The CDF and quantile function of $\mathcal{N}(\mu,\sigma^2)$ are:
|
|
\[
|
|
F(x) = \tfrac{1}{2}\!\left[1 + \operatorname{erf}\!\left(\tfrac{x-\mu}{\sigma\sqrt{2}}\right)\right],
|
|
\qquad
|
|
F^{-1}(p) = \mu + \sigma\sqrt{2}\,\operatorname{ierf}(2p-1)
|
|
\]
|
|
where $\operatorname{erf}(x)=\frac{2}{\sqrt{\pi}}\int_0^x e^{-t^2}\,\mathrm{d}t$ is the Gauss
|
|
error function and $\operatorname{ierf}$ its functional inverse.
|
|
Substituting into the general ROLL objective \eqref{eq:roll-tpr-at-fpr}:
|
|
\begin{align}
|
|
\tau &= \mu_0 + \sigma_0\sqrt{2}\,\operatorname{ierf}(1-2\alpha) \nonumber \\
|
|
\mathcal{L}_{\text{ROLL-TPR@FPR}}^{\text{GAUSSIAN}}(f_\theta(\mathcal{B});\,\alpha)
|
|
&= \tfrac{1}{2}\!\left[1 + \operatorname{erf}\!\left(\tfrac{\tau-\mu_1}{\sigma_1\sqrt{2}}\right)\right]
|
|
\label{eq:roll-tpr-at-fpr-gaussian}
|
|
\end{align}
|
|
|
|
\paragraph{Closed-form AOC.}
|
|
|
|
Unlike the general (KDE) case, the full AOC integral admits a closed form for the
|
|
Gaussian instantiation, bypassing the 99-point quadrature of \Cref{sec:roll-aoc}
|
|
entirely. Writing $\Phi$ for the standard normal CDF
|
|
($\Phi(x) = \tfrac{1}{2}[1+\operatorname{erf}(x/\sqrt{2})]$) and $\varphi$ for its
|
|
density, substitute \eqref{eq:roll-tpr-at-fpr-gaussian} into
|
|
$\mathcal{L}_{\text{AOC}} = \int_0^1 \mathcal{L}_{\text{ROLL-TPR@FPR}}(\alpha)\,d\alpha$
|
|
and change variables via $u = \Phi^{-1}(1-\alpha)$, so $\tau = \mu_0 + \sigma_0 u$:
|
|
\[
|
|
\mathcal{L}_{\text{AOC}}^{\text{Gaussian}}
|
|
= \int_{-\infty}^{\infty} \Phi\!\left(\frac{\mu_0-\mu_1+\sigma_0 u}{\sigma_1}\right)\varphi(u)\,du.
|
|
\]
|
|
Applying the standard identity
|
|
$\int_{-\infty}^{\infty}\Phi(a+bu)\,\varphi(u)\,du = \Phi\!\bigl(a/\sqrt{1+b^2}\bigr)$
|
|
with $a = (\mu_0-\mu_1)/\sigma_1$, $b = \sigma_0/\sigma_1$:
|
|
\begin{equation}
|
|
\mathcal{L}_{\text{AOC}}^{\text{Gaussian}} = \Phi\!\left(\frac{\mu_0-\mu_1}{\sqrt{\sigma_0^2+\sigma_1^2}}\right).
|
|
\label{eq:gauss-aoc-closed-form}
|
|
\end{equation}
|
|
(Verified numerically against direct quadrature of the integral.) We nonetheless use
|
|
the same 99-point average as every other instantiation in this thesis, for
|
|
implementation uniformity across distribution families rather than necessity
|
|
(\Cref{sec:roll-aoc}).
|
|
|
|
\paragraph{Gradient derivation.}
|
|
|
|
We derive the gradient of \eqref{eq:roll-tpr-at-fpr-gaussian} with respect to
|
|
$f_\theta(\mathbf{x}_i)$, instantiating the general structure~\eqref{eq:roll-gradient}
|
|
for the Gaussian case. Throughout, $\tau = \hat{F}_0^{-1}(1-\alpha) =
|
|
\mu_0 + \sigma_0\sqrt{2}\,\operatorname{ierf}(1-2\alpha)$.
|
|
|
|
\paragraph{Case $y_i = 1$.}
|
|
|
|
The score $f_\theta(\mathbf{x}_i)$ enters through $\mu_1$ and $\sigma_1$.
|
|
Differentiating $\hat{F}_1(\tau) = \frac{1}{2}\bigl[1+\operatorname{erf}\!\bigl(\frac{\tau-\mu_1}{\sigma_1\sqrt{2}}\bigr)\bigr]$:
|
|
|
|
\[
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial \mu_1} =
|
|
-\frac{1}{\sigma_1\sqrt{2\pi}}\exp\!\left(-\frac{(\tau - \mu_1)^2}{2\sigma_1^2}\right)
|
|
\]
|
|
|
|
\[
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial \sigma_1} =
|
|
\frac{\mu_1 - \tau}{\sigma_1^2\sqrt{2\pi}}\exp\!\left(-\frac{(\tau - \mu_1)^2}{2\sigma_1^2}\right)
|
|
\]
|
|
|
|
Combined with $\frac{\partial \mu_1}{\partial f_\theta(\mathbf{x}_i)} = \frac{1}{|\mathcal{B}_1|}$
|
|
and $\frac{\partial \sigma_1}{\partial f_\theta(\mathbf{x}_i)} = \frac{f_\theta(\mathbf{x}_i)-\mu_1}{|\mathcal{B}_1|\sigma_1}$,
|
|
the chain rule gives:
|
|
|
|
\begin{equation}
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial f_{\theta}(\mathbf{x}_i)}
|
|
= -\frac{1}{|\mathcal{B}_1|\,\sigma_1\sqrt{2\pi}}
|
|
\exp\!\left(-\frac{(\tau - \mu_1)^2}{2\sigma_1^2}\right)
|
|
\left(1 + \frac{(\tau - \mu_1)(f_\theta(\mathbf{x}_i) - \mu_1)}{\sigma_1^2}\right)
|
|
\label{eq:gauss-grad-y1}
|
|
\end{equation}
|
|
|
|
\paragraph{Case $y_i = 0$.}
|
|
|
|
The score $f_\theta(\mathbf{x}_i)$ affects the loss only through $\tau$. By the chain rule:
|
|
|
|
\[
|
|
\frac{\partial \hat{F}_1(\tau)}{\partial f_\theta(\mathbf{x}_i)}
|
|
= \underbrace{\frac{\partial \hat{F}_1(\tau)}{\partial \tau}}_{\text{PDF of class 1 at }\tau}
|
|
\cdot \frac{\partial \tau}{\partial f_\theta(\mathbf{x}_i)}
|
|
\]
|
|
|
|
The first factor is the Gaussian PDF at $\tau$:
|
|
\[
|
|
\frac{\partial \hat{F}_1(\tau)}{\partial \tau}
|
|
= \frac{1}{\sigma_1\sqrt{2\pi}}\exp\!\left(-\frac{(\tau - \mu_1)^2}{2\sigma_1^2}\right)
|
|
\]
|
|
|
|
For the second factor, since $\tau = \mu_0 + \sigma_0\sqrt{2}\,\operatorname{ierf}(1-2\alpha)$:
|
|
$\frac{\partial\tau}{\partial\mu_0}=1$ and
|
|
$\frac{\partial\tau}{\partial\sigma_0}=\frac{\tau-\mu_0}{\sigma_0}$.
|
|
Combined with $\frac{\partial\mu_0}{\partial f_\theta(\mathbf{x}_i)}=\frac{1}{|\mathcal{B}_0|}$
|
|
and $\frac{\partial\sigma_0}{\partial f_\theta(\mathbf{x}_i)}=\frac{f_\theta(\mathbf{x}_i)-\mu_0}{|\mathcal{B}_0|\sigma_0}$:
|
|
\[
|
|
\frac{\partial\tau}{\partial f_\theta(\mathbf{x}_i)}
|
|
= \frac{1}{|\mathcal{B}_0|}\!\left(1+\frac{(\tau-\mu_0)(f_\theta(\mathbf{x}_i)-\mu_0)}{\sigma_0^2}\right)
|
|
\]
|
|
|
|
Therefore:
|
|
\begin{equation}
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial f_\theta(\mathbf{x}_i)}
|
|
= \frac{1}{|\mathcal{B}_0|\,\sigma_1\sqrt{2\pi}}
|
|
\exp\!\left(-\frac{(\tau - \mu_1)^2}{2\sigma_1^2}\right)
|
|
\left(1 + \frac{(\tau - \mu_0)(f_\theta(\mathbf{x}_i) - \mu_0)}{\sigma_0^2}\right)
|
|
\label{eq:gauss-grad-y0}
|
|
\end{equation}
|
|
|
|
Substituting \eqref{eq:gauss-grad-y1} and \eqref{eq:gauss-grad-y0} into \eqref{eq:roll-gradient}:
|
|
\begin{equation}
|
|
\frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}^{\text{GAUSSIAN}}}{\partial f_\theta(\mathbf{x}_i)} =
|
|
\begin{cases}
|
|
\dfrac{-1}{|\mathcal{B}_1|\,\sigma_1\sqrt{2\pi}}
|
|
\exp\!\left(-\dfrac{(\tau-\mu_1)^2}{2\sigma_1^2}\right)
|
|
\!\left(1 + \dfrac{(\tau-\mu_1)(f_\theta(\mathbf{x}_i)-\mu_1)}{\sigma_1^2}\right)
|
|
& \text{if } y_i = 1 \\[18pt]
|
|
\dfrac{1}{|\mathcal{B}_0|\,\sigma_1\sqrt{2\pi}}
|
|
\exp\!\left(-\dfrac{(\tau-\mu_1)^2}{2\sigma_1^2}\right)
|
|
\!\left(1 + \dfrac{(\tau-\mu_0)(f_\theta(\mathbf{x}_i)-\mu_0)}{\sigma_0^2}\right)
|
|
& \text{if } y_i = 0
|
|
\end{cases}
|
|
\label{eq:gauss-grad-combined}
|
|
\end{equation}
|