114 lines
5.0 KiB
TeX
114 lines
5.0 KiB
TeX
We instantiate the general ROLL framework (\Cref{sec:roll-formulation}) with
|
|
Gaussian-distributed scores. Each class's scores are modelled as
|
|
$\mathcal{N}(\mu_k, \sigma_k^2)$, fitted by maximum-likelihood estimation.
|
|
|
|
\paragraph{Forward pass.}
|
|
|
|
The MLE parameters are:
|
|
\begin{equation}
|
|
\mu_k = \frac{1}{|\mathcal{B}_k|}\sum_{\mathbf{x}_j \in \mathcal{B}_k} f_\theta(\mathbf{x}_j),
|
|
\qquad
|
|
\sigma_k^2 = \frac{1}{|\mathcal{B}_k|}\sum_{\mathbf{x}_j \in \mathcal{B}_k}(f_\theta(\mathbf{x}_j)-\mu_k)^2
|
|
\end{equation}
|
|
|
|
The CDF and quantile function of $\mathcal{N}(\mu,\sigma^2)$ are:
|
|
\[
|
|
F(x) = \tfrac{1}{2}\!\left[1 + \operatorname{erf}\!\left(\tfrac{x-\mu}{\sigma\sqrt{2}}\right)\right],
|
|
\qquad
|
|
F^{-1}(p) = \mu + \sigma\sqrt{2}\,\operatorname{ierf}(2p-1)
|
|
\]
|
|
where $\operatorname{erf}(x)=\frac{2}{\sqrt{\pi}}\int_0^x e^{-t^2}\,\mathrm{d}t$ is the Gauss
|
|
error function and $\operatorname{ierf}$ its functional inverse.
|
|
Substituting into the general ROLL objective \eqref{eq:roll-tpr-at-fpr}:
|
|
\begin{align}
|
|
\tau &= \mu_0 + \sigma_0\sqrt{2}\,\operatorname{ierf}(1-2\alpha) \nonumber \\
|
|
\mathcal{L}_{\text{ROLL-TPR@FPR}}^{\text{GAUSSIAN}}(f_\theta(\mathcal{B});\,\alpha)
|
|
&= \tfrac{1}{2}\!\left[1 + \operatorname{erf}\!\left(\tfrac{\tau-\mu_1}{\sigma_1\sqrt{2}}\right)\right]
|
|
\label{eq:roll-tpr-at-fpr-gaussian}
|
|
\end{align}
|
|
|
|
\paragraph{Gradient derivation.}
|
|
|
|
We derive the gradient of \eqref{eq:roll-tpr-at-fpr-gaussian} with respect to
|
|
$f_\theta(\mathbf{x}_i)$, instantiating the general structure~\eqref{eq:roll-gradient}
|
|
for the Gaussian case. Throughout, $\tau = \hat{F}_0^{-1}(1-\alpha) =
|
|
\mu_0 + \sigma_0\sqrt{2}\,\operatorname{ierf}(1-2\alpha)$.
|
|
|
|
\paragraph{Case $y_i = 1$.}
|
|
|
|
The score $f_\theta(\mathbf{x}_i)$ enters through $\mu_1$ and $\sigma_1$.
|
|
Differentiating $\hat{F}_1(\tau) = \frac{1}{2}\bigl[1+\operatorname{erf}\!\bigl(\frac{\tau-\mu_1}{\sigma_1\sqrt{2}}\bigr)\bigr]$:
|
|
|
|
\[
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial \mu_1} =
|
|
-\frac{1}{\sigma_1\sqrt{2\pi}}\exp\!\left(-\frac{(\tau - \mu_1)^2}{2\sigma_1^2}\right)
|
|
\]
|
|
|
|
\[
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial \sigma_1} =
|
|
\frac{\mu_1 - \tau}{\sigma_1^2\sqrt{2\pi}}\exp\!\left(-\frac{(\tau - \mu_1)^2}{2\sigma_1^2}\right)
|
|
\]
|
|
|
|
Combined with $\frac{\partial \mu_1}{\partial f_\theta(\mathbf{x}_i)} = \frac{1}{|\mathcal{B}_1|}$
|
|
and $\frac{\partial \sigma_1}{\partial f_\theta(\mathbf{x}_i)} = \frac{f_\theta(\mathbf{x}_i)-\mu_1}{|\mathcal{B}_1|\sigma_1}$,
|
|
the chain rule gives:
|
|
|
|
\begin{equation}
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial f_{\theta}(\mathbf{x}_i)}
|
|
= -\frac{1}{|\mathcal{B}_1|\,\sigma_1\sqrt{2\pi}}
|
|
\exp\!\left(-\frac{(\tau - \mu_1)^2}{2\sigma_1^2}\right)
|
|
\left(1 + \frac{(\tau - \mu_1)(f_\theta(\mathbf{x}_i) - \mu_1)}{\sigma_1^2}\right)
|
|
\label{eq:gauss-grad-y1}
|
|
\end{equation}
|
|
|
|
\paragraph{Case $y_i = 0$.}
|
|
|
|
The score $f_\theta(\mathbf{x}_i)$ affects the loss only through $\tau$. By the chain rule:
|
|
|
|
\[
|
|
\frac{\partial \hat{F}_1(\tau)}{\partial f_\theta(\mathbf{x}_i)}
|
|
= \underbrace{\frac{\partial \hat{F}_1(\tau)}{\partial \tau}}_{\text{PDF of class 1 at }\tau}
|
|
\cdot \frac{\partial \tau}{\partial f_\theta(\mathbf{x}_i)}
|
|
\]
|
|
|
|
The first factor is the Gaussian PDF at $\tau$:
|
|
\[
|
|
\frac{\partial \hat{F}_1(\tau)}{\partial \tau}
|
|
= \frac{1}{\sigma_1\sqrt{2\pi}}\exp\!\left(-\frac{(\tau - \mu_1)^2}{2\sigma_1^2}\right)
|
|
\]
|
|
|
|
For the second factor, since $\tau = \mu_0 + \sigma_0\sqrt{2}\,\operatorname{ierf}(1-2\alpha)$:
|
|
$\frac{\partial\tau}{\partial\mu_0}=1$ and
|
|
$\frac{\partial\tau}{\partial\sigma_0}=\frac{\tau-\mu_0}{\sigma_0}$.
|
|
Combined with $\frac{\partial\mu_0}{\partial f_\theta(\mathbf{x}_i)}=\frac{1}{|\mathcal{B}_0|}$
|
|
and $\frac{\partial\sigma_0}{\partial f_\theta(\mathbf{x}_i)}=\frac{f_\theta(\mathbf{x}_i)-\mu_0}{|\mathcal{B}_0|\sigma_0}$:
|
|
\[
|
|
\frac{\partial\tau}{\partial f_\theta(\mathbf{x}_i)}
|
|
= \frac{1}{|\mathcal{B}_0|}\!\left(1+\frac{(\tau-\mu_0)(f_\theta(\mathbf{x}_i)-\mu_0)}{\sigma_0^2}\right)
|
|
\]
|
|
|
|
Therefore:
|
|
\begin{equation}
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial f_\theta(\mathbf{x}_i)}
|
|
= \frac{1}{|\mathcal{B}_0|\,\sigma_1\sqrt{2\pi}}
|
|
\exp\!\left(-\frac{(\tau - \mu_1)^2}{2\sigma_1^2}\right)
|
|
\left(1 + \frac{(\tau - \mu_0)(f_\theta(\mathbf{x}_i) - \mu_0)}{\sigma_0^2}\right)
|
|
\label{eq:gauss-grad-y0}
|
|
\end{equation}
|
|
|
|
Substituting \eqref{eq:gauss-grad-y1} and \eqref{eq:gauss-grad-y0} into \eqref{eq:roll-gradient}:
|
|
\begin{equation}
|
|
\frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}^{\text{GAUSSIAN}}}{\partial f_\theta(\mathbf{x}_i)} =
|
|
\begin{cases}
|
|
\dfrac{-1}{|\mathcal{B}_1|\,\sigma_1\sqrt{2\pi}}
|
|
\exp\!\left(-\dfrac{(\tau-\mu_1)^2}{2\sigma_1^2}\right)
|
|
\!\left(1 + \dfrac{(\tau-\mu_1)(f_\theta(\mathbf{x}_i)-\mu_1)}{\sigma_1^2}\right)
|
|
& \text{if } y_i = 1 \\[18pt]
|
|
\dfrac{1}{|\mathcal{B}_0|\,\sigma_1\sqrt{2\pi}}
|
|
\exp\!\left(-\dfrac{(\tau-\mu_1)^2}{2\sigma_1^2}\right)
|
|
\!\left(1 + \dfrac{(\tau-\mu_0)(f_\theta(\mathbf{x}_i)-\mu_0)}{\sigma_0^2}\right)
|
|
& \text{if } y_i = 0
|
|
\end{cases}
|
|
\label{eq:gauss-grad-combined}
|
|
\end{equation}
|