791 lines
45 KiB
TeX
791 lines
45 KiB
TeX
|
|
\section{Problem Formulation}
|
|
\label{sec:problem-formulation}
|
|
|
|
% Motivation for this section — why accuracy fails under class imbalance, why
|
|
% AUC is the natural objective under label noise and imbalance, and the
|
|
% Neyman-Pearson framing for single-threshold optimization — is developed in
|
|
% \Cref{chapter:related_work} (\Cref{sec:imbalanced-tpr-fpr,sec:neyman-pearson}).
|
|
% This section establishes the notation and primary objective used throughout the chapter.
|
|
Given a dataset $\mathcal{D} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{n}$ with inputs
|
|
$\mathbf{x}_i \in \mathcal{X}$ and binary labels $y_i \in \{0, 1\}$, we learn a
|
|
scoring function $f_\theta : \mathcal{X} \to \mathbb{R}$ by minimising a surrogate loss on $\mathcal{D}$.
|
|
The positive and negative subsets of $\mathcal{D}$ are:
|
|
\[
|
|
\mathcal{D}_1 = \{(\mathbf{x}_i, y_i) \in \mathcal{D} \mid y_i = 1\},
|
|
\qquad
|
|
\mathcal{D}_0 = \{(\mathbf{x}_i, y_i) \in \mathcal{D} \mid y_i = 0\}.
|
|
\]
|
|
Applying a threshold $\tau$ produces predictions $\hat{y} = \mathbf{1}[f_\theta(\mathbf{x}) > \tau]$.
|
|
The \emph{true positive rate} (TPR) and \emph{false positive rate} (FPR) at threshold $\tau$ are:
|
|
\begin{align}
|
|
\operatorname{TPR}(\tau) &= P\!\bigl(f_\theta(\mathbf{x}) > \tau \mid y = 1\bigr), \\
|
|
\operatorname{FPR}(\tau) &= P\!\bigl(f_\theta(\mathbf{x}) > \tau \mid y = 0\bigr).
|
|
\end{align}
|
|
|
|
Conversely, we may denote the inverse operations $\operatorname{FPR}^{-1}$ and $\operatorname{TPR}^{-1}$ to be:
|
|
|
|
\begin{align}
|
|
\operatorname{TPR^{-1}}\!\bigl(P\!\bigl(f_\theta(\mathbf{x}) > \tau \mid y = 1\bigr)\bigr) &= \tau, \\
|
|
\operatorname{FPR^{-1}}\!\bigl(P\!\bigl(f_\theta(\mathbf{x}) > \tau \mid y = 0\bigr)\bigr) &= \tau .
|
|
\end{align}
|
|
|
|
|
|
For $\alpha \in (0,1)$, let $\tau = \operatorname{FPR}^{-1}(\alpha)$ denote the threshold achieving
|
|
$\operatorname{FPR}(\tau) = \alpha$, assuming $\operatorname{FPR}$ is continuous and strictly
|
|
decreasing in $\tau$ so that this threshold exists and is unique.\footnote{This requires the
|
|
negative-class score distribution to have no discontinuous steps. For an empirical distribution
|
|
this doesn't need to hold, but every distribution family $\operatorname{ROLL}$ is instantiated with in
|
|
this thesis (KDE with positive bandwidth (\Cref{sec:roll-kde}), Gaussian
|
|
(\Cref{appendix:gauss-grad}), and Beta (\Cref{appendix:beta-roll})) has a continuous,
|
|
strictly increasing CDF, so the statement holds for the
|
|
fitted distributions ROLL actually optimizes over.}
|
|
The curve $\alpha \mapsto \operatorname{TPR}(\operatorname{FPR^{-1}}(\alpha))$ is the ROC curve.
|
|
The primary learning objective is to maximize its area:
|
|
\begin{equation}
|
|
\max_\theta\;\operatorname{AUC}(f_\theta) = \int_0^1 \operatorname{TPR}(\operatorname{FPR}^{-1}(\alpha))\,d\alpha.
|
|
\label{eq:objective-auc}
|
|
\end{equation}
|
|
Maximizing AUC is the natural objective under class imbalance and label noise, where no
|
|
single operating threshold is fixed in advance.
|
|
|
|
As a special case, one may instead optimize a single operating point: maximize
|
|
$\operatorname{TPR}$ at a fixed $\operatorname{FPR} = \alpha_0$, or minimize
|
|
$\operatorname{FPR}$ at a fixed $\operatorname{TPR} = \alpha_0$.
|
|
The ROLL framework, introduced in \Cref{sec:roll-framework}, provides differentiable
|
|
surrogates for both the AUC objective \eqref{eq:objective-auc} and these point objectives.
|
|
|
|
%------------------------------------------------
|
|
|
|
\section{The ROLL Framework}
|
|
\label{sec:roll-framework}
|
|
|
|
In order to allow gradient-based learning methods to optimize AUC directly, we introduce
|
|
ROLL, Rate Optimized Likelihood Loss.
|
|
|
|
The core idea of ROLL is as follows: at each training step, differentiable distributions
|
|
are fitted to the positive- and negative-class score outputs of $f_\theta$. AUC is then
|
|
approximated by evaluating the estimated $\operatorname{TPR}$ at 99 evenly-spaced
|
|
$\operatorname{FPR}$ values across $(0,1)$ and averaging, a numerical integration of
|
|
the ROC curve. Because each per-threshold evaluation is differentiable through the fitted
|
|
distributions, this AUC approximation can be maximized directly by gradient descent, with
|
|
no threshold selection required at inference time.
|
|
|
|
ROLL also supports single-threshold optimization: selecting a target
|
|
$\operatorname{FPR} = \alpha_0$ and maximizing $\operatorname{TPR}$ at exactly that point.
|
|
In Neyman-Pearson terms~\cite{tong2016neyman}, this corresponds to minimizing Type-II error
|
|
at a fixed Type-I error bound.
|
|
|
|
In practice, however, making these evaluations differentiable is not straightforward.
|
|
|
|
\subsection{The Differentiability Problem}
|
|
\label{sec:differentiability-problem}
|
|
|
|
% TPR and FPR are defined via threshold comparisons on model scores, making them
|
|
% piecewise-constant functions of the model parameters with zero gradient almost
|
|
% everywhere — standard backpropagation cannot move through them. Show this
|
|
% concretely with the step-function definition of TPR/FPR and explain why
|
|
% common surrogates (AUC proxies, hinge-based losses) only indirectly target
|
|
% the operating point a practitioner cares about.
|
|
|
|
Directly optimizing TPR for FPR, or vice versa, is a non-differentiable problem.
|
|
|
|
For this sub-section, we will refer to an example of maximizing TPR given a fixed FPR,
|
|
though this can be reversed.
|
|
|
|
For any loss $\mathcal{L}(\theta)$, in order
|
|
to continue fitting our model, we must derive some gradient
|
|
|
|
$\frac{\partial \mathcal{L}(\theta)}{\partial f_\theta(\mathbf{x}_i)}$ where $f_\theta(\mathbf{x}_i)$ is the score
|
|
given to data point $\mathbf{x}_i$ by our model $f_\theta$.
|
|
There are two inherent steps built-in to our methods:
|
|
|
|
\begin{itemize}
|
|
\item Based on fixed rate, calculate threshold that gives said rate.
|
|
For example, when fixing FPR = $\alpha$, find some threshold $\tau$ such that $\mathbb{E}[\mathbf{1}[f_\theta(\mathbf{x}_i) > \tau \mid y_i = 0]] = \alpha$
|
|
\item Given the calculated threshold, calculate the rate we would like to maximize
|
|
(in our example, maximize TPR)
|
|
\end{itemize}
|
|
|
|
TPR or FPR calculation, given a threshold, is non-differentiable.
|
|
The reason for that is that the TPR and FPR as a function of $f_\theta(\mathbf{x}_i)$ are
|
|
non-continuous. If $f_\theta(\mathbf{x}_i) \neq \tau$, that is, the score is not exactly on the threshold, then changing it
|
|
by some $\varepsilon$ will not affect either TPR or FPR at all, and the derivative of the function w.r.t.\ $f_\theta(\mathbf{x}_i)$ will be zero. Likewise,
|
|
if $f_\theta(\mathbf{x}_i) = \tau$, the one-sided derivatives of TPR/FPR w.r.t.\ $f_\theta(\mathbf{x}_i)$ disagree: approaching from below leaves the indicator at $0$, while approaching from above sends the difference quotient to $+\infty$. The derivative at the jump is therefore \emph{undefined}, not an ordinary derivative equal to infinity, and either way there is no usable gradient to fit our model with.
|
|
|
|
It is important to distinguish this finite-sample indicator objective from the population rate it estimates. The TPR and FPR terms above are indicator averages over the finite sample $\mathcal{D}$: each component of the sum, $\mathbf{1}[f_\theta(\mathbf{x}_i) > \tau]$, is a step function of that sample's own score, and it is this per-sample step that is non-differentiable. The distribution the dataset $\mathcal{D}$ draws from might itself have a smooth CDF, therefore an FPR or TPR term derived from said smooth CDF might be differentiable. However, in practice, we must always train off of a finite sample $\mathcal{D}$, and so the differentiability problem remains.
|
|
|
|
This necessitates a different approach, to make these calculations differentiable w.r.t.\ the scores themselves, in order to fit this type of optimization into a learning platform.
|
|
|
|
\subsection{Score Distribution Fitting as a Solution}
|
|
\label{sec:score-distribution-fitting}
|
|
|
|
% The key insight: instead of thresholding scores directly, fit a differentiable
|
|
% probability distribution to the positive-class and negative-class score outputs.
|
|
% TPR and FPR then become integrals of that distribution above/below a threshold,
|
|
% which are smooth and differentiable in the distribution parameters — and through
|
|
% them, in the model weights. Explain intuitively why this preserves the semantics
|
|
% of TPR/FPR while making the objective amenable to gradient descent.
|
|
|
|
The way to overcome this problem in differentiability is not to calculate the TPR and FPR
|
|
directly, but to estimate them by fitting probability distributions to the score output of the model at
|
|
every step. Continuous probability distributions are continuous with respect to their defining parameters, and a derivation of the parameters with respect to the distributions themselves can be made.
|
|
Additionally, we can calculate the TPR, FPR given the probability distributions CDF and ICDF functions. If we can guarantee these functions to be differentiable, which proved to be the main challenge of this work, we can guarantee the entire process to be differentiable. Once we do so, a derivative of the scores themselves with respect to the loss can be achieved.
|
|
|
|
|
|
\subsection{Per-Threshold ROLL Loss}
|
|
\label{sec:roll-formulation}
|
|
|
|
% Present the abstract ROLL loss in terms of a generic distribution $\mathcal{F}$
|
|
% fitted to model scores, without yet committing to a specific family. Derive the
|
|
% expression for TPR@FPR as a function of $\mathcal{F}$'s parameters. This
|
|
% per-threshold loss is the building block that ROLL-AOC averages over 99 FPR values
|
|
% to obtain the primary AUC objective.
|
|
|
|
Using the dataset $\mathcal{D} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{n}$ as introduced in
|
|
\Cref{sec:problem-formulation}, let $\mathcal{B} \subseteq \mathcal{D}$ denote a training batch
|
|
over which we compute the loss.
|
|
|
|
We then subdivide the batch into positive and negative samples:
|
|
%
|
|
\begin{align*}
|
|
\mathcal{B}_1 &= \{(\mathbf{x}_i, y_i) \in \mathcal{B} \mid y_i = 1\} \\
|
|
\mathcal{B}_0 &= \{(\mathbf{x}_i, y_i) \in \mathcal{B} \mid y_i = 0\}
|
|
\end{align*}
|
|
%
|
|
and extract the corresponding model score vectors:
|
|
%
|
|
\begin{equation*}
|
|
\mathbf{s}^{(1)} = \bigl(f_\theta(\mathbf{x}_i)\bigr)_{(\mathbf{x}_i,\, y_i)\,\in\,\mathcal{B}_1},
|
|
\qquad
|
|
\mathbf{s}^{(0)} = \bigl(f_\theta(\mathbf{x}_i)\bigr)_{(\mathbf{x}_i,\, y_i)\,\in\,\mathcal{B}_0}
|
|
\end{equation*}
|
|
|
|
We then fit a distribution family $\mathcal{F} = \{P_\psi : \psi \in \Psi\}$ separately to each
|
|
score vector in order to obtain a differentiable model of each class's score distribution:
|
|
%
|
|
\begin{equation*}
|
|
\hat{\psi}_1 = \operatorname{Fit}_{\mathcal{F}}\!\left(\mathbf{s}^{(1)}\right),
|
|
\qquad
|
|
\hat{\psi}_0 = \operatorname{Fit}_{\mathcal{F}}\!\left(\mathbf{s}^{(0)}\right)
|
|
\end{equation*}
|
|
%
|
|
Let $\hat{F}_1$ and $\hat{F}_0$ denote the CDFs of $P_{\hat{\psi}_1}$ and $P_{\hat{\psi}_0}$,
|
|
respectively. The KDE instantiation is developed in \Cref{sec:roll-instantiations};
|
|
the Gaussian and Beta instantiations, including full derivations, are in
|
|
\Cref{appendix:gauss-grad,appendix:beta-roll}.
|
|
|
|
Using this notation, at a threshold $\tau$:
|
|
\begin{align}
|
|
\operatorname{TPR}(\tau) &= 1 - \hat{F}_1(\tau), \label{eq:tpr-cdf} \\
|
|
\operatorname{FPR}(\tau) &= 1 - \hat{F}_0(\tau). \label{eq:fpr-cdf}
|
|
\end{align}
|
|
|
|
Let $\hat{F}_1^{-1}$ and $\hat{F}_0^{-1}$ denote the quantile functions of $P_{\hat{\psi}_1}$ and
|
|
$P_{\hat{\psi}_0}$, respectively. The threshold achieving a target TPR of $\alpha$ follows directly
|
|
from \eqref{eq:tpr-cdf}:
|
|
\[
|
|
\operatorname{TPR}(\tau) = \alpha \;\Longrightarrow\; 1 - \hat{F}_1(\tau) = \alpha
|
|
\;\Longrightarrow\; \tau = \hat{F}_1^{-1}(1 - \alpha)
|
|
\]
|
|
and similarly for a target FPR of $\alpha$, from \eqref{eq:fpr-cdf}:
|
|
\[
|
|
\operatorname{FPR}(\tau) = \alpha \;\Longrightarrow\; 1 - \hat{F}_0(\tau) = \alpha
|
|
\;\Longrightarrow\; \tau = \hat{F}_0^{-1}(1 - \alpha)
|
|
\]
|
|
|
|
This yields two versions of our objective. Substituting the ICDF threshold into the rate
|
|
expressions, our estimated TPR at a fixed FPR of $\alpha$ is $1 - \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))$.
|
|
Therefore, to maximise TPR we minimise:
|
|
\begin{equation}
|
|
\mathcal{L}_{\text{ROLL-TPR@FPR}}(f_{\theta}(\mathcal{B}) ; \alpha) = \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))
|
|
\label{eq:roll-tpr-at-fpr}
|
|
\end{equation}
|
|
|
|
Our estimated FPR at a fixed TPR of $\alpha$ is $1 - \hat{F}_0(\hat{F}_1^{-1}(1 - \alpha))$.
|
|
Therefore, to minimise FPR:
|
|
\begin{equation}
|
|
\mathcal{L}_{\text{ROLL-FPR@TPR}}(f_{\theta}(\mathbf{X}) ; \alpha) = 1 - \hat{F}_0(\hat{F}_1^{-1}(1 - \alpha))
|
|
\label{eq:roll-fpr-at-tpr}
|
|
\end{equation}
|
|
|
|
Observe that \eqref{eq:roll-tpr-at-fpr} and \eqref{eq:roll-fpr-at-tpr} share the same
|
|
inner structure: each evaluates one empirical CDF at the $(1-\alpha)$-quantile of the other.
|
|
The two objectives are interchangeable at the implementation level: replacing
|
|
$f_\theta(\mathbf{x})$ with $-f_\theta(\mathbf{x})$ and $y$ with $1 - y$ swaps the roles
|
|
of the two populations, and passing target rate $1-\alpha$ to the resulting
|
|
TPR@FPR objective recovers the FPR@TPR objective at rate $\alpha$. All derivations
|
|
and implementation details below are therefore given for \eqref{eq:roll-tpr-at-fpr} only;
|
|
\eqref{eq:roll-fpr-at-tpr} follows by the same transformation applied to the inputs, with
|
|
the target rate complemented to $1-\alpha$.\footnote{This negation is valid whenever the
|
|
fitted distribution family is closed under score negation, as is the case for the Gaussian
|
|
and symmetric-KDE instantiations used in this thesis. It is not valid for Beta ROLL
|
|
(\Cref{appendix:beta-roll}), whose scores are constrained to $(0,1)$: negating a score
|
|
leaves this support entirely. There, the roles of the two populations are instead swapped
|
|
by replacing $f_\theta(\mathbf{x})$ with $1-f_\theta(\mathbf{x})$, which stays within
|
|
$(0,1)$ and is already computed in the Beta forward pass as the complementary score
|
|
$\mathbf{t}^{(k)} = \mathbf{1}-\mathbf{s}^{(k)}$. This substitution is exact for the same
|
|
reason the negation is exact for reflection-equivariant families: if $X\sim\operatorname{Beta}(a,b)$
|
|
then $1-X\sim\operatorname{Beta}(b,a)$, and the log-moment estimator \eqref{eq:beta-params}
|
|
respects this swap exactly, since replacing $\mathbf{s}^{(k)}$ with $\mathbf{t}^{(k)}$ in
|
|
\eqref{eq:beta-denom}--\eqref{eq:beta-params} leaves $D_k$ unchanged and exchanges
|
|
$\hat{a}_k \leftrightarrow \hat{b}_k$.}
|
|
\Cref{fig:roll-principle} provides a geometric view of \eqref{eq:roll-tpr-at-fpr}.
|
|
|
|
\begin{figure}[H]
|
|
\centering
|
|
\includegraphics[width=0.85\textwidth]{content/method/figures/roll_principle.png}
|
|
\caption{Geometric picture of the ROLL TPR@FPR objective. \textit{Upper}: score PDFs
|
|
for the negative (blue, $y=0$) and positive (orange, $y=1$) classes. The operating
|
|
threshold $\tau$ (dashed) is placed at the $(1-\alpha)$-quantile of the negative
|
|
class, so that exactly $\alpha$ fraction of negative scores exceed it ($\operatorname{FPR}=\alpha$).
|
|
The shaded area under the positive-class PDF to the left of $\tau$ is
|
|
$\hat{F}_1(\tau)$, the ROLL loss we minimize. Tick marks along the bottom represent
|
|
observed positive-class scores. \textit{Lower}: corresponding CDFs $\hat{F}_0$ (blue)
|
|
and $\hat{F}_1$ (orange). Reading horizontally at height $1-\alpha$ on $\hat{F}_0$
|
|
gives $\tau$; reading $\hat{F}_1(\tau)$ off $\hat{F}_1$ at the same $\tau$ gives the
|
|
loss; the distance from $\hat{F}_1(\tau)$ to $1$ is the resulting TPR.}
|
|
\label{fig:roll-principle}
|
|
\end{figure}
|
|
|
|
We now derive the gradient of \eqref{eq:roll-tpr-at-fpr} with respect to $f_\theta(\mathbf{x}_i)$.
|
|
|
|
If $y_i = 1$, then $(\mathbf{x}_i, y_i) \in \mathcal{B}_1$ and $f_\theta(\mathbf{x}_i)$ has no
|
|
effect on $\hat{F}_0^{-1}(1-\alpha)$. Therefore:
|
|
\begin{equation}
|
|
\left.\frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}(f_\theta(\mathcal{B}) ; \alpha)}
|
|
{\partial f_\theta(\mathbf{x}_i)}\right|_{y_i = 1}
|
|
= \frac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial f_\theta(\mathbf{x}_i)}
|
|
\end{equation}
|
|
|
|
If $y_i = 0$, then $(\mathbf{x}_i, y_i) \in \mathcal{B}_0$ and $f_\theta(\mathbf{x}_i)$ affects
|
|
$\hat{F}_0^{-1}(1-\alpha)$ instead. Applying the chain rule:
|
|
\begin{equation}
|
|
\left.\frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}(f_\theta(\mathbf{X}) ; \alpha)}
|
|
{\partial f_\theta(\mathbf{x}_i)}\right|_{y_i = 0}
|
|
= \frac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial \hat{F}_0^{-1}(1-\alpha)}
|
|
\cdot \frac{\partial \hat{F}_0^{-1}(1-\alpha)}{\partial f_\theta(\mathbf{x}_i)}
|
|
\end{equation}
|
|
|
|
Unifying both cases, the general gradient applicable to all ROLL instantiations is:
|
|
\begin{equation}
|
|
\frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}(f_\theta(\mathbf{X}) ; \alpha)}
|
|
{\partial f_\theta(\mathbf{x}_i)} =
|
|
\begin{cases}
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial f_\theta(\mathbf{x}_i)}
|
|
& \text{if } y_i = 1 \\[10pt]
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))}{\partial \hat{F}_0^{-1}(1-\alpha)}
|
|
\cdot \dfrac{\partial \hat{F}_0^{-1}(1-\alpha)}{\partial f_\theta(\mathbf{x}_i)}
|
|
& \text{if } y_i = 0
|
|
\end{cases}
|
|
\label{eq:roll-gradient}
|
|
\end{equation}
|
|
The concrete form of each partial derivative depends on the choice of distribution family
|
|
$\mathcal{F}$; the KDE case is derived in \Cref{sec:roll-instantiations}, and the
|
|
parametric cases are in \Cref{appendix:gauss-grad,appendix:beta-roll}.
|
|
|
|
\paragraph{Regularity assumptions.} The derivation above assumes
|
|
$\mathcal{B}_0, \mathcal{B}_1 \neq \emptyset$, both classes are represented in the
|
|
batch. This is not a peculiarity of ROLL: any ROC-based objective is undefined without
|
|
both classes present, since $\operatorname{TPR}$ and $\operatorname{FPR}$ are each
|
|
conditional on a single class. We enforce this operationally, via a batch sampler that
|
|
guarantees both classes appear in every batch, rather than inside the loss itself; a
|
|
batch missing a class causes an explicit error rather than a silently degraded gradient.
|
|
|
|
\subsection{ROLL-AOC: AUC Optimization}
|
|
\label{sec:roll-aoc}
|
|
|
|
The primary objective of ROLL is to maximize AUC directly, using the per-threshold
|
|
loss \eqref{eq:roll-tpr-at-fpr} as a building block. Recall from
|
|
\Cref{sec:problem-formulation} that AUC is the integral of $\operatorname{TPR}(\operatorname{FPR}^{-1}(\alpha))$
|
|
over all FPR values:
|
|
\begin{equation}
|
|
\operatorname{AUC} = \int_{0}^{1}\operatorname{TPR}(\operatorname{FPR}^{-1}(\alpha))\,d\alpha.
|
|
\end{equation}
|
|
Maximizing AUC is equivalent to minimizing the \emph{Area Over the Curve} (AOC),
|
|
$1 - \operatorname{AUC}$. Since $1 - \operatorname{TPR}(\operatorname{FPR}^{-1}(\alpha)) =
|
|
\mathcal{L}_{\text{ROLL-TPR@FPR}}(\alpha)$ by definition \eqref{eq:roll-tpr-at-fpr}:
|
|
\begin{equation}
|
|
\mathcal{L}_{\text{AOC}} = \int_0^1\mathcal{L}_{\text{ROLL-TPR@FPR}}(\alpha)\,d\alpha.
|
|
\end{equation}
|
|
Direct integration is not tractable in general. An arbitrary fitted distribution admits
|
|
no closed form, but the integral is well-approximated by averaging sampled points. (The
|
|
Gaussian instantiation is an exception: it admits the exact closed form
|
|
$\mathcal{L}_{\text{AOC}}^{\text{Gaussian}} = \Phi\!\left(\frac{\mu_0-\mu_1}{\sqrt{\sigma_0^2+\sigma_1^2}}\right)$,
|
|
\Cref{appendix:gauss-grad}. We use the same 99-point quadrature uniformly across all
|
|
instantiations in this thesis for implementation consistency, not because it is
|
|
necessary for every distribution family.) Evaluating at 99 evenly-spaced FPR values
|
|
$\alpha = i/100$ for $i = 1, \ldots, 99$
|
|
(excluding the degenerate endpoints 0 and 1), the ROLL-AOC loss is:
|
|
\begin{equation}
|
|
\mathcal{L}_{\text{ROLL-AOC}} = \frac{1}{100}\sum_{i=1}^{99}\mathcal{L}_{\text{ROLL-TPR@FPR}}\!\left(\frac{i}{100}\right)
|
|
\label{eq:roll-aoc}
|
|
\end{equation}
|
|
Since $\operatorname{TPR}(\operatorname{FPR}^{-1}(\alpha))$ is monotone in $\alpha$ along the ROC curve,
|
|
$g(\alpha) := 1-\operatorname{TPR}(\operatorname{FPR}^{-1}(\alpha))$ is monotone and bounded in $[0,1]$.
|
|
Moreover $g(1) = 1-\operatorname{TPR}(\operatorname{FPR}^{-1}(1)) = 1-\operatorname{TPR}(-\infty) = 0$
|
|
exactly, since thresholding at $-\infty$ classifies every sample positive. This means
|
|
\eqref{eq:roll-aoc} is not merely a 99-point average standing in for the integral. It is
|
|
the exact right-Riemann sum of $\mathcal{L}_{\text{AOC}} = \int_0^1 g(\alpha)\,d\alpha$ over
|
|
$100$ equal-width subintervals of $[0,1]$, with the $100$th term, $g(1)$, known to vanish
|
|
rather than omitted. For a monotone function, the right-Riemann-sum error over $N$ equal
|
|
subintervals is bounded by the function's total range times the step size; here that gives
|
|
\[
|
|
\bigl|\mathcal{L}_{\text{ROLL-AOC}} - \mathcal{L}_{\text{AOC}}\bigr|
|
|
\;\le\; \bigl(g(0)-g(1)\bigr)\cdot\frac{1}{100}
|
|
\;\le\; \frac{1}{100} = 0.01,
|
|
\]
|
|
using $g(0)\le 1$ and $g(1)=0$. This bound covers only the quadrature step: it does not
|
|
extend to the gradient \eqref{eq:roll-aoc-gradient}, nor does it account for the gap
|
|
between the KDE-fitted rates and the raw empirical scores (\Cref{sec:roll-instantiations})
|
|
or any residual numerical error in the per-threshold bisection for $\tau_k$.
|
|
|
|
The gradient follows by linearity of the sum:
|
|
\begin{equation}
|
|
\frac{\partial \mathcal{L}_{\text{ROLL-AOC}}(f_\theta(\mathbf{X}))}
|
|
{\partial f_\theta(\mathbf{x}_i)} = \frac{1}{100}\sum_{r=1}^{99}\frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}\!\left(f_\theta(\mathbf{X}) ; \frac{r}{100}\right)}
|
|
{\partial f_\theta(\mathbf{x}_i)}
|
|
\label{eq:roll-aoc-gradient}
|
|
\end{equation}
|
|
(the summation index $r$ ranges over the $99$ thresholds; $i$, fixed throughout the sum,
|
|
indexes the sample being differentiated. Distinct roles that \eqref{eq:roll-aoc} does
|
|
not need to distinguish, since it has no outer per-sample index to collide with.)
|
|
The concrete per-threshold gradient for the KDE instantiation is derived in
|
|
\Cref{sec:roll-instantiations}; gradients for the parametric cases are in
|
|
\Cref{appendix:gauss-grad,appendix:beta-roll}.
|
|
|
|
\Cref{fig:roll-aoc-roc} gives a geometric view of this objective.
|
|
|
|
\begin{figure}[H]
|
|
\centering
|
|
\includegraphics[width=0.62\textwidth]{content/method/figures/roll_aoc_roc.pdf}
|
|
\caption{Geometric interpretation of ROLL-AOC. The ROC curve (orange) maps
|
|
each FPR value $\alpha$ to $\operatorname{TPR}(\operatorname{FPR}^{-1}(\alpha))$.
|
|
\textbf{Green}: area under the curve (AUC), the quantity we maximize.
|
|
\textbf{Red}: area over the curve
|
|
($\mathrm{AOC} = 1 - \mathrm{AUC} = \mathcal{L}_{\text{ROLL-AOC}}$),
|
|
the loss we minimize.
|
|
Dots mark 10 evenly-spaced FPR sample points $\alpha_i$; vertical
|
|
segments show the per-threshold loss contribution
|
|
$1 - \operatorname{TPR}(\alpha_i) = \mathcal{L}_{\text{ROLL-TPR@FPR}}(\alpha_i)$
|
|
at each point. Averaging these contributions (99 in practice, 10 shown
|
|
for clarity) approximates the continuous integral, giving the value of
|
|
\eqref{eq:roll-aoc}.}
|
|
\label{fig:roll-aoc-roc}
|
|
\end{figure}
|
|
|
|
|
|
%------------------------------------------------
|
|
|
|
\section{Distribution Instantiations}
|
|
\label{sec:roll-instantiations}
|
|
|
|
The abstract ROLL framework (\Cref{sec:roll-formulation}) accommodates any distribution
|
|
family $\mathcal{F}$ whose CDF and quantile function are differentiable with respect to
|
|
the model scores. As part of this work, we instantiated the framework for two parametric
|
|
families: a Gaussian model, fitted by maximum likelihood, and a Beta model, fitted by a
|
|
closed-form log-moment estimator~\cite{chen2022beta}. Both admit fast parameter estimation
|
|
and differentiable CDFs; for the Gaussian case we derive analytic gradients in full, and
|
|
for the Beta case gradients propagate via automatic differentiation. Complete derivations
|
|
are given in \Cref{appendix:gauss-grad,appendix:beta-roll}.
|
|
|
|
Parametric families, however, impose distributional assumptions on the score output that
|
|
no trained model can be guaranteed to satisfy. The primary instantiation in this thesis
|
|
is therefore non-parametric: Kernel Density Estimation (KDE) places no assumption on the
|
|
shape of the score distribution and adapts to whatever the model produces at each training
|
|
step. The cost is a numerical CDF inversion and a custom backward pass. The gain is a
|
|
surrogate that adapts to the score distribution's shape as it evolves during training,
|
|
without committing to a parametric assumption that may not hold.
|
|
|
|
\subsection{KDE ROLL}
|
|
\label{sec:roll-kde}
|
|
|
|
KDE estimates the score distribution directly from the observed batch scores, placing no
|
|
assumption on their shape. It is the primary instantiation of ROLL used throughout this
|
|
thesis: its non-parametric flexibility makes it the most adaptable surrogate for the ROC
|
|
curve when score distributions are unknown, at the cost of a numerical CDF inversion and
|
|
a custom backward pass evaluated 99 times per training step under ROLL-AOC.
|
|
|
|
\subsubsection{Forward}
|
|
\label{sec:roll-kde-forward}
|
|
|
|
% Instantiate \eqref{eq:roll-tpr-at-fpr} under KDE. The CDF becomes a sum of
|
|
% kernel CDFs; the ICDF is computed numerically. Derive the loss expression
|
|
% explicitly and discuss computational cost relative to the parametric cases.
|
|
|
|
As before, to calculate the loss, we calculate:
|
|
|
|
|
|
\[
|
|
\mathcal{L}_{\text{ROLL-TPR@FPR}}^{\text{KDE}}(f_{\theta}(\mathcal{B}) ; \alpha) = \hat{F}_1(\hat{F}_0^{-1}(1-\alpha))
|
|
\]
|
|
|
|
To calculate this, we first define the kernel function as $\sigma '$, and the CDF of the kernel function as $\sigma$. The approach used here is to define $\sigma$ as a sigmoid function.
|
|
|
|
\begin{equation}
|
|
\sigma(x ; v) = \left( 1 + \exp(-vx)\right)^{-1} \qquad \sigma'(x ; v) = \frac{v \cdot \exp(-v \cdot x)}{(1 + \exp(-vx))^2}
|
|
\end{equation}
|
|
|
|
Where $v$ is the bandwidth parameter. This affects the calculation. We calculate $v_0$ as the bandwidth for the false population and $v_1$ for the true population. Bandwidth calculation
|
|
will be discussed later.
|
|
|
|
We then denote $\sigma_1(x) = \sigma(x ; v_1), \sigma_0(x) = \sigma(x ; v_0)$ to differentiate between the kernel functions for the true and false population, which only differentiate by bandwidth.
|
|
|
|
The CDF of the KDE function then becomes:
|
|
|
|
\[
|
|
\hat{F}_{KDE}(\tau ; \mathbf{X}) = \frac{1}{|X|}\sum_i \sigma(\tau - x_i)
|
|
\]
|
|
|
|
While the CDF has a closed form, its inverse has none and must be found numerically.
|
|
We use bisection, which requires no derivative evaluation and is guaranteed to converge
|
|
for any strictly monotone CDF; the bracket construction and iteration are detailed in
|
|
Appendix~\ref{appendix:kde-bisection}.
|
|
|
|
We can now trivially compute $\hat{F}_1(\tau ; \mathbf{X}_1)$ using the formula for $\hat{F}_{KDE}(\tau ; \mathbf{X})$ above.
|
|
|
|
Note that $\tau$ is cached for use in the backward pass, along with the computed bandwidths $v_1, v_0$.
|
|
|
|
|
|
\subsubsection{Gradient Derivation}
|
|
\label{sec:roll-kde-backward}
|
|
|
|
% The primary technical contribution. Standard autograd cannot differentiate
|
|
% through the numerical ICDF inversion; describe the custom backward pass.
|
|
% Gradients flow through the kernel evaluations back to the model scores.
|
|
% This is the section that connects to \Cref{sec:roll-backward} motivation.
|
|
|
|
The derivation is given in Appendix~\ref{appendix:kde-grad}; letting $\tau = \hat{F}_0^{-1}(1-\alpha ; \mathcal{B}_0)$, the gradient is:
|
|
|
|
\begin{equation}
|
|
\frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}}{\partial f_\theta(\mathbf{x}_i)} =
|
|
\begin{cases}
|
|
-\dfrac{1}{|\mathcal{B}_1|}\,\sigma_1'(\tau - f_{\theta}(\mathbf{x}_i))
|
|
& \text{if } y_i = 1 \\[14pt]
|
|
+\dfrac{1}{|\mathcal{B}_1|}\!\displaystyle\sum_{\mathbf{x}_j \in \mathcal{B}_1}\!\sigma_1'(\tau - f_{\theta}(\mathbf{x}_j))
|
|
\;\cdot\;
|
|
\dfrac{\sigma_0'(\tau - f_{\theta}(\mathbf{x}_i))}{\displaystyle\sum_{\mathbf{x}_j \in \mathcal{B}_0}\sigma_0'(\tau - f_{\theta}(\mathbf{x}_j))}
|
|
& \text{if } y_i = 0
|
|
\end{cases}
|
|
\label{eq:kde-grad-combined}
|
|
\end{equation}
|
|
|
|
where $\tau = \hat{F}_0^{-1}(1-\alpha ; \mathcal{B}_0)$.
|
|
|
|
The explicit sigmoid-kernel form of \eqref{eq:kde-grad-combined}, forward pass CDF, efficient gradient, and implementation notes, is given in Appendix~\ref{appendix:kde-sigmoid}.
|
|
Positive-class samples are weighted by their individual kernel
|
|
PDF at $\tau$; negative-class samples are weighted by the mean positive-class kernel
|
|
PDF scaled by each sample's share of the negative-class kernel mass.
|
|
|
|
Equation~\eqref{eq:kde-grad-combined} is derived by treating the bandwidths $v_0, v_1$
|
|
as fixed constants, even though \Cref{sec:kde-bandwidth} re-estimates them from the
|
|
batch scores on every forward pass. We adopt this as a deliberate stop-gradient
|
|
convention: as \Cref{obs:gradient-sign-correctness} shows, it is exactly what
|
|
guarantees every per-sample gradient points in the direction that improves separation.
|
|
Differentiating fully through the bandwidth estimator can break that guarantee, as
|
|
Appendix~\ref{appendix:bandwidth-freeze} shows by counterexample.
|
|
|
|
%------------------------------------------------
|
|
|
|
\subsection{Properties of the ROLL Formulation}
|
|
\label{sec:roll-properties}
|
|
|
|
% State and prove (or sketch) key properties: consistency of the estimator as
|
|
% the number of samples grows, monotonicity of the loss with respect to the
|
|
% TPR/FPR trade-off, and behaviour at the constraint boundary. Discuss what
|
|
% guarantees ROLL provides and under what assumptions they hold.
|
|
% NOTE - these properties are auto-generated, consider suggestions.
|
|
|
|
A structural consequence of the ROLL formulation is that the total signed
|
|
gradient contribution from each class cancels exactly, regardless of their relative
|
|
sizes in the batch. This stands in contrast to standard losses such as cross-entropy,
|
|
where the total gradient contribution from each class is proportional to its count,
|
|
making heavy class imbalance a direct source of biased updates.
|
|
|
|
\begin{proposition}[Gradient balance]
|
|
\label{prop:gradient-balance}
|
|
Let $\mathcal{F}$ be any distribution family whose CDF estimator is translation-invariant,
|
|
meaning $\hat{F}_k(t\,;\,\mathbf{s}+\delta) = \hat{F}_k(t-\delta\,;\,\mathbf{s})$
|
|
for all $\delta \in \mathbb{R}$. Then for any batch $\mathcal{B} = \mathcal{B}_0 \cup \mathcal{B}_1$:
|
|
\begin{equation}
|
|
\sum_{\mathbf{x}_i \in \mathcal{B}_1}
|
|
\frac{\partial \mathcal{L}_{\emph{\text{ROLL-TPR@FPR}}}}{\partial f_\theta(\mathbf{x}_i)}
|
|
\;=\;
|
|
-\!\sum_{\mathbf{x}_i \in \mathcal{B}_0}
|
|
\frac{\partial \mathcal{L}_{\emph{\text{ROLL-TPR@FPR}}}}{\partial f_\theta(\mathbf{x}_i)}
|
|
\label{eq:grad-balance}
|
|
\end{equation}
|
|
with both sides equal in magnitude to $\dfrac{\partial \hat{F}_1(\tau)}{\partial \tau}$,
|
|
independently of $|\mathcal{B}_0|$ and $|\mathcal{B}_1|$.
|
|
\end{proposition}
|
|
|
|
\begin{proof}
|
|
See \Cref{appendix:grad-balance}.
|
|
\end{proof}
|
|
|
|
The translation-invariance assumption holds for both the Gaussian distribution,
|
|
as well as the KDE-based distribution, but not for the Beta distribution.
|
|
For Gaussian MLE, a uniform shift in scores moves $\mu_k$ by the same amount while
|
|
leaving $\sigma_k$ unchanged, which translates $\hat{F}_k$; for KDE, each kernel term is
|
|
evaluated at $\tau - f_\theta(\mathbf{x}_j)$, so a uniform shift in the $f_\theta(\mathbf{x}_j)$
|
|
is equivalent to an equal shift in $\tau$. The condition is sufficient, not necessary: it is
|
|
what the proof uses to guarantee a rigid translation of $\hat{F}_k$, but the balance identity
|
|
can hold without it. For example, $\hat{F}_k(t;\mathbf{s}) = \Phi(t - 2\bar{s})$ (with
|
|
$\bar{s}$ the class mean) does not satisfy $\hat{F}_k(t;\mathbf{s}+\delta) = \hat{F}_k(t-\delta;\mathbf{s})$
|
|
— shifting the scores by $\delta$ shifts $\hat{F}_k$ by $2\delta$, not $\delta$ — yet applying
|
|
this same construction to both classes still yields $\sum_{\mathcal{B}_1}\partial L/\partial f_i
|
|
= -\sum_{\mathcal{B}_0}\partial L/\partial f_i$ exactly, since the extra factor of $2$ enters
|
|
both classes' gradients identically and cancels in the same shift argument. Translation
|
|
invariance is a convenient sufficient condition, not the weakest one that would do.
|
|
|
|
The Beta distribution only accepts values in the interval $[0, 1]$. Since it
|
|
is mapped only over an interval, the translation-invariance assumption does not hold
|
|
for this distribution.
|
|
|
|
\begin{corollary}[Gradient balance under ROLL-AOC]
|
|
\label{cor:grad-balance-aoc}
|
|
$\mathcal{L}_{\emph{\text{ROLL-AOC}}}$ also satisfies gradient balance: since it is a uniform
|
|
average of 99 per-threshold objectives each satisfying \eqref{eq:grad-balance}, the property
|
|
is preserved by linearity of summation.
|
|
\end{corollary}
|
|
|
|
The identity \eqref{eq:grad-balance} constrains only the signed sum per class. It
|
|
does not by itself say that each class contributes equal gradient mass,
|
|
$\sum_i |\partial\mathcal{L}/\partial f_\theta(\mathbf{x}_i)|$. Equal gradient-mass is satisfied
|
|
when every sample within a class shares the same gradient sign, so that the signed and
|
|
absolute sums are equal in magnitude. For KDE-ROLL,
|
|
this extra property does hold (\Cref{obs:gradient-sign-correctness} below), so the
|
|
signed identity does translate into balanced gradient mass for the method used
|
|
throughout this thesis. However, notably, this does not hold true for the Gaussian formulation.
|
|
|
|
Additionally, \Cref{prop:gradient-balance} is a statement about the per-sample loss
|
|
gradients $\partial\mathcal{L}/\partial f_\theta(\mathbf{x}_i)$ at the model's output. It
|
|
does not extend to the resulting parameter gradients $\partial\mathcal{L}/\partial\theta$,
|
|
as this depends on the model's structure.
|
|
\paragraph{Gradient sign correctness.}
|
|
\label{para:gradient-sign-correctness}
|
|
|
|
A third structural consequence of \eqref{eq:kde-grad-combined} is that every per-sample
|
|
gradient points in the direction that improves separation, never against it.
|
|
|
|
\begin{observation}[Gradient sign correctness]
|
|
\label{obs:gradient-sign-correctness}
|
|
For every $\mathbf{x}_i \in \mathcal{B}$,
|
|
\[
|
|
y_i = 1 \;\Longrightarrow\; \frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}}{\partial f_\theta(\mathbf{x}_i)} \leq 0,
|
|
\qquad
|
|
y_i = 0 \;\Longrightarrow\; \frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}}{\partial f_\theta(\mathbf{x}_i)} \geq 0.
|
|
\]
|
|
\end{observation}
|
|
|
|
This follows directly from \eqref{eq:kde-grad-combined}: for $y_i = 1$ the gradient is
|
|
$-\sigma_1'(\tau-f_\theta(\mathbf{x}_i))/|\mathcal{B}_1|$, and $\sigma_1' \geq 0$
|
|
everywhere; for $y_i = 0$ the gradient is a product of two nonnegative quantities (a
|
|
mean of $\sigma_1'$ values and a normalized $\sigma_0'$ ratio). Gradient descent
|
|
therefore never pushes a positive-class score down, nor a negative-class score up, a
|
|
minimal requirement for any loss intended to drive separation.
|
|
|
|
This guarantee is specific to the frozen-bandwidth gradient~\eqref{eq:kde-grad-combined}.
|
|
Differentiating fully through the adaptive bandwidth of \Cref{sec:kde-bandwidth} can
|
|
break it: Appendix~\ref{appendix:bandwidth-freeze} gives an exact counterexample where
|
|
the fully adaptive gradient violates this sign guarantee.
|
|
|
|
\paragraph{Gradient locality.}
|
|
\label{para:gradient-locality}
|
|
|
|
A second structural consequence of the ROLL-KDE formulation, when optimizing at a
|
|
single operating threshold, is that gradient magnitude concentrates near $\tau$, but
|
|
the mechanism differs between the two classes, and so does how literally the claim
|
|
holds.
|
|
|
|
This follows from \Cref{eq:kde-grad-combined}. For the positive class ($y_i = 1$), the
|
|
gradient magnitude is an absolute, unnormalised function of the sample's own
|
|
distance to $\tau$:
|
|
\[
|
|
\left|\frac{\partial \mathcal{L}}{\partial f_\theta(\mathbf{x}_i)}\right|
|
|
= \frac{1}{|\mathcal{B}_1|}\,\sigma_1'(f_\theta(\mathbf{x}_i) - \tau),
|
|
\]
|
|
a bell-shaped kernel PDF that peaks when $f_\theta(\mathbf{x}_i) = \tau$ and decays to
|
|
exactly $0$ as $|f_\theta(\mathbf{x}_i)-\tau|\to\infty$, regardless of where any other
|
|
sample lies. For this class, "far from $\tau$ implies negligible gradient" holds without
|
|
qualification.
|
|
|
|
For the negative class ($y_i = 0$), the gradient is instead a \emph{normalised share} of
|
|
the class's total kernel mass, $\sigma_0'(\tau-f_\theta(\mathbf{x}_i)) \big/
|
|
\sum_j \sigma_0'(\tau-f_\theta(\mathbf{x}_j))$, and this ratio doesn't vanish just
|
|
because $f_\theta(\mathbf{x}_i)$ is far from $\tau$: what determines
|
|
sample $i$'s share of the gradient mass is its position relative to the rest of $\mathcal{B}_0$,
|
|
not its absolute distance to $\tau$. If the whole negative class drifts far from $\tau$
|
|
together, every individual $\sigma_0'$ value shrinks, but so does their sum, and the
|
|
ratio can converge to a nonzero constant no matter how far the class has drifted.
|
|
Concretely, for two negative samples at $\tau+M$ and $\tau+M+1$ (a fixed unit
|
|
separation), the closer sample's normalised weight converges, as $M\to\infty$, to
|
|
$1/(1+e^{-v_0})$, $\approx 0.731$ at $v_0=1$, not to $0$.
|
|
However, if a single score drifts far from $\tau$ while the rest of it's class scores,
|
|
stay close, its individual score will converge to zero.
|
|
|
|
The effect, for both classes, is that each gradient step focuses the model's attention
|
|
on the samples whose re-ranking relative to $\tau$ would most change the operating-point
|
|
performance. For the positive class this is an absolute statement about each sample's
|
|
own distance to $\tau$, and for the negative class, it is a statement about each sample's
|
|
distance relative to its classmates. Typically, but not necessarily, this correlates
|
|
with its absolute distance to $\tau$.
|
|
|
|
This stands in contrast to standard losses such as cross-entropy, where the per-sample
|
|
gradient is determined by the predicted probability, with no direct dependence on an
|
|
operating threshold.
|
|
|
|
\begin{observation}[Gradient locality, single-threshold]
|
|
\label{obs:gradient-locality}
|
|
For KDE-ROLL at a single operating threshold: (a) a positive-class sample's gradient
|
|
magnitude is the kernel PDF evaluated at its own distance to $\tau$, and vanishes as
|
|
that distance grows, independent of the rest of the batch. (b) a negative-class
|
|
sample's gradient magnitude is its kernel-PDF value \emph{relative to the sum over
|
|
$\mathcal{B}_0$}. Samples near $\tau$ receive a larger share than those far from it
|
|
\emph{within the same batch}, but this relative weighting does not by itself vanish
|
|
for an individual sample merely because the whole class is far from $\tau$.
|
|
\end{observation}
|
|
|
|
\Cref{fig:threshold-locality} illustrates this for a simulated two-class setting in
|
|
which both classes are broadly spread around $\tau$: the typical case, and the one in
|
|
which the negative class's relative weighting (\Cref{obs:gradient-locality}b) and the
|
|
positive class's absolute decay (\Cref{obs:gradient-locality}a) happen to agree. The
|
|
upper panel shows the score distributions and the operating threshold; the lower panel
|
|
shows the gradient magnitude of each sample, with a clear bell-shaped concentration
|
|
around $\tau$ for both classes in this regime.
|
|
|
|
\begin{figure}[H]
|
|
\centering
|
|
\includegraphics[width=0.82\textwidth]{content/method/figures/threshold_weighting.pdf}
|
|
\caption{Gradient locality in KDE-ROLL, for a batch where both classes are spread
|
|
broadly around $\tau$. \textit{Upper}: score distributions for the negative (blue,
|
|
$y=0$) and positive (orange, $y=1$) classes; dashed line marks the operating
|
|
threshold $\tau$ at FPR $= \alpha$. \textit{Lower}: gradient magnitude
|
|
$|\partial\mathcal{L}/\partial f_\theta(\mathbf{x})|$ for each sample. Both classes
|
|
show a bell-shaped concentration of gradient mass near $\tau$ here; for the negative
|
|
class this reflects relative weighting within $\mathcal{B}_0$
|
|
(\Cref{obs:gradient-locality}b), which need not track absolute distance to $\tau$ if
|
|
the whole class is displaced together. Smooth curves are the kernel PDFs scaled for
|
|
reference.}
|
|
\label{fig:threshold-locality}
|
|
\end{figure}
|
|
|
|
\begin{remark}[Gradient coverage under ROLL-AOC]
|
|
\label{rem:aoc-coverage}
|
|
\Cref{obs:gradient-locality} describes gradient concentration at a single threshold.
|
|
Under ROLL-AOC, 99 distinct thresholds spanning $\operatorname{FPR} \in \{0.01,\ldots,0.99\}$
|
|
are active simultaneously, so the gradient is a superposition of 99 bell-shaped kernels
|
|
spread across the full score range. This broadens gradient coverage relative to
|
|
single-threshold optimization, making ROLL-AOC more suitable when no specific operating
|
|
point is fixed in advance.
|
|
\end{remark}
|
|
|
|
%------------------------------------------------
|
|
|
|
\section{Implementation Considerations}
|
|
\label{sec:roll-implementation}
|
|
|
|
\subsection{Combatting Numerical Instability With Score Scaling}
|
|
\label{sec:roll-numerical-stability}
|
|
|
|
Before computing the KDE, all batch scores are divided by their overall standard
|
|
deviation:
|
|
\[
|
|
\tilde{s}_i = \frac{f_\theta(\mathbf{x}_i)}{\hat\sigma_{\mathcal{B}}},
|
|
\qquad
|
|
\hat\sigma_{\mathcal{B}} = \sqrt{\operatorname{Var}(f_\theta(\mathbf{x}_i) : (\mathbf{x}_i, y_i) \in \mathcal{B})}.
|
|
\]
|
|
This keeps the values given to the sigmoid kernel at a consistent scale. Why that
|
|
matters depends entirely on which bandwidth estimator is in play.
|
|
|
|
\begin{proposition}[Normalisation invariance under scale-equivariant bandwidth]
|
|
\label{prop:normalization-invariance}
|
|
Let $\hat{h}$ be any bandwidth estimator satisfying $\hat{h}(a\cdot x) = a\cdot\hat{h}(x)$
|
|
for all $a>0$ (e.g.\ Silverman's rule, $h \propto \hat\sigma \cdot n^{-1/5}$). Under the
|
|
frozen-bandwidth convention of \Cref{sec:roll-kde-backward} and a frozen normalisation
|
|
constant $\hat\sigma_\mathcal{B}$, the loss value and every per-sample gradient
|
|
$\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}/\partial f_\theta(\mathbf{x}_i)$ computed from
|
|
the normalised scores $\tilde{s}$ are exactly equal to those computed directly from the
|
|
raw scores $s$.
|
|
\end{proposition}
|
|
\begin{proof}
|
|
See Appendix~\ref{appendix:score-normalization}.
|
|
\end{proof}
|
|
|
|
For Silverman's rule, normalisation is therefore a pure floating-point reparametrisation:
|
|
it changes neither the loss nor the gradient used for training, so it cannot be what
|
|
prevents overflow for that estimator. There is nothing left for it to change.
|
|
ISJ, the primary bandwidth estimator (\Cref{sec:kde-bandwidth}), does \emph{not} satisfy
|
|
the proposition's hypothesis (Appendix~\ref{appendix:score-normalization} measures its
|
|
scale-dependence directly). For ISJ, both the loss value and the gradient genuinely
|
|
depend on whether the scores are normalised, so normalisation is doing real work: it
|
|
pins ISJ's input to the one scale regime it behaves consistently in, rather than leaving
|
|
it to drift with the model's raw output scale over training.
|
|
|
|
As with the bandwidths $v_0,v_1$, the normalisation constant $\hat\sigma_\mathcal{B}$ is
|
|
treated as frozen (stop-gradient) in the backward pass: gradients with respect to the
|
|
normalised scores are simply multiplied by $1/\hat\sigma_\mathcal{B}$. Differentiating
|
|
fully through $\hat\sigma_\mathcal{B}$ instead introduces a further correction term that,
|
|
like the bandwidth correction of \Cref{sec:roll-kde-backward}, can violate
|
|
\Cref{obs:gradient-sign-correctness} (Appendix~\ref{appendix:score-normalization}).
|
|
|
|
\subsection{Bandwidth Estimation for KDE}
|
|
\label{sec:kde-bandwidth}
|
|
|
|
% The KDE bandwidth $h$ controls how smoothly the score distribution is estimated:
|
|
% too small and the loss becomes noisy; too large and the gradient signal is
|
|
% washed out. Describe the bandwidth estimation strategy used in ROLL (e.g.
|
|
% Silverman's rule, a learned or scheduled $h$), justify the choice, and discuss
|
|
% sensitivity of results to this hyperparameter.
|
|
% NOTE - not silverman's rule, we do use scheduling, but we also use different
|
|
% estimator - see impl.
|
|
|
|
When computing KDE, bandwidth estimation strongly affects the result. Large bandwidths
|
|
produce an overly smooth CDF that fails to track the true score distribution; small
|
|
bandwidths produce a jagged CDF, yielding negligible gradients for all but the
|
|
nearest-threshold samples.
|
|
|
|
We estimate the bandwidth using the Improved Sheather-Jones (ISJ)
|
|
method~\cite{botev2010kde}, which minimises the asymptotic mean integrated squared
|
|
error via an iterative plug-in procedure. ISJ tracks the true score distribution more
|
|
closely than closed-form rules. ISJ's optimality guarantee is derived for a Gaussian
|
|
kernel, whereas the kernel used throughout this work is logistic (\Cref{sec:roll-kde});
|
|
applying it here is a plug-in heuristic approximation, chosen for its ready
|
|
availability in the existing implementation rather than a proven-optimal fit to the
|
|
logistic kernel specifically. Kernel-appropriate bandwidth selection is left to future
|
|
work (\Cref{sec:conclusions:future}).
|
|
Because ISJ is iterative, it can fail to converge on small or degenerate batches early
|
|
in training. In those cases we fall back to Silverman's rule of
|
|
thumb~\cite{silverman1986density},
|
|
\[
|
|
h = 1.06\,\hat{\sigma}\,n^{-1/5},
|
|
\]
|
|
which is closed-form. Silverman's rule is itself undefined at $\hat\sigma=0$. i.e.\ if a
|
|
class's scores are literally constant, which does not arise with real, continuously-valued
|
|
model outputs but is not ruled out in principle. The underlying bandwidth-selection
|
|
library substitutes a fixed fallback ($h=1$) in that case rather than propagating a
|
|
division by zero.\footnote{As a further safety net, $v=1/h$ is capped at $10{,}000$
|
|
regardless of source. This last-resort cap technically breaks the exact
|
|
scale-equivariance assumed by \Cref{prop:normalization-invariance} once triggered,
|
|
though in practice. Especially after the numerical-stability improvements of
|
|
\Cref{sec:kde-grad-stability,appendix:bandwidth-freeze,appendix:score-normalization}
|
|
it is essentially never reached.} More generally, the ROLL derivation throughout assumes
|
|
each fitted density is nonzero at the quantile being inverted, so that the
|
|
inverse-function-theorem step of \Cref{sec:roll-kde-backward} is well-defined; the
|
|
safeguards above are exactly what keep this assumption from failing in the KDE
|
|
instantiation.
|
|
The kernel parameter is then set to $v = 1/h$.
|
|
|
|
\subsubsection{Bandwidth Scaling for KDE}
|
|
\label{sec:kde-bandwidth-scaling}
|
|
|
|
Empirically, training without a bandwidth scheduler stalls more often near the start of
|
|
training. A plausible explanation is that the classes are not yet well separated this
|
|
early: the model has not yet learned to distinguish them — so the fitted score
|
|
distributions overlap substantially relative to their spread, and the resulting
|
|
gradient carries little discriminative signal regardless of the estimated bandwidth's
|
|
absolute value.
|
|
|
|
The solution was to use a bandwidth scheduler. Starting with large bandwidths resulted in gradients that spread out more evenly across all scores, and fewer non-convergence runs occurred. The bandwidth scheduler artificially scales up the bandwidth at the beginning to speed up initial training, and then tapers back to restore the intended bandwidth in order to maximise results.
|
|
|