Hopefully final draft
This commit is contained in:
+99
-34
@@ -32,7 +32,14 @@ Conversely, we may denote the inverse operations $\operatorname{FPR}^{-1}$ and $
|
||||
|
||||
|
||||
For $\alpha \in (0,1)$, let $\tau = \operatorname{FPR}^{-1}(\alpha)$ denote the threshold achieving
|
||||
$\operatorname{FPR}(\tau) = \alpha$.
|
||||
$\operatorname{FPR}(\tau) = \alpha$, assuming $\operatorname{FPR}$ is continuous and strictly
|
||||
decreasing in $\tau$ so that this threshold exists and is unique.\footnote{This requires the
|
||||
negative-class score distribution to have no discontinuous steps. For an empirical distribution
|
||||
this doesn't need to hold, but every distribution family $\operatorname{ROLL}$ is instantiated with in
|
||||
this thesis (KDE with positive bandwidth (\Cref{sec:roll-kde}), Gaussian
|
||||
(\Cref{appendix:gauss-grad}), and Beta (\Cref{appendix:beta-roll})) has a continuous,
|
||||
strictly increasing CDF, so the statement holds for the
|
||||
fitted distributions ROLL actually optimizes over.}
|
||||
The curve $\alpha \mapsto \operatorname{TPR}(\operatorname{FPR^{-1}}(\alpha))$ is the ROC curve.
|
||||
The primary learning objective is to maximize its area:
|
||||
\begin{equation}
|
||||
@@ -66,8 +73,8 @@ no threshold selection required at inference time.
|
||||
|
||||
ROLL also supports single-threshold optimization: selecting a target
|
||||
$\operatorname{FPR} = \alpha_0$ and maximizing $\operatorname{TPR}$ at exactly that point.
|
||||
In Neyman-Pearson terms~\cite{tong2016neyman}, this corresponds to minimizing Type-I error
|
||||
at a fixed Type-II error bound.
|
||||
In Neyman-Pearson terms~\cite{tong2016neyman}, this corresponds to minimizing Type-II error
|
||||
at a fixed Type-I error bound.
|
||||
|
||||
In practice, however, making these evaluations differentiable is not straightforward.
|
||||
|
||||
@@ -104,8 +111,9 @@ TPR or FPR calculation, given a threshold, is non-differentiable.
|
||||
The reason for that is that the TPR and FPR as a function of $f_\theta(\mathbf{x}_i)$ are
|
||||
non-continuous. If $f_\theta(\mathbf{x}_i) \neq \tau$, that is, the score is not exactly on the threshold, then changing it
|
||||
by some $\varepsilon$ will not affect either TPR or FPR at all, and the derivative of the function w.r.t.\ $f_\theta(\mathbf{x}_i)$ will be zero. Likewise,
|
||||
if $f_\theta(\mathbf{x}_i) = \tau$, the derivative of TPR/FPR w.r.t.\ $f_\theta(\mathbf{x}_i)$ becomes infinite. This derivative cannot be used
|
||||
to continuously fit our model.
|
||||
if $f_\theta(\mathbf{x}_i) = \tau$, the one-sided derivatives of TPR/FPR w.r.t.\ $f_\theta(\mathbf{x}_i)$ disagree: approaching from below leaves the indicator at $0$, while approaching from above sends the difference quotient to $+\infty$. The derivative at the jump is therefore \emph{undefined}, not an ordinary derivative equal to infinity, and either way there is no usable gradient to fit our model with.
|
||||
|
||||
It is important to distinguish this finite-sample indicator objective from the population rate it estimates. The TPR and FPR terms above are indicator averages over the finite sample $\mathcal{D}$: each component of the sum, $\mathbf{1}[f_\theta(\mathbf{x}_i) > \tau]$, is a step function of that sample's own score, and it is this per-sample step that is non-differentiable. The distribution the dataset $\mathcal{D}$ draws from might itself have a smooth CDF, therefore an FPR or TPR term derived from said smooth CDF might be differentiable. However, in practice, we must always train off of a finite sample $\mathcal{D}$, and so the differentiability problem remains.
|
||||
|
||||
This necessitates a different approach, to make these calculations differentiable w.r.t.\ the scores themselves, in order to fit this type of optimization into a learning platform.
|
||||
|
||||
@@ -153,16 +161,16 @@ and extract the corresponding model score vectors:
|
||||
\mathbf{s}^{(0)} = \bigl(f_\theta(\mathbf{x}_i)\bigr)_{(\mathbf{x}_i,\, y_i)\,\in\,\mathcal{B}_0}
|
||||
\end{equation*}
|
||||
|
||||
We then fit a distribution family $\mathcal{F} = \{P_\phi : \phi \in \Phi\}$ separately to each
|
||||
We then fit a distribution family $\mathcal{F} = \{P_\psi : \psi \in \Psi\}$ separately to each
|
||||
score vector in order to obtain a differentiable model of each class's score distribution:
|
||||
%
|
||||
\begin{equation*}
|
||||
\hat{\phi}_1 = \operatorname{Fit}_{\mathcal{F}}\!\left(\mathbf{s}^{(1)}\right),
|
||||
\hat{\psi}_1 = \operatorname{Fit}_{\mathcal{F}}\!\left(\mathbf{s}^{(1)}\right),
|
||||
\qquad
|
||||
\hat{\phi}_0 = \operatorname{Fit}_{\mathcal{F}}\!\left(\mathbf{s}^{(0)}\right)
|
||||
\hat{\psi}_0 = \operatorname{Fit}_{\mathcal{F}}\!\left(\mathbf{s}^{(0)}\right)
|
||||
\end{equation*}
|
||||
%
|
||||
Let $\hat{F}_1$ and $\hat{F}_0$ denote the CDFs of $P_{\hat{\phi}_1}$ and $P_{\hat{\phi}_0}$,
|
||||
Let $\hat{F}_1$ and $\hat{F}_0$ denote the CDFs of $P_{\hat{\psi}_1}$ and $P_{\hat{\psi}_0}$,
|
||||
respectively. The KDE instantiation is developed in \Cref{sec:roll-instantiations};
|
||||
the Gaussian and Beta instantiations, including full derivations, are in
|
||||
\Cref{appendix:gauss-grad,appendix:beta-roll}.
|
||||
@@ -173,8 +181,8 @@ Using this notation, at a threshold $\tau$:
|
||||
\operatorname{FPR}(\tau) &= 1 - \hat{F}_0(\tau). \label{eq:fpr-cdf}
|
||||
\end{align}
|
||||
|
||||
Let $\hat{F}_1^{-1}$ and $\hat{F}_0^{-1}$ denote the quantile functions of $P_{\hat{\phi}_1}$ and
|
||||
$P_{\hat{\phi}_0}$, respectively. The threshold achieving a target TPR of $\alpha$ follows directly
|
||||
Let $\hat{F}_1^{-1}$ and $\hat{F}_0^{-1}$ denote the quantile functions of $P_{\hat{\psi}_1}$ and
|
||||
$P_{\hat{\psi}_0}$, respectively. The threshold achieving a target TPR of $\alpha$ follows directly
|
||||
from \eqref{eq:tpr-cdf}:
|
||||
\[
|
||||
\operatorname{TPR}(\tau) = \alpha \;\Longrightarrow\; 1 - \hat{F}_1(\tau) = \alpha
|
||||
@@ -209,7 +217,19 @@ of the two populations, and passing target rate $1-\alpha$ to the resulting
|
||||
TPR@FPR objective recovers the FPR@TPR objective at rate $\alpha$. All derivations
|
||||
and implementation details below are therefore given for \eqref{eq:roll-tpr-at-fpr} only;
|
||||
\eqref{eq:roll-fpr-at-tpr} follows by the same transformation applied to the inputs, with
|
||||
the target rate complemented to $1-\alpha$.
|
||||
the target rate complemented to $1-\alpha$.\footnote{This negation is valid whenever the
|
||||
fitted distribution family is closed under score negation, as is the case for the Gaussian
|
||||
and symmetric-KDE instantiations used in this thesis. It is not valid for Beta ROLL
|
||||
(\Cref{appendix:beta-roll}), whose scores are constrained to $(0,1)$: negating a score
|
||||
leaves this support entirely. There, the roles of the two populations are instead swapped
|
||||
by replacing $f_\theta(\mathbf{x})$ with $1-f_\theta(\mathbf{x})$, which stays within
|
||||
$(0,1)$ and is already computed in the Beta forward pass as the complementary score
|
||||
$\mathbf{t}^{(k)} = \mathbf{1}-\mathbf{s}^{(k)}$. This substitution is exact for the same
|
||||
reason the negation is exact for reflection-equivariant families: if $X\sim\operatorname{Beta}(a,b)$
|
||||
then $1-X\sim\operatorname{Beta}(b,a)$, and the log-moment estimator \eqref{eq:beta-params}
|
||||
respects this swap exactly, since replacing $\mathbf{s}^{(k)}$ with $\mathbf{t}^{(k)}$ in
|
||||
\eqref{eq:beta-denom}--\eqref{eq:beta-params} leaves $D_k$ unchanged and exchanges
|
||||
$\hat{a}_k \leftrightarrow \hat{b}_k$.}
|
||||
\Cref{fig:roll-principle} provides a geometric view of \eqref{eq:roll-tpr-at-fpr}.
|
||||
|
||||
\begin{figure}[H]
|
||||
@@ -264,33 +284,58 @@ The concrete form of each partial derivative depends on the choice of distributi
|
||||
$\mathcal{F}$; the KDE case is derived in \Cref{sec:roll-instantiations}, and the
|
||||
parametric cases are in \Cref{appendix:gauss-grad,appendix:beta-roll}.
|
||||
|
||||
\paragraph{Regularity assumptions.} The derivation above assumes
|
||||
$\mathcal{B}_0, \mathcal{B}_1 \neq \emptyset$, both classes are represented in the
|
||||
batch. This is not a peculiarity of ROLL: any ROC-based objective is undefined without
|
||||
both classes present, since $\operatorname{TPR}$ and $\operatorname{FPR}$ are each
|
||||
conditional on a single class. We enforce this operationally, via a batch sampler that
|
||||
guarantees both classes appear in every batch, rather than inside the loss itself; a
|
||||
batch missing a class causes an explicit error rather than a silently degraded gradient.
|
||||
|
||||
\subsection{ROLL-AOC: AUC Optimization}
|
||||
\label{sec:roll-aoc}
|
||||
|
||||
The primary objective of ROLL is to maximize AUC directly, using the per-threshold
|
||||
loss \eqref{eq:roll-tpr-at-fpr} as a building block. Recall from
|
||||
\Cref{sec:problem-formulation} that AUC is the integral of $\operatorname{TPR}(\alpha)$
|
||||
\Cref{sec:problem-formulation} that AUC is the integral of $\operatorname{TPR}(\operatorname{FPR}^{-1}(\alpha))$
|
||||
over all FPR values:
|
||||
\begin{equation}
|
||||
\operatorname{AUC} = \int_{0}^{1}\operatorname{TPR}(\alpha)\,d\alpha.
|
||||
\operatorname{AUC} = \int_{0}^{1}\operatorname{TPR}(\operatorname{FPR}^{-1}(\alpha))\,d\alpha.
|
||||
\end{equation}
|
||||
Maximizing AUC is equivalent to minimizing the \emph{Area Over the Curve} (AOC),
|
||||
$1 - \operatorname{AUC}$. Since $1 - \operatorname{TPR}(\alpha) =
|
||||
$1 - \operatorname{AUC}$. Since $1 - \operatorname{TPR}(\operatorname{FPR}^{-1}(\alpha)) =
|
||||
\mathcal{L}_{\text{ROLL-TPR@FPR}}(\alpha)$ by definition \eqref{eq:roll-tpr-at-fpr}:
|
||||
\begin{equation}
|
||||
\mathcal{L}_{\text{AOC}} = \int_0^1\mathcal{L}_{\text{ROLL-TPR@FPR}}(\alpha)\,d\alpha.
|
||||
\end{equation}
|
||||
Direct integration is not tractable, but the integral is well-approximated by averaging out sampled points. Evaluating at 99 evenly-spaced FPR values $\alpha = i/100$ for $i = 1, \ldots, 99$
|
||||
Direct integration is not tractable in general. An arbitrary fitted distribution admits
|
||||
no closed form, but the integral is well-approximated by averaging sampled points. (The
|
||||
Gaussian instantiation is an exception: it admits the exact closed form
|
||||
$\mathcal{L}_{\text{AOC}}^{\text{Gaussian}} = \Phi\!\left(\frac{\mu_0-\mu_1}{\sqrt{\sigma_0^2+\sigma_1^2}}\right)$,
|
||||
\Cref{appendix:gauss-grad}. We use the same 99-point quadrature uniformly across all
|
||||
instantiations in this thesis for implementation consistency, not because it is
|
||||
necessary for every distribution family.) Evaluating at 99 evenly-spaced FPR values
|
||||
$\alpha = i/100$ for $i = 1, \ldots, 99$
|
||||
(excluding the degenerate endpoints 0 and 1), the ROLL-AOC loss is:
|
||||
\begin{equation}
|
||||
\mathcal{L}_{\text{ROLL-AOC}} = \frac{1}{99}\sum_{i=1}^{99}\mathcal{L}_{\text{ROLL-TPR@FPR}}\!\left(\frac{i}{100}\right)
|
||||
\mathcal{L}_{\text{ROLL-AOC}} = \frac{1}{100}\sum_{i=1}^{99}\mathcal{L}_{\text{ROLL-TPR@FPR}}\!\left(\frac{i}{100}\right)
|
||||
\label{eq:roll-aoc}
|
||||
\end{equation}
|
||||
Since $\operatorname{TPR}(\alpha)$ is monotone in $\alpha$ along the ROC curve,
|
||||
$g(\alpha) := 1-\operatorname{TPR}(\alpha)$ is monotone and bounded in $[0,1]$, so
|
||||
\eqref{eq:roll-aoc} approximates $\int_0^1 g(\alpha)\,d\alpha$ to within
|
||||
$1/99 \approx 0.01$ by a standard Darboux-sum argument, assuming each per-threshold
|
||||
term is evaluated exactly. This bound covers only the quadrature step: it does not
|
||||
Since $\operatorname{TPR}(\operatorname{FPR}^{-1}(\alpha))$ is monotone in $\alpha$ along the ROC curve,
|
||||
$g(\alpha) := 1-\operatorname{TPR}(\operatorname{FPR}^{-1}(\alpha))$ is monotone and bounded in $[0,1]$.
|
||||
Moreover $g(1) = 1-\operatorname{TPR}(\operatorname{FPR}^{-1}(1)) = 1-\operatorname{TPR}(-\infty) = 0$
|
||||
exactly, since thresholding at $-\infty$ classifies every sample positive. This means
|
||||
\eqref{eq:roll-aoc} is not merely a 99-point average standing in for the integral. It is
|
||||
the exact right-Riemann sum of $\mathcal{L}_{\text{AOC}} = \int_0^1 g(\alpha)\,d\alpha$ over
|
||||
$100$ equal-width subintervals of $[0,1]$, with the $100$th term, $g(1)$, known to vanish
|
||||
rather than omitted. For a monotone function, the right-Riemann-sum error over $N$ equal
|
||||
subintervals is bounded by the function's total range times the step size; here that gives
|
||||
\[
|
||||
\bigl|\mathcal{L}_{\text{ROLL-AOC}} - \mathcal{L}_{\text{AOC}}\bigr|
|
||||
\;\le\; \bigl(g(0)-g(1)\bigr)\cdot\frac{1}{100}
|
||||
\;\le\; \frac{1}{100} = 0.01,
|
||||
\]
|
||||
using $g(0)\le 1$ and $g(1)=0$. This bound covers only the quadrature step: it does not
|
||||
extend to the gradient \eqref{eq:roll-aoc-gradient}, nor does it account for the gap
|
||||
between the KDE-fitted rates and the raw empirical scores (\Cref{sec:roll-instantiations})
|
||||
or any residual numerical error in the per-threshold bisection for $\tau_k$.
|
||||
@@ -298,12 +343,12 @@ or any residual numerical error in the per-threshold bisection for $\tau_k$.
|
||||
The gradient follows by linearity of the sum:
|
||||
\begin{equation}
|
||||
\frac{\partial \mathcal{L}_{\text{ROLL-AOC}}(f_\theta(\mathbf{X}))}
|
||||
{\partial f_\theta(\mathbf{x}_i)} = \frac{1}{99}\sum_{r=1}^{99}\frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}\!\left(f_\theta(\mathbf{X}) ; \frac{r}{100}\right)}
|
||||
{\partial f_\theta(\mathbf{x}_i)} = \frac{1}{100}\sum_{r=1}^{99}\frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}\!\left(f_\theta(\mathbf{X}) ; \frac{r}{100}\right)}
|
||||
{\partial f_\theta(\mathbf{x}_i)}
|
||||
\label{eq:roll-aoc-gradient}
|
||||
\end{equation}
|
||||
(the summation index $r$ ranges over the $99$ thresholds; $i$, fixed throughout the sum,
|
||||
indexes the sample being differentiated — distinct roles that \eqref{eq:roll-aoc} does
|
||||
indexes the sample being differentiated. Distinct roles that \eqref{eq:roll-aoc} does
|
||||
not need to distinguish, since it has no outer per-sample index to collide with.)
|
||||
The concrete per-threshold gradient for the KDE instantiation is derived in
|
||||
\Cref{sec:roll-instantiations}; gradients for the parametric cases are in
|
||||
@@ -315,7 +360,7 @@ The concrete per-threshold gradient for the KDE instantiation is derived in
|
||||
\centering
|
||||
\includegraphics[width=0.62\textwidth]{content/method/figures/roll_aoc_roc.pdf}
|
||||
\caption{Geometric interpretation of ROLL-AOC. The ROC curve (orange) maps
|
||||
each FPR value $\alpha$ to $\operatorname{TPR}(\alpha)$.
|
||||
each FPR value $\alpha$ to $\operatorname{TPR}(\operatorname{FPR}^{-1}(\alpha))$.
|
||||
\textbf{Green}: area under the curve (AUC), the quantity we maximize.
|
||||
\textbf{Red}: area over the curve
|
||||
($\mathrm{AOC} = 1 - \mathrm{AUC} = \mathcal{L}_{\text{ROLL-AOC}}$),
|
||||
@@ -596,13 +641,13 @@ For KDE-ROLL at a single operating threshold: (a) a positive-class sample's grad
|
||||
magnitude is the kernel PDF evaluated at its own distance to $\tau$, and vanishes as
|
||||
that distance grows, independent of the rest of the batch. (b) a negative-class
|
||||
sample's gradient magnitude is its kernel-PDF value \emph{relative to the sum over
|
||||
$\mathcal{B}_0$} — samples near $\tau$ receive a larger share than those far from it
|
||||
$\mathcal{B}_0$}. Samples near $\tau$ receive a larger share than those far from it
|
||||
\emph{within the same batch}, but this relative weighting does not by itself vanish
|
||||
for an individual sample merely because the whole class is far from $\tau$.
|
||||
\end{observation}
|
||||
|
||||
\Cref{fig:threshold-locality} illustrates this for a simulated two-class setting in
|
||||
which both classes are broadly spread around $\tau$ — the typical case, and the one in
|
||||
which both classes are broadly spread around $\tau$: the typical case, and the one in
|
||||
which the negative class's relative weighting (\Cref{obs:gradient-locality}b) and the
|
||||
positive class's absolute decay (\Cref{obs:gradient-locality}a) happen to agree. The
|
||||
upper panel shows the score distributions and the operating threshold; the lower panel
|
||||
@@ -669,7 +714,7 @@ See Appendix~\ref{appendix:score-normalization}.
|
||||
|
||||
For Silverman's rule, normalisation is therefore a pure floating-point reparametrisation:
|
||||
it changes neither the loss nor the gradient used for training, so it cannot be what
|
||||
prevents overflow for that estimator — there is nothing left for it to change.
|
||||
prevents overflow for that estimator. There is nothing left for it to change.
|
||||
ISJ, the primary bandwidth estimator (\Cref{sec:kde-bandwidth}), does \emph{not} satisfy
|
||||
the proposition's hypothesis (Appendix~\ref{appendix:score-normalization} measures its
|
||||
scale-dependence directly). For ISJ, both the loss value and the gradient genuinely
|
||||
@@ -703,23 +748,43 @@ nearest-threshold samples.
|
||||
We estimate the bandwidth using the Improved Sheather-Jones (ISJ)
|
||||
method~\cite{botev2010kde}, which minimises the asymptotic mean integrated squared
|
||||
error via an iterative plug-in procedure. ISJ tracks the true score distribution more
|
||||
closely than closed-form rules.
|
||||
closely than closed-form rules. ISJ's optimality guarantee is derived for a Gaussian
|
||||
kernel, whereas the kernel used throughout this work is logistic (\Cref{sec:roll-kde});
|
||||
applying it here is a plug-in heuristic approximation, chosen for its ready
|
||||
availability in the existing implementation rather than a proven-optimal fit to the
|
||||
logistic kernel specifically. Kernel-appropriate bandwidth selection is left to future
|
||||
work (\Cref{sec:conclusions:future}).
|
||||
Because ISJ is iterative, it can fail to converge on small or degenerate batches early
|
||||
in training. In those cases we fall back to Silverman's rule of
|
||||
thumb~\cite{silverman1986density},
|
||||
\[
|
||||
h = 1.06\,\hat{\sigma}\,n^{-1/5},
|
||||
\]
|
||||
which is closed-form and always produces a valid bandwidth.
|
||||
which is closed-form. Silverman's rule is itself undefined at $\hat\sigma=0$. i.e.\ if a
|
||||
class's scores are literally constant, which does not arise with real, continuously-valued
|
||||
model outputs but is not ruled out in principle. The underlying bandwidth-selection
|
||||
library substitutes a fixed fallback ($h=1$) in that case rather than propagating a
|
||||
division by zero.\footnote{As a further safety net, $v=1/h$ is capped at $10{,}000$
|
||||
regardless of source. This last-resort cap technically breaks the exact
|
||||
scale-equivariance assumed by \Cref{prop:normalization-invariance} once triggered,
|
||||
though in practice. Especially after the numerical-stability improvements of
|
||||
\Cref{sec:kde-grad-stability,appendix:bandwidth-freeze,appendix:score-normalization}
|
||||
it is essentially never reached.} More generally, the ROLL derivation throughout assumes
|
||||
each fitted density is nonzero at the quantile being inverted, so that the
|
||||
inverse-function-theorem step of \Cref{sec:roll-kde-backward} is well-defined; the
|
||||
safeguards above are exactly what keep this assumption from failing in the KDE
|
||||
instantiation.
|
||||
The kernel parameter is then set to $v = 1/h$.
|
||||
|
||||
\subsubsection{Bandwidth Scaling for KDE}
|
||||
\label{sec:kde-bandwidth-scaling}
|
||||
|
||||
Although we may select a valid bandwidth at each step, it tends to be
|
||||
too small near the start of training, when scores are not yet
|
||||
spread out and $\hat{\sigma}$ is small. This causes the KDE to undersmooth the score
|
||||
distribution, producing very low gradient magnitudes and stalling convergence.
|
||||
Empirically, training without a bandwidth scheduler stalls more often near the start of
|
||||
training. A plausible explanation is that the classes are not yet well separated this
|
||||
early: the model has not yet learned to distinguish them — so the fitted score
|
||||
distributions overlap substantially relative to their spread, and the resulting
|
||||
gradient carries little discriminative signal regardless of the estimated bandwidth's
|
||||
absolute value.
|
||||
|
||||
The solution was to use a bandwidth scheduler. Starting with large bandwidths resulted in gradients that spread out more evenly across all scores, and fewer non-convergence runs occurred. The bandwidth scheduler artificially scales up the bandwidth at the beginning to speed up initial training, and then tapers back to restore the intended bandwidth in order to maximise results.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user