649 lines
32 KiB
TeX
649 lines
32 KiB
TeX
|
|
\section{Problem Formulation}
|
|
\label{sec:problem-formulation}
|
|
|
|
\subsection{Imbalanced Classification and The TPR/FPR Objective}
|
|
\label{sec:imbalanced-tpr-fpr}
|
|
|
|
% IMPROVEMENT SUGGESTIONS:
|
|
% - Section title promises TPR/FPR but neither is formally defined here; the section
|
|
% must land on a concrete statement of the TPR@FPR objective with proper notation.
|
|
% - The inseparability tangent (paragraphs 2-3) dilutes the argument; trim to one
|
|
% sentence or move to Related Work. The cascade example in particular reads as
|
|
% background material unrelated to the TPR/FPR framing.
|
|
% - Narrative arc is currently backwards: the comment block near the end (lines
|
|
% starting 'Motivate why standard loss functions...') is a better outline than
|
|
% the prose above it. Drive the section with: accuracy is gameable under imbalance
|
|
% -> class weighting is the standard fix -> it gives no direct control over the
|
|
% operating point -> therefore we need TPR@FPR directly.
|
|
% - $\mathcal{D}_0$ and $\mathcal{D}_1$ are used but never defined; add definitions
|
|
% for the positive/negative subsets of $\mathcal{D}$ before first use.
|
|
% - Suggested arc: (1) standard setup + accuracy objective; (2) why accuracy fails
|
|
% under imbalance -- the trivial majority-class classifier; (3) standard remedies
|
|
% and why they give only indirect control; (4) formally define TPR, FPR, ROC, and
|
|
% state the target objective: maximise TPR at a fixed FPR budget $\alpha$.
|
|
|
|
When fitting a classifier, it is most common to try and maximize the probability
|
|
of success. That is, given some dataset
|
|
$\mathcal{D} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{n}$
|
|
with inputs $\mathbf{x}_i \in \mathcal{X}$ and binary labels $y_i \in \{0, 1\}$, we learn a scoring function $f_\theta : \mathcal{X} \to \mathbb{R}$ and apply a threshold $\tau$ to produce predictions $\hat{y} = \mathbf{1}[f_\theta(\mathbf{x}) > \tau]$. The standard objective is to minimize the probability of classification error:
|
|
|
|
$$\mathcal{L}(\theta) = \frac{1}{n} \sum_{i=1}^{n} \mathbf{1}[\hat{y}_i \neq y_i] = P(\hat{y} \neq y)$$
|
|
|
|
which is equivalent to maximizing classification accuracy $P(\hat{y} = y)$.
|
|
|
|
For most applications, this optimization is warranted. However, there are many real-world settings in which
|
|
this type of fitting is ill-suited.
|
|
|
|
One such application is in the case of inseparable classes. In many cases, no meaningful
|
|
separation boundary can be found between $\mathcal{D}_0$ and $\mathcal{D}_1$. In many cases,
|
|
especially in tabular datasets, there can be two data elements $i,j$ for which $\mathbf{x}_i = \mathbf{x}_j$,
|
|
however, $y_i \neq y_j$. Any deterministic classifier will have to decide whether to classify
|
|
inseparable points as one class or the other, but it remains hard to control the behavior of
|
|
a classifier while it is fitting to suggest one outcome or the other.
|
|
|
|
The issue of inseparability becomes more apparent in certain applications where runtime is a key
|
|
and limiting factor. In many classification pipelines, a ``quick and dirty'' classifier may
|
|
filter out many candidates before an ``in depth'' classifier gives the final answer. This may be done
|
|
in events where running the full in-depth classification on all data points is not a cost-effective or time-efficient solution.
|
|
The ``quick and dirty'' classifier can be either low in parameters, making it run quickly. It may also only
|
|
make a quick decision based on part of the data point. For example, in the case of a classifier looking to flag
|
|
images of the ocean, a ``quick and dirty'' classifier may look at a very-low resolution version of the image and simply check if it is relatively ``blue''. It filters out many certain false candidates this way, then passes on the remainder to an in-depth, higher accuracy classifier. Solutions like this allow to run at near-optimal performance with lower compute resources in limited-time applications. These situations are common in large-volume image applications on the internet, as well as in astronomy/microscopy.
|
|
|
|
Another such application is a case of highly imbalanced data. Even with separability, it is very
|
|
easy to a classifier to simply ``ignore'' the minority class. This may very likely be a local minimum
|
|
while fitting, which may be inescapable. As was once said, ``a classifier that always returns 0 is
|
|
well fitted for flagging pictures of Michael Jordan bathing in green M\&Ms''. Combined with inseparability, we see that in extreme cases, classifiers may choose to ignore, or otherwise penalize, the minority class.
|
|
|
|
For every trained classifier, the output per model is a score (TODO rewrite and formalize).
|
|
|
|
TODO finish this subsection
|
|
|
|
\begin{itemize}
|
|
\item What usually happens
|
|
\begin{itemize}
|
|
\item Fitting
|
|
\item Graphing ROC and picking threshold based on tradeoff
|
|
\end{itemize}
|
|
\item Usually we compensate for poor performance with
|
|
\begin{itemize}
|
|
\item Weighting between classes
|
|
\item Possible other methods
|
|
\end{itemize}
|
|
\item Problem with this method is lack of control.
|
|
\end{itemize}
|
|
|
|
|
|
% Motivate why standard loss functions (cross-entropy, MSE) break down under class
|
|
% imbalance: they optimize average accuracy, which a model can game by predicting
|
|
% the majority class. Introduce concrete examples (e.g. medical diagnosis, fraud
|
|
% detection) where the cost of a false negative vastly outweighs a false positive,
|
|
% and where the practitioner needs to operate at a specific FPR budget. Argue that
|
|
% what is actually needed is direct control over TPR at a fixed FPR, not a
|
|
% surrogate that only loosely correlates with it.
|
|
|
|
|
|
|
|
|
|
\subsection{Connection to Neyman-Pearson}
|
|
\label{sec:neyman-pearson}
|
|
|
|
% Introduce the Neyman-Pearson lemma: among all tests at a given false-positive rate,
|
|
% the likelihood-ratio test maximizes TPR. Frame ROLL as a learnable, gradient-based
|
|
% realization of this principle — rather than assuming known class-conditional
|
|
% distributions (as classical NP does), ROLL estimates them from model scores during
|
|
% training. Briefly note that existing NP-inspired methods (covered in
|
|
% \Cref{chapter:related_work}) do not support deep learning / gradient-based
|
|
% optimization; this is the gap ROLL fills.
|
|
|
|
|
|
%------------------------------------------------
|
|
|
|
\section{The ROLL Framework}
|
|
\label{sec:roll-framework}
|
|
|
|
In order to allow gradient-based learning methods to optimize TPR or FPR directly, we introduce
|
|
ROLL - Rate Optimized Likelyhood Loss.
|
|
|
|
The core idea of ROLL is as follows: when training, the user selects that they would like to optimize
|
|
TPR given some fixed FPR. Then, during training, on every training step, the actual TPR is calculated for
|
|
the given FPR, and the fitting algorithm tries to maximize the TPR directly. This way, while training, we gain direct control over the performance at the threshold that we care about - avoiding needing to pick out a threshold later.
|
|
|
|
In Neyman-Pearson terms, our algorithm wishes to directly minimize the Type-1 error given a fixed Type-2 error, or vice versa.
|
|
|
|
In practice, however, things are not so simple.
|
|
|
|
\subsection{The Differentiability Problem}
|
|
\label{sec:differentiability-problem}
|
|
|
|
% TPR and FPR are defined via threshold comparisons on model scores, making them
|
|
% piecewise-constant functions of the model parameters with zero gradient almost
|
|
% everywhere — standard backpropagation cannot move through them. Show this
|
|
% concretely with the step-function definition of TPR/FPR and explain why
|
|
% common surrogates (AUC proxies, hinge-based losses) only indirectly target
|
|
% the operating point a practitioner cares about.
|
|
|
|
Directly optimizing TPR for FPR, or vice versa, is an indefferentiable problem.
|
|
|
|
For this sub-section, we will reffer to an example of maximizing TPR given a fixed FPR,
|
|
though this can be reversed.
|
|
|
|
For any loss $\mathcal{L}(\theta)$, in order
|
|
to continue fitting our model, we must derive some gradient
|
|
|
|
$\frac{\partial \mathcal{L}(\theta)}{\partial f_\theta(\mathbf{x}_i)}$ where $f_\theta(\mathbf{x}_i)$ is the score
|
|
given to data point $\mathbf{x}_i$ by our model $f_\theta$.
|
|
There are two inherent steps built-in to our methods:
|
|
|
|
\begin{itemize}
|
|
\item Based on fixed rate, calculate threshold that gives said rate.
|
|
For example, when fixing FPR = $\alpha$, find some threshold $\tau$ s.t. $\mathbb{E}[1_{f_{\theta}\mathbf{x}_i > \tau | y_i = 0}] = \alpha$
|
|
\item Given the calculated threshold, calculate the rate we would like to maximize
|
|
(in our example, maximize TPR)
|
|
\end{itemize}
|
|
|
|
TPR or FPR calculation, given a threshold, is non-differentiable.
|
|
The reason for that is that the TPR and FPR as a function of $f_\theta(\mathbf{x}_i)$ are
|
|
non-continuous. If $f_\theta(\mathbf{x}_i) \neq \tau$, that is, the score is not exactly on the threshold, then changing it
|
|
by some $\varepsilon$ will not affect either TPR or FPR at all, and the derivative of the function w.r.t.\ $f_\theta(\mathbf{x}_i)$ will be zero. Likewise,
|
|
if $f_\theta(\mathbf{x}_i) = \tau$, the derivative of TPR/FPR w.r.t.\ $f_\theta(\mathbf{x}_i)$ becomes infinite. This derivative cannot be used
|
|
to continuously fit our model.
|
|
|
|
This necessitates a different approach, to make these calculations differentiable w.r.t the scores themselves, in order to fit this type of optimization into a learning platform.
|
|
|
|
\subsection{Score Distribution Fitting as a Solution}
|
|
\label{sec:score-distribution-fitting}
|
|
|
|
% The key insight: instead of thresholding scores directly, fit a differentiable
|
|
% probability distribution to the positive-class and negative-class score outputs.
|
|
% TPR and FPR then become integrals of that distribution above/below a threshold,
|
|
% which are smooth and differentiable in the distribution parameters — and through
|
|
% them, in the model weights. Explain intuitively why this preserves the semantics
|
|
% of TPR/FPR while making the objective amenable to gradient descent.
|
|
|
|
The way to overcome this problem in differentiability is not to calculate the TPR and FPR
|
|
directly, but to estimate them by fitting probability distributions to the score output of the model at
|
|
every step. Contiguous probability distributions are contiguous w.r.t their inputs. Additionally, we can calculate the TPR, FPR given the probability distributions CDF and ICDF functions. If we can gaurantee these functions to be differentiable, which proved to be the main challenge of this work, we can gaurantee this process to be differentiable.
|
|
|
|
|
|
\subsection{General ROLL Formulation and Derivation}
|
|
\label{sec:roll-formulation}
|
|
|
|
% Present the abstract ROLL loss in terms of a generic distribution $\mathcal{F}$
|
|
% fitted to model scores, without yet committing to a specific family. Derive the
|
|
% expression for TPR@FPR$_0$ as a function of $\mathcal{F}$'s parameters, and show
|
|
% that minimizing the loss is equivalent to maximizing TPR subject to the FPR
|
|
% constraint. Include the full derivation here; this is the core theoretical
|
|
% contribution of the chapter.
|
|
|
|
Using the dataset $\mathcal{D} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{n}$ as introduced in
|
|
\Cref{sec:problem-formulation}, let $\mathcal{B} \subseteq \mathcal{D}$ denote a training batch
|
|
over which we compute the loss.
|
|
|
|
We then subdivide the batch into positive and negative samples:
|
|
%
|
|
\begin{align*}
|
|
\mathcal{B}_1 &= \{(\mathbf{x}_i, y_i) \in \mathcal{B} \mid y_i = 1\} \\
|
|
\mathcal{B}_0 &= \{(\mathbf{x}_i, y_i) \in \mathcal{B} \mid y_i = 0\}
|
|
\end{align*}
|
|
%
|
|
and extract the corresponding model score vectors:
|
|
%
|
|
\begin{equation*}
|
|
\mathbf{s}^{(1)} = \bigl(f_\theta(\mathbf{x}_i)\bigr)_{(\mathbf{x}_i,\, y_i)\,\in\,\mathcal{B}_1},
|
|
\qquad
|
|
\mathbf{s}^{(0)} = \bigl(f_\theta(\mathbf{x}_i)\bigr)_{(\mathbf{x}_i,\, y_i)\,\in\,\mathcal{B}_0}
|
|
\end{equation*}
|
|
|
|
We then fit a distribution family $\mathcal{F} = \{P_\phi : \phi \in \Phi\}$ separately to each
|
|
score vector in order to obtain a differentiable model of each class's score distribution:
|
|
%
|
|
\begin{equation*}
|
|
\hat{\phi}_1 = \operatorname{Fit}_{\mathcal{F}}\!\left(\mathbf{s}^{(1)}\right),
|
|
\qquad
|
|
\hat{\phi}_0 = \operatorname{Fit}_{\mathcal{F}}\!\left(\mathbf{s}^{(0)}\right)
|
|
\end{equation*}
|
|
%
|
|
Let $\hat{F}_1$ and $\hat{F}_0$ denote the CDFs of $P_{\hat{\phi}_1}$ and $P_{\hat{\phi}_0}$,
|
|
respectively. Specific choices of $\mathcal{F}$ (Gaussian, Beta, KDE) are discussed in
|
|
\Cref{sec:roll-instantiations}.
|
|
|
|
Using this notation, at a threshold $\alpha$:
|
|
\begin{align}
|
|
\operatorname{TPR}(\alpha) &= 1 - \hat{F}_1(\alpha), \label{eq:tpr-cdf} \\
|
|
\operatorname{FPR}(\alpha) &= \hat{F}_0(\alpha). \label{eq:fpr-cdf}
|
|
\end{align}
|
|
|
|
Let $\hat{F}_1^{-1}$ and $\hat{F}_0^{-1}$ denote the quantile functions of $P_{\hat{\phi}_1}$ and
|
|
$P_{\hat{\phi}_0}$, respectively. The threshold achieving a target TPR of $\alpha$ follows directly
|
|
from \eqref{eq:tpr-cdf}:
|
|
\[
|
|
\operatorname{TPR}(\tau) = \alpha \;\Longrightarrow\; 1 - \hat{F}_1(\tau) = \alpha
|
|
\;\Longrightarrow\; \tau = \hat{F}_1^{-1}(1 - \alpha)
|
|
\]
|
|
and similarly for a target FPR of $\alpha$, from \eqref{eq:fpr-cdf}:
|
|
\[
|
|
\operatorname{FPR}(\tau) = \alpha \;\Longrightarrow\; \hat{F}_0(\tau) = \alpha
|
|
\;\Longrightarrow\; \tau = \hat{F}_0^{-1}(\alpha)
|
|
\]
|
|
|
|
This yields two versions of our objective. Substituting the ICDF threshold into the rate
|
|
expressions, our estimated TPR at a fixed FPR of $\alpha$ is $1 - \hat{F}_1(\hat{F}_0^{-1}(\alpha))$.
|
|
Therefore, to maximise TPR we minimise:
|
|
\begin{equation}
|
|
\mathcal{L}_{\text{ROLL-TPR@FPR}}(f_{\theta}(\mathcal{B}) ; \alpha) = \hat{F}_1(\hat{F}_0^{-1}(\alpha))
|
|
\label{eq:roll-tpr-at-fpr}
|
|
\end{equation}
|
|
|
|
Our estimated FPR at a fixed TPR of $\alpha$ is $\hat{F}_0(\hat{F}_1^{-1}(1 - \alpha))$.
|
|
Therefore, to minimise FPR:
|
|
\begin{equation}
|
|
\mathcal{L}_{\text{ROLL-FPR@TPR}}(f_{\theta}(\mathbf{X}) ; \alpha) = \hat{F}_0(\hat{F}_1^{-1}(1 - \alpha))
|
|
\label{eq:roll-fpr-at-tpr}
|
|
\end{equation}
|
|
|
|
TODO explain the trick where we can only use one version - base off code.
|
|
|
|
We now derive the gradient of \eqref{eq:roll-tpr-at-fpr} with respect to $f_\theta(\mathbf{x}_i)$.
|
|
As both loss forms are symmetric in structure, it is sufficient to treat the TPR@FPR case.
|
|
|
|
If $y_i = 1$, then $(\mathbf{x}_i, y_i) \in \mathcal{B}_1$ and $f_\theta(\mathbf{x}_i)$ has no
|
|
effect on $\hat{F}_0^{-1}(\alpha)$. Therefore:
|
|
\begin{equation}
|
|
\left.\frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}(f_\theta(\mathcal{B}) ; \alpha)}
|
|
{\partial f_\theta(\mathbf{x}_i)}\right|_{y_i = 1}
|
|
= \frac{\partial \hat{F}_1(\hat{F}_0^{-1}(\alpha))}{\partial f_\theta(\mathbf{x}_i)}
|
|
\end{equation}
|
|
|
|
If $y_i = 0$, then $(\mathbf{x}_i, y_i) \in \mathcal{B}_0$ and $f_\theta(\mathbf{x}_i)$ affects
|
|
$\hat{F}_0^{-1}(\alpha)$ instead. Applying the chain rule:
|
|
\begin{equation}
|
|
\left.\frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}(f_\theta(\mathbf{X}) ; \alpha)}
|
|
{\partial f_\theta(\mathbf{x}_i)}\right|_{y_i = 0}
|
|
= \frac{\partial \hat{F}_1(\hat{F}_0^{-1}(\alpha))}{\partial \hat{F}_0^{-1}(\alpha)}
|
|
\cdot \frac{\partial \hat{F}_0^{-1}(\alpha)}{\partial f_\theta(\mathbf{x}_i)}
|
|
\end{equation}
|
|
|
|
Unifying both cases, the general gradient applicable to all ROLL instantiations is:
|
|
\begin{equation}
|
|
\frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}(f_\theta(\mathbf{X}) ; \alpha)}
|
|
{\partial f_\theta(\mathbf{x}_i)} =
|
|
\begin{cases}
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(\alpha))}{\partial f_\theta(\mathbf{x}_i)}
|
|
& \text{if } y_i = 1 \\[10pt]
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(\alpha))}{\partial \hat{F}_0^{-1}(\alpha)}
|
|
\cdot \dfrac{\partial \hat{F}_0^{-1}(\alpha)}{\partial f_\theta(\mathbf{x}_i)}
|
|
& \text{if } y_i = 0
|
|
\end{cases}
|
|
\label{eq:roll-gradient}
|
|
\end{equation}
|
|
The concrete form of each partial derivative depends on the choice of distribution family
|
|
$\mathcal{F}$, and is derived for each instantiation in \Cref{sec:roll-instantiations}.
|
|
|
|
|
|
\subsection{Properties of the ROLL Formulation}
|
|
\label{sec:roll-properties}
|
|
|
|
% State and prove (or sketch) key properties: consistency of the estimator as
|
|
% the number of samples grows, monotonicity of the loss with respect to the
|
|
% TPR/FPR trade-off, and behaviour at the constraint boundary. Discuss what
|
|
% guarantees ROLL provides and under what assumptions they hold.
|
|
% NOTE - these properties are auto-generated, consider suggestions.
|
|
|
|
|
|
\subsection{Gradient Computation and the Custom Backward Pass}
|
|
\label{sec:roll-backward}
|
|
|
|
% Because the distribution parameters are themselves functions of model scores
|
|
% (not fixed), the backward pass must differentiate through the distribution
|
|
% fitting step. Describe the custom autograd implementation: how gradients flow
|
|
% from the ROLL loss back through the distribution parameters to the model
|
|
% weights. Highlight where standard autograd would fail and what the manual
|
|
% backward pass does differently.
|
|
|
|
|
|
%------------------------------------------------
|
|
|
|
\section{Distribution Instantiations}
|
|
\label{sec:roll-instantiations}
|
|
|
|
% Each subsection below instantiates the general ROLL framework with a specific
|
|
% distributional assumption. Gaussian and Beta are included as parametric
|
|
% special cases; KDE is the primary contribution and receives the fullest treatment.
|
|
|
|
\subsection{Gaussian ROLL}
|
|
\label{sec:roll-gaussian}
|
|
|
|
% Assume positive- and negative-class scores are Gaussian distributed. The
|
|
% TPR/FPR integrals reduce to closed-form expressions in terms of the normal
|
|
% CDF, giving a simple and fast-to-compute instantiation of ROLL. Note the
|
|
% limitations: the Gaussian assumption can be violated in practice, motivating
|
|
% the more flexible instantiations below.
|
|
|
|
A simple method of implementing the ROLL loss is to approximate the distribution
|
|
of the score distributions as gaussian. This method is simple, as there exist
|
|
simple closed-form estimations for the parameters of gaussian distributions, and
|
|
they are differentiable.
|
|
|
|
While the distributions of scores cannot be gauranteed to follow any form, there are
|
|
some instances where the score distributions might be similar or exactly gaussian.
|
|
For example, when finding linear separability between gaussian distributions, the resulting
|
|
scores will be distributed normally, and this method will fit exactly.
|
|
|
|
\subsubsection{Forward}
|
|
\label{sec:roll-gaussian-forward}
|
|
|
|
% Instantiate \eqref{eq:roll-tpr-at-fpr} under Gaussian score distributions.
|
|
% Each class's scores are modelled as $\mathcal{N}(\mu_k, \sigma_k^2)$, fitted
|
|
% by MLE. The CDF is the standard normal CDF $\Phi$, giving a closed-form loss.
|
|
|
|
Firstly, we must estimate the parameters to fit the gaussian distributions
|
|
to the true and false populations of the score output.
|
|
|
|
\begin{equation}
|
|
\mu_0 = \frac{\sum_{\mathbf{x}_0 \in \mathcal{B}_0}{f_{\theta}(\mathbf{x}_0)}}{|\mathcal{B}_0|}
|
|
\qquad
|
|
\mu_1 = \frac{\sum_{\mathbf{x}_0 \in \mathcal{B}_1}{f_{\theta}(\mathbf{x}_1)}}{|\mathcal{B}_1|}
|
|
\end{equation}
|
|
|
|
\begin{equation}
|
|
\sigma_0^2 = \frac{1}{|\mathcal{B}_0|}\sum_{\mathbf{x}_0 \in \mathcal{B}_0}(\mathbf{x}_0 - \mu_0)^2
|
|
\qquad
|
|
\sigma_1^2 = \frac{1}{|\mathcal{B}_1|}\sum_{\mathbf{x}_1 \in \mathcal{B}_1}(\mathbf{x}_1 - \mu_1)^2
|
|
\end{equation}
|
|
|
|
Once these parameters are estimated, we can then define our ROLL objective as required.
|
|
|
|
The CDF of the normal distribution with mean $\mu$ and standard deviation $\sigma$ is:
|
|
\[
|
|
F(x) = \frac{1}{2}\left[1 + \text{erf}\!\left(\frac{x - \mu}{\sigma\sqrt{2}}\right)\right]
|
|
\]
|
|
and its inverse (quantile function) is:
|
|
\[
|
|
F^{-1}(p) = \mu + \sigma\sqrt{2}\,\text{ierf}(2p - 1)
|
|
\]
|
|
where TODO EXPLAIN ERF AND IERF. Plugging into the general ROLL framework, the threshold
|
|
$\tau$ achieving FPR $= \alpha$ and the resulting loss are:
|
|
\begin{align}
|
|
\tau &= \mu_0 + \sigma_0\sqrt{2}\,\text{ierf}(2\alpha - 1) \nonumber \\
|
|
\mathcal{L}_{\text{ROLL-TPR@FPR}}^{\text{GAUSSIAN}}(f_{\theta}(\mathcal{B}) ; \alpha)
|
|
&= \frac{1}{2}\left[1 + \text{erf}\!\left(\frac{\tau - \mu_1}{\sigma_1\sqrt{2}}\right)\right]
|
|
\label{eq:roll-tpr-at-fpr-gaussian}
|
|
\end{align}
|
|
|
|
\subsubsection{Gradient Derivation}
|
|
\label{sec:roll-gaussian-backward}
|
|
|
|
% Differentiate the Gaussian loss with respect to $f_\theta(\mathbf{x}_i)$.
|
|
% Gradients flow through $\mu_k$ and $\sigma_k^2$ (which are differentiable
|
|
% functions of the scores), and through $\Phi$ and $\Phi^{-1}$, which have
|
|
% simple closed-form derivatives.
|
|
|
|
In order to compute the gradient derivation for the gaussian estimates we must first compute the gradient to each of the parameters.
|
|
Firstly, for $y_i = 1$.
|
|
|
|
\[
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(\alpha))}{\partial \mu_1} =
|
|
\frac{\partial \frac{1}{2}\left[1 + \text{erf}\!\left(\frac{\hat{F}_0^{-1}(\alpha) - \mu_1}{\sigma_1\sqrt{2}}\right)\right]}{\partial \mu_1}
|
|
\]
|
|
|
|
\[
|
|
= \frac{1}{\sqrt{2}\sigma_1}\exp\left(\frac{\hat{F}_0^{-1}(\alpha) - \mu_1}{\sqrt{2}\sigma_1}\right)
|
|
\]
|
|
|
|
\[
|
|
\frac{\partial \mu_1}{\partial f_{\theta}(\mathbf{x}_i)} = \frac{1}{|\mathcal{B}_1|}
|
|
\]
|
|
|
|
\[
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(\alpha))}{\partial \sigma_1} =
|
|
\frac{\partial \frac{1}{2}\left[1 + \text{erf}\!\left(\frac{\hat{F}_0^{-1}(\alpha) - \mu_1}{\sigma_1\sqrt{2}}\right)\right]}{\partial \sigma_1}
|
|
\]
|
|
|
|
\[
|
|
= \frac{\hat{F}_0^{-1}(\alpha)-\mu_1}{\sqrt{2\pi}\sigma_1^2} \exp\left( - {\left( \frac{\hat{F}_0^{-1}(\alpha) - \mu_1}{\sqrt{2}\sigma_1} \right)}^2 \right)
|
|
\]
|
|
\[
|
|
\frac{\partial \sigma_1}{\partial f_{\theta}(\mathbf{x}_i)} = \frac{\mathbf{x}_i - \mu_1}{|\mathcal{B}_1| \sigma_1}
|
|
\]
|
|
|
|
The final derivation, given $y_i = 1$, is given as:
|
|
|
|
\[
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(\alpha))}{\partial f_{\theta}(\mathbf{x}_i)} = \dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(\alpha))}{\partial \mu_1} \cdot \frac{1}{|\mathcal{B}_1|} +
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(\alpha))}{\partial \sigma_1} \cdot \frac{\partial \sigma_1}{\partial f_{\theta}(\mathbf{x}_i)} = \frac{\mathbf{x}_i - \mu_1}{|\mathcal{B}_1| \sigma_1}
|
|
\]
|
|
|
|
TODO - above equation is based off of mathexchange derivation. Link and reference properly!
|
|
|
|
Given $y_i = 0$ we first must derive:
|
|
|
|
\[
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(\alpha))}{\partial \hat{F}_0^{-1}(\alpha)} = \frac{1}{\sqrt{2 \pi \sigma_1^2}}\exp\left( -{\left( \frac{\hat{F}_0^{-1}(\alpha) - \mu_1}{2\sigma_1^2}\right)}^2\right)
|
|
\]
|
|
|
|
TODO finish somehow - later!
|
|
|
|
\subsection{Beta ROLL}
|
|
\label{sec:roll-beta}
|
|
|
|
% When scores are passed through a sigmoid they live in $[0,1]$ and a Beta
|
|
% distribution is a natural parametric fit. The integrals involve the
|
|
% regularised incomplete beta function. Discuss when Beta ROLL is preferable
|
|
% to Gaussian ROLL and what its failure modes are.
|
|
|
|
\subsubsection{Forward}
|
|
\label{sec:roll-beta-forward}
|
|
|
|
% Instantiate \eqref{eq:roll-tpr-at-fpr} under Beta-distributed scores.
|
|
% Parameters $a_k, b_k > 0$ are fitted by MLE (or method of moments).
|
|
% The CDF is the regularised incomplete beta function $I_x(a, b)$.
|
|
|
|
\subsubsection{Gradient Derivation}
|
|
\label{sec:roll-beta-backward}
|
|
|
|
% Differentiate through $I_x$ and its inverse. Both have known derivatives;
|
|
% note any numerical considerations when $a$ or $b$ are near zero.
|
|
|
|
|
|
\subsection{KDE ROLL}
|
|
\label{sec:roll-kde}
|
|
|
|
% KDE background: see \Cref{sec:kde-background} in \Cref{chapter:related_work}.
|
|
% The primary instantiation: non-parametric, no assumption on score shape.
|
|
% Discuss the trade-off between flexibility and computational cost, and how
|
|
% the bandwidth (see \Cref{sec:kde-bandwidth}) controls bias-variance.
|
|
|
|
Unfortunately, the score output of trained models remains unpredictable. A solution for this is to use a Kernel Density Estimation (KDE) for probability
|
|
estimation. This benefits from being far more adaptable to real-world distributions of outputs of our trained models. The tradeoff of course is computational
|
|
complexity - computing the gradient derivation in such a case is non-trivial.
|
|
|
|
\subsubsection{Forward}
|
|
\label{sec:roll-kde-forward}
|
|
|
|
% Instantiate \eqref{eq:roll-tpr-at-fpr} under KDE. The CDF becomes a sum of
|
|
% kernel CDFs; the ICDF is computed numerically. Derive the loss expression
|
|
% explicitly and discuss computational cost relative to the parametric cases.
|
|
|
|
As before, to calculate the loss, we calculate:
|
|
|
|
|
|
\[
|
|
\mathcal{L}_{\text{ROLL-TPR@FPR}}^{\text{KDE}}(f_{\theta}(\mathcal{B}) ; \alpha) = \hat{F}_1(\hat{F}_0^{-1}(\alpha))
|
|
\label{eq:roll-tpr-at-fpr}
|
|
\]
|
|
|
|
To calculate this, we first define the kernel function as $\sigma '$, and the CDF of the kernel function as $\sigma$. The approach used here is to define $\sigma$ as a sigmoid function,.
|
|
|
|
\begin{equation}
|
|
\sigma(x ; v) = \left( 1 + \exp(-vx)\right)^{-1} \qquad \sigma'(x ; v) = \frac{v \cdot \exp(-v \cdot x)}{(1 + \exp(-vx))^2}
|
|
\end{equation}
|
|
|
|
Where $v$ is the bandwidth parameter. This affects the calculation. We calculate $v_0$ as the bandwidth for the false population and $v_1$ for the true population. Bandwidth calculation
|
|
will be discussed later.
|
|
|
|
We then denote $\sigma_1(x) = \sigma(x ; v_1), \sigma_0(x) = \sigma(x ; v_0)$ to differentiate between the kernel functions for the true and false population, which only differentiate by bandwidth.
|
|
|
|
The CDF of the KDE function then becomes:
|
|
|
|
\[
|
|
\hat{F}_{KDE}(\tau ; \mathbf{X}) = \frac{1}{|X|}\sum_i \sigma(\tau - x_i)
|
|
\]
|
|
|
|
While the CDF has a nice formula, the inverse CDF has no known closed formula. It instead must be calculated numerically. Thus, the calculation of the decision threshold $\tau$ must be achevied
|
|
using a numerical calculation algorithm. In this work, Newton Raphson is chosen.
|
|
|
|
Firstly we must calculate $\hat{F}_0^{-1}(\alpha)$ = \tau. We start with our initial guess, $\tau_0$, and for each step, calculate:
|
|
|
|
\[
|
|
\tau_{n+1} = \tau_n - \frac{\hat{F}_0(\tau_n ; \mathcal{X}_0)}{\frac{\partial \hat{F}_0(\tau_n ; \mathcal{X}_0)}{\partial \tau}}
|
|
\]
|
|
|
|
This step is taken until $\hat{F}_0(\tau_n ; \mathcal{B}_0) \approx \alpha$ within some acceptable error (in our case $1e-3$).
|
|
|
|
Our derivative w.r.t. $\tau$ is
|
|
|
|
\[ %
|
|
\frac{\partial \hat{F}_0(\tau_n ; \mathcal{X}_0)}{\partial \tau} = \frac{1}{\mathcal{X}_0}\sum_i \sigma'(\tau - x_i)
|
|
\]
|
|
|
|
At this point, we have succesfully calculated $\tau$ for which $\tau = \hat{F}_0^{-1}(\alpha ; \mathbf{X}_0)$.
|
|
|
|
We can now relatively easily calculate $\hat{F}_1(\tau ; \mathbf{X}_1)$ using the formulat for $\hat{F}_{KDE}(\tau ; \mathbf{X})$ above.
|
|
|
|
Importantly, we must save the calculation of $\tau$ for the backward derivation, as well as the calculated bandwidths $v_1, v_0$.
|
|
|
|
|
|
\subsubsection{Gradient Derivation}
|
|
\label{sec:roll-kde-backward}
|
|
|
|
% The primary technical contribution. Standard autograd cannot differentiate
|
|
% through the numerical ICDF inversion; describe the custom backward pass.
|
|
% Gradients flow through the kernel evaluations back to the model scores.
|
|
% This is the section that connects to \Cref{sec:roll-backward} motivation.
|
|
|
|
Firstly, for the true population, we calculate the derivatives.
|
|
|
|
\[
|
|
\dfrac{\partial \hat{F}_1(\hat{F}_0^{-1}(\alpha))}{\partial f_{\theta}(\mathbf{x}_i)} =
|
|
\frac{\partial \frac{1}{|\mathcal{B}_1|}\sum_{\mathbf{x}_j \in \mathcal{B}_1} (\sigma_1(f_{\theta}(\mathbf{x}_j) - \tau))}{\partial f_{\theta}(\mathbf{x}_i)}
|
|
\]
|
|
|
|
\[
|
|
= \frac{\partial \left(\sum_{\mathbf{x}_j \in \mathcal{B}_1 ; j \neq i} \frac{1}{|\mathcal{B}_1|}(\sigma_1(f_{\theta}(\mathbf{x}_j) - \tau)) + \frac{1}{|\mathcal{B}_1|}\sigma_1(f_{\theta}(\mathbf{x}_i) - \tau)\right)}{\partial f_{\theta}(\mathbf{x}_i)}
|
|
\]
|
|
|
|
\[
|
|
= \frac{1}{|\mathcal{B}_1|}\sigma_1'(f_{\theta}(\mathbf{x}_i) - \tau)
|
|
\]
|
|
|
|
Now, for the false population. As stated in the roll framework equation above (TODO LINK) we must first compute the derivative w.r.t the threshold $\tau = \hat{F}_{0}^{-1}(\alpha)$:
|
|
|
|
|
|
\[
|
|
\frac{\partial \hat{F}_1(\hat{F}_0^{-1}(\alpha))}{\partial \hat{F}_0^{-1}(\alpha)} = \frac{\partial \frac{1}{|\mathcal{B}_1|}\sum_{\mathbf{x}_j \in \mathcal{B}_1} (\sigma_1(f_{\theta}(\mathbf{x}_j) - \tau))}{\partial \tau}
|
|
\]
|
|
|
|
\[
|
|
= -\frac{1}{|\mathcal{B}_1|}\sum_{\mathbf{x}_j \in \mathcal{B}_1}\sigma_1'(f_{\theta}(\mathbf{x}_j) - \tau)
|
|
\]
|
|
|
|
At this point, we must calculate the derivative of the threshold w.r.t each score from the false population.
|
|
|
|
Unfortunately for us, this involves calculating the derivative of a calculation we computed numerically.
|
|
|
|
In order to derive this, we can utilize our earlier computation, along with the inverse derivative rule:
|
|
|
|
\[
|
|
f'(x) = \frac{1}{f^{-1}'(f(x))}
|
|
\]
|
|
|
|
This equation shows the relationship between a function's derivative and its inverse. It shows that we may use the derivative of the
|
|
inverse function. However, in order to do this, we must first define what is the inverse we are looking for.
|
|
|
|
As a reminder we are trying to calculate $\frac{\partial \hat{F}_0^{-1}(\alpha ; \mathcal{B}_0)}{\partial f_\theta(\mathbf{x}_i)}$. That is, we have a function that takes $\alpha, \mathcal{B}_0$ as parameters and calculates
|
|
our threshold $\tau$. We are trying to calculate the derivative of this function w.r.t $\mathbf{x}_i$.
|
|
|
|
Therefore, we must invert the function in order to calculate $\mathbf{x}_i$ given $\alpha, \tau$ and all $\mathcal{B}_0 \backslash \{\mathbf{x}_i\}$.
|
|
|
|
This inverse equation is given by:
|
|
|
|
\[
|
|
\tau - \sigma^{-1}\left( |\mathcal{B}_0|\cdot\alpha - \sum_{\mathbf{x}_j \in \mathcal{B}_0 ; j \neq i}\sigma(\tau - f_{\theta}(\mathbf{x}_j))\right)
|
|
\]
|
|
|
|
Applying the inverse derivative rule to this inverse:
|
|
|
|
\[
|
|
\frac{\partial \hat{F}_0^{-1}(\alpha ; \mathcal{B}_0)}{\partial f_\theta(\mathbf{x}_i)} = \frac{1}{\frac{\partial \left(\tau - \sigma^{-1}\left( |\mathcal{B}_0|\cdot \alpha - \sum_{\mathbf{x}_j \in \mathcal{B}_0 ; j \neq i}\sigma(\tau - f_{\theta}(\mathbf{x}_j))\right)\right)}{\partial \tau}}
|
|
\]
|
|
|
|
Applying the chain rule:
|
|
|
|
|
|
\[
|
|
= \frac{1}{\sigma^{-1}'\left( |\mathcal{B}_0|\cdot \alpha - \sum_{\mathbf{x}_j \in \mathcal{B}_0 ; j \neq i}\sigma(\tau - f_{\theta}(\mathbf{x}_j))\right)\cdot\left( \sum_{\mathbf{x}_j \in \mathcal{B}_0 ; j \neq i}\sigma'(\tau - f_{\theta}(\mathbf{x}_j))\right) + 1}
|
|
\]
|
|
|
|
Remembering the calculation of $\alpha$ we get:
|
|
|
|
\[
|
|
\alpha = \sum_{\mathbf{x}_j \in \mathcal{B}_0}\sigma(\tau - f_{\theta}(\mathbf{x}_j)) \Rightarrow
|
|
\]
|
|
|
|
\[
|
|
|\mathcal{B}_0|\cdot \alpha - \sum_{\mathbf{x}_j \in \mathcal{B}_0 ; j \neq i}\sigma(\tau - f_{\theta}(\mathbf{x}_j)) = \sigma(\tau - f_{\theta}(\mathbf{x}_i))
|
|
\]
|
|
|
|
Therefore, our derivative to calculate from before is equal to:
|
|
|
|
|
|
\[
|
|
\frac{\partial \hat{F}_0^{-1}(\alpha ; \mathcal{B}_0)}{\partial f_\theta(\mathbf{x}_i)} = \frac{1}{\sigma^{-1}'\left( \sigma(\tau - f_{\theta}(\mathbf{x}_i)) \right)\cdot\left( \sum_{\mathbf{x}_j \in \mathcal{B}_0 ; j \neq i}\sigma'(\tau - f_{\theta}(\mathbf{x}_j))\right) + 1 }
|
|
\]
|
|
|
|
Substituting $\sigma^{-1}'(\sigma(u)) = \frac{1}{\sigma'(u)}$ and multiplying numerator and denominator by $\sigma'(\tau - f_{\theta}(\mathbf{x}_i))$:
|
|
|
|
\[
|
|
\frac{\partial \hat{F}_0^{-1}(\alpha ; \mathcal{B}_0)}{\partial f_\theta(\mathbf{x}_i)} = \frac{\sigma'(\tau - f_{\theta}(\mathbf{x}_i))}{\displaystyle\sum_{\mathbf{x}_j \in \mathcal{B}_0}\sigma'(\tau - f_{\theta}(\mathbf{x}_j))}
|
|
\]
|
|
|
|
Which we can plug back into the ROLL derivation framework.
|
|
|
|
%------------------------------------------------
|
|
|
|
\section{Implementation Considerations}
|
|
\label{sec:roll-implementation}
|
|
|
|
\subsection{Numerical Stability and the Scaling Trick}
|
|
\label{sec:roll-numerical-stability}
|
|
|
|
% The KDE loss involves sums of kernel evaluations that can span many orders of
|
|
% magnitude, leading to floating-point underflow or overflow during training.
|
|
% Describe the scaling trick that normalizes these sums to a numerically stable
|
|
% range without changing the gradient direction, and show that it is equivalent
|
|
% to computing in log-space with a max-subtraction stabilizer (analogous to the
|
|
% log-sum-exp trick).
|
|
|
|
|
|
\subsection{Bandwidth Estimation for KDE}
|
|
\label{sec:kde-bandwidth}
|
|
|
|
% The KDE bandwidth $h$ controls how smoothly the score distribution is estimated:
|
|
% too small and the loss becomes noisy; too large and the gradient signal is
|
|
% washed out. Describe the bandwidth estimation strategy used in ROLL (e.g.
|
|
% Silverman's rule, a learned or scheduled $h$), justify the choice, and discuss
|
|
% sensitivity of results to this hyperparameter.
|
|
% NOTE - not silverman's rule, we do use scheduling, but we also use different
|
|
% estimator - see impl.
|
|
|
|
|
|
% TODO: algorithm box — full training loop (forward → ROLL loss → custom backward → weight update)
|
|
|
|
|
|
%%% Local Variables:
|
|
%%% mode: latex
|
|
%%% TeX-master: "../../main"
|
|
%%% TeX-engine: xetex
|
|
%%% End:
|