This appendix gives a Neyman-Pearson-theoretic account of why order-dependent losses (objectives that depend on the ranking of scores rather than on comparing each score
to a fixed threshold) avoid a real limitation of class-weighted
per-sample losses. \Cref{sec:np-class-weighting} shows that a weighted per-sample loss
 commits to a Neyman-Pearson threshold $\eta$ chosen indirectly through the class
weights.
\Cref{sec:np-noise-invariance} shows that under label noise this implicit $\eta$ is
further, unpredictably distorted. Neither limitation is specific to \textsc{Roll}: they
apply to any per-sample-weighted loss, and the way around them (targeting an operating
point or optimizing AUC directly, without ever forming a class weight or comparing to a
fixed $\eta$) applies equally to \textsc{Roll}'s TPR@FPR and ROLL-AOC objectives and to
other AUC-optimizing methods such as \textsc{LibAUC}~\cite{yuan2021auc}.

\section{Class Weighting as an Implicit Choice of $\eta$}
\label{sec:np-class-weighting}

Recall from \Cref{sec:neyman-pearson} that the Neyman-Pearson lemma characterizes the
optimal test at a fixed false-positive rate as thresholding the likelihood ratio
$p(\mathbf{x}\mid y{=}1)/p(\mathbf{x}\mid y{=}0)$ against some constant $\eta$: a single
threshold, fixed once for the whole test, the same for every $\mathbf{x}$. We will not
need $\eta$'s exact value below, only that it exists, that it is one constant shared by
every sample, and that a weighted per-sample loss determines it indirectly through the
class weights.

\Cref{sec:imbalanced-tpr-fpr} noted that per-sample losses typically correct for class
imbalance by multiplying the loss on one class by a weight. This practice has a direct
reading in Neyman-Pearson terms: for a weighted per-sample loss, minimizing the pointwise
loss and thresholding the result is equivalent to thresholding the posterior
odds,\footnote{For weighted cross-entropy with weights $w_1,w_0$, minimizing
$w_1 P(y{=}1\mid\mathbf{x})(-\log q) + w_0 P(y{=}0\mid\mathbf{x})(-\log(1-q))$ over the
predicted probability $q$ gives $q^\star = w_1 P(y{=}1\mid\mathbf{x}) \big/
\bigl[w_1 P(y{=}1\mid\mathbf{x}) + w_0 P(y{=}0\mid\mathbf{x})\bigr]$. Thresholding
$q^\star > \tfrac12$ is equivalent to $w_1 P(y{=}1\mid\mathbf{x}) > w_0 P(y{=}0\mid\mathbf{x})$,
i.e. to the threshold on $O(\mathbf{x})$ stated below.} $O(\mathbf{x}) = P(y{=}1\mid\mathbf{x})/P(y{=}0\mid\mathbf{x})$,
against a constant set by the weight ratio:
\[
  O(\mathbf{x}) > \eta, \qquad \eta = \frac{w_0}{w_1}.
\]
This exact value of $\eta$ follows from this particular (weighted cross-entropy) loss;
other weighted per-sample losses would fix it slightly differently. What matters below is
not the exact value but the qualitative fact common to all of them: $\eta$ is one constant,
the same for every sample, that moves with the chosen weight ratio. Choosing per-class
weights is therefore, knowingly or not, equivalent to choosing $\eta$ indirectly, and
without prior knowledge of the actual separability of the data.

Concretely, for any point $\mathbf{x}$, this means the classifier ends up treating
$\mathbf{x}$ as positive or negative purely according to which side of $\eta$ its own
posterior odds $O(\mathbf{x})$ falls on: samples with $O(\mathbf{x}) > \eta$ are pushed
toward the positive class, samples with $O(\mathbf{x}) < \eta$ toward the negative class,
regardless of how separable $\mathbf{x}$ actually is from the other class. Two samples with
nearly identical posterior odds straddling $\eta$ can therefore end up on opposite sides of
the decision rule purely because of where $\eta$ happens to land, and since $\eta$ is
fixed entirely by the chosen weight ratio, it is the weight choice, not any property of the
data, that decides the fate of every such borderline sample. This is the central difference
from an order-dependent loss: a per-sample loss must commit to some $\eta$ before training
even begins, and every sample's outcome is filtered through that fixed, weight-determined
cutoff.

Ordering-based objectives, such as our ROLL instantiations, avoid this altogether: rather
than fixing $\eta$ via class weights, they optimize for AUC or a target operating point
directly, and whatever $\eta$ this implies is discovered from the fitted score
distributions during training. No class weight, and no assumption linking class counts to
$\eta$, is needed. The same is true of any objective that optimizes AUC or a fixed
operating point directly, including \textsc{LibAUC}: none of these ever form a
class-weight ratio or compare a probability to a fixed $\eta$ in the first place.

\section{Class Weighting Under Label Noise: An Implicit, Distorted $\eta$}
\label{sec:np-noise-invariance}

\Cref{sec:np-class-weighting} showed that a weighted per-sample loss trained on clean
labels implicitly targets a threshold $\eta$ on the true posterior odds
$O(\mathbf{x}) = P(y{=}1\mid\mathbf{x})/P(y{=}0\mid\mathbf{x})$, set by the weight ratio.
In practice, however, training data is rarely clean. This section shows that under label
noise, the threshold actually reached is not the one chosen, regardless of how $\eta$
itself is defined.

Suppose labels are corrupted by class-conditional noise, the same model used in
\Cref{sec:noisy-labels} and in this thesis's noise experiments: each label is flipped with
probability $\rho \in [0,\tfrac12)$ depending only on its true class, independent of
$\mathbf{x}$. A model trained on the resulting noisy labels $\tilde y$ estimates the noisy
posterior $P(\tilde y{=}1\mid\mathbf{x})$, not the true one, so a weight ratio $w = w_0/w_1$
now thresholds $P(\tilde y{=}1\mid\mathbf{x})/P(\tilde y{=}0\mid\mathbf{x})$, not
$O(\mathbf{x})$. These two quantities are related by a fixed function of $\rho$
alone,\footnote{Since the noise depends only on $y$, $P(\tilde y{=}1\mid\mathbf{x}) =
(1-\rho)P(y{=}1\mid\mathbf{x}) + \rho P(y{=}0\mid\mathbf{x})$, and similarly
$P(\tilde y{=}0\mid\mathbf{x}) = \rho P(y{=}1\mid\mathbf{x}) + (1-\rho)P(y{=}0\mid\mathbf{x})$.
Dividing both by $P(y{=}0\mid\mathbf{x})$ and taking the ratio gives
$P(\tilde y{=}1\mid\mathbf{x})/P(\tilde y{=}0\mid\mathbf{x}) = h(O(\mathbf{x}))$ with
$h(O) = [\rho + O(1-\rho)] / [(1-\rho) + O\rho]$, and $h'(O) = (1-2\rho) \big/
[(1-\rho)+O\rho]^2 > 0$ for $\rho < \tfrac12$, independent of $O$.} namely
\[
  \frac{P(\tilde y{=}1\mid\mathbf{x})}{P(\tilde y{=}0\mid\mathbf{x})} = h\bigl(O(\mathbf{x})\bigr),
  \qquad
  h(O) = \frac{\rho + O(1-\rho)}{(1-\rho) + O\rho},
\]
and $h$ is strictly increasing for every $\rho < \tfrac12$.

Training with weight ratio $w$ on the noisy labels therefore does not threshold
$O(\mathbf{x})$ at $w$, it thresholds $h(O(\mathbf{x}))$ at $w$, which is equivalent to
thresholding the \emph{true} posterior odds at
\[
  \eta_{\text{true}} = h^{-1}(w) = \frac{w(1-\rho) - \rho}{(1-\rho) - w\rho}.
\]
Whatever $w$ is chosen, the classifier ends up targeting $\eta_{\text{true}}$, not $w$,
and $\eta_{\text{true}}$ depends on $\rho$, which is generally unknown. Choosing $w$ no
longer means choosing $\eta$; it means choosing $\eta$ up to an unknown,
noise-dependent distortion.

\textsc{Roll}'s objective, and AUC-optimizing objectives more generally, sidestep
this mechanism entirely: they never form a weight ratio or compare a probability to a
fixed $\eta$. They target a metric directly from whichever labels they are given.

This relies specifically on the noise being class-conditional, $\rho$ a function of $y$
alone, independent of $\mathbf{x}$, as assumed throughout \Cref{sec:noisy-labels} and in
this thesis's noise experiments. The argument above only shows that this particular
distortion has this closed form under that assumption.
This does not show that likelihood-ratio ranking actually fails
under instance-dependent noise, only that the analysis given here does not, by itself,
extend to that setting.