From f4de5a9e8a027fd02af06aa31722a0f3e20bc05e Mon Sep 17 00:00:00 2001 From: Aner Zakobar Date: Wed, 9 Sep 2026 18:34:20 +0300 Subject: [PATCH] Fixes fixes fixes --- content/appendixs/appendix_kde_bisection.tex | 85 +++++++- content/appendixs/appendix_kde_grad.tex | 35 ++++ .../appendix_score_normalization.tex | 143 +++++++++++++ content/method/method.tex | 188 ++++++++++++------ main.tex | 5 + 5 files changed, 391 insertions(+), 65 deletions(-) create mode 100644 content/appendixs/appendix_score_normalization.tex diff --git a/content/appendixs/appendix_kde_bisection.tex b/content/appendixs/appendix_kde_bisection.tex index 7c930f2..978a8fd 100644 --- a/content/appendixs/appendix_kde_bisection.tex +++ b/content/appendixs/appendix_kde_bisection.tex @@ -2,15 +2,51 @@ We describe the bisection procedure used to compute $\tau = \hat{F}_0^{-1}(1-\al in the KDE-ROLL forward pass (\Cref{sec:roll-kde-forward}). \paragraph{Bracket initialisation.} -Because $\sigma(u;\,v) \to 0$ exponentially as $u \to -\infty$ and $\sigma(u;\,v) \to 1$ -as $u \to +\infty$, with tails decaying within roughly $10/v_0$ of the data range, the -interval +A fixed margin of $10/v_0 = 10h_0$ beyond the data range, \[ - \mathrm{lo} = \min(\mathbf{s}^{(0)}) - \tfrac{10}{v_0}, + \mathrm{lo} = \min(\mathbf{s}^{(0)}) - 10h_0, \qquad - \mathrm{hi} = \max(\mathbf{s}^{(0)}) + \tfrac{10}{v_0} + \mathrm{hi} = \max(\mathbf{s}^{(0)}) + 10h_0, \] -is guaranteed to satisfy $\hat{F}_0(\mathrm{lo}) < 1-\alpha < \hat{F}_0(\mathrm{hi})$. +only guarantees $\hat{F}_0(\mathrm{lo}) < 1-\alpha < \hat{F}_0(\mathrm{hi})$ for +$1-\alpha \in (\sigma(-10;1),\,\sigma(10;1)) \approx (4.5\times10^{-5},\,1-4.5\times10^{-5})$, +since that is the range the sigmoid tails have decayed into at distance $10h_0$ — not for +every $\alpha \in (0,1)$ as a fixed margin might suggest. + +Instead we use an exact, target-dependent bracket. For any $p \in (0,1)$, define +\[ + \mathrm{lo}(p) = \min(\mathbf{s}^{(0)}) + h_0\log\!\frac{p}{1-p}, + \qquad + \mathrm{hi}(p) = \max(\mathbf{s}^{(0)}) + h_0\log\!\frac{p}{1-p}. +\] + +\begin{proposition}[Exact bracket containment] +\label{prop:bisection-bracket} +If $\min(\mathbf{s}^{(0)}) < \max(\mathbf{s}^{(0)})$, then for every $p \in (0,1)$, +\[ + \hat{F}_0(\mathrm{lo}(p)) < p < \hat{F}_0(\mathrm{hi}(p)). +\] +\end{proposition} + +\begin{proof} +$\sigma(u;v_0)$ is strictly increasing in $u$ (its derivative $v_0\,\sigma(1-\sigma) > 0$). +Direct substitution, using $v_0 h_0 = 1$, gives +$\sigma\bigl(\mathrm{lo}(p) - \min(\mathbf{s}^{(0)});\,v_0\bigr) + = \sigma\bigl(\log\tfrac{p}{1-p};\,1\bigr) + = \bigl(1+\tfrac{1-p}{p}\bigr)^{-1} = p$ +exactly. For any other score $s_j > \min(\mathbf{s}^{(0)})$, +$\mathrm{lo}(p) - s_j < \mathrm{lo}(p) - \min(\mathbf{s}^{(0)})$, so monotonicity gives +$\sigma(\mathrm{lo}(p)-s_j;v_0) < p$. Averaging over $\mathcal{B}_0$, with at least one +score exceeding the minimum, $\hat{F}_0(\mathrm{lo}(p)) < p$ strictly. The symmetric +argument at $\max(\mathbf{s}^{(0)})$ gives $\hat{F}_0(\mathrm{hi}(p)) > p$. +\end{proof} + +Setting $p = 1-\alpha$ gives a bracket valid for every $\alpha \in (0,1)$, unlike the +fixed-margin version above. The degenerate case $\min(\mathbf{s}^{(0)}) = \max(\mathbf{s}^{(0)})$ +— a batch with constant negative-class scores — is excluded from +\Cref{prop:bisection-bracket} above; it also breaks the Silverman bandwidth estimate +itself ($h_0 = 0$), and is addressed as part of the degenerate-batch handling still +outstanding for \Cref{sec:kde-bandwidth}. \paragraph{Bisection iterations.} At each step we evaluate the midpoint, @@ -24,7 +60,40 @@ and update the bracket: \mathrm{hi} \leftarrow \tau_{\mathrm{mid}} & \text{otherwise.} \end{cases} \] -Iteration continues until $\lvert\hat{F}_0(\tau_{\mathrm{mid}};\,\mathcal{B}_0)-(1-\alpha)\rvert < \varepsilon$ -(with $\varepsilon = 10^{-4}$), or until a maximum iteration count is reached. +\paragraph{Stopping criterion.} +A residual check on $\hat{F}_0$ alone, $\lvert\hat{F}_0(\tau_{\mathrm{mid}})-(1-\alpha)\rvert < \varepsilon$, +certifies that the \emph{threshold} $\tau$ has been located accurately, but not that the +\emph{loss} $\hat{F}_1(\tau)$ has: if $\hat{F}_0$ is comparatively flat near the target +(e.g.\ $v_0 \ll v_1$, or the negative-class kernel density is low at $\tau$), a wide range +of $\tau$ values can satisfy the residual bound while $\hat{F}_1(\tau)$ still varies +substantially across that range. + +Instead, we exploit that $\hat{F}_1$ is itself a CDF — monotonically increasing — and +that the bracket invariant $\hat{F}_0(\mathrm{lo}) < 1-\alpha < \hat{F}_0(\mathrm{hi})$ +keeps the true root $\tau^\star$ inside $[\mathrm{lo},\mathrm{hi}]$ throughout. Hence +\[ + \hat{F}_1(\mathrm{lo}) \;\leq\; \hat{F}_1(\tau^\star) \;\leq\; \hat{F}_1(\mathrm{hi}) +\] +regardless of exactly where $\tau^\star$ falls in the bracket. Iteration continues until +\begin{equation} + \hat{F}_1(\mathrm{hi};\,\mathcal{B}_1) - \hat{F}_1(\mathrm{lo};\,\mathcal{B}_1) < \varepsilon_L + \label{eq:bisection-loss-cert} +\end{equation} +(with $\varepsilon_L = 10^{-4}$), or until a maximum iteration count is reached. Unlike a +Lipschitz-based bound (e.g.\ using $\max_u \sigma'(u;v_1) = v_1/4$ to bound $\hat{F}_1$'s +slope and stopping purely on bracket width), \eqref{eq:bisection-loss-cert} is exact +rather than a worst-case estimate, at the cost of two extra $\hat{F}_1$ evaluations per +iteration. + +\paragraph{Why the residual-only rule fails.} Take $\mathcal{B}_0 = \{-30,-25,5,45,60\}$ +with $v_0 = 0.01$ (wide, smooth kernel) and $\alpha=\tfrac12$; the residual-only rule with +$\varepsilon=10^{-4}$ stops after 8 iterations at $\tau \approx 10.918$, versus the +converged root $\tau^\star \approx 10.932$, a gap of only $0.014$ in $\tau$. But with a +sharp positive-class kernel ($v_1 = 1000$) and a cluster of $\mathcal{B}_1$ sitting inside +that gap, $\hat{F}_1(\tau)$ differs by $0.40$ between the two: the reported loss is wrong +by nearly half its range, despite the threshold residual being $8\times$ tighter than +$\varepsilon$. Certifying via \eqref{eq:bisection-loss-cert} instead catches this +directly, converging to $\hat{F}_1$ accurate to $3\times 10^{-5}$. + The method converges unconditionally and requires no derivative evaluation, in contrast to the Newton--Raphson alternative documented in Appendix~\ref{appendix:kde-nr}. diff --git a/content/appendixs/appendix_kde_grad.tex b/content/appendixs/appendix_kde_grad.tex index aba0213..43652b9 100644 --- a/content/appendixs/appendix_kde_grad.tex +++ b/content/appendixs/appendix_kde_grad.tex @@ -140,3 +140,38 @@ entirely in terms of sigmoid values already cached from the forward pass: Substituting \eqref{eq:kde-grad-y1}, \eqref{eq:kde-dF1-dtau}, and \eqref{eq:kde-grad-tau} into \eqref{eq:roll-gradient} yields \eqref{eq:kde-grad-combined}. + +\paragraph{Numerical stability of the efficient form.} +\label{sec:kde-grad-stability} + +\eqref{eq:kde-grad-tau-efficient} is vulnerable in floating point: $\sigma_0(\tau-f_i)$ +rounds to exactly $1$ once its argument $v_0(\tau-f_i)$ exceeds $\approx 17$ in single +precision, so $1-\sigma_0(\tau-f_i)$ evaluates to exactly $0$. +$\sigma_0'$ itself stays representable until the argument exceeds $\approx 88$, meaning we are giving +up orders of magnitude of valid numerical representation. In extreme cases, such as those +where a class is well-separated from the threshold $\tau$, and multiple scores fall +over $5$ deviations away from said threshold, it is possible for many multiple scorewise +gradients to fail, returning a value of $0$ instead of returning a low magnitude, yet usable, gradient. + +The ratio being computed is unchanged; only the arithmetic path is. Writing +$z_i = v_0(\tau-f_\theta(\mathbf{x}_i))$ and using +$\sigma'(z)=\sigma(z)(1-\sigma(z))$ together with the stable log-sigmoid identity +$\log\sigma(z) = -\log(1+e^{-z})$: +\begin{equation} + \log\sigma_0'(z_i) = -|z_i| - 2\log\bigl(1+e^{-|z_i|}\bigr) =: \ell_i + \label{eq:kde-grad-log-sigma-prime} +\end{equation} +(valid for either sign of $z_i$, since $\sigma'$ is even), which is safe to evaluate for +any $|z_i|$: the only large-magnitude term is the linear $-|z_i|$, while +$\log(1+e^{-|z_i|}) \in [0,\log 2]$ regardless of $|z_i|$. The normalised weight is then +recovered by exponentiating exactly once, inside a $\operatorname{softmax}$-style +normalisation that is itself overflow-safe: +\begin{equation} + \frac{\sigma_0'(z_i)}{\sum_j \sigma_0'(z_j)} = \exp\Bigl(\ell_i - \operatorname*{logsumexp}_j \ell_j\Bigr). + \label{eq:kde-grad-softmax} +\end{equation} +This has no failure point short of the representable exponent range. The denominator +can never collapse as it can under \eqref{eq:kde-grad-tau-efficient}. +\eqref{eq:kde-grad-tau-efficient} is retained above as the clearest statement of the +derivation; \eqref{eq:kde-grad-log-sigma-prime}, \eqref{eq:kde-grad-softmax} is what is +implemented. diff --git a/content/appendixs/appendix_score_normalization.tex b/content/appendixs/appendix_score_normalization.tex new file mode 100644 index 0000000..d05493f --- /dev/null +++ b/content/appendixs/appendix_score_normalization.tex @@ -0,0 +1,143 @@ +This appendix proves \Cref{prop:normalization-invariance} (\Cref{sec:score-normalization-proof}), +shows numerically that ISJ does not satisfy its scale-equivariance hypothesis +(\Cref{sec:score-normalization-isj}), and derives the chain-rule term that a frozen +normalisation constant $\hat\sigma_\mathcal{B}$ omits, together with a concrete instance +where omitting it is necessary to preserve \Cref{obs:gradient-sign-correctness} +(\Cref{sec:score-normalization-missing-term}). + +\section{Proof of Proposition~\ref{prop:normalization-invariance}} +\label{sec:score-normalization-proof} + +Write $s$ for the raw batch scores, $c := \hat\sigma_\mathcal{B}(s) > 0$, and +$\tilde{s} = s/c$. Let $v_k^A := 1/\hat{h}(s^{(k)})$ (raw path) and +$v_k^B := 1/\hat{h}(\tilde{s}^{(k)})$ (normalised path), and let $\tau^A, \tau^B$ solve +$\hat{F}_0(\tau; s^{(0)}, v_0^A) = 1-\alpha$ and +$\hat{F}_0(\tau; \tilde{s}^{(0)}, v_0^B) = 1-\alpha$ respectively. + +By hypothesis $\hat{h}(\tilde{s}^{(k)}) = \hat{h}(s^{(k)}/c) = \hat{h}(s^{(k)})/c$, so +\begin{equation} + v_k^B = c \cdot v_k^A. + \label{eq:sn-v-scale} +\end{equation} +Substituting $\tau = \tau^A/c$ into $\hat{F}_0(\tau; \tilde{s}^{(0)}, v_0^B)$: +\[ + v_0^B\bigl(\tau^A/c - \tilde{s}_j^{(0)}\bigr) + = c v_0^A\Bigl(\frac{\tau^A - s_j^{(0)}}{c}\Bigr) + = v_0^A\bigl(\tau^A - s_j^{(0)}\bigr) +\] +for every $j$, so $\hat{F}_0(\tau^A/c; \tilde{s}^{(0)}, v_0^B) = \hat{F}_0(\tau^A; s^{(0)}, v_0^A) = 1-\alpha$ +— i.e.\ $\tau^A/c$ solves the normalised-path equation, and by uniqueness of the +bisection root, $\tau^B = \tau^A/c$ exactly. The same substitution gives, for every $j$, +\begin{equation} + v_k^B\bigl(\tau^B - \tilde{s}_j^{(k)}\bigr) = v_k^A\bigl(\tau^A - s_j^{(k)}\bigr), + \label{eq:sn-argument-match} +\end{equation} +i.e.\ every sigmoid-kernel argument is identical between the two paths. Consequently +$\sigma(\cdot)$ evaluates identically in both paths (so the loss value matches: +$\hat{F}_1(\tau^B;\tilde{s}^{(1)},v_1^B) = \hat{F}_1(\tau^A;s^{(1)},v_1^A)$), while +$\sigma_k'(u;v) = v\,\sigma(u;v)(1-\sigma(u;v))$ picks up exactly a factor of +$v_k^B/v_k^A = c$ by \eqref{eq:sn-v-scale}: +\begin{equation} + \sigma_k'\bigl(\tau^B - \tilde{s}_j^{(k)}; v_k^B\bigr) + = c \cdot \sigma_k'\bigl(\tau^A - s_j^{(k)}; v_k^A\bigr). + \label{eq:sn-derivative-scale} +\end{equation} + +\paragraph{Case $y_i=1$.} By \eqref{eq:kde-grad-y1} and the frozen-$c$ chain rule +$\partial/\partial s_i = (1/c)\,\partial/\partial\tilde{s}_i$: +\[ + \frac{\partial L^B}{\partial s_i} + = \frac{1}{c}\left(-\frac{1}{|\mathcal{B}_1|}\sigma_1'\bigl(\tau^B-\tilde{s}_i;v_1^B\bigr)\right) + \overset{\eqref{eq:sn-derivative-scale}}{=} + \frac{1}{c}\left(-\frac{c}{|\mathcal{B}_1|}\sigma_1'\bigl(\tau^A-s_i;v_1^A\bigr)\right) + = \frac{\partial L^A}{\partial s_i}. +\] + +\paragraph{Case $y_i=0$.} By \eqref{eq:kde-grad-combined}, the gradient is +$\frac{1}{c}\cdot D_1^B \cdot +\dfrac{\sigma_0'(\tau^B-\tilde{s}_i;v_0^B)}{\sum_j \sigma_0'(\tau^B-\tilde{s}_j;v_0^B)}$, +where $D_1^B := \frac{1}{|\mathcal{B}_1|}\sum_j\sigma_1'(\tau^B-\tilde{s}_j;v_1^B) = c\cdot D_1^A$ +by \eqref{eq:sn-derivative-scale}. The ratio's numerator and denominator both pick up the +same factor $c$ (again by \eqref{eq:sn-derivative-scale}), so the ratio itself is +unchanged: +\[ + \frac{\partial L^B}{\partial s_i} + = \frac{1}{c}\cdot c D_1^A \cdot + \frac{\sigma_0'(\tau^A-s_i;v_0^A)}{\sum_j\sigma_0'(\tau^A-s_j;v_0^A)} + = D_1^A \cdot \frac{\sigma_0'(\tau^A-s_i;v_0^A)}{\sum_j\sigma_0'(\tau^A-s_j;v_0^A)} + = \frac{\partial L^A}{\partial s_i}. +\] + +In both cases the outer $1/c$ from the chain rule is exactly cancelled by the $c$ +introduced into $v_k^B$, confirming \Cref{prop:normalization-invariance}. (Verified +numerically to float32 precision — $\max_i|\partial L^A/\partial s_i - \partial L^B/\partial s_i| +\approx 5.6\times10^{-9}$ on a random batch, using Silverman's rule for $\hat{h}$.) + +\section{ISJ Does Not Satisfy the Hypothesis} +\label{sec:score-normalization-isj} + +Silverman's rule satisfies $\hat{h}(a\cdot x)=a\cdot\hat{h}(x)$ exactly, since +$\hat{h}\propto\hat\sigma(x)$ and $\hat\sigma(ax)=a\hat\sigma(x)$. ISJ, computed via +\texttt{KDEpy}'s \texttt{improved\_sheather\_jones} (the primary estimator, +\Cref{sec:kde-bandwidth}), does not: rescaling a fixed sample by $a$ and re-running ISJ +gives $\hat{h}(ax)/a$ ratios that vary with $a$ rather than staying constant: + +\begin{center} +\begin{tabular}{rrr} +\toprule +scale $a$ & $\hat{h}_{\text{Silverman}}(ax)/a$ & $\hat{h}_{\text{ISJ}}(ax)/a$ \\ +\midrule +$10^{-6}$ & $0.353732$ & $0.001157$ \\ +$10^{-3}$ & $0.353732$ & $0.001155$ \\ +$10^{0}$ & $0.353732$ & $0.116810$ \\ +$10^{3}$ & $0.353732$ & $0.217671$ \\ +$10^{6}$ & $0.353732$ & $0.217671$ \\ +\bottomrule +\end{tabular} +\end{center} + +Consequently, for a batch where ISJ (rather than the Silverman fallback) is used, +neither the loss value nor the gradient is invariant to normalisation: a directly +measured example (random batch, $n{=}10$) gives loss $0.734$ on raw scores versus +$0.767$ on normalised scores, and a maximum per-sample gradient difference of $0.014$ +— both far from the exact invariance \Cref{prop:normalization-invariance} guarantees +for Silverman. This is the genuine numerical-stability role of normalisation for the +method as actually used: not ``preventing overflow via an unchanged, merely-rescaled +argument'' (\Cref{prop:normalization-invariance} shows that mechanism is a no-op under +exact scale-equivariance), but pinning ISJ's input to the fixed scale regime — roughly +unit variance — it behaves consistently at, rather than leaving it to whatever scale the +model's raw outputs happen to have at a given point in training. + +\section{The Missing Chain-Rule Term for a Differentiated $\hat\sigma_\mathcal{B}$} +\label{sec:score-normalization-missing-term} + +\Cref{sec:roll-numerical-stability} treats $c=\hat\sigma_\mathcal{B}$ as frozen: +gradients with respect to normalised scores are multiplied by $1/c$ alone. Since $c$ is +itself a function of every score in the batch, $\tilde{s}_j = s_j/c(s_1,\ldots,s_n)$, +the fully differentiated chain rule is +\begin{equation} + \frac{\partial L}{\partial s_i} + = \sum_j \frac{\partial L}{\partial \tilde{s}_j}\frac{\partial \tilde{s}_j}{\partial s_i} + = \frac{1}{c}\frac{\partial L}{\partial \tilde{s}_i} + \;-\; \frac{1}{c^2}\frac{\partial c}{\partial s_i}\sum_j s_j\, q_j, + \qquad q_j := \frac{\partial L}{\partial \tilde{s}_j}, + \label{eq:sn-missing-term} +\end{equation} +using $\partial \tilde{s}_j/\partial s_i = \delta_{ij}/c - (s_j/c^2)\,\partial c/\partial s_i$. +For the sample standard deviation used in \Cref{sec:roll-numerical-stability}, +$\partial c/\partial s_i = (s_i-\bar{s})/((n-1)c)$. The frozen-$c$ convention sets +$\partial c/\partial s_i := 0$ by declaration, which is exactly why the implemented +gradient (\Cref{sec:roll-numerical-stability}) has no such term. + +If instead $\hat\sigma_\mathcal{B}$ is differentiated through, the term in +\eqref{eq:sn-missing-term} does not generally vanish. Unlike the bandwidth case, there +is no exact-cancellation argument available here: the symmetry that would zero it out +via Euler's identity ($\sum_j \tilde{s}_j\, q_j = 0$, from $L$ being homogeneous of +degree $0$ in $\tilde{s}$) requires the bandwidth to be differentiated through the same +rescaling too — which \Cref{sec:roll-kde-backward} deliberately does not do. Numerically +(random batch, $n{=}10$, one $y_i{=}1$ sample): the frozen-$c$ gradient for that sample +is $-0.00181$ — correctly satisfying \Cref{obs:gradient-sign-correctness} ($\leq 0$ +required) — while the fully-differentiated-$c$ gradient for the same sample is +$+0.00020$, violating it. Freezing $c$ is therefore adopted for the same reason as +freezing $v_0,v_1$ (Appendix~\ref{appendix:bandwidth-freeze}): the alternative is not +merely more complex, it can produce gradients of the wrong sign. diff --git a/content/method/method.tex b/content/method/method.tex index bed6042..491f1c6 100644 --- a/content/method/method.tex +++ b/content/method/method.tex @@ -286,20 +286,32 @@ Direct integration is not tractable, but the integral is well-approximated by av \mathcal{L}_{\text{ROLL-AOC}} = \frac{1}{99}\sum_{i=1}^{99}\mathcal{L}_{\text{ROLL-TPR@FPR}}\!\left(\frac{i}{100}\right) \label{eq:roll-aoc} \end{equation} +Since $\operatorname{TPR}(\alpha)$ is monotone in $\alpha$ along the ROC curve, +$g(\alpha) := 1-\operatorname{TPR}(\alpha)$ is monotone and bounded in $[0,1]$, so +\eqref{eq:roll-aoc} approximates $\int_0^1 g(\alpha)\,d\alpha$ to within +$1/99 \approx 0.01$ by a standard Darboux-sum argument, assuming each per-threshold +term is evaluated exactly. This bound covers only the quadrature step: it does not +extend to the gradient \eqref{eq:roll-aoc-gradient}, nor does it account for the gap +between the KDE-fitted rates and the raw empirical scores (\Cref{sec:roll-instantiations}) +or any residual numerical error in the per-threshold bisection for $\tau_k$. + The gradient follows by linearity of the sum: \begin{equation} \frac{\partial \mathcal{L}_{\text{ROLL-AOC}}(f_\theta(\mathbf{X}))} - {\partial f_\theta(\mathbf{x}_i)} = \frac{1}{99}\sum_{i=1}^{99}\frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}\!\left(f_\theta(\mathbf{X}) ; \frac{i}{100}\right)} + {\partial f_\theta(\mathbf{x}_i)} = \frac{1}{99}\sum_{r=1}^{99}\frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}\!\left(f_\theta(\mathbf{X}) ; \frac{r}{100}\right)} {\partial f_\theta(\mathbf{x}_i)} \label{eq:roll-aoc-gradient} \end{equation} +(the summation index $r$ ranges over the $99$ thresholds; $i$, fixed throughout the sum, +indexes the sample being differentiated — distinct roles that \eqref{eq:roll-aoc} does +not need to distinguish, since it has no outer per-sample index to collide with.) The concrete per-threshold gradient for the KDE instantiation is derived in \Cref{sec:roll-instantiations}; gradients for the parametric cases are in \Cref{appendix:gauss-grad,appendix:beta-roll}. \Cref{fig:roll-aoc-roc} gives a geometric view of this objective. -\begin{figure}[H] +\begin{figure}[H] \centering \includegraphics[width=0.62\textwidth]{content/method/figures/roll_aoc_roc.pdf} \caption{Geometric interpretation of ROLL-AOC. The ROC curve (orange) maps @@ -312,7 +324,8 @@ The concrete per-threshold gradient for the KDE instantiation is derived in segments show the per-threshold loss contribution $1 - \operatorname{TPR}(\alpha_i) = \mathcal{L}_{\text{ROLL-TPR@FPR}}(\alpha_i)$ at each point. Averaging these contributions (99 in practice, 10 shown - for clarity) gives the value of \eqref{eq:roll-aoc}.} + for clarity) approximates the continuous integral, giving the value of + \eqref{eq:roll-aoc}.} \label{fig:roll-aoc-roc} \end{figure} @@ -335,15 +348,16 @@ Parametric families, however, impose distributional assumptions on the score out no trained model can be guaranteed to satisfy. The primary instantiation in this thesis is therefore non-parametric: Kernel Density Estimation (KDE) places no assumption on the shape of the score distribution and adapts to whatever the model produces at each training -step. The cost is a numerical CDF inversion and a custom backward pass; the gain is a -faithful surrogate for the ROC curve regardless of how scores evolve during training. +step. The cost is a numerical CDF inversion and a custom backward pass. The gain is a +surrogate that adapts to the score distribution's shape as it evolves during training, +without committing to a parametric assumption that may not hold. \subsection{KDE ROLL} \label{sec:roll-kde} KDE estimates the score distribution directly from the observed batch scores, placing no assumption on their shape. It is the primary instantiation of ROLL used throughout this -thesis: its non-parametric flexibility makes it the most faithful surrogate for the ROC +thesis: its non-parametric flexibility makes it the most adaptable surrogate for the ROC curve when score distributions are unknown, at the cost of a numerical CDF inversion and a custom backward pass evaluated 99 times per training step under ROLL-AOC. @@ -437,13 +451,11 @@ Appendix~\ref{appendix:bandwidth-freeze} shows by counterexample. % guarantees ROLL provides and under what assumptions they hold. % NOTE - these properties are auto-generated, consider suggestions. -A structural consequence of the ROLL formulation is that gradient mass is automatically -balanced between the two classes, regardless of their relative sizes in the batch. -This stands in contrast to standard losses such as cross-entropy, where the total gradient -contribution from each class is proportional to its count, making heavy class imbalance -a direct source of biased updates. Therefore, instead of needing to compensate for this -with loss-weighting per-class, fitting updates are made based equally off the results -of both classes. +A structural consequence of the ROLL formulation is that the total signed +gradient contribution from each class cancels exactly, regardless of their relative +sizes in the batch. This stands in contrast to standard losses such as cross-entropy, +where the total gradient contribution from each class is proportional to its count, +making heavy class imbalance a direct source of biased updates. \begin{proposition}[Gradient balance] \label{prop:gradient-balance} @@ -471,9 +483,15 @@ as well as the KDE-based distribution, but not for the Beta distribution. For Gaussian MLE, a uniform shift in scores moves $\mu_k$ by the same amount while leaving $\sigma_k$ unchanged, which translates $\hat{F}_k$; for KDE, each kernel term is evaluated at $\tau - f_\theta(\mathbf{x}_j)$, so a uniform shift in the $f_\theta(\mathbf{x}_j)$ -is equivalent to an equal shift in $\tau$. The condition is necessary: without it, a uniform -score shift need not produce a rigid translation of $\hat{F}_k$, so the shift argument in the -proof breaks down and the gradient sums need not be equal. +is equivalent to an equal shift in $\tau$. The condition is sufficient, not necessary: it is +what the proof uses to guarantee a rigid translation of $\hat{F}_k$, but the balance identity +can hold without it. For example, $\hat{F}_k(t;\mathbf{s}) = \Phi(t - 2\bar{s})$ (with +$\bar{s}$ the class mean) does not satisfy $\hat{F}_k(t;\mathbf{s}+\delta) = \hat{F}_k(t-\delta;\mathbf{s})$ +— shifting the scores by $\delta$ shifts $\hat{F}_k$ by $2\delta$, not $\delta$ — yet applying +this same construction to both classes still yields $\sum_{\mathcal{B}_1}\partial L/\partial f_i += -\sum_{\mathcal{B}_0}\partial L/\partial f_i$ exactly, since the extra factor of $2$ enters +both classes' gradients identically and cancels in the same shift argument. Translation +invariance is a convenient sufficient condition, not the weakest one that would do. The Beta distribution only accepts values in the interval $[0, 1]$. Since it is mapped only over an interval, the translation-invariance assumption does not hold @@ -486,6 +504,19 @@ average of 99 per-threshold objectives each satisfying \eqref{eq:grad-balance}, is preserved by linearity of summation. \end{corollary} +The identity \eqref{eq:grad-balance} constrains only the signed sum per class. It +does not by itself say that each class contributes equal gradient mass, +$\sum_i |\partial\mathcal{L}/\partial f_\theta(\mathbf{x}_i)|$. Equal gradient-mass is satisfied +when every sample within a class shares the same gradient sign, so that the signed and +absolute sums are equal in magnitude. For KDE-ROLL, +this extra property does hold (\Cref{obs:gradient-sign-correctness} below), so the +signed identity does translate into balanced gradient mass for the method used +throughout this thesis. However, notably, this does not hold true for the Gaussian formulation. + +Additionally, \Cref{prop:gradient-balance} is a statement about the per-sample loss +gradients $\partial\mathcal{L}/\partial f_\theta(\mathbf{x}_i)$ at the model's output. It +does not extend to the resulting parameter gradients $\partial\mathcal{L}/\partial\theta$, +as this depends on the model's structure. \paragraph{Gradient sign correctness.} \label{para:gradient-sign-correctness} @@ -518,28 +549,42 @@ the fully adaptive gradient violates this sign guarantee. \label{para:gradient-locality} A second structural consequence of the ROLL-KDE formulation, when optimizing at a -single operating threshold, is that gradient mass is \emph{concentrated near that -threshold $\tau$}. Samples whose scores lie close to $\tau$ receive larger gradient -updates, while samples already far on the correct side contribute very little. +single operating threshold, is that gradient magnitude concentrates near $\tau$, but +the mechanism differs between the two classes, and so does how literally the claim +holds. -This follows directly from \Cref{eq:kde-grad-combined}. For the positive class ($y_i = 1$), -the gradient magnitude is +This follows from \Cref{eq:kde-grad-combined}. For the positive class ($y_i = 1$), the +gradient magnitude is an absolute, unnormalised function of the sample's own +distance to $\tau$: \[ \left|\frac{\partial \mathcal{L}}{\partial f_\theta(\mathbf{x}_i)}\right| = \frac{1}{|\mathcal{B}_1|}\,\sigma_1'(f_\theta(\mathbf{x}_i) - \tau), \] -which is a bell-shaped kernel PDF that peaks when $f_\theta(\mathbf{x}_i) = \tau$ and -decays to zero as the score moves away from the threshold in either direction. -For the negative class ($y_i = 0$), the gradient is proportional to -$\sigma_0'(\tau - f_\theta(\mathbf{x}_i))$ (normalized over $\mathcal{B}_0$), which is -again bell-shaped and centred on $\tau$. +a bell-shaped kernel PDF that peaks when $f_\theta(\mathbf{x}_i) = \tau$ and decays to +exactly $0$ as $|f_\theta(\mathbf{x}_i)-\tau|\to\infty$, regardless of where any other +sample lies. For this class, "far from $\tau$ implies negligible gradient" holds without +qualification. -The effect is that each gradient step focuses the model's attention on the region that -directly determines the operating-point performance: samples that could plausibly be -re-ranked relative to the threshold at the current step. Samples classified with strong -confidence relative to the threshold, with scores far enough away from said threshold, -receive small gradient updates, since a small change to the score of these samples will not -bring any change to the placement of the threshold itself, or the performance at said threshold. +For the negative class ($y_i = 0$), the gradient is instead a \emph{normalised share} of +the class's total kernel mass, $\sigma_0'(\tau-f_\theta(\mathbf{x}_i)) \big/ +\sum_j \sigma_0'(\tau-f_\theta(\mathbf{x}_j))$, and this ratio doesn't vanish just +because $f_\theta(\mathbf{x}_i)$ is far from $\tau$: what determines +sample $i$'s share of the gradient mass is its position relative to the rest of $\mathcal{B}_0$, +not its absolute distance to $\tau$. If the whole negative class drifts far from $\tau$ +together, every individual $\sigma_0'$ value shrinks, but so does their sum, and the +ratio can converge to a nonzero constant no matter how far the class has drifted. +Concretely, for two negative samples at $\tau+M$ and $\tau+M+1$ (a fixed unit +separation), the closer sample's normalised weight converges, as $M\to\infty$, to +$1/(1+e^{-v_0})$, $\approx 0.731$ at $v_0=1$, not to $0$. +However, if a single score drifts far from $\tau$ while the rest of it's class scores, +stay close, its individual score will converge to zero. + +The effect, for both classes, is that each gradient step focuses the model's attention +on the samples whose re-ranking relative to $\tau$ would most change the operating-point +performance. For the positive class this is an absolute statement about each sample's +own distance to $\tau$, and for the negative class, it is a statement about each sample's +distance relative to its classmates. Typically, but not necessarily, this correlates +with its absolute distance to $\tau$. This stands in contrast to standard losses such as cross-entropy, where the per-sample gradient is determined by the predicted probability, with no direct dependence on an @@ -547,27 +592,36 @@ operating threshold. \begin{observation}[Gradient locality, single-threshold] \label{obs:gradient-locality} -For KDE-ROLL at a single operating threshold, the gradient magnitude of sample -$\mathbf{x}_i$ is proportional to the kernel PDF evaluated at the distance between -$f_\theta(\mathbf{x}_i)$ and $\tau$. Samples nearest to $\tau$ receive the largest -updates; samples far from $\tau$ in either direction contribute negligible gradient. +For KDE-ROLL at a single operating threshold: (a) a positive-class sample's gradient +magnitude is the kernel PDF evaluated at its own distance to $\tau$, and vanishes as +that distance grows, independent of the rest of the batch. (b) a negative-class +sample's gradient magnitude is its kernel-PDF value \emph{relative to the sum over +$\mathcal{B}_0$} — samples near $\tau$ receive a larger share than those far from it +\emph{within the same batch}, but this relative weighting does not by itself vanish +for an individual sample merely because the whole class is far from $\tau$. \end{observation} -\Cref{fig:threshold-locality} illustrates this for a simulated two-class setting. The +\Cref{fig:threshold-locality} illustrates this for a simulated two-class setting in +which both classes are broadly spread around $\tau$ — the typical case, and the one in +which the negative class's relative weighting (\Cref{obs:gradient-locality}b) and the +positive class's absolute decay (\Cref{obs:gradient-locality}a) happen to agree. The upper panel shows the score distributions and the operating threshold; the lower panel -shows the gradient magnitude of each sample. Both classes exhibit a clear bell-shaped -concentration of gradient mass around $\tau$, with smooth decay on either side. +shows the gradient magnitude of each sample, with a clear bell-shaped concentration +around $\tau$ for both classes in this regime. \begin{figure}[H] \centering \includegraphics[width=0.82\textwidth]{content/method/figures/threshold_weighting.pdf} - \caption{Gradient locality in KDE-ROLL. \textit{Upper}: score distributions for the - negative (blue, $y=0$) and positive (orange, $y=1$) classes; dashed line marks the - operating threshold $\tau$ at FPR $= \alpha$. \textit{Lower}: gradient magnitude + \caption{Gradient locality in KDE-ROLL, for a batch where both classes are spread + broadly around $\tau$. \textit{Upper}: score distributions for the negative (blue, + $y=0$) and positive (orange, $y=1$) classes; dashed line marks the operating + threshold $\tau$ at FPR $= \alpha$. \textit{Lower}: gradient magnitude $|\partial\mathcal{L}/\partial f_\theta(\mathbf{x})|$ for each sample. Both classes - show a bell-shaped concentration of gradient mass near $\tau$, with samples far from - the threshold contributing negligible updates. Smooth curves are the kernel PDFs scaled - for reference.} + show a bell-shaped concentration of gradient mass near $\tau$ here; for the negative + class this reflects relative weighting within $\mathcal{B}_0$ + (\Cref{obs:gradient-locality}b), which need not track absolute distance to $\tau$ if + the whole class is displaced together. Smooth curves are the kernel PDFs scaled for + reference.} \label{fig:threshold-locality} \end{figure} @@ -596,19 +650,39 @@ deviation: \qquad \hat\sigma_{\mathcal{B}} = \sqrt{\operatorname{Var}(f_\theta(\mathbf{x}_i) : (\mathbf{x}_i, y_i) \in \mathcal{B})}. \] -This keeps the values given to the sigmoid kernel ($v(\tau - \tilde{s}_j)$) -near unit scale, preventing floating-point overflow and underflow in the exponential -evaluations. +This keeps the values given to the sigmoid kernel at a consistent scale. Why that +matters depends entirely on which bandwidth estimator is in play. -The scaling has no effect on the loss value itself or derived gradients. -Silverman's rule derives the bandwidth proportionally to the data standard deviation, -$h \propto \hat\sigma \cdot n^{-1/5}$, so dividing all scores by $\hat\sigma_\mathcal{B}$ -causes the bandwidth to scale by the same factor. -The product $v \cdot (\tau - s_j)$ (the only quantity entering the sigmoid) is -therefore invariant under the rescaling. -In the backward pass, gradients with respect to the normalised scores are multiplied -by $1/\hat\sigma_\mathcal{B}$ to apply the chain rule, providing gradients mathematically -identical to those that would have been obtained without normalisation. +\begin{proposition}[Normalisation invariance under scale-equivariant bandwidth] +\label{prop:normalization-invariance} +Let $\hat{h}$ be any bandwidth estimator satisfying $\hat{h}(a\cdot x) = a\cdot\hat{h}(x)$ +for all $a>0$ (e.g.\ Silverman's rule, $h \propto \hat\sigma \cdot n^{-1/5}$). Under the +frozen-bandwidth convention of \Cref{sec:roll-kde-backward} and a frozen normalisation +constant $\hat\sigma_\mathcal{B}$, the loss value and every per-sample gradient +$\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}/\partial f_\theta(\mathbf{x}_i)$ computed from +the normalised scores $\tilde{s}$ are exactly equal to those computed directly from the +raw scores $s$. +\end{proposition} +\begin{proof} +See Appendix~\ref{appendix:score-normalization}. +\end{proof} + +For Silverman's rule, normalisation is therefore a pure floating-point reparametrisation: +it changes neither the loss nor the gradient used for training, so it cannot be what +prevents overflow for that estimator — there is nothing left for it to change. +ISJ, the primary bandwidth estimator (\Cref{sec:kde-bandwidth}), does \emph{not} satisfy +the proposition's hypothesis (Appendix~\ref{appendix:score-normalization} measures its +scale-dependence directly). For ISJ, both the loss value and the gradient genuinely +depend on whether the scores are normalised, so normalisation is doing real work: it +pins ISJ's input to the one scale regime it behaves consistently in, rather than leaving +it to drift with the model's raw output scale over training. + +As with the bandwidths $v_0,v_1$, the normalisation constant $\hat\sigma_\mathcal{B}$ is +treated as frozen (stop-gradient) in the backward pass: gradients with respect to the +normalised scores are simply multiplied by $1/\hat\sigma_\mathcal{B}$. Differentiating +fully through $\hat\sigma_\mathcal{B}$ instead introduces a further correction term that, +like the bandwidth correction of \Cref{sec:roll-kde-backward}, can violate +\Cref{obs:gradient-sign-correctness} (Appendix~\ref{appendix:score-normalization}). \subsection{Bandwidth Estimation for KDE} \label{sec:kde-bandwidth} diff --git a/main.tex b/main.tex index 80093fa..1869f05 100644 --- a/main.tex +++ b/main.tex @@ -311,6 +311,11 @@ \addtocontents{tocheb}{\contentsline{chapter}{\protect\numberline{נספח י}{\quad\qquadKDE ROLL: גרדיאנט רוחב פס אדפטיבי ומוסכמת ההקפאה}}{{\beginL\thepage\endL}}} \input{content/appendixs/appendix_bandwidth_freeze} +\chapter{Score Normalisation: Invariance, ISJ, and the Frozen Chain Rule} +\label{appendix:score-normalization} +\addtocontents{tocheb}{\contentsline{chapter}{\protect\numberline{נספח יא}{\quad\qquadנרמול ציונים: שקילות, ISJ וכלל השרשרת הקפוא}}{{\beginL\thepage\endL}}} +\input{content/appendixs/appendix_score_normalization} + \end{appendices} %~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~%