Bandwidth freeze

content/method/method.tex

@@ -22,13 +22,21 @@ The \emph{true positive rate} (TPR) and \emph{false positive rate} (FPR) at thre
   \operatorname{TPR}(\tau) &= P\!\bigl(f_\theta(\mathbf{x}) > \tau \mid y = 1\bigr), \\
   \operatorname{FPR}(\tau) &= P\!\bigl(f_\theta(\mathbf{x}) > \tau \mid y = 0\bigr).
 \end{align}
-For $\alpha \in (0,1)$, let $\tau^*(\alpha)$ denote the threshold achieving
-$\operatorname{FPR}(\tau^*(\alpha)) = \alpha$, and write
-$\operatorname{TPR}(\alpha) = \operatorname{TPR}(\tau^*(\alpha))$ for the true positive rate
-at that operating point. The curve $\alpha \mapsto \operatorname{TPR}(\alpha)$ is the ROC curve.
+
+Conversely, we may denote the inverse operations $\operatorname{FPR}^{-1}$ and $\operatorname{TPR}^{-1}$ to be:
+
+\begin{align}
+  \operatorname{TPR^{-1}}\!\bigl(P\!\bigl(f_\theta(\mathbf{x}) > \tau \mid y = 1\bigr)\bigr) &= \tau, \\
+  \operatorname{FPR^{-1}}\!\bigl(P\!\bigl(f_\theta(\mathbf{x}) > \tau \mid y = 0\bigr)\bigr) &= \tau .
+\end{align}
+
+
+For $\alpha \in (0,1)$, let $\tau = \operatorname{FPR}^{-1}(\alpha)$ denote the threshold achieving
+$\operatorname{FPR}(\tau) = \alpha$.
+The curve $\alpha \mapsto \operatorname{TPR}(\operatorname{FPR^{-1}}(\alpha))$ is the ROC curve.
 The primary learning objective is to maximize its area:
 \begin{equation}
-  \max_\theta\;\operatorname{AUC}(f_\theta) = \int_0^1 \operatorname{TPR}(\alpha)\,d\alpha.
+  \max_\theta\;\operatorname{AUC}(f_\theta) = \int_0^1 \operatorname{TPR}(\operatorname{FPR}^{-1}(\alpha))\,d\alpha.
   \label{eq:objective-auc}
 \end{equation}
 Maximizing AUC is the natural objective under class imbalance and label noise, where no
@@ -410,6 +418,14 @@ Positive-class samples are weighted by their individual kernel
 PDF at $\tau$; negative-class samples are weighted by the mean positive-class kernel
 PDF scaled by each sample's share of the negative-class kernel mass.
 
+Equation~\eqref{eq:kde-grad-combined} is derived by treating the bandwidths $v_0, v_1$
+as fixed constants, even though \Cref{sec:kde-bandwidth} re-estimates them from the
+batch scores on every forward pass. We adopt this as a deliberate stop-gradient
+convention: as \Cref{obs:gradient-sign-correctness} shows, it is exactly what
+guarantees every per-sample gradient points in the direction that improves separation.
+Differentiating fully through the bandwidth estimator can break that guarantee, as
+Appendix~\ref{appendix:bandwidth-freeze} shows by counterexample.
+
 %------------------------------------------------
 
 \subsection{Properties of the ROLL Formulation}
@@ -470,6 +486,34 @@ average of 99 per-threshold objectives each satisfying \eqref{eq:grad-balance},
 is preserved by linearity of summation.
 \end{corollary}
 
+\paragraph{Gradient sign correctness.}
+\label{para:gradient-sign-correctness}
+
+A third structural consequence of \eqref{eq:kde-grad-combined} is that every per-sample
+gradient points in the direction that improves separation, never against it.
+
+\begin{observation}[Gradient sign correctness]
+\label{obs:gradient-sign-correctness}
+For every $\mathbf{x}_i \in \mathcal{B}$,
+\[
+  y_i = 1 \;\Longrightarrow\; \frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}}{\partial f_\theta(\mathbf{x}_i)} \leq 0,
+  \qquad
+  y_i = 0 \;\Longrightarrow\; \frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}}{\partial f_\theta(\mathbf{x}_i)} \geq 0.
+\]
+\end{observation}
+
+This follows directly from \eqref{eq:kde-grad-combined}: for $y_i = 1$ the gradient is
+$-\sigma_1'(\tau-f_\theta(\mathbf{x}_i))/|\mathcal{B}_1|$, and $\sigma_1' \geq 0$
+everywhere; for $y_i = 0$ the gradient is a product of two nonnegative quantities (a
+mean of $\sigma_1'$ values and a normalized $\sigma_0'$ ratio). Gradient descent
+therefore never pushes a positive-class score down, nor a negative-class score up, a
+minimal requirement for any loss intended to drive separation.
+
+This guarantee is specific to the frozen-bandwidth gradient~\eqref{eq:kde-grad-combined}.
+Differentiating fully through the adaptive bandwidth of \Cref{sec:kde-bandwidth} can
+break it: Appendix~\ref{appendix:bandwidth-freeze} gives an exact counterexample where
+the fully adaptive gradient violates this sign guarantee.
+
 \paragraph{Gradient locality.}
 \label{para:gradient-locality}