Hopefully final draft

content/discussion_and_conclusions/discussion_and_conclusions.tex

@@ -3,6 +3,21 @@
 \label{sec:discussion}
 \addcontentsline{tocheb}{section}{\protect\numberline{\secnumforhebrewtoc}{דיון}}
 
+\subsection{Theoretical Justification for AUC-Optimizing Losses}
+\label{sec:discussion:np-justification}
+\addcontentsline{tocheb}{subsection}{\protect\numberline{\subsecnumforhebrewtoc}{הצדקה תיאורטית לפונקציות הפסד ממטבות AUC}}
+
+A theoretical justification for AUC-optimizing losses can be based on the implicit
+assumptions made by class weighting, as interpreted through the Neyman-Pearson lemma: a
+weighted per-sample loss implicitly commits to a fixed decision threshold $\eta$ chosen
+indirectly through the class weights, with no reference to the operating point the task
+actually requires, and this implicit threshold is further, unpredictably distorted under
+label noise. Objectives that target an operating point or optimize AUC directly,
+\textsc{Roll}'s TPR@FPR and ROLL-AOC objectives, as well as other AUC-optimizing methods
+such as \textsc{LibAUC}~\cite{yuan2021auc}, sidestep this altogether, since none of them
+ever form a class weight or compare a probability to a fixed $\eta$. The full analysis is
+given in \Cref{appendix:np-theory}.
+
 \subsection{Noise Robustness of \textsc{ROLL}-AOC}
 \label{sec:discussion:noise}
 \addcontentsline{tocheb}{subsection}{\protect\numberline{\subsecnumforhebrewtoc}{עמידות לרעש של ROLL-AOC}}
@@ -29,14 +44,12 @@ Having the ability to optimize the desired metric (AUC) directly, rather than in
 our method to overcome challenging data. When using a per-sample loss, optimizing for a mislabeled sample
 might decrease the overall metric, but would do so at the cost of the performance of the model. Using AUC
 based methods allows this to be factored in already at the point of gradient weighting, so that the importance
-of this erred sample is factored in immediately. More precisely, in KDE-ROLL a single flipped label shifts
-the fitted score distribution by $1/n$, diluting its influence smoothly across the full distribution rather
-than producing a concentrated gradient outlier as occurs in per-sample losses.
+of this erred sample is factored in immediately.
 
 This noise-robustness pattern is consistent across all three benchmarks used in this work: KEEL (controlled
 symmetric label noise), CIFAR-10N (real-world human annotation noise), and ChestXray14 (structural noise
 from NLP-extracted labels). The same advantage appearing across tabular data, natural images, and medical
-images strengthens the case that noise robustness is a structural property of the distributional objective,
+images strengthens the case that noise robustness is a property of the distributional objective,
 rather than an artifact of any particular dataset or noise type.
 
 \subsection{\textsc{ROLL}-AOC Versus \textsc{GCE} and \textsc{MAE}}
@@ -235,8 +248,16 @@ Finally, the framework currently lacks formal convergence guarantees; the empiri
 \paragraph{Stabilizing single-threshold optimization.}
 The failure of \textsc{Roll}-TPR suggests that concentrating all gradient mass on a single operating threshold is too aggressive. A softer objective would weight each threshold's contribution according to a distribution centred on the target operating point, rather than a hard indicator at one FPR value. Unlike \textsc{LibAUC}, whose pairwise formulation has no notion of per-threshold weighting, \textsc{Roll}'s grid-based structure makes this modification straightforward.
 
+\paragraph{Kernel-appropriate bandwidth selection.}
+As noted in \Cref{sec:kde-bandwidth}, the ISJ bandwidth estimator is a plug-in
+heuristic here: its optimality guarantee is derived for a Gaussian kernel, while
+\textsc{Roll}-KDE uses a logistic kernel throughout. It was adopted for its ready
+availability in the existing implementation, not because it is proven optimal for
+this kernel. Deriving or empirically validating a bandwidth selection rule matched to
+the logistic kernel specifically remains a further point of research.
+
 \paragraph{Adaptive bandwidth estimation.}
-Variable-bandwidth (adaptive) KDE methods address the single-bandwidth limitation by assigning a per-sample or per-region bandwidth inversely proportional to local density~\cite{abramson1982bandwidth,terrell1992variable}. Adapting such methods to the \textsc{Roll} framework is a natural extension: the threshold computation and gradient derivations carry through with per-sample bandwidths with only local changes to the kernel evaluation.
+Variable-bandwidth (adaptive) KDE methods address the single-bandwidth limitation by assigning a per-sample or per-region bandwidth inversely proportional to local density~\cite{abramson1982bandwidth,terrell1992variable}. Adapting such methods to the \textsc{Roll} framework is a natural extension: the threshold computation and gradient derivations carry through with per-sample bandwidths with only local changes to the kernel evaluation, assuming a frozen gradient for bandwidth, as currently implemented.
 
 \paragraph{Convergence theory.}
 Formal convergence guarantees for KDE-ROLL under standard assumptions (batch size, bandwidth decay schedule, smoothness) remain an open problem, and would also clarify how the bandwidth scheduler interacts with gradient variance across training.