% ============================================================ % PLACEMENT: near the START of Ch. 2, before KDE and AUC/lit % review sections. This is the reader's first encounter with % the problem setting and the TPR@FPR objective. The arc % should be: (1) standard accuracy under imbalance fails; % (2) class-weighting is the usual fix but gives no direct % operating-point control; (3) define TPR, FPR, ROC formally; % (4) state the target: maximise TPR at a fixed FPR budget α. % ============================================================ \section{Imbalanced Classification and The TPR/FPR Objective} \label{sec:imbalanced-tpr-fpr} % IMPROVEMENT SUGGESTIONS (carried over from original draft): % - Narrative arc is currently backwards. Drive the section % with: accuracy is gameable under imbalance -> class % weighting is the standard fix -> it gives no direct % control over the operating point -> therefore we need % TPR@FPR directly. % - The inseparability tangent (cascade-classifier example) % dilutes the argument; trim to one sentence or cut. % - $\mathcal{D}_0$ and $\mathcal{D}_1$ must be defined % before first use. % - Land the section on a concrete, formal statement of the % TPR@FPR objective with proper notation. When fitting a classifier, it is most common to try and maximize the probability of success. That is, given some dataset $\mathcal{D} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{n}$ with inputs $\mathbf{x}_i \in \mathcal{X}$ and binary labels $y_i \in \{0, 1\}$, we learn a scoring function $f_\theta : \mathcal{X} \to \mathbb{R}$ and apply a threshold $\tau$ to produce predictions $\hat{y} = \mathbf{1}[f_\theta(\mathbf{x}) > \tau]$. The standard objective is to minimize the probability of classification error: $$\mathcal{L}(\theta) = \frac{1}{n} \sum_{i=1}^{n} \mathbf{1}[\hat{y}_i \neq y_i] = P(\hat{y} \neq y)$$ which is equivalent to maximizing classification accuracy $P(\hat{y} = y)$. For most applications, this optimization is warranted. However, there are many real-world settings in which this type of fitting is ill-suited. One such application is in the case of inseparable classes. In many cases, no meaningful separation boundary can be found between $\mathcal{D}_0$ and $\mathcal{D}_1$. In many cases, especially in tabular datasets, there can be two data elements $i,j$ for which $\mathbf{x}_i = \mathbf{x}_j$, however, $y_i \neq y_j$. Any deterministic classifier will have to decide whether to classify inseparable points as one class or the other, but it remains hard to control the behavior of a classifier while it is fitting to suggest one outcome or the other. The issue of inseparability becomes more apparent in certain applications where runtime is a key and limiting factor. In many classification pipelines, a ``quick and dirty'' classifier may filter out many candidates before an ``in depth'' classifier gives the final answer. This may be done in events where running the full in-depth classification on all data points is not a cost-effective or time-efficient solution. The ``quick and dirty'' classifier can be either low in parameters, making it run quickly. It may also only make a quick decision based on part of the data point. For example, in the case of a classifier looking to flag images of the ocean, a ``quick and dirty'' classifier may look at a very-low resolution version of the image and simply check if it is relatively ``blue''. It filters out many certain false candidates this way, then passes on the remainder to an in-depth, higher accuracy classifier. Solutions like this allow to run at near-optimal performance with lower compute resources in limited-time applications. These situations are common in large-volume image applications on the internet, as well as in astronomy/microscopy. Another such application is a case of highly imbalanced data. Even with separability, it is very easy to a classifier to simply ``ignore'' the minority class. This may very likely be a local minimum while fitting, which may be inescapable. As was once said, ``a classifier that always returns 0 is well fitted for flagging pictures of Michael Jordan bathing in green M\&Ms''. Combined with inseparability, we see that in extreme cases, classifiers may choose to ignore, or otherwise penalize, the minority class. For every trained classifier, the output per model is a score (TODO rewrite and formalize). TODO finish this section \begin{itemize} \item What usually happens \begin{itemize} \item Fitting \item Graphing ROC and picking threshold based on tradeoff \end{itemize} \item Usually we compensate for poor performance with \begin{itemize} \item Weighting between classes \item Possible other methods \end{itemize} \item Problem with this method is lack of control. \end{itemize} % Motivate why standard loss functions (cross-entropy, MSE) break down under class % imbalance: they optimize average accuracy, which a model can game by predicting % the majority class. Introduce concrete examples (e.g. medical diagnosis, fraud % detection) where the cost of a false negative vastly outweighs a false positive, % and where the practitioner needs to operate at a specific FPR budget. Argue that % what is actually needed is direct control over TPR at a fixed FPR, not a % surrogate that only loosely correlates with it. % ============================================================ % PLACEMENT: Section 2.2, immediately after the imbalanced % classification section (sec:imbalanced-tpr-fpr). Establishes % AUC as the standard measure and surveys how the literature % optimizes it, before narrowing to partial AUC / single-point % objectives in the next section. % ============================================================ \section{ROC Curves and AUC Optimization} \label{sec:roc-auc} \addcontentsline{tocheb}{section}{\protect\numberline{\secnumforhebrewtoc}{עקומות ROC ואופטימיזציה של AUC}} % \subsection{The ROC Curve} % Define the ROC curve as the plot of TPR vs.\ FPR as the threshold sweeps its full range. % Show that AUC equals $P(f(x^+) > f(x^-))$ for a random positive/negative pair (the % Wilcoxon--Mann--Whitney statistic): AUC is threshold-independent and measures ranking % quality, not calibration. Cite Hanley \& McNeil (1982). % \subsection{Surrogate AUC Losses} % Explain why directly maximizing the empirical AUC is hard: the WMW indicator is pairwise % and discontinuous ($O(n^2)$ pairs). Introduce smooth surrogates --- sigmoid approximation % to the pairwise indicator, squared loss on score differences --- and their consistency % guarantees. Cite Cortes \& Mohri (2003/2004). % \subsection{Stochastic AUC Maximization} % Describe mini-batch and compositional stochastic methods that make AUC optimization % practical at scale. Cover PESG (Yuan et al., NeurIPS 2020) and LibAUC (Yuan et al., % ICML 2021), which is used as a baseline in \Cref{chapter:experiments}. % Key limitation shared by all: they optimize the \emph{whole-curve} AUC, not a specific % operating point on the ROC --- motivating the next section. A Receiver Operating Characteristic (ROC) is a graph that represents the performance of a classifier. The classifier in question is evaluated on binary-labeled data. The labeled data is ordered based off of the scores given by the classifier. Given any numerical threshold, it is then easy to see which labeled data points fall above and below. For each threshold, the true positive rate and the false positive rate is given. The results for each threshold, and so for each possible TPR and FPR, are put on a graph. The resulting curve is the ROC curve. The AUC is a number in the range $[0, 1]$ representing the total area under the curve. It is calculated by integrating over all TPRs and taking the resulting average FPRs. A result of $1$ represents a perfect classifier, while a result of $0$ represents a classifier with a perfect opposite result. A result of $0.5$ represents a classifier that found no separation whatsoever. The AUC is a direct representation of the performance of a classifier, as opposed to a loss. There are instances where the loss may be minimized, yet seperation might be minimal. These two metrics might correlate, but do not necessarily correspond. An example of this is a result for which all true samples receive a score of $0.51$ and all false samples receie a score of $0.49$. Using many losses, including BCE, this would result in a relatively high loss value. On this score however, the AUC is $1$. The model perfectly seperates the samples, so further training is not necessary. Direct AUC optimization can be ieal with optimizing for imbalanced datasets. With most losses such as BCE or focal, the loss is calculated per-sample and then summed (or averaged). In order to properly train for both true and false populations, the scores are weighted, so that equal weight falls on both the true and the false populations. With direct AUC optimization, the imbalance does not affect the calculation, and the reweighting can be forgone. Additionally, direct AUC methods are more resilient to label noise, as opposed to classical methods. Due to inseperability, correctly classifying a sample might require a model to incorrectly classify similar or even identical samples that were mislabeled. This is only worth it for the model in training if the total loss of the correctly labeled samples outweighs the total loss of the incorrectly labeled samples. For most methods, the weight per class is given by the class imbalance in the dataset, but this doesn't take into account the label noise in any way. Optimizing for AUC specifically would eliminate this risk. % ============================================================ % PLACEMENT: Section 2.3, after ROC/AUC section above and % before the KDE background section. Narrows from whole-curve % AUC to the single operating-point objective that ROLL targets. % ============================================================ \section{Partial AUC and TPR at Fixed FPR} \label{sec:partial-auc} \addcontentsline{tocheb}{section}{\protect\numberline{\secnumforhebrewtoc}{AUC חלקי ו-TPR בנקודת FPR קבועה}} % \subsection{Partial AUC} % Define partial AUC as the area under the ROC curve restricted to $\mathrm{FPR} \in [0, \alpha]$. % Motivate: in high-stakes settings (fraud, medical triage) only the low-FPR region matters. % Survey recent pAUC optimizers that extend the PESG/compositional framework % (Lin et al., 2022; Yang \& Ying, 2022). % \subsection{TPR at a Fixed FPR: The Single-Point Objective} % Distinguish pAUC (integral over an interval) from the exact single-point objective: % maximize $\mathrm{TPR}(f, \alpha)$ for a fixed budget $\alpha$. % Argue that post-hoc threshold search and pAUC integration both fail to provide a % differentiable end-to-end training signal for this precise target --- this is the gap % that ROLL fills. % \subsection{Neyman-Pearson Classification (Brief)} % Introduce NP classification (Cannon et al., 2002; Tong, 2013) as the statistical % framing: minimize type-II error subject to $\mathrm{FPR} \le \alpha$. % Note that classical NP methods are not gradient-based and cannot be applied to DNNs. % Full treatment deferred to \Cref{sec:neyman-pearson} at the end of this chapter. TODO write this section. % ============================================================ % PLACEMENT: Section 2.4, after partial AUC and before the % noisy labels section. Provides the technical KDE background % that Ch.\ 3 (sec:roll-kde) builds on directly. End with a % forward pointer to sec:roll-kde. % ============================================================ \section{Kernel Density Estimation} \label{sec:kde-background} \addcontentsline{tocheb}{section}{\protect\numberline{\secnumforhebrewtoc}{אמידת צפיפות גרעין (KDE)}} % \subsection{The Parzen Window Estimator} % Define KDE: $\hat{p}(x) = \frac{1}{nh}\sum_i K\!\bigl(\frac{x-x_i}{h}\bigr)$, % where $K$ is a kernel and $h > 0$ is the bandwidth. % Present common kernels: Gaussian, Epanechnikov, sigmoid. Discuss how kernel % smoothness affects gradient properties. Cite Parzen (1962), Silverman (1986). % \subsection{Bandwidth Selection} % Describe Silverman's rule-of-thumb ($h = 1.06\,\hat\sigma\,n^{-1/5}$) as the default. % Mention cross-validation as the principled alternative, expensive at training time. % In ROLL, bandwidth is fixed (Silverman) or treated as a learnable parameter; % details deferred to Ch.\ 3. % \subsection{Differentiability and Use as a Loss Ingredient} % Explain why KDE with a smooth kernel is differentiable w.r.t.\ the support points $x_i$: % the density at any query is a smooth weighted sum over all support points, so gradients % flow back to network parameters. This property is why KDE is suitable as a building % block inside a loss function. Forward pointer: \Cref{sec:roll-kde} exploits this to % construct ROLL's differentiable threshold estimate from the estimated score distributions. TODO write this section. % ============================================================ % PLACEMENT: Section 2.5, AFTER the KDE background section % (sec:kde-background) and BEFORE the Neyman-Pearson bridge % (sec:neyman-pearson, which becomes 2.6). % % Purpose: establish that existing pointwise losses are % fragile under label corruption, motivating both the % practical robustness of ROLL and the poisoning experiment % results in Ch. 4. % % Narrative arc: % 1. Define label noise types: symmetric (uniform flip), % asymmetric / class-conditional (only certain class % transitions occur), instance-dependent. Our poisoning % experiment is asymmetric: P(\tilde{y}=0 \mid y=1)>0, % P(\tilde{y}=1 \mid y=0)=0. % 2. Why BCE is fragile: the symmetry condition (Ghosh % et al., AAAI 2017) — a loss \ell is noise-tolerant % iff \sum_k \ell(f(x),k) = C (constant). CE fails % this; MAE satisfies it but converges slowly. Cite % Feng et al. (IJCAI 2020) for "Can CE be robust?" % (only under very strict conditions). Implication: % mislabeled samples produce large unmitigated % gradients; DNNs memorize noise late in training % (memorization effect). % 3. Family of existing fixes — keep brief; these are what % ROLL sidesteps structurally: % (a) Robust loss functions: MAE (noise-tolerant, % slow), GCE (Zhang & Sabuncu, NeurIPS 2018; % interpolates MAE↔CE via q), asymmetric losses % (Zhou et al., ICML 2021), symmetrization % (arXiv:2605.20347). % (b) Label correction via noise transition matrix % (Patrini et al., CVPR 2017): estimate % T[i,j]=P(\tilde{y}=j|y=i), correct logits by % T^{-1}. Hard for asymmetric noise without a % clean reference set. % (c) Sample selection / co-teaching (Han et al., % NeurIPS 2018): two networks select small-loss % examples for each other, exploiting the % memorization effect. Requires dual-model % training. % 4. Ranking / distributional objectives and noise % robustness: pairwise losses depend on score % differences, not absolute labels — one mislabeled % sample affects O(n) pairs, each with a small gradient % weight. Wang et al. (arXiv:2305.14258) formally show % noise robustness of partial-AUC objectives. % 5. Bridge (observation, not oversell): ROLL's KDE-based % distributional objective shares this structural % property. Fake negatives (copies of positives with % flipped labels) shift the estimated negative density % but do not erase distributional separation when true % positive scores remain clustered. ROLL achieves this % without any explicit noise-handling mechanism; the % poisoning experiment in Ch. 4 provides empirical % evidence. % ============================================================ \section{Learning with Noisy Labels} \label{sec:noisy-labels} % SUBSECTION STRUCTURE: % % \subsection{Types of Label Noise} % - Symmetric / uniform: each label flipped with % probability \varepsilon across all classes. % - Asymmetric / class-conditional: only certain class % transitions occur. Our poisoning experiment is of % this type: P(\tilde{y}=0 \mid y=1)>0, % P(\tilde{y}=1 \mid y=0)=0. % - Instance-dependent: flip probability varies per % sample (hardest to handle; mention only briefly). % - Note: even moderate noise rates significantly hurt % pointwise losses in the imbalanced setting, because % mislabeled positives are rare to begin with. % % \subsection{Why Cross-Entropy Is Fragile Under Label Noise} % - Introduce symmetry condition \cite{ghosh2017robust}: % a loss is noise-tolerant iff % \sum_k \ell(f(\mathbf{x}), k) = C (constant). % - CE does NOT satisfy this; MAE does (but is slow). % - Memorization effect: DNNs learn clean patterns early, % then memorize noisy labels late in training. % - Cite: \cite{ghosh2017robust}, \cite{feng2020can}. % % \subsection{Approaches to Robust Learning} % - Paragraph 1 — Robust loss functions: % MAE \cite{ghosh2017robust}, % GCE \cite{zhang2018generalized} (L_q interpolation), % asymmetric losses \cite{zhou2021asymmetric}. % Key tradeoff: noise-tolerance vs.\ convergence speed. % - Paragraph 2 — Transition matrix correction % \cite{patrini2017making}: forward correction % multiplies logits by T^{-1}. Canonical method but % requires estimating T, which is hard for asymmetric % noise without a clean anchor set. % - Paragraph 3 — Sample selection / co-teaching % \cite{han2018coteaching}: two nets select small-loss % samples for each other each epoch. Bypasses T % estimation but requires dual-model training. % % \subsection{Ranking Objectives and Noise Robustness} % - Pairwise ranking losses are structurally more % noise-robust: a single mislabeled sample affects O(n) % pairs, each with small gradient weight, vs.\ one large % pointwise gradient in BCE. % - Cite Wang et al.\ \cite{wang2023weakly} for formal % treatment in the partial-AUC setting. % - Close with bridge sentence: ROLL's distributional % threshold objective operates in the same structural % regime — not pairwise, but also not pointwise — % conferring noise robustness without explicit correction. % Empirical evidence deferred to the Experiments chapter. TODO write this section. % ============================================================ % PLACEMENT: Section 2.6 (was 2.5), near the END of Ch. 2, % after KDE and Noisy Labels sections, just before the % chapter summary. This section bridges from prior work to % the method chapter by framing ROLL as a learnable, % gradient-based NP test. % NOTE: the label sec:neyman-pearson is unchanged; existing % \Cref references in Ch. 3 (sec:problem-formulation) still % work without modification. % ============================================================ \section{Connection to Neyman-Pearson} \label{sec:neyman-pearson} % Introduce the Neyman-Pearson lemma: among all tests at a % given false-positive rate, the likelihood-ratio test % maximizes TPR. Frame ROLL as a learnable, gradient-based % realization of this principle — rather than assuming known % class-conditional distributions (as classical NP does), % ROLL estimates them from model scores during training. % Briefly note that existing NP-inspired methods do not % support deep learning / gradient-based optimization; this % is the gap ROLL fills. TODO write this section. %------------------------------------------------ \section{topic a} \label{sec:related_work:jigsaw_puzzles} \addcontentsline{tocheb}{section}{\protect\numberline{\secnumforhebrewtoc}{נושא א}} To be continued. \section{topic b} \label{sec:related_work:relaxation_labeling} \addcontentsline{tocheb}{section}{\protect\numberline{\secnumforhebrewtoc}{נושא ב}} To be continued. \subsection{sub topic b.1} \label{subsec:formulation_as_rl:rationale:type_2} \addcontentsline{tocheb}{subsection}{\protect\numberline{\subsecnumforhebrewtoc}{תת נושא ב1}} To be continued. \begin{figure}[H] \centering \begin{subfigure}[b]{0.3\textwidth} \begin{tikzpicture} \node[anchor=south west, inner sep=0] at (0,0) {\includegraphics[width=\textwidth]{content/related_work/images/2x2_puzzle_grid.png}}; \draw[step=0.5\textwidth] (0,0) grid (\textwidth,\textwidth); \node[font=\large] at (0.25\textwidth,0.75\textwidth) {(1,1)}; \node[font=\large] at (0.75\textwidth,0.75\textwidth) {(1,2)}; \node[font=\large] at (0.25\textwidth,0.25\textwidth) {(2,1)}; \node[font=\large] at (0.75\textwidth,0.25\textwidth) {(2,2)}; \end{tikzpicture} \caption{} \label{fig:type_1_goal_and_labeling:dimensions} \end{subfigure} \hfill \begin{subfigure}[b]{0.3\textwidth} \begin{tikzpicture} \node[anchor=south west, inner sep=0] at (0,0) {\includegraphics[width=\textwidth]{content/related_work/images/2x2puzzle_type_1.png}}; \draw[step=0.5\textwidth] (0,0) grid (\textwidth,\textwidth); \node[font=\large, color=red] at (0.25\textwidth,0.75\textwidth) {Piece 1}; \node[font=\large, color=red] at (0.75\textwidth,0.75\textwidth) {Piece 2}; \node[font=\large, color=red] at (0.25\textwidth,0.25\textwidth) {Piece 3}; \node[font=\large, color=red] at (0.75\textwidth,0.25\textwidth) {Piece 4}; \end{tikzpicture} \caption{} \label{fig:type_1_goal_and_labeling:pieces} \end{subfigure} \hfill \begin{subfigure}[b]{0.3\textwidth} \begin{tikzpicture} \node[anchor=south west, inner sep=0] at (0,0) {\includegraphics[width=\textwidth]{content/related_work/images/2x2puzzle_solution.png}}; \draw[step=0.5\textwidth] (0,0) grid (\textwidth,\textwidth); \node[font=\large, color=red] at (0.25\textwidth,0.75\textwidth) {Piece 3}; \node[font=\large, color=red] at (0.75\textwidth,0.75\textwidth) {Piece 2}; \node[font=\large, color=red] at (0.25\textwidth,0.25\textwidth) {Piece 4}; \node[font=\large, color=red] at (0.75\textwidth,0.25\textwidth) {Piece 1}; \end{tikzpicture} \caption{} \label{fig:type_1_goal_and_labeling:solution} \end{subfigure} \vfill \begin{subfigure}[b]{1\textwidth} \centering \begin{tikzpicture} \def\scaletitles{0.88} \def\minimumEntrySize{0.95cm} \matrix[matrix of nodes, nodes={draw, align=center, minimum size=\minimumEntrySize}, row 1/.style={nodes={draw=none, gray, font=\footnotesize, scale=\scaletitles}}, column 1/.style={nodes={draw=none, gray, font=\footnotesize, scale=\scaletitles}}] { \node{}; & \node{(1,1)}; & \node{(1,2)}; & \node{(2,1)}; & \node{(2,2)};\\ %----------% \node{Piece 1}; & \node{0}; & \node{0}; & \node{0}; & \node[text=blue]{1};\\ %----------% \node{Piece 2}; & \node{0}; & \node[text=blue]{1}; & \node{0}; & \node{0};\\ %----------% \node{Piece 3}; & \node[text=blue]{1}; & \node{0}; & \node{0}; & \node{0};\\ %----------% \node{Piece 4}; & \node{0}; & \node{0}; & \node[text=blue]{1}; & \node{0};\\ }; \end{tikzpicture} \caption{} \label{fig:type_1_goal_and_labeling:labeling} \end{subfigure} \caption[fig A - Example for fig]{Some example} \label{fig:type_1_goal_and_labeling} \end{figure} \begin{table}[H] \centering \begin{tabular}{ |c|c|c|c|c|c| } \hline \multicolumn{1}{|c|}{Puzzle Type} & Direct & Neighbor & Perfect & Occupied & Feasible \\ \hline Type 1 & {0\%} & {0.1\%} & 0 & {1\%} & 0 \\ \hline Type 2 & {0.001\%} & {0.1\%} & 0 & {1.2\%} & 0 \\ \hline \end{tabular} \caption[Table A - Example for table]{Some example} \label{table:plain_rl_results} \end{table}