Thesis - wip

content/related_work/related_work.tex

@@ -84,10 +84,130 @@ TODO finish this section
 
 
 % ============================================================
-% PLACEMENT: near the END of Ch. 2, after KDE and AUC/lit
-% review sections, just before the chapter summary.  This
-% section bridges from prior work to the method chapter by
-% framing ROLL as a learnable, gradient-based NP test.
+% PLACEMENT: Section 2.5, AFTER the KDE background section
+% (sec:kde-background) and BEFORE the Neyman-Pearson bridge
+% (sec:neyman-pearson, which becomes 2.6).
+%
+% Purpose: establish that existing pointwise losses are
+% fragile under label corruption, motivating both the
+% practical robustness of ROLL and the poisoning experiment
+% results in Ch. 4.
+%
+% Narrative arc:
+%   1. Define label noise types: symmetric (uniform flip),
+%      asymmetric / class-conditional (only certain class
+%      transitions occur), instance-dependent. Our poisoning
+%      experiment is asymmetric: P(\tilde{y}=0 \mid y=1)>0,
+%      P(\tilde{y}=1 \mid y=0)=0.
+%   2. Why BCE is fragile: the symmetry condition (Ghosh
+%      et al., AAAI 2017) — a loss \ell is noise-tolerant
+%      iff \sum_k \ell(f(x),k) = C (constant).  CE fails
+%      this; MAE satisfies it but converges slowly.  Cite
+%      Feng et al. (IJCAI 2020) for "Can CE be robust?"
+%      (only under very strict conditions). Implication:
+%      mislabeled samples produce large unmitigated
+%      gradients; DNNs memorize noise late in training
+%      (memorization effect).
+%   3. Family of existing fixes — keep brief; these are what
+%      ROLL sidesteps structurally:
+%      (a) Robust loss functions: MAE (noise-tolerant,
+%          slow), GCE (Zhang & Sabuncu, NeurIPS 2018;
+%          interpolates MAE↔CE via q), asymmetric losses
+%          (Zhou et al., ICML 2021), symmetrization
+%          (arXiv:2605.20347).
+%      (b) Label correction via noise transition matrix
+%          (Patrini et al., CVPR 2017): estimate
+%          T[i,j]=P(\tilde{y}=j|y=i), correct logits by
+%          T^{-1}.  Hard for asymmetric noise without a
+%          clean reference set.
+%      (c) Sample selection / co-teaching (Han et al.,
+%          NeurIPS 2018): two networks select small-loss
+%          examples for each other, exploiting the
+%          memorization effect.  Requires dual-model
+%          training.
+%   4. Ranking / distributional objectives and noise
+%      robustness: pairwise losses depend on score
+%      differences, not absolute labels — one mislabeled
+%      sample affects O(n) pairs, each with a small gradient
+%      weight.  Wang et al. (arXiv:2305.14258) formally show
+%      noise robustness of partial-AUC objectives.
+%   5. Bridge (observation, not oversell): ROLL's KDE-based
+%      distributional objective shares this structural
+%      property.  Fake negatives (copies of positives with
+%      flipped labels) shift the estimated negative density
+%      but do not erase distributional separation when true
+%      positive scores remain clustered.  ROLL achieves this
+%      without any explicit noise-handling mechanism; the
+%      poisoning experiment in Ch. 4 provides empirical
+%      evidence.
+% ============================================================
+\section{Learning with Noisy Labels}
+\label{sec:noisy-labels}
+
+% SUBSECTION STRUCTURE:
+%
+% \subsection{Types of Label Noise}
+%   - Symmetric / uniform: each label flipped with
+%     probability \varepsilon across all classes.
+%   - Asymmetric / class-conditional: only certain class
+%     transitions occur.  Our poisoning experiment is of
+%     this type: P(\tilde{y}=0 \mid y=1)>0,
+%     P(\tilde{y}=1 \mid y=0)=0.
+%   - Instance-dependent: flip probability varies per
+%     sample (hardest to handle; mention only briefly).
+%   - Note: even moderate noise rates significantly hurt
+%     pointwise losses in the imbalanced setting, because
+%     mislabeled positives are rare to begin with.
+%
+% \subsection{Why Cross-Entropy Is Fragile Under Label Noise}
+%   - Introduce symmetry condition \cite{ghosh2017robust}:
+%     a loss is noise-tolerant iff
+%     \sum_k \ell(f(\mathbf{x}), k) = C (constant).
+%   - CE does NOT satisfy this; MAE does (but is slow).
+%   - Memorization effect: DNNs learn clean patterns early,
+%     then memorize noisy labels late in training.
+%   - Cite: \cite{ghosh2017robust}, \cite{feng2020can}.
+%
+% \subsection{Approaches to Robust Learning}
+%   - Paragraph 1 — Robust loss functions:
+%     MAE \cite{ghosh2017robust},
+%     GCE \cite{zhang2018generalized} (L_q interpolation),
+%     asymmetric losses \cite{zhou2021asymmetric}.
+%     Key tradeoff: noise-tolerance vs.\ convergence speed.
+%   - Paragraph 2 — Transition matrix correction
+%     \cite{patrini2017making}: forward correction
+%     multiplies logits by T^{-1}.  Canonical method but
+%     requires estimating T, which is hard for asymmetric
+%     noise without a clean anchor set.
+%   - Paragraph 3 — Sample selection / co-teaching
+%     \cite{han2018coteaching}: two nets select small-loss
+%     samples for each other each epoch.  Bypasses T
+%     estimation but requires dual-model training.
+%
+% \subsection{Ranking Objectives and Noise Robustness}
+%   - Pairwise ranking losses are structurally more
+%     noise-robust: a single mislabeled sample affects O(n)
+%     pairs, each with small gradient weight, vs.\ one large
+%     pointwise gradient in BCE.
+%   - Cite Wang et al.\ \cite{wang2023weakly} for formal
+%     treatment in the partial-AUC setting.
+%   - Close with bridge sentence: ROLL's distributional
+%     threshold objective operates in the same structural
+%     regime — not pairwise, but also not pointwise —
+%     conferring noise robustness without explicit correction.
+%     Empirical evidence deferred to the Experiments chapter.
+
+TODO write this section.
+
+% ============================================================
+% PLACEMENT: Section 2.6 (was 2.5), near the END of Ch. 2,
+% after KDE and Noisy Labels sections, just before the
+% chapter summary.  This section bridges from prior work to
+% the method chapter by framing ROLL as a learnable,
+% gradient-based NP test.
+% NOTE: the label sec:neyman-pearson is unchanged; existing
+% \Cref references in Ch. 3 (sec:problem-formulation) still
+% work without modification.
 % ============================================================
 \section{Connection to Neyman-Pearson}
 \label{sec:neyman-pearson}