Citations, better flake.nix, better figure

content/related_work/related_work.tex

@@ -1,3 +1,112 @@
+% ============================================================
+% PLACEMENT: near the START of Ch. 2, before KDE and AUC/lit
+% review sections.  This is the reader's first encounter with
+% the problem setting and the TPR@FPR objective.  The arc
+% should be: (1) standard accuracy under imbalance fails;
+% (2) class-weighting is the usual fix but gives no direct
+% operating-point control; (3) define TPR, FPR, ROC formally;
+% (4) state the target: maximise TPR at a fixed FPR budget α.
+% ============================================================
+\section{Imbalanced Classification and The TPR/FPR Objective}
+\label{sec:imbalanced-tpr-fpr}
+
+% IMPROVEMENT SUGGESTIONS (carried over from original draft):
+% - Narrative arc is currently backwards.  Drive the section
+%   with: accuracy is gameable under imbalance -> class
+%   weighting is the standard fix -> it gives no direct
+%   control over the operating point -> therefore we need
+%   TPR@FPR directly.
+% - The inseparability tangent (cascade-classifier example)
+%   dilutes the argument; trim to one sentence or cut.
+% - $\mathcal{D}_0$ and $\mathcal{D}_1$ must be defined
+%   before first use.
+% - Land the section on a concrete, formal statement of the
+%   TPR@FPR objective with proper notation.
+
+When fitting a classifier, it is most common to try and maximize the probability
+of success. That is, given some dataset
+$\mathcal{D} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{n}$
+with inputs $\mathbf{x}_i \in \mathcal{X}$ and binary labels $y_i \in \{0, 1\}$, we learn a scoring function $f_\theta : \mathcal{X} \to \mathbb{R}$ and apply a threshold $\tau$ to produce predictions $\hat{y} = \mathbf{1}[f_\theta(\mathbf{x}) > \tau]$. The standard objective is to minimize the probability of classification error:
+
+$$\mathcal{L}(\theta) = \frac{1}{n} \sum_{i=1}^{n} \mathbf{1}[\hat{y}_i \neq y_i] = P(\hat{y} \neq y)$$
+
+which is equivalent to maximizing classification accuracy $P(\hat{y} = y)$.
+
+For most applications, this optimization is warranted. However, there are many real-world settings in which
+this type of fitting is ill-suited.
+
+One such application is in the case of inseparable classes. In many cases, no meaningful
+separation boundary can be found between $\mathcal{D}_0$ and $\mathcal{D}_1$. In many cases,
+especially in tabular datasets, there can be two data elements $i,j$ for which $\mathbf{x}_i = \mathbf{x}_j$,
+however, $y_i \neq y_j$. Any deterministic classifier will have to decide whether to classify
+inseparable points as one class or the other, but it remains hard to control the behavior of
+a classifier while it is fitting to suggest one outcome or the other.
+
+The issue of inseparability becomes more apparent in certain applications where runtime is a key
+and limiting factor. In many classification pipelines, a ``quick and dirty'' classifier may
+filter out many candidates before an ``in depth'' classifier gives the final answer. This may be done
+in events where running the full in-depth classification on all data points is not a cost-effective or time-efficient solution.
+The ``quick and dirty'' classifier can be either low in parameters, making it run quickly. It may also only
+make a quick decision based on part of the data point. For example, in the case of a classifier looking to flag
+images of the ocean, a ``quick and dirty'' classifier may look at a very-low resolution version of the image and simply check if it is relatively ``blue''. It filters out many certain false candidates this way, then passes on the remainder to an in-depth, higher accuracy classifier. Solutions like this allow to run at near-optimal performance with lower compute resources in limited-time applications. These situations are common in large-volume image applications on the internet, as well as in astronomy/microscopy.
+
+Another such application is a case of highly imbalanced data. Even with separability, it is very
+easy to a classifier to simply ``ignore'' the minority class. This may very likely be a local minimum
+while fitting, which may be inescapable. As was once said, ``a classifier that always returns 0 is
+well fitted for flagging pictures of Michael Jordan bathing in green M\&Ms''. Combined with inseparability, we see that in extreme cases, classifiers may choose to ignore, or otherwise penalize, the minority class.
+
+For every trained classifier, the output per model is a score (TODO rewrite and formalize).
+
+TODO finish this section
+
+\begin{itemize}
+    \item What usually happens
+    \begin{itemize}
+        \item Fitting
+        \item Graphing ROC and picking threshold based on tradeoff
+    \end{itemize}
+    \item Usually we compensate for poor performance with
+    \begin{itemize}
+        \item Weighting between classes
+        \item Possible other methods
+    \end{itemize}
+    \item Problem with this method is lack of control.
+\end{itemize}
+
+
+% Motivate why standard loss functions (cross-entropy, MSE) break down under class
+% imbalance: they optimize average accuracy, which a model can game by predicting
+% the majority class. Introduce concrete examples (e.g. medical diagnosis, fraud
+% detection) where the cost of a false negative vastly outweighs a false positive,
+% and where the practitioner needs to operate at a specific FPR budget. Argue that
+% what is actually needed is direct control over TPR at a fixed FPR, not a
+% surrogate that only loosely correlates with it.
+
+
+% ============================================================
+% PLACEMENT: near the END of Ch. 2, after KDE and AUC/lit
+% review sections, just before the chapter summary.  This
+% section bridges from prior work to the method chapter by
+% framing ROLL as a learnable, gradient-based NP test.
+% ============================================================
+\section{Connection to Neyman-Pearson}
+\label{sec:neyman-pearson}
+
+% Introduce the Neyman-Pearson lemma: among all tests at a
+% given false-positive rate, the likelihood-ratio test
+% maximizes TPR.  Frame ROLL as a learnable, gradient-based
+% realization of this principle — rather than assuming known
+% class-conditional distributions (as classical NP does),
+% ROLL estimates them from model scores during training.
+% Briefly note that existing NP-inspired methods do not
+% support deep learning / gradient-based optimization; this
+% is the gap ROLL fills.
+
+TODO write this section.
+
+
+%------------------------------------------------
+
 \section{topic a}
 \label{sec:related_work:jigsaw_puzzles}
 \addcontentsline{tocheb}{section}{\protect\numberline{\secnumforhebrewtoc}{נושא א}}