backup: 2026-10-06 10:50
This commit is contained in:
+19
-19
@@ -7,41 +7,41 @@
|
||||
|
||||
* Writing — chapters
|
||||
|
||||
* TODO Write Ch. 1 (Introduction)
|
||||
* DONE Write Ch. 1 (Introduction)
|
||||
- Hook: imbalance + noise are everywhere, standard losses fail silently
|
||||
- Gap: no existing method does AUC optimization + noise robustness together
|
||||
- ROLL's claim: distributional objective, robustness as structural consequence
|
||||
- Close with chapter roadmap
|
||||
|
||||
* TODO Write Ch. 5 (Discussion)
|
||||
* DONE Write Ch. 5 (Discussion)
|
||||
- Explain WHY ROLL-AOC is noise robust (KDE distributional fit vs isolated gradients)
|
||||
- Explain WHY Bank Marketing fails (open question, speculate honestly)
|
||||
- Explain WHY GCE/MAE fail under asymmetric imbalanced noise
|
||||
- Compare mechanism with LibAUC (same objective, different path)
|
||||
|
||||
* TODO Write Ch. 6 (Conclusion)
|
||||
* DONE Write Ch. 6 (Conclusion)
|
||||
- Summary: differentiable AUC surrogate via score-distribution fitting
|
||||
- Key result: noise robustness validated on KEEL (5/10/20%) and CIFAR-10N (≤40%)
|
||||
- Limitations: Bank Marketing, 99-pass cost, bandwidth scheduler hyperparameter
|
||||
- Future work: learned bandwidth, partial-AUC variant, multi-class extension
|
||||
|
||||
* TODO Finish Ch. 4 (Results) — remaining stubs
|
||||
* DONE Finish Ch. 4 (Results) — remaining stubs
|
||||
- Write Summary section (sec:experiments:summary) — key points noted in comments
|
||||
- Decide: write or drop Ablation Study and Sensitivity Analysis sections
|
||||
- Fill all \cite{TODO} placeholders in experiments.tex
|
||||
|
||||
* TODO Review Ch. 4 (Results) end-to-end - Check narrative flow after noise section was added - Confirm KEEL/CIFAR-10N/large-dataset table interpretations are accurate - Verify roll-tpr90 is correctly described as secondary/special-case throughout
|
||||
* DONE Review Ch. 4 (Results) end-to-end - Check narrative flow after noise section was added - Confirm KEEL/CIFAR-10N/large-dataset table interpretations are accurate - Verify roll-tpr90 is correctly described as secondary/special-case throughout
|
||||
* Writing — front/back matter
|
||||
|
||||
* TODO Write English abstract
|
||||
* TODO Write Hebrew abstract
|
||||
* TODO Fill Acknowledgements (Main_pages/acknowledgements.tex)
|
||||
* DONE Write English abstract
|
||||
* DONE Write Hebrew abstract
|
||||
* DONE Fill Acknowledgements (Main_pages/acknowledgements.tex)
|
||||
|
||||
* Writing — Ch. 2 & 3 loose ends
|
||||
|
||||
* TODO Verify alpha notation w.r.t. FPR — eq:fpr-cdf writes FPR(τ) = F̂₀(τ) and threshold as τ = F̂₀⁻¹(α), but standard FPR = P(score > τ | y=0) = 1−F₀(τ) requires τ = F̂₀⁻¹(1−α). Check for sign/notation inconsistency throughout method.tex and appendix_gauss_grad.tex.
|
||||
* TODO Add graphs of CDF tracking for gaussian, KDE, roll.
|
||||
* TODO Cite in sec:related-work (Ch. 2):
|
||||
* DONE Verify alpha notation w.r.t. FPR — eq:fpr-cdf writes FPR(τ) = F̂₀(τ) and threshold as τ = F̂₀⁻¹(α), but standard FPR = P(score > τ | y=0) = 1−F₀(τ) requires τ = F̂₀⁻¹(1−α). Check for sign/notation inconsistency throughout method.tex and appendix_gauss_grad.tex.
|
||||
* DONE Add graphs of CDF tracking for gaussian, KDE, roll.
|
||||
* DONE Cite in sec:related-work (Ch. 2):
|
||||
- Benchmarking Deep AUROC Optimization (arXiv 2203.14177) — cite in 2.2 as survey of AUROC losses; finding: composite losses best overall, barrier hinge best under noise
|
||||
- LibAUC pAUCLoss / SOPAs (arXiv 2203.01505) — cite in 2.3 as partial AUC baseline
|
||||
- "Closing the Approximation Gap" (arXiv 2512.01213, Dec 2025) — cite in 2.3 as recent concurrent work on partial AUC
|
||||
@@ -49,8 +49,8 @@
|
||||
|
||||
* Experiments — open questions
|
||||
|
||||
* TODO Debug banking dataset — results unexpectedly low (roll-aoc 0.511 vs GCE 0.746), investigate root cause
|
||||
* TODO Check if LibAUC pAUCLoss (one-way partial AUC / SOPAs) should be added as baseline
|
||||
* DONE Debug banking dataset — results unexpectedly low (roll-aoc 0.511 vs GCE 0.746), investigate root cause
|
||||
* DONE Check if LibAUC pAUCLoss (one-way partial AUC / SOPAs) should be added as baseline
|
||||
- Already have LibAUC as dependency; low-cost to add
|
||||
- Most direct competitor to roll-tpr90 (both target a fixed FPR region)
|
||||
- See: https://docs.libauc.org/examples/pauc.html
|
||||
@@ -149,13 +149,13 @@
|
||||
- Beta fix (user's proposal, verified before writing up): replace $f_\theta(\mathbf{x}) \to -f_\theta(\mathbf{x})$ with $f_\theta(\mathbf{x}) \to 1-f_\theta(\mathbf{x})$ — stays in (0,1), and is already computed in the Beta forward pass as the complementary score $t^{(k)}=1-s^{(k)}$ (appendix_beta_roll.tex eq:beta-denom/eq:beta-params).
|
||||
- Verified with sympy/numpy before writing anything (never trusted mentally, per usual convention): (1) numerically confirmed $X\sim\text{Beta}(a,b) \Rightarrow 1-X\sim\text{Beta}(b,a)$ exactly (sympy's symbolic beta() didn't auto-cancel $B(a,b)$ vs $B(b,a)$, so checked numerically instead — matched to 1e-9 across 5 random parameter draws); (2) confirmed the log-moment estimator $D_k$ (eq:beta-denom) is exactly symmetric under swapping which vector is called $s^{(k)}$ vs $t^{(k)}$, so applying the estimator to $1-s^{(k)}$ exactly swaps $\hat a_k \leftrightarrow \hat b_k$ (numerically exact, not approximate); (3) end-to-end check that the same "target rate complemented to $1-\alpha$" recovery claim used for the Gaussian/negation case holds identically for the Beta/$1-s$ case — built both a Gaussian direct-vs-transform check (sanity check of the *existing* thesis claim, confirmed exact match) and a Beta direct-vs-transform check using scipy.stats.beta (also confirmed exact match to float precision) using the swapped $(\hat b,\hat a)$ params as the "new" class parameters.
|
||||
- Rebuilt thesis clean (nix build exit 0), footnote renders correctly with the appendix_beta_roll.tex cross-references (\Cref{appendix:beta-roll}, eq:beta-params) resolving.
|
||||
** TODO Remove/soften claim that gradient balance implies noise robustness or population FPR control (general, throughout Ch. 3 prose) — these require additional statistical assumptions/arguments or appropriately scoped empirical evidence; should not be presented as direct consequences of the audited identities.
|
||||
** DONE Remove/soften claim that gradient balance implies noise robustness or population FPR control (general, throughout Ch. 3 prose) — these require additional statistical assumptions/arguments or appropriately scoped empirical evidence; should not be presented as direct consequences of the audited identities.
|
||||
|
||||
* Final pass
|
||||
|
||||
* TODO Build full PDF and proofread end-to-end
|
||||
* TODO Check all cross-references (\Cref, \label) are consistent after restructuring
|
||||
* TODO Verify Hebrew ToC entries match English section titles
|
||||
* DONE Build full PDF and proofread end-to-end
|
||||
* DONE Check all cross-references (\Cref, \label) are consistent after restructuring
|
||||
* DONE Verify Hebrew ToC entries match English section titles
|
||||
|
||||
* DONE [#A] Rerun keel noised experiments — data exists in results-final/keel-noise-05/10/20
|
||||
* DONE Restructure thesis per mentor guidelines (2026-08-17)
|
||||
@@ -165,5 +165,5 @@
|
||||
- [x] Ch. 3 restructured: AUC as primary objective, ROLL-AOC promoted
|
||||
- [x] Noise results table added to Ch. 4
|
||||
* DONE Get rid of TPR results
|
||||
* TODO re-integrate results for banking
|
||||
* TODO integrate food101n results when done
|
||||
* DONE re-integrate results for banking
|
||||
* DONE integrate food101n results when done
|
||||
|
||||
Reference in New Issue
Block a user