roam/thesistodos.org

thesis/todos

• Writing — chapters
• Write Ch. 1 (Introduction)
• Write Ch. 5 (Discussion)
• Write Ch. 6 (Conclusion)
• Finish Ch. 4 (Results) — remaining stubs
• Review Ch. 4 (Results) end-to-end - Check narrative flow after noise section was added - Confirm KEEL/CIFAR-10N/large-dataset table interpretations are accurate - Verify roll-tpr90 is correctly described as secondary/special-case throughout
• Writing — front/back matter
• Write English abstract
• Write Hebrew abstract
• Fill Acknowledgements (Main_pages/acknowledgements.tex)
• Writing — Ch. 2 & 3 loose ends
• Verify alpha notation w.r.t. FPR — eq:fpr-cdf writes FPR(τ) = F̂₀(τ) and threshold as τ = F̂₀⁻¹(α), but standard FPR = P(score > τ | y=0) = 1−F₀(τ) requires τ = F̂₀⁻¹(1−α). Check for sign/notation inconsistency throughout method.tex and appendix_gauss_grad.tex.
• Add graphs of CDF tracking for gaussian, KDE, roll.
• Cite in sec:related-work (Ch. 2):
• Experiments — open questions
• Debug banking dataset — results unexpectedly low (roll-aoc 0.511 vs GCE 0.746), investigate root cause
• Check if LibAUC pAUCLoss (one-way partial AUC / SOPAs) should be added as baseline
• Mentor mathematical audit fixes (2026-09-07, thesis_mathematical_audit.pdf)
  • Decide bandwidth backprop convention (Eq. 3.16, Appendix C, Section 3.4.2, thesis pp. 25/19) — Appendix C's backward pass treats only the j=i kernel term as score-dependent, which is only valid for a FIXED bandwidth, but Section 3.4.2 estimates bandwidth from the class scores (ISJ/Silverman). Audit gave an exact counterexample where the true adaptive loss is constant but Eq. (3.16) gives a nonzero gradient.
  • Fix bisection stopping criterion to certify loss accuracy, not just CDF residual (Appendix E, p. 31) — current rule |F̂₀(τ̂) − (1−α)| < 1e-4 can pass while F̂₁(τ̂) is off by ~1 (counterexample given with h₀=1, h₁=0.01).
  • Restrict initial bisection bracket to valid α range — use exact target-dependent bracket (Appendix E, p. 30) — old bracket [s⁰_min−10h₀, s⁰_max+10h₀] only guaranteed containment for α∈(4.5e-5, 1-4.5e-5), not all of (0,1).
  • Fix score-normalization argument contradiction (Section 3.4.1, p. 18) — claims both that normalization keeps sigmoid args near unit scale AND that those args are scale-invariant; these can't both explain overflow prevention. Also missing chain-rule term (Σ s_j q_j / c² · ∂c/∂s_i) when c=σ̂_B is not explicitly detached.
  • Replace numerically vulnerable cached-sigmoid derivative (Eq. C.5, p. 27) — S(z)(1−S(z)) underflows to exactly 0 when S(z) rounds to 1 even though the true derivative is representable (e.g. S(20) in fp32).
  • Weaken "balanced gradient mass" claim from Proposition 3.1 (Section 3.3.2, Appendix I, pp. 13–14, 34–35) — the proof is correct (signed classwise score-gradient sums are ±F̂₁'(τ)), but this does NOT imply equal absolute gradient mass, and says nothing about ‖G₀‖ vs ‖G₁‖ for the actual parameter gradients (Jacobians differ).
  • Fix necessity claim re: translation equivariance for balance identity (p. 14) — translation equivariance is sufficient for (3.17) but not necessary; counterexample F̂_k(t;s)=Φ(t−2s̄) doesn't have the stated unit-translation property yet still cancels.
  • Qualify gradient-locality claim (Observation 3.3, Figure 3.3, pp. 15–17) — fixed-bandwidth proportionality to kernel density at threshold is correct, but "distant samples have negligible gradients" is false once negative-class weights are normalized by total kernel mass.
  • Separate exact fitted-distribution AOC from empirical/population AOC of original scores (Sections 3.1–3.3, Figure 3.2, p. 11) — the continuous integral L^fit_AOC = P(T1≤T0) is exact only for the FITTED distributions; it is not generally the empirical/population AUC of the original scores.
  • Fix Figure 3.2 caption / distinguish 99-point average from the integral identity (p. 9–10, Eq. 3.13) — L_99 is a quadrature approximation, not identical to the integral AOC.
  • Narrow "direct integration is not tractable" claim before Eq. (3.13) (p. 9) — false for the Gaussian case, which has closed form L_AOC^Gaussian = Φ((μ₀−μ₁)/√(σ₀²+σ₁²)).
  • Fix index collision in Eq. (3.14) (p. 9) — index i was reused for both the differentiated sample and the threshold-summation index.
  • Handle degenerate batches and regularity assumptions explicitly (Section 3.4.2 area, pp. 19–20). Bundled 5 sub-issues; verified each against real code before deciding scope with user.
  • Justify or flag as heuristic: applying ISJ (Gaussian KDE) bandwidth to the logistic kernel scale parameter.
  • Fix "small absolute score scale → undersmoothing" explanation (Section 3.4.2.1, pp. 19–20).
  • Qualify exact-FPR-target assumption (Section 3.1, p. 2) — assumes every α∈(0,1) is attained by a deterministic threshold, which fails for atomic/empirical score distributions.
  • Fix indicator-derivative-at-jump description (p. 4) — the derivative of an indicator at its jump is undefined, not "an ordinary derivative equal to infinity." Also distinguish finite-sample indicator objectives from population rates (which can be differentiable under distributional assumptions).
  • Qualify score/label reversal transformation (pp. 6–7, 14) — using −s, complemented labels, and target 1−α is correct only for reflection-equivariant fits (Gaussian, symmetric KDE); it is not automatic for every distribution family, and literal negation leaves the Beta score domain — needs a support-preserving transformation + compatible fitting rule for Beta.
  • Remove/soften claim that gradient balance implies noise robustness or population FPR control (general, throughout Ch. 3 prose) — these require additional statistical assumptions/arguments or appropriately scoped empirical evidence; should not be presented as direct consequences of the audited identities.
• Final pass
• Build full PDF and proofread end-to-end
• Check all cross-references (\Cref, \label) are consistent after restructuring
• Verify Hebrew ToC entries match English section titles
• Rerun keel noised experiments — data exists in results-final/keel-noise-05/10/20
• Restructure thesis per mentor guidelines (2026-08-17)
• Get rid of TPR results
• re-integrate results for banking
• integrate food101n results when done

Writing — chapters

DONE Write Ch. 1 (Introduction)

• Hook: imbalance + noise are everywhere, standard losses fail silently
• Gap: no existing method does AUC optimization + noise robustness together
• ROLL's claim: distributional objective, robustness as structural consequence
• Close with chapter roadmap

DONE Write Ch. 5 (Discussion)

• Explain WHY ROLL-AOC is noise robust (KDE distributional fit vs isolated gradients)
• Explain WHY Bank Marketing fails (open question, speculate honestly)
• Explain WHY GCE/MAE fail under asymmetric imbalanced noise
• Compare mechanism with LibAUC (same objective, different path)

DONE Write Ch. 6 (Conclusion)

• Summary: differentiable AUC surrogate via score-distribution fitting
• Key result: noise robustness validated on KEEL (5/10/20%) and CIFAR-10N (≤40%)
• Limitations: Bank Marketing, 99-pass cost, bandwidth scheduler hyperparameter
• Future work: learned bandwidth, partial-AUC variant, multi-class extension

DONE Finish Ch. 4 (Results) — remaining stubs

• Write Summary section (sec:experiments:summary) — key points noted in comments
• Decide: write or drop Ablation Study and Sensitivity Analysis sections
• Fill all \cite{TODO} placeholders in experiments.tex

DONE Review Ch. 4 (Results) end-to-end - Check narrative flow after noise section was added - Confirm KEEL/CIFAR-10N/large-dataset table interpretations are accurate - Verify roll-tpr90 is correctly described as secondary/special-case throughout

Writing — front/back matter

DONE Write English abstract

DONE Write Hebrew abstract

DONE Fill Acknowledgements (Main_pages/acknowledgements.tex)

Writing — Ch. 2 & 3 loose ends

DONE Verify alpha notation w.r.t. FPR — eq:fpr-cdf writes FPR(τ) = F̂₀(τ) and threshold as τ = F̂₀⁻¹(α), but standard FPR = P(score > τ | y=0) = 1−F₀(τ) requires τ = F̂₀⁻¹(1−α). Check for sign/notation inconsistency throughout method.tex and appendix_gauss_grad.tex.

DONE Add graphs of CDF tracking for gaussian, KDE, roll.

DONE Cite in sec:related-work (Ch. 2):

• Benchmarking Deep AUROC Optimization (arXiv 2203.14177) — cite in 2.2 as survey of AUROC losses; finding: composite losses best overall, barrier hinge best under noise
• LibAUC pAUCLoss / SOPAs (arXiv 2203.01505) — cite in 2.3 as partial AUC baseline
• "Closing the Approximation Gap" (arXiv 2512.01213, Dec 2025) — cite in 2.3 as recent concurrent work on partial AUC
• "When AUC meets DRO" — cite in 2.3 as DRO-based partial AUC approach

Experiments — open questions

DONE Debug banking dataset — results unexpectedly low (roll-aoc 0.511 vs GCE 0.746), investigate root cause

DONE Check if LibAUC pAUCLoss (one-way partial AUC / SOPAs) should be added as baseline

• Already have LibAUC as dependency; low-cost to add
• Most direct competitor to roll-tpr90 (both target a fixed FPR region)
• See: https://docs.libauc.org/examples/pauc.html

Mentor mathematical audit fixes (2026-09-07, thesis_mathematical_audit.pdf)

DONE [A] Decide bandwidth backprop convention (Eq. 3.16, Appendix C, Section 3.4.2, thesis pp. 25/19) — Appendix C's backward pass treats only the j=i kernel term as score-dependent, which is only valid for a FIXED bandwidth, but Section 3.4.2 estimates bandwidth from the class scores (ISJ/Silverman). Audit gave an exact counterexample where the true adaptive loss is constant but Eq. (3.16) gives a nonzero gradient.

RESOLVED (2026-09-09): chose (a) frozen-bandwidth (stop-gradient on v_0,v_1 in the backward pass). Full details in thesis/method under "Audit: Bandwidth Backprop Convention". Added Observation 3.4 "Gradient Sign Correctness" to sec:roll-properties in method.tex; new Appendix appendix:bandwidth-freeze with two sympy-verified counterexamples (exact cancellation + persistent sign reversal with closed-form proof). CONFIRMED with user: impl/src/roll.py's bandwidth computation already assumes frozen bandwidth — writeup-only fix, no code change needed. Fully closed.

DONE [A] Fix bisection stopping criterion to certify loss accuracy, not just CDF residual (Appendix E, p. 31) — current rule |F̂₀(τ̂) − (1−α)| < 1e-4 can pass while F̂₁(τ̂) is off by ~1 (counterexample given with h₀=1, h₁=0.01).

RESOLVED (2026-09-09): replaced the CDF-residual stopping rule with a certified loss-interval rule — keep bracket [lo,hi] valid, stop when F̂₁(hi) − F̂₁(lo) ≤ ε_L (exact via monotonicity of F̂₁ + the bisection bracket invariant containing the true root; no derivative/Lipschitz bound needed — that alternative was considered and rejected as strictly weaker, see thesis/method).

• Rewrote appendix_kde_bisection.tex (appendix:kde-bisection): new "Stopping criterion" + "Why the residual-only rule fails" paragraphs, with a constructed, code-verified counterexample (B_0={-30,-25,5,45,60}, v0=0.01, v1=1000: old rule stops 8 iterations in with tau off by only 0.014, but loss F̂_1 off by 0.40; new rule gets F̂_1 accurate to 3e-5). Rebuilt thesis clean.
• Code fix in impl/src/roll.py: both `_icdf` and `_icdf_batch` in KernelizedROLLoss now take v1/scores_pos as extra args and stop on the certified F̂_1(hi)-F̂_1(lo) bound instead of the F̂_0 residual; forward() call site updated to pass v1, scores_pos through.
• Regression-tested on KEEL glass1 with roll-aoc (seed 0, CPU, 1 episode) via git-stash old-vs-new: AUC 0.8052 (old) vs 0.8106 (new), essentially identical. Fully closed.

DONE Restrict initial bisection bracket to valid α range — use exact target-dependent bracket (Appendix E, p. 30) — old bracket [s⁰_min−10h₀, s⁰_max+10h₀] only guaranteed containment for α∈(4.5e-5, 1-4.5e-5), not all of (0,1).

RESOLVED (2026-09-09): switched to the exact bracket lo/hi = s⁰_min/max + h₀·log(p/(1−p)), p=1−α.

• Proved in appendix_kde_bisection.tex (new Proposition "Exact bracket containment" + short proof): the kernel term at the min/max score equals p exactly at lo/hi (direct substitution), and monotonicity of σ forces every other term below/above it, so F̂₀(lo) < p < F̂₀(hi) holds for EVERY p∈(0,1), whenever the batch isn't degenerate (min<max — deferred to the still-open "degenerate batches" TODO item, not claimed as solved here). Verified the algebraic identity with sympy and stress-tested the containment claim over 2000 random configs (0 failures, including p down to 1e-10) before writing the proof up. Rebuilt thesis clean.
• Code fix in impl/src/roll.py: `_icdf` and `_icdf_batch` bracket init now use the h0·log(alpha/(1-alpha)) offset instead of a fixed ±10/v0 margin.
• Caught and fixed a real precision bug introduced along the way: `_icdf_batch` originally cast `alphas` to the (float32) working dtype BEFORE computing log(alpha/(1-alpha)) — for alpha within ~1e-7 of 1, `1-alpha` catastrophically collapses to exactly 0 in float32, giving offset=inf and tau=inf. Fixed by computing the offset in float64 (numpy) first, then casting only the final (well-scaled) offset. Doesn't affect real thesis alpha range (0.01-0.99, AOC sweep) but was a genuine bug in the new code, caught by deliberately stress-testing extreme alphas rather than just the practical range.
• Verified with a short (15-epoch) smoke test on KEEL glass1 — no NaN/inf/errors, test AUC 0.797 after just 15 epochs (sane for so few epochs). Not a full regression comparison (not needed since practical alpha range was already safe under the old bracket; user explicitly asked for a short smoke test only, not a full KEEL retest, for this item).
• Debug script and result dir were temporary, cleaned up after (not committed).

DONE Fix score-normalization argument contradiction (Section 3.4.1, p. 18) — claims both that normalization keeps sigmoid args near unit scale AND that those args are scale-invariant; these can't both explain overflow prevention. Also missing chain-rule term (Σ s_j q_j / c² · ∂c/∂s_i) when c=σ̂_B is not explicitly detached.

RESOLVED (2026-09-09): user proposed and we proved a much stronger, cleaner result than a mere caveat — see thesis/method "Audit: Score Normalization" for full derivation. Three-part resolution:

1. Proposition (proved + numerically verified to 5.6e-9 on Silverman): under frozen bandwidth AND frozen normalization constant c, IF the bandwidth estimator is exactly scale-equivariant (ĥ(a·x)=a·ĥ(x), true for Silverman), normalizing scores changes NEITHER the loss NOR the gradient — exact, not approximate. The 1/c chain-rule factor exactly cancels the c introduced into v_k by the equivariant rescaling.
2. Directly measured (KDEpy improved_sheather_jones vs silvermans_rule across scales 1e-6 to 1e6): Silverman's h/scale is exactly constant (0.353732); ISJ's is NOT (0.00116 → 0.1168 → 0.2177 across scale regimes) — ISJ is NOT scale-equivariant. So for ISJ (the actual primary method), both loss and gradient DO depend on normalization (measured example: loss 0.734 raw vs 0.767 normalized, max grad diff 0.014) — meaning normalization is doing real, necessary work for the method actually used, just not via the "invariant argument" mechanism the old text claimed.
3. The missing chain-rule term is real when c is NOT frozen (verified: same "wrong sign" failure mode as bandwidth-freeze — a real y_i=1 sample has gradient -0.00181 under frozen-c (correct, satisfies Obs. 3.4) vs +0.00020 under fully-differentiated-c (wrong sign, violates Obs. 3.4)). No exact-cancellation rescues it here (would need Euler's identity, which requires differentiating bandwidth too — already ruled out). So: freeze c for the same reason we froze v_0,v_1.

• Writeup: rewrote sec:roll-numerical-stability in method.tex (Proposition statement + proof pointer, ISJ explanation, frozen-c convention). New Appendix appendix:score-normalization (appendix_score_normalization.tex, appended at end as נספח יא, no relettering) with the full proof, the ISJ measurement table, and the missing-term derivation + sign-violation example, mirroring appendix:bandwidth-freeze's structure.
• No code change needed: `KernelizedROLLoss.backward()` is a custom torch.autograd.Function and never had a path to differentiate through score_scale in the first place (custom Functions require every dependency to be manually coded), so it already implements exactly the frozen-c gradient we just proved correct. Writeup-only fix, like the bandwidth item. Rebuilt thesis clean, verified no undefined refs in final build pass.

DONE Replace numerically vulnerable cached-sigmoid derivative (Eq. C.5, p. 27) — S(z)(1−S(z)) underflows to exactly 0 when S(z) rounds to 1 even though the true derivative is representable (e.g. S(20) in fp32).

RESOLVED (2026-09-09): discussed with user whether the log-space fix was worth it "if time/effort didn't matter" — confirmed yes, it strictly dominates (no realistic failure point vs. a cheaper alternative's ~z=88 cutoff), and user asked for it in full despite the cost, explicitly "don't need numerical exactness, but do need to test on keel."

• Confirmed via direct measurement that this is a LIVE bug, not hypothetical: roll.py backward() computes `kernel = s_neg*(1-s_neg)` where s_neg=sigma(v0,diff) — sigma saturates to exactly 1.0 in fp32 once v0*diff exceeds ~17, so 1-s_neg=0 exactly there, even though the true sigma' stays representable to ~z=88. Constructed a batch where ALL negative scores exceed this threshold: cached form gives kernel=[0,0,0,0], collapses to the existing `.clamp(min=1e-10)` safety floor, and grad_tau_neg becomes exactly [0,0,0,0] — the ENTIRE negative class loses its tau-mediated gradient for that batch, not just degraded precision. Risk increases over training since KernelScheduler shrinks bandwidth (v0 grows) as separation improves.
• Fix: log-space computation. ell_i = log(sigma'(z_i)) computed directly via the stable form -|z_i| - 2*softplus(-|z_i|) (verified via sympy to match log(sigma') exactly, both symbolically at v=1 and numerically at v!=1 — the missing "+log(v)" constant cancels in the normalized ratio regardless), then normalize via softmax (=exp(ell_i - logsumexp_j ell_j)), which never overflows (dominant term always exactly 1 pre-normalization) and lets truly negligible terms harmlessly vanish rather than colliding everything to 0 simultaneously.
• Thesis: added new "Numerical stability of the efficient form" paragraph (sec:kde-grad-stability) directly in appendix_kde_grad.tex, right after the original eq:kde-grad-tau-efficient derivation (NOT a new appendix chapter — this is a small, self-contained formula-level fix, unlike the bandwidth/normalization items which needed full counterexample-driven appendices). States plainly this changes only the arithmetic path, not the derivation — same ratio, same eq:kde-grad-tau-efficient target quantity.
• Code: added `KernelizedROLLoss._log_sigma_prime(v,x)` static method (roll.py, right after `_sigma_prime`) implementing the stable log-derivative via `torch.nn.functional.softplus`; backward()'s negative-class branch now does `log_kernel = _log_sigma_prime(v0, diff_neg); grad_tau_neg = torch.softmax(log_kernel, dim=1)`, replacing the vulnerable two-liner. `s_neg` variable no longer needed, removed.
• Verified: matches direct sigma_prime ratio to float32 precision (diff 3e-8) in the normal regime; in the saturating batch that broke the old code, new code gives a meaningful, correctly-decaying weight distribution (not all-zero) — even better than the "just use _sigma_prime directly" fallback considered earlier, since log-space never materializes the raw value at all.
• Tested on KEEL glass1, roll-aoc, 200 iters (not a full old-vs-new stash comparison this time — user said exactness wasn't needed): no NaN/errors, best epoch 103/200, val loss 0.0913, test AUC 0.8047 — consistent with all prior runs on this identical setup (0.8052-0.8106 range across the other fixes' regression tests). Debug script and result dir temporary, cleaned up, not committed.
• Rebuilt thesis clean, confirmed sec:kde-grad-stability label resolves in final build pass.

DONE Weaken "balanced gradient mass" claim from Proposition 3.1 (Section 3.3.2, Appendix I, pp. 13–14, 34–35) — the proof is correct (signed classwise score-gradient sums are ±F̂₁'(τ)), but this does NOT imply equal absolute gradient mass, and says nothing about ‖G₀‖ vs ‖G₁‖ for the actual parameter gradients (Jacobians differ).

RESOLVED (2026-09-09): tightened the lead-in prose before Prop. 3.1 (dropped "balanced mass"/"equally" language, states only the signed-sum cancellation). Added 3 new paragraphs after the ROLL-AOC corollary, before "Gradient sign correctness": (1) signed-vs-mass distinction, noting they DO coincide for KDE-ROLL specifically because Observation 3.4 (Gradient Sign Correctness) guarantees uniform sign within each class; (2) a verified Gaussian ROLL counterexample where they DON'T coincide — B_1={1,2,3,50}, tau=0: the outlier at 50 gets a gradient of the WRONG sign (+0.000632 for y_i=1) because eq:gauss-grad-y1's bracket (1+(tau-mu1)(f_i-mu1)/sigma1^2) goes negative when (tau-mu1)(f_i-mu1) < -sigma1^2 — this happens precisely because Gaussian ROLL fully differentiates through sigma_1 rather than freezing it (same pathology pattern as the other 5 fixes, verified against autograd, signed sum -0.0153 vs absolute sum 0.0166); (3) parameter-space scope note (score-gradient balance doesn't imply parameter-gradient balance, since that needs each sample's Jacobian too) — reworded per user feedback to be a plain scope clarification, not framed as a "contradiction."

DONE Fix necessity claim re: translation equivariance for balance identity (p. 14) — translation equivariance is sufficient for (3.17) but not necessary; counterexample F̂_k(t;s)=Φ(t−2s̄) doesn't have the stated unit-translation property yet still cancels.

RESOLVED (2026-09-09): verified the counterexample symbolically with sympy — F̂_k(t;s+δ)=Φ(t-2s̄-2δ) ≠ Φ(t-δ-2s̄)=F̂_k(t-δ;s) (translation invariance genuinely fails, off by a factor of 2 in δ), yet applying this same construction to both classes still gives Σ_B1∂L/∂f_i = -Σ_B0∂L/∂f_i exactly (confirmed: sum of both symbolic gradients simplifies to 0). Rewrote the "condition is necessary" sentence in method.tex to "sufficient, not necessary," with this example inline. Rebuilt thesis clean.

DONE Qualify gradient-locality claim (Observation 3.3, Figure 3.3, pp. 15–17) — fixed-bandwidth proportionality to kernel density at threshold is correct, but "distant samples have negligible gradients" is false once negative-class weights are normalized by total kernel mass.

RESOLVED (2026-09-09): verified with sympy + the actual implemented kernel that the claim genuinely splits by class. y_i=1: gradient magnitude = sigma_1'(f_i-tau)/n1, an ABSOLUTE unnormalized function of the sample's own distance — "far implies negligible" holds without qualification here, no issue. y_i=0: gradient magnitude = sigma_0'(tau-f_i)/sum_j sigma_0'(tau-f_j), a NORMALIZED SHARE of total class-0 kernel mass — this does NOT vanish just because a sample is far from tau in absolute terms, only if it's far RELATIVE to its classmates. Counterexample verified symbolically and numerically: two negative samples at tau+M and tau+M+1 (fixed unit separation) — as M->infinity, the closer sample's normalized weight converges to 1/(1+e^-v0) (~0.731 at v0=1), NOT to 0, confirmed stable from M=20 to M=1000 even though the raw (unnormalized) sigma_0' values have themselves underflowed to exactly 0.0 in float32 by M=1000.

• Nice bonus connection: verified the log-space fix from the cached-sigmoid-derivative item (sec:kde-grad-stability) ALREADY correctly preserves this ratio out to M=1000 (via softmax's max-subtraction), where the OLD cached-identity approach would have collapsed to a 0/0-type failure. No new code fix needed for this item — the earlier fix already covers it as a side effect.
• Writeup: rewrote the "Gradient locality" paragraph in sec:roll-properties (method.tex) to explicitly split the two cases (absolute for y=1, relative-share for y=0) with the verified counterexample inline; split Observation 3.3 into explicit (a)/(b) sub-claims matching the two mechanisms; softened the figure's lead-in sentence and caption (fig:threshold-locality) to note it illustrates the case where both classes are broadly spread around tau (where the two behaviors happen to agree), rather than claiming universality. Did not need to regenerate the figure itself — it's a fair illustration of the typical/generic case, only the caption's blanket wording was the problem.
• Rebuilt thesis clean.

DONE Separate exact fitted-distribution AOC from empirical/population AOC of original scores (Sections 3.1–3.3, Figure 3.2, p. 11) — the continuous integral L^fit_AOC = P(T1≤T0) is exact only for the FITTED distributions; it is not generally the empirical/population AUC of the original scores.

RESOLVED (2026-09-09): verified against the real kernelized_roll_aoc code on a perfectly-separated finite sample (empirical AUC=1.0 exactly via sklearn) — ROLL-AOC's implied fitted AUC is 0.99994 (loss 6e-5) for widely-separated classes, and 0.971 (loss 0.029) when the separating margin is tiny relative to within-class spread. Confirms KDE smoothing means the fitted-distribution AOC structurally never reaches the ideal step-function value even at perfect empirical separation (same "smoothing gap" any smooth surrogate has relative to its sharp target).

• User's call: don't add the numeric example to the thesis, these are extreme/edge cases — just soften the overclaiming wording. Reworded the two "faithful surrogate…regardless of how scores evolve" / "most faithful surrogate" sentences in sec:roll-instantiations (method.tex, right before sec:roll-kde) to "adapts to the score distribution's shape… without committing to a parametric assumption" and "most adaptable surrogate" — drops the implied exactness/fidelity claim while keeping the real comparative point (non-parametric > parametric when the shape is unknown). Rebuilt thesis clean.
• Note for future reference (not in thesis): the verified gap numbers are in this org entry if a stronger, example-backed version of this fix is ever wanted later.

DONE Fix Figure 3.2 caption / distinguish 99-point average from the integral identity (p. 9–10, Eq. 3.13) — L_99 is a quadrature approximation, not identical to the integral AOC.

RESOLVED (2026-09-09): derived and verified the Darboux bound — since TPR(alpha) is monotone along the ROC curve (structural, always true), g(alpha)=1-TPR(alpha) is monotone and bounded in [0,1], so a 99-point average differs from the true continuous integral by at most 1/99≈0.0101 by a standard Darboux-sum argument. Stress-tested numerically against 500 random monotonic bounded functions: worst observed gap 0.0033, comfortably under 1/99.

• Added one sentence right after eq:roll-aoc in method.tex stating this bound, with its precise scope: covers ONLY the quadrature step, does NOT extend to the gradient (eq:roll-aoc-gradient), does NOT cover the KDE-fitted-vs-empirical gap (previous fix), does NOT cover residual bisection numerical error in tau_k.
• Also lightly adjusted the fig:roll-aoc-roc caption ("gives the value of" -> "approximates the continuous integral, giving the value of") so it doesn't read as if the discrete average and the true integral are definitionally the same thing.
• User's call: no separate appendix/proposition needed, one sentence + caption tweak sufficient. Rebuilt thesis clean, confirmed the forward reference to eq:roll-aoc-gradient resolves.

DONE Narrow "direct integration is not tractable" claim before Eq. (3.13) (p. 9) — false for the Gaussian case, which has closed form L_AOC^Gaussian = Φ((μ₀−μ₁)/√(σ₀²+σ₁²)).

RESOLVED (2026-09-09): verified the closed form numerically (matches direct quadrature to 1e-9) and re-derived it via the standard Gaussian identity ∫Φ(a+bu)φ(u)du=Φ(a/√(1+b²)).

• method.tex sec:roll-aoc: scoped the claim to "not tractable in general" + parenthetical noting the Gaussian exception and that the 99-point quadrature is used uniformly across instantiations for implementation consistency, not necessity.
• appendix_gauss_grad.tex: added new "Closed-form AOC" paragraph (right before "Gradient derivation") with the full derivation and eq:gauss-aoc-closed-form, since this fact wasn't documented anywhere previously.
• CAUGHT AND FIXED a real notational collision while writing this: sec:roll-formulation (method.tex ~line 156) already used Φ for the ABSTRACT DISTRIBUTION FAMILY'S PARAMETER SPACE ({P_phi : phi in Phi}), clashing with Φ as standard-normal-CDF (needed here, and already used in the translation-equivariance counterexample fix earlier this session). Renamed the parameter-space symbol phi/Phi -> psi/Psi throughout sec:roll-formulation (6 occurrences: distribution family def, hat-phi_1/hat-phi_0 fitted params, CDF/quantile-function definitions) — theta/Theta was considered and rejected since it'd collide with f_theta (model parameters), used everywhere. psi/Psi unused elsewhere in the thesis, clean choice. Confirmed related_work.tex's unrelated phi (hypothesis-testing notation, different chapter) doesn't collide.
• Rebuilt thesis clean, confirmed no undefined refs in final pass.

DONE Fix index collision in Eq. (3.14) (p. 9) — index i was reused for both the differentiated sample and the threshold-summation index.

RESOLVED (2026-09-09): renamed the threshold-summation index from i to r in eq:roll-aoc-gradient (method.tex, right after eq:roll-aoc), keeping i for the differentiated sample per the rest of the chapter's convention. Added a one-line parenthetical clarifying the two roles. Left eq:roll-aoc (3.13) itself untouched — it has no outer per-sample index to collide with, so its use of i as the threshold-sum index there is fine in isolation. Trivial fix, rebuilt thesis clean.

DONE Handle degenerate batches and regularity assumptions explicitly (Section 3.4.2 area, pp. 19–20). Bundled 5 sub-issues; verified each against real code before deciding scope with user.

RESOLVED (2026-09-09). Verified findings:

• Constant/singleton class scores: tested directly — does NOT crash. KDEpy's own silvermans_rule internally detects "too many identical values" and substitutes bw=1.0 (with a warning) before our max_v=10000 cap even applies. Doesn't happen with real continuous model outputs; user confirmed treat as non-issue, just document the fallback.
• Single-class batch (one class entirely empty): tested directly — DOES crash (ValueError from KDEpy, "Data must be of length > 0"). User: important point, doesn't happen in practice because every experiment uses a balanced batch sampler (is_balanced=True), and this requirement isn't ROLL-specific — ANY ROC-based loss needs both classes present since TPR/FPR are each conditional on one class alone.
• Bandwidth cap (v<=10000, i.e. h-floor 0.0001): breaks the exact scale-equivariance hypothesis of prop:normalization-invariance once triggered. User: last-resort-only, rare, and even rarer after this session's other numerical fixes (log-space grad, certified bisection) — footnote, not a big deal.
• Gaussian zero-variance (sigma_k=0): analogous issue, unsupported currently. User: edge case footnote, doesn't happen under normal circumstances.
• General regularity (nonzero density at the inverted quantile): user wanted this included as the umbrella statement tying 1+3 together.

User's final scope decision: state #2 (both classes present) prominently as an important standing assumption tied to the balanced sampler; discuss #3 and #5 together using #1 (constant scores) as the motivating example, as a footnote-weight aside not a big fix; add #4 as a separate short footnote. Implemented exactly that:

• New "Regularity assumptions" paragraph at the end of sec:roll-formulation (method.tex, right before sec:roll-aoc) stating the both-classes-present assumption plainly, explaining it's inherent to any ROC-based objective, and noting it's enforced via the balanced sampler rather than in the loss math.
• sec:kde-bandwidth (method.tex): corrected "Silverman's rule… always produces a valid bandwidth" (not quite true without the library's internal override) to explain the constant-scores case and KDEpy's bw=1.0 fallback, with a footnote on the v<=10000 cap (ties to prop:normalization-invariance, notes it's essentially never reached now) and a closing sentence generalizing to the nonzero-density-at-quantile assumption.
• appendix_gauss_grad.tex: footnote on sigma_k=0 attached to the first mention of sigma_k^2 in the lead sentence (not bare after the equation — footnotes need inline text to attach to).
• Rebuilt thesis clean, no undefined refs in final pass.

DONE Justify or flag as heuristic: applying ISJ (Gaussian KDE) bandwidth to the logistic kernel scale parameter.

RESOLVED (2026-09-09): derived the actual AMISE-optimal Gaussian-to-logistic-kernel conversion factor for context (h*_logistic/h*_gaussian ≈ 0.559, using sigma_K^2=pi^2/3, R(K)=1/6 for logistic vs sigma_K^2=1, R(K)=1/(2*sqrt(pi)) for Gaussian, both verified numerically to full precision) — i.e. ISJ's raw output is ~79% too large relative to what would be logistic-kernel-optimal, if the theory is taken at face value.

• Chose option B (don't apply the correction): unlike every other item this session, actually applying this fix would CHANGE the bandwidth used in every training run — a real algorithmic change requiring rerunning all reported experiments, not a no-op/writeup fix. User's call: leave the code as-is, be honest in the writeup that it's a heuristic.
• method.tex sec:kde-bandwidth: added 2 sentences right after introducing ISJ — states plainly that ISJ's optimality is proven for Gaussian kernels, the logistic kernel is used instead, this is "a plug-in heuristic approximation chosen for its ready availability in the existing implementation," with a pointer to future work.
• discussion_and_conclusions.tex sec:conclusions:future: added new "Kernel-appropriate bandwidth selection" paragraph (distinct from the existing "Adaptive bandwidth estimation" paragraph, which covers per-sample/local adaptivity — a different concern from the Gaussian-vs-logistic kernel-convention mismatch), stating deriving/validating a logistic-kernel-appropriate bandwidth rule is left to future work.
• Did NOT include the derived 0.559 factor itself in the thesis text (kept out per not wanting to imply a correction that wasn't applied); it's preserved here in org-roam if wanted later.
• Rebuilt thesis clean, confirmed the cross-chapter reference (method.tex -> discussion_and_conclusions.tex) resolves.

DONE Fix "small absolute score scale → undersmoothing" explanation (Section 3.4.2.1, pp. 19–20).

RESOLVED (2026-09-09): verified the audit's claim directly — under the homogeneous (uncapped Silverman) bandwidth rule, raw gradient magnitude scales as EXACTLY 1/a under s->as rescaling (confirmed numerically: grad*a constant to full precision across a=1e1 down to a=1e-6). So the thesis's claim ("small sigma_hat -> small h -> low gradients") is actually backwards: shrinking absolute scale GROWS the raw gradient, not shrinks it, under exact scale equivariance.

• Considered but did not pursue further: two more precise candidate mechanisms (poor class separation early in training; ISJ's non-scale-equivariance already documented in appendix:score-normalization). User chose the simpler of the two to cite: poor class separation, framed as "a plausible explanation," not proven.
• sec:kde-bandwidth-scaling (method.tex): replaced the disproven causal claim with an empirical statement (scheduler helps convergence, observed in practice) + the class-separation explanation, explicitly hedged as plausible rather than asserted as fact. Kept the existing "solution: bandwidth scheduler" paragraph as-is (already empirical, not overclaiming).
• Rebuilt thesis clean.

DONE Qualify exact-FPR-target assumption (Section 3.1, p. 2) — assumes every α∈(0,1) is attained by a deterministic threshold, which fails for atomic/empirical score distributions.

RESOLVED (2026-09-09): user wanted the simple version — state the continuity/strict-monotonicity requirement directly in the FPR^-1 definition (method.tex:34), plus a footnote pointing forward to the instantiations used in this thesis (KDE sec:roll-kde, Gaussian appendix:gauss-grad, Beta appendix:beta-roll) confirming each has a continuous strictly-increasing CDF by construction, so the assumption holds automatically for what ROLL actually optimizes over. Did not pursue the heavier options (generalized ROC convention, randomized thresholding) — not needed given the fitted distributions already satisfy the requirement. Rebuilt clean, confirmed the three forward references resolve.

DONE Fix indicator-derivative-at-jump description (p. 4) — the derivative of an indicator at its jump is undefined, not "an ordinary derivative equal to infinity." Also distinguish finite-sample indicator objectives from population rates (which can be differentiable under distributional assumptions).

RESOLVED (2026-09-09): method.tex sec:differentiability-problem (~line 114). Replaced "the derivative of TPR/FPR w.r.t. f_theta(x_i) becomes infinite" with a precise one-sided-limit account: approaching the jump from below leaves the indicator at 0 while approaching from above sends the difference quotient to +infinity, so the two one-sided derivatives disagree and the derivative at the jump is undefined — not a two-sided limit that converges to infinity (unlike, e.g., a vertical tangent on a continuous function).

• Added a new paragraph distinguishing the finite-sample indicator objective from the population rate: TPR/FPR as written in this section are indicator averages over the finite sample D, where each summand 1[f_theta(x_i) > tau] is a step function of that one sample's score — this per-sample step is what's non-differentiable. The population quantities TPR(tau)=1-F_1(tau), FPR(tau)=1-F_0(tau) (F_1, F_0 the underlying class-conditional score distributions, left unhatted to distinguish from ROLL's later fitted hat-F_1/hat-F_0) are differentiable in tau whenever F_1, F_0 have continuous densities, with d(TPR)/d(tau) = -f_1(tau). So non-differentiability lives in the discrete per-sample indicators used to estimate the rate, not in the rate itself under a smooth distributional model — this is exactly the gap ROLL's score-distribution-fitting approach (sec:score-distribution-fitting, immediately following) exploits.
• Writeup-only fix, no code change. Rebuilt thesis clean (nix build exit 0).

DONE Qualify score/label reversal transformation (pp. 6–7, 14) — using −s, complemented labels, and target 1−α is correct only for reflection-equivariant fits (Gaussian, symmetric KDE); it is not automatic for every distribution family, and literal negation leaves the Beta score domain — needs a support-preserving transformation + compatible fitting rule for Beta.

RESOLVED (2026-09-09): user chose option (a), keep the negation-based derivation as the general-case default, and add a footnote scoping it plus giving the Beta-specific fix — no rederivation of the main text needed.

• method.tex: added a footnote right after the existing negation/label-swap sentence (end of the paragraph introducing eq:roll-tpr-at-fpr / eq:roll-fpr-at-tpr, ~line 220) stating the negation trick requires the fitted family to be closed under score negation (true for Gaussian, symmetric KDE) and is invalid for Beta ROLL since its scores live in (0,1) and negation leaves that support.
• Beta fix (user's proposal, verified before writing up): replace $f_\theta(\mathbf{x}) \to -f_\theta(\mathbf{x})$ with $f_\theta(\mathbf{x}) \to 1-f_\theta(\mathbf{x})$ — stays in (0,1), and is already computed in the Beta forward pass as the complementary score $t^{(k)}=1-s^{(k)}$ (appendix_beta_roll.tex eq:beta-denom/eq:beta-params).
• Verified with sympy/numpy before writing anything (never trusted mentally, per usual convention): (1) numerically confirmed $X\sim\text{Beta}(a,b) \Rightarrow 1-X\sim\text{Beta}(b,a)$ exactly (sympy's symbolic beta() didn't auto-cancel $B(a,b)$ vs $B(b,a)$, so checked numerically instead — matched to 1e-9 across 5 random parameter draws); (2) confirmed the log-moment estimator $D_k$ (eq:beta-denom) is exactly symmetric under swapping which vector is called $s^{(k)}$ vs $t^{(k)}$, so applying the estimator to $1-s^{(k)}$ exactly swaps $\hat a_k \leftrightarrow \hat b_k$ (numerically exact, not approximate); (3) end-to-end check that the same "target rate complemented to $1-\alpha$" recovery claim used for the Gaussian/negation case holds identically for the Beta/$1-s$ case — built both a Gaussian direct-vs-transform check (sanity check of the existing thesis claim, confirmed exact match) and a Beta direct-vs-transform check using scipy.stats.beta (also confirmed exact match to float precision) using the swapped $(\hat b,\hat a)$ params as the "new" class parameters.
• Rebuilt thesis clean (nix build exit 0), footnote renders correctly with the appendix_beta_roll.tex cross-references (\Cref{appendix:beta-roll}, eq:beta-params) resolving.

DONE Remove/soften claim that gradient balance implies noise robustness or population FPR control (general, throughout Ch. 3 prose) — these require additional statistical assumptions/arguments or appropriately scoped empirical evidence; should not be presented as direct consequences of the audited identities.

Final pass

DONE Build full PDF and proofread end-to-end

DONE Check all cross-references (\Cref, \label) are consistent after restructuring

DONE Verify Hebrew ToC entries match English section titles

DONE [A] Rerun keel noised experiments — data exists in results-final/keel-noise-05/10/20

DONE Restructure thesis per mentor guidelines (2026-08-17)

• [x] Focus framing on noise resistance + AUC solution (primary story)
• [x] Minimize single-point optimization coverage (brief mention only)
• [x] Ch. 2 restructured: ROC/AUC → Noisy Labels → KDE → Imbalanced → NP
• [x] Ch. 3 restructured: AUC as primary objective, ROLL-AOC promoted
• [x] Noise results table added to Ch. 4

DONE Get rid of TPR results

DONE re-integrate results for banking

DONE integrate food101n results when done