:PROPERTIES:
:ID: 001430d5-e1e7-4e72-baf6-17399bfd6447
:END:

#+title: impl/loss-functions
#+filetags: :project: :knowledge: :loss-functions:

* impl/loss-functions

** KernelizedROLLoss — Key Design Decisions

- =_icdf= uses **bisection** (not Newton-Raphson). NR was replaced because =F_prime= goes near-zero when =v= is large and =tau= drifts, causing divergence. Bisection bracket is =[min(scores)−10/v, max(scores)+10/v]=, guaranteed to converge.
- Score normalisation (=normalize= flag): divides =yh= by its std before KDE. Can be disabled via =kernelized_roll_fpr(fpr, normalize=False)=. The chain-rule correction (multiply grad by =score_scale=) is applied in backward.
- =_grad_tau_scores_neg= denominator clamped at =1e-10= to prevent 0/0 when all sigmoid kernels saturate.
- Variance clamped at =1e-8= in normalisation to prevent =inf= when all batch scores are identical.

** KernelizedROLLoss Unification (2026-07-14)

=KernelizedROLLoss= and =KernelizedROLLossAOC= merged into a single class accepting =alphas= (list of CDF targets) and =divisor=:
- Point-FPR: =KernelizedROLLoss.apply([1−fpr], 1, yh, y, gamma, normalize)=
- AOC: =KernelizedROLLoss.apply(_AOC_ALPHAS, _AOC_DIVISOR, yh, y, gamma, normalize)=

Backward always vectorised (K-dim); K=1 reduces to the scalar case. =ctx.divisor= stores the non-tensor divisor.

** BCE Combination

Both =kernelized_roll_fpr= and =kernelized_roll_tpr= accept =bce_weight= (default 0.0) and =bce_pos_weight= (default 1.0).
Loss = ROLL + bce_weight × BCE. Standard value: =bce_weight=0.5=, =bce_pos_weight=num_false/num_true=.

** ISJ Bandwidth Failure

ISJ fails when a class has too few samples in a split (~<20). Affects: cleveland, ecoli, glass4, glass5.

** Gamma Scheduling

=KernelScheduler= widens KDE kernels early then anneals. Default: =initial_gamma=100=, =decay=0.5=, =decay_every=500=. Gamma is a *divisor* of =v=.

** LibAUC Integration (2026-07-14, tuned 2026-07-14)

=libauc_auc_loss= in =src/roll.py= wraps =AUCMLoss= (Yuan et al., ICCV 2021) to the =(yh, y)= convention.
- Applies =torch.sigmoid(yh)= before passing to AUCMLoss (squared-hinge surrogate, no built-in sigmoid).
- AUCMLoss has learnable params =a, b, alpha= — must use PESG optimizer (not Adam).
- =opt_factory= signature: =opt_factory(model)= — does NOT take criteriorator (would be unbound at call site in =_perform_episode=).
- =opt_factory= pattern: =loss_fn.pesg_opt_factory(...)= returns picklable =_PESGFactory= sharing the same inner =AUCMLoss= instance as the criteriorator's loss func (preserved by pickle memo).
- =ExperimentConfiguration= gained =opt_factory: callable = None=; overrides =optim_class/optim_args= when set.
- =tqdm= is an undeclared dep of libauc — must be in =propagatedBuildInputs= in flake.nix.

*** Recommended settings (v2 + adam, tuned 2026-07-14)
- =version='v2'= — removes class prior from formulation; LibAUC explicitly recommends over v1; default in =libauc_auc_loss=.
- =mode='adam'= in PESG — Adam-style primal updates, converges much faster than SGD on small datasets; default in =pesg_opt_factory=.
- =imratio= passed explicitly as =num_true / (num_true + num_false)= — avoids noisy mini-batch estimates on imbalanced data. Note: v2 ignores imratio (prior-free), so this only matters if reverting to v1.
- =lr=0.1= (PESG default, much larger than Adam's 1e-3). If diverging: try =epoch_decay=0.0= or drop lr to =0.01=.

** Grace Period (BasicCriteriorator / CRBasedCriteriorator, 2026-07-14)

=grace_period: int = 0= parameter added to both criteriorators. During the first =grace_period= epochs:
- =best_flag= is always False (no model snapshot taken)
- Patience counter does not increment (early stopping cannot fire)

All keel configs use =grace_period=50=. Prevents noisy early-epoch models from being selected as best.