* impl/paper-beyond-rebalancing

[[id:151d5686-6f40-4158-a59a-b0be94cdc969][impl/research]]

*Beyond Rebalancing: Benchmarking Binary Classifiers Under Class Imbalance Without Rebalancing Techniques*
2024 — arXiv:2509.07605

** Problem

The imbalanced learning literature almost exclusively evaluates methods *with* rebalancing (SMOTE, oversampling, etc.). This paper asks: which classifiers are intrinsically robust to class imbalance, with no rebalancing at all?

** Setup

- 19 real-world UCI/Kaggle datasets (IR 0.0015–0.54) + 5 synthetic datasets
- Synthetic decision boundaries of increasing complexity: linear → moderate non-linear → non-linear+redundancy → Gaussian quantiles → XOR (hardest)
- Minority class progressively reduced to 100%, 50%, 25%, 10%, 5%, 1%, plus one-shot/few-shot (k=1,3,5)
- 12 classifiers; 2×5-fold stratified CV

** Classifiers Tested

Traditional: Decision Tree, k-NN, SVM
Ensemble: Random Forest, XGBoost, LightGBM, CatBoost, BaggingRF, RUSBoost
Advanced: [[id:bf0fc08a-e806-48df-b188-7a2c4c41c693][impl/paper-tabpfn]]
One-class: OCSVM, Isolation Forest, LOF

** Metrics

AUC-ROC, AUC-PR, F1, G-mean, Accuracy, Precision, Recall

** Key Findings

1. *TabPFN wins overall* — best performer at all imbalance levels including extreme; only method that holds up one-shot/few-shot
2. *Ensembles second* — CatBoost, XGBoost, LightGBM degrade moderately; RF degrades faster
3. *Traditional classifiers collapse* — DT and k-NN fail sharply below 25% minority
4. *Decision boundary complexity is a major factor* — on linear data, most classifiers survive extreme imbalance; on XOR, nearly all collapse
5. Practical advice: use [[id:bf0fc08a-e806-48df-b188-7a2c4c41c693][impl/paper-tabpfn]] or CatBoost/SVM when rebalancing is not feasible

** Datasets Used (real-world)

Breast Cancer, Pen Local/Global, Letter, Annthyroid, Satellite, Glass, Segment, Pima, Yeast4/5/6, Abalone/Abalone9-18, Ecoli4, PC1/CM1/KC1/KC2 — all UCI

** Relevance to ROLL

- Directly in ROLL's territory: binary tabular classification under imbalance, AUC/G-mean metrics
- Strong candidate as a baseline paper to cite
- Does *not* use any custom loss or ROC-optimization — ROLL's TPR-at-FPR objective is orthogonal and potentially more practically useful
- Dataset list is a good target for ROLL coverage: Annthyroid, Abalone9-18, Satellite, Yeast4/5/6 are missing from ROLL (see [[id:151d5686-6f40-4158-a59a-b0be94cdc969][impl/research]] gap table)