5.4 KiB
5.4 KiB
impl/datasets
impl/datasets
Datasets with Final Results
CIFAR-10N (image)
- Size: 50,000 training images (32×32 RGB, CIFAR-10 base)
- Noise: clean 0% / aggre ~9% / worse ~40% (human annotator noise)
- Source: UCSC-REAL/cifar-10-100n — standard noisy-label benchmark (Learning with Noisy Labels, NeurIPS 2021)
- Notes: Positive class = automobile (class 1) vs rest; natural IR ~9:1. Three noise conditions all have final results. Labels replaced with real human annotation noise from crowdworkers.
Credit Card Fraud (tabular)
- Size: 284,807 transactions
- Noise: 0% (clean; real confirmed fraud labels)
- Source: Kaggle / ULB (Worldline + MLG research collaboration, 2013)
- Notes: Extreme imbalance — only 492 frauds (IR ~577:1). 30 features: V1–V28 are PCA-transformed (original features confidential), plus Time and Amount. Time/Amount rescaled; V1–V28 left as-is.
Bank Marketing (tabular)
- Size: 41,188 samples; ~48 features after one-hot encoding
- Noise: 0% (clean)
- Source: UCI Machine Learning Repository (Moro et al., 2014 — Portuguese bank direct marketing campaigns)
- Notes: IR ~8:1 (~11.3% positive). Target: whether client subscribed to term deposit. Mix of demographic, contact, and economic features.
Home Credit (tabular)
- Size: 307,511 loan applications; ~120 numerical features
- Noise: 0% (clean; proxy labels — loan repayment used as credit default indicator)
- Source: Kaggle Home Credit Default Risk competition (2018)
- Notes: IR ~11.4:1 (~8% default rate). Features from application_train.csv only (no auxiliary tables). Non-numeric and ID columns dropped; NaNs filled with 0.
KEEL Datasets (tabular, 12 with final results)
- Noise: 0% (clean — standard imbalanced learning benchmarks)
- Source: KEEL repository (Knowledge Extraction based on Evolutionary Learning)
- Notes: All small/medium tabular datasets used as standard IR benchmarks in the literature. Loaded via keel_ds Python package.
| Dataset | Total | Pos | Neg | IR | Notes |
|---|---|---|---|---|---|
| glass0 | 214 | 70 | 144 | 2.1 | Glass identification (UCI) |
| glass1 | 214 | 76 | 138 | 1.8 | Glass identification (UCI) |
| glass2 | 214 | 17 | 197 | 11.6 | Glass identification (UCI) |
| glass6 | 214 | 29 | 185 | 6.4 | Glass identification (UCI) |
| haberman | 306 | 81 | 225 | 2.8 | Breast cancer survival (1958–1970) |
| iris0 | 150 | 50 | 100 | 2.0 | Iris setosa vs others |
| new-thyroid1 | 215 | 35 | 180 | 5.1 | Thyroid function classification |
| pima | 768 | 268 | 500 | 1.9 | Pima Indian diabetes (UCI) |
| vehicle2 | 846 | 218 | 628 | 2.9 | Vehicle silhouette (UCI) |
| vowel0 | 988 | 90 | 898 | 10.0 | Vowel recognition (class 0 vs rest) |
| wisconsin | 683 | 239 | 444 | 1.9 | Wisconsin breast cancer (UCI) |
| yeast3 | 1484 | 163 | 1321 | 8.1 | Yeast protein localization (UCI) |
KEEL Dataset Class Balance (full list)
Counts from KeelDataset loader. Ratio = false/true (imbalance factor).
Two datasets fail to load (likely missing env vars or files): led7digit, page-blocks.
| Dataset | Total | True | False | Ratio | Runnable |
|---|---|---|---|---|---|
| cleveland-0_vs_4 | 177 | 13 | 164 | 12.6 | NO |
| ecoli-0-1_vs_5 | 240 | 20 | 220 | 11.0 | NO |
| glass0 | 214 | 70 | 144 | 2.1 | YES |
| glass1 | 214 | 76 | 138 | 1.8 | YES |
| glass2 | 214 | 17 | 197 | 11.6 | YES |
| glass4 | 214 | 13 | 201 | 15.5 | NO |
| glass5 | 214 | 9 | 205 | 22.8 | NO |
| glass6 | 214 | 29 | 185 | 6.4 | YES |
| haberman | 306 | 81 | 225 | 2.8 | YES |
| iris0 | 150 | 50 | 100 | 2.0 | YES |
| led7digit | N/A | NO | |||
| new-thyroid1 | 215 | 35 | 180 | 5.1 | YES |
| page-blocks | N/A | NO | |||
| pima | 768 | 268 | 500 | 1.9 | YES |
| vehicle2 | 846 | 218 | 628 | 2.9 | YES |
| vowel0 | 988 | 90 | 898 | 10.0 | YES |
| wisconsin | 683 | 239 | 444 | 1.9 | YES |
| yeast3 | 1484 | 163 | 1321 | 8.1 | YES |
Gotchas
- ISJ "Need more data" failure: cleveland (13 true), ecoli (20 true), glass4 (13), glass5 (9) — after 33% test split, too few samples in minority class for ISJ root-finding. Excluded from
run_all.sh. - glass2 (17 true) is borderline — watch for failures.
- led7digit and page-blocks fail to load entirely (missing dataset files or env vars).
- Credit card V1–V28 are pre-PCA (do NOT re-scale them); only Time (col 0) and Amount (col 29) are rescaled.
- Home Credit uses application_train.csv only — no auxiliary tables (bureau, previous applications, etc.) merged in.
- CIFAR-10N aggre noise rate is ~9%, worse is ~40% — large gap; both have completed final results.