90 lines
5.4 KiB
Org Mode
90 lines
5.4 KiB
Org Mode
:PROPERTIES:
|
||
:ID: b8a9886a-d349-43e5-a745-817a148c1fd8
|
||
:END:
|
||
|
||
#+title: impl/datasets
|
||
#+filetags: :project: :knowledge: :datasets:
|
||
|
||
* impl/datasets
|
||
|
||
** Datasets with Final Results
|
||
|
||
*** CIFAR-10N (image)
|
||
- *Size*: 50,000 training images (32×32 RGB, CIFAR-10 base)
|
||
- *Noise*: clean 0% / aggre ~9% / worse ~40% (human annotator noise)
|
||
- *Source*: UCSC-REAL/cifar-10-100n — standard noisy-label benchmark (Learning with Noisy Labels, NeurIPS 2021)
|
||
- *Notes*: Positive class = automobile (class 1) vs rest; natural IR ~9:1. Three noise conditions all have final results. Labels replaced with real human annotation noise from crowdworkers.
|
||
|
||
*** Credit Card Fraud (tabular)
|
||
- *Size*: 284,807 transactions
|
||
- *Noise*: 0% (clean; real confirmed fraud labels)
|
||
- *Source*: Kaggle / ULB (Worldline + MLG research collaboration, 2013)
|
||
- *Notes*: Extreme imbalance — only 492 frauds (IR ~577:1). 30 features: V1–V28 are PCA-transformed (original features confidential), plus Time and Amount. Time/Amount rescaled; V1–V28 left as-is.
|
||
|
||
*** Bank Marketing (tabular)
|
||
- *Size*: 41,188 samples; ~48 features after one-hot encoding
|
||
- *Noise*: 0% (clean)
|
||
- *Source*: UCI Machine Learning Repository (Moro et al., 2014 — Portuguese bank direct marketing campaigns)
|
||
- *Notes*: IR ~8:1 (~11.3% positive). Target: whether client subscribed to term deposit. Mix of demographic, contact, and economic features.
|
||
|
||
*** Home Credit (tabular)
|
||
- *Size*: 307,511 loan applications; ~120 numerical features
|
||
- *Noise*: 0% (clean; proxy labels — loan repayment used as credit default indicator)
|
||
- *Source*: Kaggle Home Credit Default Risk competition (2018)
|
||
- *Notes*: IR ~11.4:1 (~8% default rate). Features from application_train.csv only (no auxiliary tables). Non-numeric and ID columns dropped; NaNs filled with 0.
|
||
|
||
*** KEEL Datasets (tabular, 12 with final results)
|
||
- *Noise*: 0% (clean — standard imbalanced learning benchmarks)
|
||
- *Source*: KEEL repository (Knowledge Extraction based on Evolutionary Learning)
|
||
- *Notes*: All small/medium tabular datasets used as standard IR benchmarks in the literature. Loaded via keel_ds Python package.
|
||
|
||
| Dataset | Total | Pos | Neg | IR | Notes |
|
||
|--------------+-------+-----+-----+------+------------------------------------|
|
||
| glass0 | 214 | 70 | 144 | 2.1 | Glass identification (UCI) |
|
||
| glass1 | 214 | 76 | 138 | 1.8 | Glass identification (UCI) |
|
||
| glass2 | 214 | 17 | 197 | 11.6 | Glass identification (UCI) |
|
||
| glass6 | 214 | 29 | 185 | 6.4 | Glass identification (UCI) |
|
||
| haberman | 306 | 81 | 225 | 2.8 | Breast cancer survival (1958–1970) |
|
||
| iris0 | 150 | 50 | 100 | 2.0 | Iris setosa vs others |
|
||
| new-thyroid1 | 215 | 35 | 180 | 5.1 | Thyroid function classification |
|
||
| pima | 768 | 268 | 500 | 1.9 | Pima Indian diabetes (UCI) |
|
||
| vehicle2 | 846 | 218 | 628 | 2.9 | Vehicle silhouette (UCI) |
|
||
| vowel0 | 988 | 90 | 898 | 10.0 | Vowel recognition (class 0 vs rest)|
|
||
| wisconsin | 683 | 239 | 444 | 1.9 | Wisconsin breast cancer (UCI) |
|
||
| yeast3 | 1484 | 163 |1321 | 8.1 | Yeast protein localization (UCI) |
|
||
|
||
** KEEL Dataset Class Balance (full list)
|
||
|
||
Counts from =KeelDataset= loader. Ratio = false/true (imbalance factor).
|
||
Two datasets fail to load (likely missing env vars or files): =led7digit=, =page-blocks=.
|
||
|
||
| Dataset | Total | True | False | Ratio | Runnable |
|
||
|----------------------+-------+------+-------+-------+----------|
|
||
| cleveland-0_vs_4 | 177 | 13 | 164 | 12.6 | NO |
|
||
| ecoli-0-1_vs_5 | 240 | 20 | 220 | 11.0 | NO |
|
||
| glass0 | 214 | 70 | 144 | 2.1 | YES |
|
||
| glass1 | 214 | 76 | 138 | 1.8 | YES |
|
||
| glass2 | 214 | 17 | 197 | 11.6 | YES |
|
||
| glass4 | 214 | 13 | 201 | 15.5 | NO |
|
||
| glass5 | 214 | 9 | 205 | 22.8 | NO |
|
||
| glass6 | 214 | 29 | 185 | 6.4 | YES |
|
||
| haberman | 306 | 81 | 225 | 2.8 | YES |
|
||
| iris0 | 150 | 50 | 100 | 2.0 | YES |
|
||
| led7digit | N/A | | | | NO |
|
||
| new-thyroid1 | 215 | 35 | 180 | 5.1 | YES |
|
||
| page-blocks | N/A | | | | NO |
|
||
| pima | 768 | 268 | 500 | 1.9 | YES |
|
||
| vehicle2 | 846 | 218 | 628 | 2.9 | YES |
|
||
| vowel0 | 988 | 90 | 898 | 10.0 | YES |
|
||
| wisconsin | 683 | 239 | 444 | 1.9 | YES |
|
||
| yeast3 | 1484 | 163 | 1321 | 8.1 | YES |
|
||
|
||
** Gotchas
|
||
|
||
- ISJ "Need more data" failure: cleveland (13 true), ecoli (20 true), glass4 (13), glass5 (9) — after 33% test split, too few samples in minority class for ISJ root-finding. Excluded from =run_all.sh=.
|
||
- glass2 (17 true) is borderline — watch for failures.
|
||
- led7digit and page-blocks fail to load entirely (missing dataset files or env vars).
|
||
- Credit card V1–V28 are pre-PCA (do NOT re-scale them); only Time (col 0) and Amount (col 29) are rescaled.
|
||
- Home Credit uses application_train.csv only — no auxiliary tables (bureau, previous applications, etc.) merged in.
|
||
- CIFAR-10N aggre noise rate is ~9%, worse is ~40% — large gap; both have completed final results.
|