backup: 2026-07-25 14:39

.gitignore

@@ -19,3 +19,4 @@ auto/
 *.pdf
 *.docx
 *.odt
+/.agent-shell/

azos.org

@@ -55,3 +55,4 @@ Skill descriptions are routing keys — they determine whether a skill activates
 
 - [[id:36fe6a01-ea1f-4785-8516-f6dcfecf05bb][azos/architecture]] — module registration, option namespace, file deployment, Claude Code wiring
 - [[id:c945da4f-de5c-4eb8-bd99-810576a2545a][azos/org-roam]] — setup, MCP server, SQLite internals, fork details
+- [[id:09ae970c-2995-43d9-a597-47bb904c98de][azos/pass]] — password store format, path mappings, mbsync conventions

azosorg-roam.org

@@ -17,3 +17,4 @@
 - org-roam-mcp is forked at =anerisgreat/org-roam-mcp= (not upstream =aserranoni/org-roam-mcp=); fork fixes: =create_node= writes directly to SQLite so new nodes are immediately searchable (no emacsclient needed), and =cli_main= is defined natively (no postPatch needed)
 - emacsql stores all Emacs strings in SQLite with surrounding ="..."= — the Python DB layer strips these with =_clean_path=/_clean_string=
 - org-roam timestamps in SQLite are Emacs =(HIGH LOW USEC PSEC)= tuples: =HIGH = secs >> 16=, =LOW = secs & 0xFFFF=
+- *MCP unavailable in projects with Python dev shells*: if a project uses =use flake= with a =python3.withPackages= dev shell (e.g. =roll/impl=), direnv sets =PYTHONPATH= to that shell's Python site-packages. org-roam-mcp uses Python 3.13; the inherited =PYTHONPATH= points to a different Python version, causing import failures (symptom: =ImportError: cannot import name 'Sentinel' from 'typing_extensions'=). Fix: =PYTHONPATH: ""= in the MCP server =env= block of =~/.claude.json= (already set in =azos-core/features/claude-memory/default.nix= as of the fix). If mcp__org-roam__ tools are missing in a session, check whether the project's direnv is polluting =PYTHONPATH=.

azospass.org

@@ -0,0 +1,42 @@
+:PROPERTIES:
+:ID: 09ae970c-2995-43d9-a597-47bb904c98de
+:END:
+
+#+title: azos/pass
+#+filetags: :project: :knowledge: :pass:
+
+* azos/pass
+
+Password store conventions and path mappings for lauretta.
+
+See root: [[id:bf4dde56-0967-4374-94ed-771301e47c66][azos]]
+
+** Format
+
+The store was migrated (2026-07-25) from a nested path scheme to a flat layout.
+Each entry is =domain.com= with the password on line 1 and =login:= on line 2:
+
+#+begin_example
+<PASSWORD>
+login: user@example.com
+#+end_example
+
+** Conventions
+
+- All =passwordCommand= fields in Nix/mbsync configs must pipe through =| head -1= to extract only the password line
+- =pass-git-helper= handles multi-line entries natively — no =head -1= needed there
+- mbsync app passwords live under the =mbsync/= namespace to keep them separate from account passwords
+
+** Path Mappings
+
+| Service | pass path | Config location |
+|---------+-----------+-----------------|
+| Gmail mbsync | =mbsync/anerisgreat@gmail.com= | =features/mail/default.nix= |
+| BGU mbsync | =mbsync/anerz@post.bgu.ac.il= | =features/mail/default.nix= |
+| Zakobar mail (privateemail.com) | =privateemail.com= | =features/mail/default.nix= |
+| Nextcloud / caldav | =zakobar.com= | =features/lauretta/emacs/config.org= |
+| Git credentials (zakobar.com) | =zakobar.com= | =features/git-config/pass-git-mapping.ini= |
+
+** Pending
+
+- =mbsync/anerisgreat@gmail.com= and =mbsync/anerz@post.bgu.ac.il= entries not yet created in the store — mail sync will fail until these are added via =pass insert=

homey.org

@@ -25,15 +25,37 @@ Eurovision Vote: Django app sourced from external flake =github:anerisgreat/euro
 
 Backup: Restic daily at 03:00 to S3 (Backblaze B2, bucket =zakobar-home-backup=). Pre-hook: Nextcloud maintenance mode on + pg_dump. Post-hook: maintenance mode off. Manual offload: =restic copy= to local disk. NAR content and media excluded.
 
-Reliability hardening in =hosts/pi-main/default.nix=:
+Reliability hardening in =hosts/pi-main/default.nix=: hardware watchdog (bcm2835_wdt), WiFi power save disabled, network watchdog timer, zramSwap zstd 25%, Nix build-dir on external HD.
-- Hardware watchdog: =bcm2835_wdt= kernel module, systemd watchdog runtimeTime=300s / rebootTime=360s
-- WiFi power save disabled: brcmfmac driver drops connections under low traffic; disabled via =iw= on interface up
-- Network watchdog: timer every 2 min (starts 5 min after boot), pings gateway, restarts wpa_supplicant, reboots if still dead after 30s
-- zramSwap: zstd, 25% RAM (~2 GB) — breathing room for PHP upload spikes
-- Nix build-dir: =/mnt/data/nix-build= — avoids small tmpfs filling during large builds
 
 Bootstrap: =pi-main-bootstrap= config builds an SD image (=sd-image-aarch64.nix=) for first flash.
 
+** Deployment
+
+*Always deploy from the dev machine using the dev shell command:*
+
+#+begin_src bash
+# Enter the dev shell first:
+nix develop
+
+# Then run:
+homey-deploy-rpi-main
+#+end_src
+
+Equivalent command (if not in dev shell):
+#+begin_src bash
+nixos-rebuild switch \
+  --flake .#pi-main \
+  --target-host admin@192.168.1.100 \
+  --build-host admin@192.168.1.100 \
+  --use-remote-sudo
+#+end_src
+
+Both =--target-host= and =--build-host= point to the Pi — the build happens ON the Pi (uses Attic cache at =attic.zakobar.com=). The dev machine only supplies the flake source; it does not build locally.
+
+*NEVER run =nixos-rebuild= directly on the Pi* (=/home/admin/homey/= is a stale mirror, not the authoritative source). The dev machine at =/home/aner/projects/selfhosted/homey/= is the source of truth.
+
+Nix evaluates git-tracked files from the flake. New/modified files must be at least =git add=-ed (staged) before deploying, or they will be invisible to Nix. Untracked files are silently ignored.
+
 ** Conventions
 
 =homeyConfig= specialArgs (passed to every module): =domain=, =organization=, =timezone=. Never hardcode domain strings.
@@ -52,33 +74,34 @@ DynamicUser services (Eurovision Vote): secrets must be mode =0444= (not =0400=)
 
 Caddy Cloudflare plugin secrets: uses =LoadCredential= + =ExecStart= override (clears list with empty string first, then sets the real start command) to export =CLOUDFLARE_API_TOKEN= before exec-ing caddy.
 
+Stirling-PDF login disable: =DOCKER_ENABLE_SECURITY=false= is build-time only. To disable the runtime login page, also set =SECURITY_ENABLELOGIN=false=.
+
 ** Gotchas
 
-hdparm APM udev rule was removed — USB-SATA bridges often don't support APM commands and hdparm hangs indefinitely, causing boot-time crashes. =hdparm= is still available as a package for manual use.
+hdparm APM udev rule was removed — USB-SATA bridges often don't support APM commands and hdparm hangs indefinitely, causing boot-time crashes.
 
 =storage.nix= config is gated on =lib.mkIf (cfg.device != "")= — if =homey.storage.device= is empty string, the mount and all tmpfiles rules are skipped. Useful during initial setup.
 
 Grafana login form disabled (=disable_login_form = true=) — recovery requires re-enabling it in the Nix config. All proxy-auth users are auto-assigned Admin role (safe because Authelia already restricts to admins group).
 
-Nextcloud preview generation: a separate =oneshot= service =nextcloud-generate-previews= (declared in =hosts/pi-main/default.nix=) fills missing thumbnails after first start. Must be triggered manually or via timer.
-
 Authelia config bind-mount gotcha: NixOS resolves the symlink to the nix store path at container start. Without =NIXOS_CONFIG_HASH= env var, a config change would not take effect until manual container restart.
 
-WiFi network name: =Zakobar=. sops secret key: =wifi/psk=. The secret file must contain exactly one line: =wifi_psk=<password>=. The =ext:wifi_psk= format is wpa_supplicant's literal substitution syntax, not an env var.
+WiFi network name: =Zakobar=. sops secret key: =wifi/psk=. The secret file must contain exactly one line: =wifi_psk=<password>=.
 
-Attic: writing ephemeral TOML config (not a real file in the store) via =ExecStartPre= shell script that writes to =/run/attic-config.toml=. JWT secret interpolated into the TOML at runtime.
+Attic: writing ephemeral TOML config via =ExecStartPre= shell script to =/run/attic-config.toml=. JWT secret interpolated into the TOML at runtime.
+
+First deployment of a new container pulls the image during =nixos-rebuild switch=, which can block the activation for several minutes (e.g. stirling-pdf ~2 GB image took ~6 min on Pi). This is expected — not a hang.
 
 ** Key Files
 
 - =flake.nix= — module list, =mkHost= builder, =homeyConfig= specialArgs, =rpi4Headless= hardware snippet
 - =hosts/pi-main/default.nix= — enabled services, static IP, WiFi, reliability hardening, Attic substituter config
-- =hosts/pi-main-bootstrap/default.nix= — SD card bootstrap image
+- =shells/defaultShell.nix= — dev shell with =homey-deploy-rpi-main= and other helper commands
 - =modules/caddy.nix= — =virtualHosts= option, dual vhost generation, Authelia forward_auth snippet
 - =modules/services/authelia.nix= — access control rule rendering, =accessControlRules= option (unconditional)
 - =modules/services/uptime-kuma.nix= — =homey.monitoring.monitors= option (unconditional), sync script
 - =modules/services/attic.nix= — Nix binary cache, JWT token config, netrc injection for Nix daemon
-- =modules/services/attic-setup.md= — post-deploy steps, token commands, client config, setup history
+- =modules/services/stirling-pdf.nix= — PDF tools (merge/split/OCR/etc), port 8084, auth disabled via =SECURITY_ENABLELOGIN=false=
-- =modules/services/eurovote.nix= — DynamicUser wrapper for external flake module
 - =modules/monitoring.nix= — Prometheus + Grafana, proxy auth wiring, Node Exporter Full dashboard
 - =modules/common.nix= — Nix settings, podman network creation, sops global config
 - =modules/storage.nix= — external HD mount, =extraDirs= option
@@ -96,7 +119,7 @@ Should be able to do this
 [store]
 # Enable the chunk‑aware upload handler
 enableChunkedUpload = true
-# Optional: limit the size of each chunk (default 10 MiB)
+# Optional: limit the size of each chunk (default 10 MiB)
-maxChunkSize = 5_000_000   # 5 MiB per chunk
+maxChunkSize = 5_000_000   # 5 MiB per chunk
 
 #+end_src

impl.org

@@ -1,3 +1,9 @@
+:PROPERTIES:
+:ID: b362dad3-19b7-486a-8d26-128a83643924
+:END:
+
+#+title: impl
+
 * impl
 
 ROLL (Rate Optimized Likelyhood-based Loss) — PyTorch research project implementing custom loss functions for binary classification using kernel density estimation (KDE) to optimize TPR at target FPR thresholds. Targets imbalanced classification problems.
@@ -6,11 +12,14 @@ ROLL (Rate Optimized Likelyhood-based Loss) — PyTorch research project impleme
 
 - =src/roll.py= — Loss function implementations (Normal/Beta/Kernelized ROLL)
 - =src/experiment.py= — Training loop, evaluation infra, =run_configurations()= entry point
-- =src/datasets.py= — 10+ dataset loaders (KEEL, UCI, Kaggle, synthetic)
+- =src/datasets.py= — Dataset loaders incl. =Cifar10NDataset=, =ImbalancedCifar10Dataset=
-- =src/networks.py= — =KeelNet= MLP architecture
+- =src/networks.py= — =KeelNet= MLP architecture + =ConvNet= (moved from experiment scripts for subprocess pickle compatibility)
-- =src/summary.py= — Plotly HTML visualization (ROC, score distributions, ECDF)
+- =src/utils.py= — Logging, =init_experiment=, =get_device()= (CUDA/MPS/CPU)
-- =src/utils.py= — Logging, output dir creation (=init_experiment=)
+- =src/_episode_worker.py= — Subprocess entry point for parallel MPS episode execution
-- =experiments/keel/=, =experiments/other/=, =experiments/large/= — experiment scripts
+- =experiments/keel/= — KEEL experiment scripts; =_base.py= shared runner
+- =experiments/other/= — CIFAR-10, CIFAR-10N, imbalanced CIFAR-10, adult, gaussian, etc.
+- =experiments/large/= — Higgs, credit card, home credit
+- =scripts/= — Remote runner: =setup_remote.sh=, =run_remote.sh=, =fetch_results.sh=, =tail_remote.sh=
 
 ** Conventions
 
@@ -19,23 +28,40 @@ ROLL (Rate Optimized Likelyhood-based Loss) — PyTorch research project impleme
 - KEEL experiments share =experiments/keel/_base.py= runner; individual files just call it
 - All datasets expose: =__getitem__=, =__len__=, =.x=, =.y= attributes
 - Episode-based eval: N independent train runs per config, results aggregated
-- GPU enabled via =cudaSupport = true= in flake.nix; =get_device()= in utils.py auto-selects GPU/CPU
+- =get_device()= auto-selects CUDA → Apple MPS → CPU
-- Beta distribution variant (=roll_beta_loss_from_fpr=) is kept for thesis writing but is not actively developed or used
+- *Two-tier results strategy*: =results/= holds ephemeral date-stamped run dirs (deletable); =results-final/= holds keeper results for the thesis — one flat copy per config per dataset, no date subfolders. Both are in =.gitignore=.
+- Resume scripts (=experiment-*-resume.py=) write directly into =results-final/<dataset>/= to land in the right place immediately. Delete after run completes.
 
 ** Gotchas
 
-- Dataset paths injected as env vars by Nix shell hook (=$keel_wisconsin_dir=, etc.) — must use =nix develop=
+- Dataset paths injected as env vars by Nix shell hook (=$keel_wisconsin_dir=, etc.) — must use =nix develop= on Linux
 - =_calc_moments()= in roll.py is unused and has a variable typo (=array= vs =arr=)
-- CIFAR-10 binary: class 1 vs rest (not class 0)
+- CIFAR-10 binary: class 1 (automobile) vs rest; natural IR ~9
-- Multiprocessing uses =spawn= method via =torch.multiprocessing=
+- MPS tensors cannot cross multiprocessing process boundaries — =run_configurations()= auto-disables =is_mp= when =device.type == 'mps'=
+- MPS concurrency sweet spot: N=3 independent subprocesses for cifar10n on 16GB M1 Pro; N=5 caused OOM after ~15h.
+- cifar10n on remote now runs with =sequential_episodes=True= (one episode at a time in-process) to avoid MPS unified memory exhaustion
+- Subprocess workers (=_episode_worker.py=) write stdout+stderr to per-episode =worker.log= — 0 bytes is normal for successful runs (only WARNING+ logged)
+- Kaggle CLI: available via =nix run nixpkgs#kaggle= or inside =nix develop= (added to =flake.nix= buildInputs). Credentials at =~/.config/kaggle/kaggle.json= (username: anerzakobar). creditcard.csv downloaded to =~/.data/creditcard/creditcard.csv=.
+- Large datasets (creditcard, homecredit, higgs) are manual downloads; =credit_card_fraud_dir= / =home_credit_dir= env vars set by shell hook to =~/.data/...=
+
+** Remote Machine
+
+- Host: =chenzakobar@192.168.1.190= (Mac OS, Python 3.13 via Homebrew)
+- SSH key: =~/.ssh/roll_remote= — dedicated ed25519 key, NOT the YubiKey/GPG key
+- Remote project dir: =~/roll-impl=; venv: =~/roll-venv=; env vars: =~/roll-env.sh=
+- Use =--detach= flag with =run_remote.sh= for long experiments (survives sleep/disconnect)
+- Kill a detached run: =ssh -i ~/.ssh/roll_remote -o IdentitiesOnly=yes chenzakobar@192.168.1.190 'pkill -f <script-name>'=
 
 ** Key Files
 
 - [[file:src/roll.py][src/roll.py]] — Core loss: =KernelizedROLLoss= custom autograd Function with KDE backward pass
 - [[file:src/experiment.py][src/experiment.py]] — =ExperimentConfiguration= dataclass, =Criteriorator= ABC, =run_configurations()=
-- [[file:experiments/keel/_base.py][experiments/keel/_base.py]] — shared KEEL runner =run_keel_experiment()=
+- [[file:experiments/keel/_base.py][experiments/keel/_base.py]] — shared KEEL runner; config suite: roll-aoc, roll-tpr90, bce-weighted, libauc-auroc, gce-0.7, mae, focal-loss, asymmetric-loss
-- [[file:flake.nix][flake.nix]] — Nix env with dataset downloads, hash-pinned, exports path env vars
+- [[file:experiments/other/experiment-cifar10n.py][experiments/other/experiment-cifar10n.py]] — CIFAR-10N (noisy labels: clean/aggre/worse), sequential_episodes=True
-- [[file:AGENTS.md][AGENTS.md]] — Project guidelines (naming conventions, env, dataset list)
+- [[file:experiments/large/experiment-creditcard.py][experiments/large/experiment-creditcard.py]] — Credit card fraud dataset
+- [[file:scripts/run_remote.sh][scripts/run_remote.sh]] — rsync + run on remote Mac; =--detach= for sleep-safe long runs; excludes =results-final/= from push, includes it in fetch
+- [[file:flake.nix][flake.nix]] — Nix env; kaggle + sshpass in buildInputs
+- =results-final/cifar10n/= — Keeper results: clean/aggre/worse × 7 configs × 3 episodes. As of 2026-07-25: clean+aggre complete; worse-roll-aoc/bce-weighted/mae complete; worse-gce-0.7/focal-loss/asymmetric-loss/libauc-auroc running on remote.
 
 ** Subnodes
 
@@ -43,3 +69,5 @@ ROLL (Rate Optimized Likelyhood-based Loss) — PyTorch research project impleme
 - [[id:a53cbe84-cd8d-45c2-a8cf-34ab520a3ea5][impl/experiments]] — Experiment structure, training flow, metrics, output layout
 - [[id:b8a9886a-d349-43e5-a745-817a148c1fd8][impl/datasets]] — Dataset catalog, KEEL list, eval metrics
 - [[id:151d5686-6f40-4158-a59a-b0be94cdc969][impl/research]] — Literature survey: competing methods, dataset gaps, key papers
+- [[id:fdc18323-e9c2-4fc6-ace5-065035d30c51][impl/todos]] — Project TODOs
+- [[id:863ca60e-6a3c-45d2-86fd-fb4ed386e31b][impl/work-history/recent]] — Daily work log

impldatasets.org

@@ -4,3 +4,37 @@
 
 #+title: impl/datasets
 #+filetags: :project: :knowledge: :datasets:
+
+* impl/datasets
+
+** KEEL Dataset Class Balance
+
+Counts from =KeelDataset= loader. Ratio = false/true (imbalance factor).
+Two datasets fail to load (likely missing env vars or files): =led7digit=, =page-blocks=.
+
+| Dataset              | Total | True | False | Ratio | Runnable |
+|----------------------+-------+------+-------+-------+----------|
+| cleveland-0_vs_4     |   177 |   13 |   164 |  12.6 | NO       |
+| ecoli-0-1_vs_5       |   240 |   20 |   220 |  11.0 | NO       |
+| glass0               |   214 |   70 |   144 |   2.1 | YES      |
+| glass1               |   214 |   76 |   138 |   1.8 | YES      |
+| glass2               |   214 |   17 |   197 |  11.6 | YES      |
+| glass4               |   214 |   13 |   201 |  15.5 | NO       |
+| glass5               |   214 |    9 |   205 |  22.8 | NO       |
+| glass6               |   214 |   29 |   185 |   6.4 | YES      |
+| haberman             |   306 |   81 |   225 |   2.8 | YES      |
+| iris0                |   150 |   50 |   100 |   2.0 | YES      |
+| led7digit            |   N/A |      |       |       | NO       |
+| new-thyroid1         |   215 |   35 |   180 |   5.1 | YES      |
+| page-blocks          |   N/A |      |       |       | NO       |
+| pima                 |   768 |  268 |   500 |   1.9 | YES      |
+| vehicle2             |   846 |  218 |   628 |   2.9 | YES      |
+| vowel0               |   988 |   90 |   898 |  10.0 | YES      |
+| wisconsin            |   683 |  239 |   444 |   1.9 | YES      |
+| yeast3               |  1484 |  163 |  1321 |   8.1 | YES      |
+
+** Gotchas
+
+- ISJ "Need more data" failure: cleveland (13 true), ecoli (20 true), glass4 (13), glass5 (9) — after 33% test split, too few samples in minority class for ISJ root-finding. Excluded from =run_all.sh=.
+- glass2 (17 true) is borderline — watch for failures.
+- led7digit and page-blocks fail to load entirely (missing dataset files or env vars).

implexperiments.org

@@ -4,3 +4,130 @@
 
 #+title: impl/experiments
 #+filetags: :project: :knowledge: :experiments:
+
+* impl/experiments
+
+** Current Config Set (as of 2026-07-08)
+
+- =roll= — =kernelized_roll_tpr(0.95)=
+- =roll+bce= — =kernelized_roll_tpr(0.95, bce_weight=0.5, bce_pos_weight=imbalance_ratio)=
+- =bce= — plain =BCEWithLogitsLoss=
+- =bce-weighted= — BCE with =pos_weight = num_false/num_true=
+
+FPR variants (=roll-fpr-0.40=, =roll-fpr-0.40+bce=) removed — less consistent, added complexity for marginal gain.
+Target metric: FPR@TPR=0.95 (lower is better), written to =tpr_summary.csv=.
+
+** CIFAR-10N Experiment (2026-07-19 → 2026-07-20)
+
+Run dir: =results/cifar10n/2026-07-19-06-52=
+Configs: =roll-aoc=, =bce-weighted=, =mae=, =gce-0.7=, =libauc-auroc= × 3 episodes × 3 noise types = 45 runs.
+Noise types: =clean= (original CIFAR-10 labels), =aggre= (human aggregated, ~moderate noise), =worse= (single worst-annotator labels, ~heavy noise).
+Architecture: ConvNet (3 conv blocks + 2 FC, trained from scratch). Positive class: automobile (class 1) vs rest, IR ~9.
+LibAUC ran on CPU (MPS unsupported); all others on MPS. PESG lr=1e-3 (not 0.1 — from-scratch training explodes at 0.1).
+
+*Results (mean test AUC over 3 episodes):*
+
+| Config       | clean  | aggre  | worse  |
+|--------------+--------+--------+--------|
+| roll-aoc     | 0.9800 | 0.9336 | 0.8200 |
+| bce-weighted | 0.9772 | 0.9267 | 0.8044 |
+| gce-0.7      | 0.9776 | 0.9289 | 0.7790 |
+| libauc-auroc | 0.9777 | 0.9228 | 0.7690 |
+| mae          | 0.6954 | 0.7756 | 0.5979 |
+
+*Key findings:*
+- =roll-aoc= is the top performer at every noise level, with the gap widening under heavy noise.
+- Under =worse= noise, =roll-aoc= beats =bce-weighted= by 1.6pp and =libauc-auroc= by 5.1pp.
+- =gce-0.7= and =libauc-auroc= degrade more steeply than =roll-aoc= under increasing noise.
+- =mae= is weak throughout; satisfies the symmetry condition theoretically but fails in practice on this task.
+
+Summaries: =results/cifar10n/2026-07-19-06-52/auc.csv= (45 rows), =roc-clean.html=, =roc-aggre.html=, =roc-worse.html=.
+Regen script: =scripts/regen_cifar10n_summaries.py= — loads MPS-saved pkls via =default_restore_location= patch (maps mps→cpu).
+
+** Poisoning Experiment (2026-07-13 → 2026-07-15)
+
+Setup: for each KEEL dataset, train positives are duplicated N times with label flipped to 0 (false).
+Originals keep their true label. Val/test splits untouched.
+Code: =LabelPoisonedDataset= + =PoisonedSplitter= in =src/experiment.py=.
+Runner: =experiments/keel/run_poisoned.py <dataset> <n_duplicates> [--no-mp]=, results go to =poison/<dataset>/=.
+Batch runner: =experiments/keel/run_all_poisoned.sh [n_duplicates]= (default 3).
+Configs run (poisoned): =roll-aoc=, =bce-weighted=, =mae=, =gce-0.7=, =libauc-auroc=.
+Episodes: 15.
+
+*Key findings (2026-07-15, full 12-dataset run):*
+- =roll-aoc= is statistically significantly better than =libauc-auroc= on glass1, pima, vehicle2 (one-sided paired t-test and Wilcoxon, p < 0.05).
+- =libauc-auroc= is numerically unstable under poisoning on some datasets: produces NaN/Inf predictions on many episodes. glass2: only 2/15 valid episodes; vowel0: only 1/15 valid episode. These are silently filtered in =write_auc_csv= (=src/summary.py:344=) — the auc.csv will have far fewer rows than expected for libauc on those datasets.
+- =mae= and =gce-0.7= perform poorly under poisoning across all datasets.
+- =roll-aoc= and =libauc-auroc= are competitive on easy datasets (glass6, new-thyroid1, vowel0, wisconsin) where both reach ~0.99 AUC.
+
+** KEEL Run 2026-07-08 — Setup
+
+Results folders: =/home/aner/projects/research-projects/roll/impl/results/*/2026-07-08-*/=
+
+Changes from 2026-07-07 run:
+- Bandwidth: ISJ → *Silverman's rule* (ISJ crashed on small/degenerate batches mid-training)
+- Kernel scheduler: =KernelScheduler(16.0, decay_every=20)= (old: 100.0/500)
+- LR: roll =1e-1=, bce =1e-3= (was both =1e-3=)
+- Weight decay: roll =0.1=, bce =1e-3= (was none)
+- Dropout: =0.1= (was =0.0=)
+- Batch size: =256= (was 128; tried 32 but too small for balanced sampling)
+- Patience: =100= (was =500=)
+- Episodes: =5= (was =1=)
+- FPR target changed =fpr-0.05= → =fpr-0.40=, then FPR variants dropped entirely
+
+Scoring: =mean(fpr) + std(fpr)= across 5 episodes (penalises variance and instability; lower is better).
+Note: mean−std was tried first but incorrectly rewarded instability — corrected to mean+std.
+
+** KEEL Run 2026-07-08 — Results (mean + std, FPR@TPR=0.95)
+
+| Dataset      | roll  | roll+bce | bce   | bce-weighted | Winner       |
+|--------------+-------+----------+-------+--------------+--------------|
+| glass0       | 0.568 | —        | 0.677 | 0.706        | roll         |
+| glass1       | 0.809 | 0.776    | 0.744 | 0.676        | bce-weighted |
+| glass2       | 0.702 | 0.642    | 0.793 | 0.683        | roll+bce     |
+| glass6       | 0.081 | 0.060    | 0.042 | 0.054        | bce          |
+| haberman     | 0.910 | 0.945    | 1.015 | 0.940        | roll         |
+| iris0        | 0.000 | 0.000    | 0.000 | 0.600        | roll/bce/roll+bce (tied) |
+| new-thyroid1 | 0.077 | 0.048    | 0.057 | 0.055        | roll+bce     |
+| pima         | 0.710 | 0.744    | 0.627 | 0.681        | bce          |
+| vehicle2     | 0.148 | 0.097    | 0.163 | 0.108        | roll+bce     |
+| vowel0       | 0.160 | 0.095    | 0.059 | 0.066        | bce          |
+| wisconsin    | 0.011 | 0.007    | 0.008 | 0.011        | roll+bce     |
+| yeast3       | 0.398 | 0.539    | 0.451 | 0.552        | roll         |
+
+Win count: roll+bce: 4, bce: 3, roll: 3, bce-weighted: 1
+
+** KEEL Run 2026-07-08 — Findings
+
+- =roll+bce= is the most consistent ROLL variant: wins glass2, new-thyroid1, vehicle2, wisconsin.
+- =roll= alone wins where it has a large mean advantage that outweighs variance penalty (haberman, glass0, yeast3).
+- =bce= wins on glass6, pima, vowel0; =bce-weighted= wins glass1 but its iris0 score is inflated by one catastrophic episode.
+- No single method dominates. ROLL adds clear value on haberman, glass0, yeast3 (hard/imbalanced datasets).
+- High episode variance on haberman, yeast3, glass1 — small test sets make results noisy.
+
+** Previous Run — KEEL Results (FPR@TPR=0.95, 2026-07-07, single episode, raw mean)
+
+| Dataset   | roll  | roll+bce | roll-fpr+bce | bce   | bce-weighted | Best          |
+|-----------+-------+----------+--------------+-------+--------------+---------------|
+| glass0    | 0.878 | 0.490    | 0.673        | 0.857 | 0.449        | bce-weighted  |
+| glass1    | 0.872 | 0.830    | 0.702        | 0.872 | 0.681        | bce-weighted  |
+| haberman  | 0.870 | 1.0      | 1.0          | 0.922 | 1.0          | roll (barely) |
+| pima      | 0.649 | 0.714    | 0.673        | 0.649 | 0.595        | bce-weighted  |
+| vehicle2  | 0.095 | 0.032    | 0.032        | 0.058 | 0.058        | roll+bce      |
+| vowel0    | 0.141 | 0.0      | 0.0          | 0.0   | 0.004        | tied          |
+| wisconsin | 0.964 | 0.0      | 0.0          | 0.0   | 0.0          | tied          |
+
+Config: 1 episode, ISJ bandwidth, no weight decay/dropout, lr=1e-3 for all, gamma=100/decay=500.
+
+** Large Datasets
+
+| Experiment            | Dataset           | Status  | Notes                                        |
+|-----------------------+-------------------+---------+----------------------------------------------|
+| experiment-higgs      | HiggsDataset      | OK      | 500k samples, ~50/50 balance                 |
+| experiment-creditcard | CreditCardFraud   | MISSING | =/home/aner/.data/creditcard/creditcard.csv= |
+| experiment-homecredit | HomeCreditDataset | MISSING | =/home/aner/.data/homecredit/=               |
+
+** run_all.sh
+
+Runs KEEL datasets via shell script in =experiments/keel/=.
+On failure, continues to next dataset.

implloss-functions.org

@@ -4,3 +4,57 @@
 
 #+title: impl/loss-functions
 #+filetags: :project: :knowledge: :loss-functions:
+
+* impl/loss-functions
+
+** KernelizedROLLoss — Key Design Decisions
+
+- =_icdf= uses **bisection** (not Newton-Raphson). NR was replaced because =F_prime= goes near-zero when =v= is large and =tau= drifts, causing divergence. Bisection bracket is =[min(scores)−10/v, max(scores)+10/v]=, guaranteed to converge.
+- Score normalisation (=normalize= flag): divides =yh= by its std before KDE. Can be disabled via =kernelized_roll_fpr(fpr, normalize=False)=. The chain-rule correction (multiply grad by =score_scale=) is applied in backward.
+- =_grad_tau_scores_neg= denominator clamped at =1e-10= to prevent 0/0 when all sigmoid kernels saturate.
+- Variance clamped at =1e-8= in normalisation to prevent =inf= when all batch scores are identical.
+
+** KernelizedROLLoss Unification (2026-07-14)
+
+=KernelizedROLLoss= and =KernelizedROLLossAOC= merged into a single class accepting =alphas= (list of CDF targets) and =divisor=:
+- Point-FPR: =KernelizedROLLoss.apply([1−fpr], 1, yh, y, gamma, normalize)=
+- AOC: =KernelizedROLLoss.apply(_AOC_ALPHAS, _AOC_DIVISOR, yh, y, gamma, normalize)=
+
+Backward always vectorised (K-dim); K=1 reduces to the scalar case. =ctx.divisor= stores the non-tensor divisor.
+
+** BCE Combination
+
+Both =kernelized_roll_fpr= and =kernelized_roll_tpr= accept =bce_weight= (default 0.0) and =bce_pos_weight= (default 1.0).
+Loss = ROLL + bce_weight × BCE. Standard value: =bce_weight=0.5=, =bce_pos_weight=num_false/num_true=.
+
+** ISJ Bandwidth Failure
+
+ISJ fails when a class has too few samples in a split (~<20). Affects: cleveland, ecoli, glass4, glass5.
+
+** Gamma Scheduling
+
+=KernelScheduler= widens KDE kernels early then anneals. Default: =initial_gamma=100=, =decay=0.5=, =decay_every=500=. Gamma is a *divisor* of =v=.
+
+** LibAUC Integration (2026-07-14, tuned 2026-07-14)
+
+=libauc_auc_loss= in =src/roll.py= wraps =AUCMLoss= (Yuan et al., ICCV 2021) to the =(yh, y)= convention.
+- Applies =torch.sigmoid(yh)= before passing to AUCMLoss (squared-hinge surrogate, no built-in sigmoid).
+- AUCMLoss has learnable params =a, b, alpha= — must use PESG optimizer (not Adam).
+- =opt_factory= signature: =opt_factory(model)= — does NOT take criteriorator (would be unbound at call site in =_perform_episode=).
+- =opt_factory= pattern: =loss_fn.pesg_opt_factory(...)= returns picklable =_PESGFactory= sharing the same inner =AUCMLoss= instance as the criteriorator's loss func (preserved by pickle memo).
+- =ExperimentConfiguration= gained =opt_factory: callable = None=; overrides =optim_class/optim_args= when set.
+- =tqdm= is an undeclared dep of libauc — must be in =propagatedBuildInputs= in flake.nix.
+
+*** Recommended settings (v2 + adam, tuned 2026-07-14)
+- =version='v2'= — removes class prior from formulation; LibAUC explicitly recommends over v1; default in =libauc_auc_loss=.
+- =mode='adam'= in PESG — Adam-style primal updates, converges much faster than SGD on small datasets; default in =pesg_opt_factory=.
+- =imratio= passed explicitly as =num_true / (num_true + num_false)= — avoids noisy mini-batch estimates on imbalanced data. Note: v2 ignores imratio (prior-free), so this only matters if reverting to v1.
+- =lr=0.1= (PESG default, much larger than Adam's 1e-3). If diverging: try =epoch_decay=0.0= or drop lr to =0.01=.
+
+** Grace Period (BasicCriteriorator / CRBasedCriteriorator, 2026-07-14)
+
+=grace_period: int = 0= parameter added to both criteriorators. During the first =grace_period= epochs:
+- =best_flag= is always False (no model snapshot taken)
+- Patience counter does not increment (early stopping cannot fire)
+
+All keel configs use =grace_period=50=. Prevents noisy early-epoch models from being selected as best.

implresearch.org

@@ -67,6 +67,215 @@ Lower priority (useful if sweeping many baselines):
 5. Satellite, Segment, Pen-local — common in full KEEL sweeps
 6. KC1/PC1 — software metrics datasets; different domain from biology/finance
 
+** Noisy Labels Literature
+
+Sources found 2026-07-13 for thesis sec:noisy-labels (section 2.5 in related work chapter).
+Poisoning finding: label-poisoning (true positives duplicated with flipped label) hurts
+bce-weighted more than roll. ROLL's distributional/threshold objective is structurally
+more robust to class-conditional label noise without any explicit noise-handling mechanism.
+
+*** Competing Method Concepts
+
+Four families per Li & Zhu survey (arXiv:2404.04159). All require knowing or estimating
+something about the noise; ROLL requires none of it.
+
+*1. Robust Loss Functions* — change the loss so mislabeled samples cannot dominate the gradient.
+- Key insight: symmetry condition (Ghosh 2017): sum_k l(f(x),k)=C => noise-tolerant.
+  CE fails this; MAE satisfies it.
+- MAE: provably tolerant, very slow to converge.
+- GCE / L_q (Zhang & Sabuncu 2018): interpolates MAE<->CE via q. Tune for tradeoff.
+- Asymmetric losses (Zhou 2021): saturate on suspected noisy samples; different margins
+  per class direction. Suited to class-conditional noise.
+- Label smoothing: a soft regularization variant; does not fully satisfy symmetry condition
+  but reduces overconfidence on noisy labels.
+- Requires: approximate noise rate to choose hyperparameters.
+
+*2. Noise Transition Matrix / Label Correction* — model the corruption explicitly.
+- Estimate T where T[i,j] = P(observed label j | true label i).
+- Forward correction (Patrini 2017): multiply model output by T^{-1} before the loss.
+- Requires: a clean anchor set or structural assumptions about T. Hard for one-sided
+  asymmetric noise (our poisoning: only positives corrupted) without an anchor.
+- Representative methods: GLC, RoG, CAN, Noise Adaptation Layer.
+
+*3. Sample Selection* — identify and train only on likely-clean samples.
+- Memorization effect: DNNs fit clean patterns first, noisy labels late in training.
+  Small-loss samples early on are likely clean.
+- Co-teaching (Han 2018): two nets cross-select small-loss samples each epoch.
+- MentorNet, JoCoR, TopoFilter, MORPH are variants.
+- Requires: two models; breaks down at high noise or with imbalance (minority samples
+  often have high loss even when clean, so they get incorrectly filtered out).
+
+*4. Semi-Supervised Learning (SSL)-Based* — treat noisy samples as unlabeled, not wrong.
+- Divide training into clean (labeled) and noisy (unlabeled) subsets, then run SSL.
+- DivideMix: fits a mixture model to per-sample loss to decide clean/noisy split,
+  then applies MixMatch. The dominant method on standard benchmarks as of 2024.
+- Other representatives: SELFIE, Jo-SRC, UNICON, TCL, SoftMatch.
+- Requires: reliable clean/noisy split; the mixture model can fail under imbalance
+  because minority-class loss distributions overlap with noisy majority distributions.
+
+*Common thread*: all four families are corrective — they assume or estimate noise
+structure. ROLL's robustness is structural (distributional objective), not corrective.
+
+*Benchmark gap (confirmed by Li & Zhu 2024)*: the entire field evaluates on multi-class
+image datasets with synthetic symmetric/asymmetric noise or a handful of real web-scraped
+image datasets (Clothing1M ~38.5% noise, Food-101N ~18.4%, ANIMAL-10N ~8%,
+WebVision ~20%, CIFAR-10N/100N human-reannotated). No tabular data, no binary
+imbalanced setting, no one-sided minority-class corruption anywhere in the standard
+benchmark suite.
+
+*** Noisy Label Datasets (real-world, from the literature)
+
+| Dataset | Size | Noise rate | Domain |
+|---------|------|------------|--------|
+| Clothing1M | ~1M | ~38.5% | clothing images (web) |
+| Food-101N | ~310K | ~18.4% | food images (web) |
+| ANIMAL-10N | 55K | ~8% | animal images |
+| WebVision | 2.5M | ~20% | general web images |
+| CIFAR-10N / CIFAR-100N | 50K | varies | CIFAR re-annotated by humans |
+
+All are image datasets. No standard noisy-label tabular benchmark exists.
+
+*** Surveys
+
+| Key | Venue | arXiv / DOI | Notes |
+|-----|-------|-------------|-------|
+| song2022survey | IEEE TNNLS 2022/23 | 10.1109/TNNLS.2022.3152527 | Main survey; 62 methods in 5 categories |
+| survey2025label | Systems Sci & Control Eng 2025 | 10.1080/21642583.2025.2488120 | Deep learning focus; 5-type taxonomy |
+| li2024noisy | arXiv:2404.04159 (Apr 2024) | — | Li & Zhu; 4-category taxonomy; confirmed no binary/imbalanced coverage |
+
+*** Robust Loss Functions
+
+| Key | Venue | Notes |
+|-----|-------|-------|
+| ghosh2017robust | AAAI 2017, arXiv:1712.09482 | Symmetry condition: sum_k l(f(x),k)=C => noise-tolerant; CE fails; MAE passes |
+| zhang2018generalized | NeurIPS 2018 | GCE: L_q interpolates MAE<->CE via q; q->0 = MAE, q->1 = CE |
+| zhou2021asymmetric | ICML 2021, PMLR v139 | Asymmetric loss; different margins per class; suited to class-conditional noise |
+| feng2020can | IJCAI 2020 | Can CE be robust? — only under very restricted conditions |
+| symmetrization2025 | arXiv:2605.20347 | Symmetrize any loss via additive complement (2025) |
+
+*** Label Correction / Transition Matrix
+
+| Key | Venue | Notes |
+|-----|-------|-------|
+| patrini2017making | CVPR 2017 | Estimate T[i,j]=P(y_tilde=j|y=i); correct logits by T^{-1}. Canonical forward-correction |
+| cannings2020classification | Biometrika 2020, arXiv:1805.11505 | kNN/SVM robust to imperfect labels; LDA not unless class priors equal |
+
+*** Sample Selection
+
+| Key | Venue | Notes |
+|-----|-------|-------|
+| han2018coteaching | NeurIPS 2018 | Co-teaching: two nets select small-loss samples for each other per epoch |
+
+*** Semi-Supervised / DivideMix Family
+
+| Key | Venue | Notes |
+|-----|-------|-------|
+| dividemix | ICLR 2020 | Dominant SSL-based method; GMM splits clean/noisy, then MixMatch |
+
+*** AUC / Ranking Objectives and Noise Robustness
+
+| Key | Venue | Notes |
+|-----|-------|-------|
+| xie2024wsauc | IEEE TPAMI 2024, arXiv:2305.14258 | WSAUC/rpAUC; unifies weak supervision for AUC; rpAUC removes high-loss pairs — more robust under noisy labels. ADDED to thesis.bib |
+| auc2025retrieval | arXiv:2510.00137 | AUC-driven learning for neural retrieval; robustness framing |
+
+*** BibTeX Entries (staged here; move to thesis.bib when writing sec:noisy-labels)
+
+#+begin_src bibtex
+@article{song2022survey,
+  title={Learning From Noisy Labels With Deep Neural Networks: A Survey},
+  author={Song, Hwanjun and Kim, Minseok and Park, Dongkwan and Shin, Yooju and Lee, Jae-Gil},
+  journal={IEEE Transactions on Neural Networks and Learning Systems},
+  volume={34},
+  number={11},
+  pages={8135--8153},
+  year={2023},
+  doi={10.1109/TNNLS.2022.3152527}
+}
+
+@article{li2024noisy,
+  title={Noisy Label Processing for Classification: A Survey},
+  author={Li, Mengting and Zhu, Chuang},
+  journal={arXiv preprint arXiv:2404.04159},
+  year={2024}
+}
+
+@inproceedings{ghosh2017robust,
+  title={Robust Loss Functions under Label Noise for Deep Neural Networks},
+  author={Ghosh, Aritra and Kumar, Himanshu and Sastry, P. S.},
+  booktitle={Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence},
+  pages={1919--1925},
+  year={2017}
+}
+
+@inproceedings{patrini2017making,
+  title={Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach},
+  author={Patrini, Giorgio and Rozza, Alessandro and Krishna Menon, Aditya and Nock, Richard and Qu, Lizhen},
+  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
+  pages={1944--1952},
+  year={2017}
+}
+
+@inproceedings{zhang2018generalized,
+  title={Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels},
+  author={Zhang, Zhilu and Sabuncu, Mert R.},
+  booktitle={Advances in Neural Information Processing Systems},
+  volume={31},
+  year={2018}
+}
+
+@inproceedings{han2018coteaching,
+  title={Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels},
+  author={Han, Bo and Yao, Quanming and Yu, Xingrui and Niu, Gang and Xu, Miao and Hu, Weihua and Tsang, Ivor W. and Sugiyama, Masashi},
+  booktitle={Advances in Neural Information Processing Systems},
+  volume={31},
+  year={2018}
+}
+
+@inproceedings{zhou2021asymmetric,
+  title={Asymmetric Loss Functions for Learning with Noisy Labels},
+  author={Zhou, Xiong and Liu, Xianming and Jiang, Junjun and Gao, Xin and Ji, Xiangyang},
+  booktitle={Proceedings of the 38th International Conference on Machine Learning},
+  pages={12846--12856},
+  year={2021},
+  volume={139},
+  series={Proceedings of Machine Learning Research},
+  publisher={PMLR}
+}
+
+@inproceedings{feng2020can,
+  title={Can Cross Entropy Loss Be Robust to Label Noise?},
+  author={Feng, Lei and Shu, Senlin and Lin, Zhuoyi and Lv, Fengmei and Li, Li and An, Bo},
+  booktitle={Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence},
+  pages={2206--2212},
+  year={2020}
+}
+
+@article{cannings2020classification,
+  title={Classification with imperfect training labels},
+  author={Cannings, Timothy I. and Fan, Yingying and Samworth, Richard J.},
+  journal={Biometrika},
+  volume={107},
+  number={2},
+  pages={311--330},
+  year={2020},
+  doi={10.1093/biomet/asaa011}
+}
+
+@article{symmetrization2025,
+  title={Symmetrization of Loss Functions for Robust Training of Neural Networks
+         in the Presence of Noisy Labels},
+  journal={arXiv preprint arXiv:2605.20347},
+  year={2025}
+}
+
+@article{auc2025retrieval,
+  title={Optimizing What Matters: {AUC}-Driven Learning for Robust Neural Retrieval},
+  journal={arXiv preprint arXiv:2510.00137},
+  year={2025}
+}
+#+end_src
+
 ** Paper Subnodes
 
 - [[id:bf0fc08a-e806-48df-b188-7a2c4c41c693][impl/paper-tabpfn]] — TabPFN: in-context learning for small tabular classification (ICLR 2023)

impltodos.org

@@ -0,0 +1,67 @@
+:PROPERTIES:
+:ID: fdc18323-e9c2-4fc6-ace5-065035d30c51
+:END:
+
+#+title: impl/todos
+#+filetags: :project: :todo:
+
+* TODO Look into previous results success/failures — understand failure modes of datasets where ROLL performs poorly (glass1, pima, glass6 from 2026-07-08 KEEL run; see [[id:a53cbe84-cd8d-45c2-a8cf-34ab520a3ea5][impl/experiments]] for full results)
+
+* TODO Run credit card fraud experiment
+Experiment file already exists: =experiments/large/experiment-creditcard.py=.
+Data at =~/.data/creditcard/creditcard.csv= (downloaded via Kaggle CLI).
+- Run all 7 configs (roll-aoc, bce-weighted, mae, gce-0.7, libauc-auroc, focal-loss, asymmetric-loss)
+- Tabular, ~285K rows, naturally imbalanced (~0.17% positive) — fast to run
+- Supports the AUC-ROLL noisy dataset story (real-world messiness, severe imbalance)
+
+* TODO Track 1: controlled noise injection on KEEL datasets
+Take 3–5 KEEL datasets with varying IR and inject label noise at 10%, 20%, 30% on minority class.
+Goal: reproduce the CIFAR-10N noise-level ablation on imbalanced tabular data.
+- Choose datasets spanning a range of IR (e.g. low/medium/high imbalance)
+- Write noise injection utility (flip minority labels randomly at given rate)
+- Run all 7 configs per dataset × noise level
+- Expected result: ROLL-AUC advantage grows with noise level (mirrors CIFAR finding)
+- Key for thesis: validates the noise-sensitivity story on tabular domain
+
+* TODO CheXpert experiment
+~224K chest X-ray images, label noise from automated NLP extraction of radiology reports.
+Severe class imbalance. Clinically relevant domain.
+Dataset: https://stanfordmlgroup.github.io/competitions/chexpert/ (requires registration)
+- Binary classification: one pathology vs rest (e.g. pleural effusion, highest prevalence)
+- Likely needs pretrained ResNet backbone (images are 224×224)
+- Medium-large compute cost; run on remote Mac or cluster
+- Priority: lower than credit card fraud and Track 1; run after those confirm the story
+
+* TODO NIH ChestX-ray14 experiment
+~112K chest X-ray images, label noise from NLP extraction of radiology reports.
+Similar story to CheXpert but smaller; could substitute if CheXpert access is delayed.
+Dataset: https://nihcc.app.box.com/v/ChestXray-NIHCC
+- Same setup considerations as CheXpert
+- Priority: same as CheXpert — run one or the other, not necessarily both
+
+* TODO ANIMAL-10N experiment
+Dataset loader written: =Animal10NDataset= in =src/datasets.py=.
+Data must be manually downloaded from https://nihalsid.github.io/animal-10n/ and extracted to =~/.data/animal10n/= (train/ and test/ subdirs).
+Next steps:
+- Download the dataset on the remote Mac
+- Write =experiments/other/experiment-animal10n.py= (all 7 configs: roll-aoc, bce-weighted, mae, gce-0.7, libauc-auroc, focal-loss, asymmetric-loss)
+- ConvNet needs adjusted Linear layer: 64×64 images → after 3 MaxPool → 8×8 spatial → 64*8*8=4096 flatten (vs 1024 for CIFAR)
+- positive_class=0 (cat vs rest), IR ~9, same setup as CIFAR-10N
+
+* TODO Food-101N experiment
+~310K training images, ~18.4% web noise. Binary: one class vs rest.
+Dataset: https://kuanghuei.github.io/Food-101N-Dataset/ (requires Google account for download)
+- Write =Food101NDataset= in =src/datasets.py= using torchvision.datasets.ImageFolder
+- Images are 224×224; may need ResNet backbone or resize to 64×64 for ConvNet
+- Lower priority than ANIMAL-10N (larger, more infra work)
+
+* TODO Clothing1M experiment
+~1M images, ~38.5% web noise. Binary: one category vs rest.
+Dataset: https://github.com/Cysu/noisy_label (requires request to authors)
+- Very large; likely needs ResNet pretrained backbone
+- Lowest priority of the noisy-label datasets
+
+* TODO WebVision experiment
+2.5M images, ~20% web noise. Very large.
+- Likely out of scope unless we get access to a GPU cluster
+- Keep as a long-term stretch goal

implwork-historyrecent.org

@@ -0,0 +1,19 @@
+:PROPERTIES:
+:ID: 863ca60e-6a3c-45d2-86fd-fb4ed386e31b
+:END:
+
+#+title: impl/work-history/recent
+#+filetags: :project: :work-history:
+
+* 2026-07-24 Thu
+
+- Investigated stalled cifar10n remote run: 5 workers from 6:45AM were still alive after 15.5h with 0 results; previous session's ps check had missed them, making the run appear dead
+- Diagnosed OOM: memory at ~46MB free at 7h mark with 5 concurrent MPS workers on 16GB; new run with 3 workers also OOM'd after 8.5h (26GB compressed pages, 5B decompressions)
+- Fixed root cause: added =sequential_episodes= flag to =run_configurations= and =_perform_multiple_episodes= in =src/experiment.py= — when True, bypasses the subprocess-parallel path and runs episodes one at a time in-process on MPS; cifar10n now uses this
+- Killed 3-worker subprocess run on remote; restarted cifar10n with =sequential_episodes=True= — now running single-process, one episode at a time
+- Added =kaggle= to =flake.nix= buildInputs; set up =~/.config/kaggle/kaggle.json= (user: anerzakobar); downloaded creditcard.csv (144MB) to =~/.data/creditcard/=
+- Rewrote =experiments/large/experiment-creditcard.py=: replaced old ad-hoc FPR configs with full baseline suite matching KEEL/_base.py — roll-aoc, roll-tpr90, bce-weighted, libauc-auroc, gce-0.7, mae, focal-loss, asymmetric-loss; StepLR on all configs; N_EPISODES=5
+- Launched creditcard experiment locally in background (5 episodes × 8 configs, CPU multiprocessing)
+- Reviewed KEEL AUC results table: libauc-auroc wins 6/12 datasets (mean 0.890); asymmetric-loss strong second (0.887); roll-aoc 6th (0.872); roll-tpr90 last (0.842, dragged by new-thyroid1 instability)
+
+** Archives

thesis.org

@@ -7,7 +7,7 @@
 
 * thesis
 
-BGU MSc thesis on the ROLL method (Ranking via Optimized Label Learning). See [[id:d23011c5-d925-4f14-b05a-0a1f4bdbe860][impl]] for the implementation project this thesis documents.
+BGU MSc thesis on the ROLL method (Rate Optimized Likelihood Loss). See [[id:d23011c5-d925-4f14-b05a-0a1f4bdbe860][impl]] for the implementation project this thesis documents.
 
 ** Architecture
 
@@ -23,8 +23,23 @@ BGU MSc thesis on the ROLL method (Ranking via Optimized Label Learning). See [[
 - Thesis metadata (title, author, supervisor, month, year) defined as commands at top of =main.tex=
 - Hebrew title/author/supervisor have =he= suffix commands (=\thesistitlehe=, etc.)
 - Both English and Hebrew ToC, abstract, and cover pages are required by BGU
-- Chapters added via =\input{content/<chapter>/<chapter>}= with a matching =\addtocontents{tocheb}= line for the Hebrew ToC
+- Chapters added via =\input{content/<chapter>/<chapter>}= with a matching =\addtocontents{tocheb}= line
 - Build engine: pdflatex (uses =\usepackage[utf8x]{inputenc}=, not xelatex)
+- Math notation: data points are bold (=\mathbf{x}_i=); batches are =\mathcal{B}_0= / =\mathcal{B}_1=
+- KDE background (general technique) in Ch. 2 (=sec:kde-background=); Ch. 3 back-references it; Ch. 2 ends with forward pointer to =sec:roll-kde=
+- =sec:imbalanced-tpr-fpr= and =sec:neyman-pearson= live in Ch. 2; Ch. 3 =sec:problem-formulation= \Cref's them — do not rename
+
+** Chapter Structure
+
+See subnodes for per-chapter guidelines and section details.
+
+| Ch. | Title | File | Target |
+|-----|-------|------|--------|
+| 1 | Introduction | =content/introduction/introduction.tex= | 4–6 pp |
+| 2 | Background / Related Work | =content/related_work/related_work.tex= | 15–20 pp |
+| 3 | Method | =content/method/method.tex= | ~30 pp |
+| 4 | Experiments | =content/experiments/experiments.tex= | 15–20 pp |
+| 5–6 | Discussion + Conclusion | =content/discussion_and_conclusions/= | 8–12 pp |
 
 ** Gotchas
 
@@ -32,6 +47,10 @@ BGU MSc thesis on the ROLL method (Ranking via Optimized Label Learning). See [[
 - Hebrew rendering via =babel-hebrew= (pdflatex approach, not fontspec/xelatex)
 - Example images in =content/related_work/images/= are from the template — safe to delete
 - =flake.nix= uses =rec= on the derivation so =buildInputs= is in scope for PATH export
+- =nix develop= devShell includes python3+sympy — use it to verify math claims, never check mentally
+- KDE backward pass uses inverse-function-theorem approach deliberately (user must explain it verbally)
+- Correct gradient of threshold w.r.t. negative-class score: σ'(τ−x_i) / Σ_j σ'(τ−x_j) — sympy-verified
+- **(NIX BUILD) New .tex files must be =git add='ed before =nix build=.** Nix flakes use git to enumerate source files; untracked files are invisible to the build. Symptom: silent LaTeX failure — TOC, LoF, and LoT all appear empty in the output PDF. Fix: =git add <new-file>=, then rebuild.
 
 ** Key Files
 
@@ -41,7 +60,13 @@ BGU MSc thesis on the ROLL method (Ranking via Optimized Label Learning). See [[
 
 ** Subnodes
 
-(none yet)
+- [[id:c4a0d60f-49d3-4185-8c5b-f8a5a10a2ccc][thesis/introduction]] — Ch. 1 guidelines: structure, length, what to include
+- [[id:cd785954-008e-4b64-957d-25d1dbeac127][thesis/related-work]] — Ch. 2 section structure, placement markers, label conventions
+- [[id:0543ac39-af9b-40fd-9973-e60576a20695][thesis/method]] — Ch. 3 section status table, derivation gotchas
+- [[id:59f84838-71a9-4aab-a494-8e6bea2080fb][thesis/experiments]] — Ch. 4 guidelines: datasets, baselines, metrics
+- [[id:9b50caf2-d259-4848-b7f5-22a9288e8ba2][thesis/discussion]] — Ch. 5–6 guidelines: failure modes, future work
+- [[id:151159e8-1105-4a84-af9c-72d31c5e0660][thesis/todos]] — project TODOs
+
 * todo
 
 ** TODO Basic structure - take it into account

thesisdiscussion.org

@@ -0,0 +1,24 @@
+:PROPERTIES:
+:ID: 9b50caf2-d259-4848-b7f5-22a9288e8ba2
+:END:
+
+#+title: thesis/discussion
+#+filetags: :project: :knowledge: :discussion:
+
+:PROPERTIES:
+:ID: thesis-discussion
+:END:
+
+#+title: thesis/discussion
+#+filetags: :project: :knowledge: :discussion:
+
+Chapters 5–6 of the BGU MSc thesis on ROLL (Discussion + Conclusion). Not yet written.
+Parent: [[id:6294e2be-6189-4473-b363-a1dd9a75fb9b][thesis]]
+
+** Guidelines
+
+- Target length: 8–12 pages combined
+- Discussion: results interpretation, failure modes (when does KDE ROLL underperform?), limitations
+- Conclusion: summary of contributions, future work
+- Future work candidates: learned/adaptive bandwidth, extension beyond binary classification,
+  theoretical convergence guarantees, multi-class generalization

thesisexperiments.org

@@ -0,0 +1,24 @@
+:PROPERTIES:
+:ID: 59f84838-71a9-4aab-a494-8e6bae2080fb
+:END:
+
+#+title: thesis/experiments
+#+filetags: :project: :knowledge: :experiments:
+
+:PROPERTIES:
+:ID: thesis-experiments
+:END:
+
+#+title: thesis/experiments
+#+filetags: :project: :knowledge: :experiments:
+
+Chapter 4 of the BGU MSc thesis on ROLL. Not yet written.
+Parent: [[id:6294e2be-6189-4473-b363-a1dd9a75fb9b][thesis]]
+
+** Guidelines
+
+- Target length: 15–20 pages
+- Datasets: KEEL imbalanced benchmark suite + larger datasets TBD
+- Baselines should include: cross-entropy + class weighting, AUC-surrogate methods, NP-inspired methods
+- Primary metric: TPR at fixed FPR; secondary: partial AUC, comparison across operating points
+- Report results for all three ROLL instantiations (Gaussian, Beta, KDE) to justify the progression

thesisintroduction.org

@@ -0,0 +1,43 @@
+:PROPERTIES:
+:ID: c4a0d60f-49d3-4185-8c5b-f8a5a10a2ccc
+:END:
+
+#+title: thesis/introduction
+#+filetags: :project: :knowledge: :introduction:
+
+:PROPERTIES:
+:ID: thesis-introduction
+:END:
+
+#+title: thesis/introduction
+#+filetags: :project: :knowledge: :introduction:
+
+Chapter 1 of the BGU MSc thesis on ROLL. Written last, after Ch. 2–4 are stable.
+Parent: [[id:6294e2be-6189-4473-b363-a1dd9a75fb9b][thesis]]
+
+** Guidelines
+
+- Target length: 4–6 pages
+- Write last — contributions list and roadmap must match what the thesis actually delivers
+- Structure: Hook → Problem statement → Contributions → Roadmap
+- Do NOT explain the method — introduction is a promise, not a delivery
+
+** Structure
+
+1. *Hook* (~1–2 paragraphs) — concrete application where a wrong operating point is costly.
+   Candidates from existing prose: medical screening, fraud detection, astronomy/microscopy pipelines.
+   Make the reader feel the problem before naming it.
+2. *Problem statement* (~1 paragraph) — class imbalance + why accuracy is the wrong metric +
+   the TPR@FPR objective in one crisp sentence.
+3. *Contributions* (bulleted list, ~5 items) — what ROLL does that prior work does not:
+   - Differentiable TPR@FPR loss (and FPR@TPR by label/score symmetry)
+   - Gaussian, Beta, KDE instantiations with closed-form gradients
+   - Gradient-balance property (updates independent of class-size ratio)
+   - Gradient-locality property (threshold-focused gradient mass)
+   - Empirical evaluation on KEEL benchmark + larger datasets
+4. *Roadmap* (~1 paragraph) — one sentence per chapter, matching final chapter titles exactly.
+
+** Gotchas
+
+- Roadmap must exactly match final chapter titles — write it last
+- Don't use the inseparability / cascade-classifier tangent from the old draft; it dilutes the argument

thesismethod.org

@@ -0,0 +1,51 @@
+:PROPERTIES:
+:ID: 0543ac39-af9b-40fd-9973-e60576a20695
+:END:
+
+#+title: thesis/method
+#+filetags: :project: :knowledge: :method:
+
+:PROPERTIES:
+:ID: thesis-method
+:END:
+
+#+title: thesis/method
+#+filetags: :project: :knowledge: :method:
+
+Chapter 3 of the BGU MSc thesis on ROLL. Core contribution chapter. Currently ~30 pages.
+Parent: [[id:6294e2be-6189-4473-b363-a1dd9a75fb9b][thesis]]
+
+** Guidelines
+
+- sec:problem-formulation is now notation-only — motivation deferred to Ch. 2
+  via \Cref{sec:imbalanced-tpr-fpr,sec:neyman-pearson}
+- sec:imbalanced-tpr-fpr and sec:neyman-pearson were removed from this chapter and live in Ch. 2
+- Both objectives (TPR@FPR and FPR@TPR) are equivalent by negating scores and swapping labels;
+  all derivations given for TPR@FPR only — FPR@TPR follows by transformation
+
+** Section Structure
+
+| Section | Label | Status |
+|---------|-------|--------|
+| Problem Formulation (notation only) | sec:problem-formulation | Done |
+| The ROLL Framework | sec:roll-framework | Written |
+| Differentiability Problem | sec:differentiability-problem | Written |
+| Score Distribution Fitting | sec:score-distribution-fitting | Written |
+| General ROLL Formulation and Derivation | sec:roll-formulation | Written — core theoretical contribution |
+| Gaussian ROLL (forward + gradient) | sec:roll-gaussian | Written |
+| Beta ROLL (forward + gradient) | sec:roll-beta | Stub — TBD |
+| KDE ROLL (forward + gradient) | sec:roll-kde | Written |
+| Properties | sec:roll-properties | Written — gradient balance + locality |
+| Custom Backward Pass | sec:roll-backward | Stub — TBD |
+| Numerical Stability | sec:roll-numerical-stability | Stub — TBD |
+| Bandwidth Estimation (ISJ + scheduling) | sec:kde-bandwidth | Written |
+
+** Gotchas
+
+- KDE backward pass uses inverse-function-theorem approach deliberately; implicit differentiation
+  was considered and rejected — user needs to explain the derivation in their own words
+- Correct gradient of threshold w.r.t. negative-class score:
+  σ'(τ − x_i) / Σ_j σ'(τ − x_j) where sum is over ALL of B_0 — sympy-verified
+- Beta ROLL: translation-invariance does NOT hold when scores pass through sigmoid before fitting;
+  gradient-balance property breaks for Beta — documented in sec:roll-properties
+- Gradient-balance proof uses a uniform-shift argument; requires translation-invariant CDF estimator

thesisrelated-work.org

@@ -0,0 +1,70 @@
+:PROPERTIES:
+:ID: cd785954-008e-4b64-957d-25d1dbeac127
+:END:
+
+#+title: thesis/related-work
+#+filetags: :project: :knowledge: :related-work:
+
+Chapter 2 of the BGU MSc thesis on ROLL. Background and prior work. Target: 15–20 pages.
+Parent: [[id:6294e2be-6189-4473-b363-a1dd9a75fb9b][thesis]]
+
+** Guidelines
+
+- Narrows from broad background toward the specific gap ROLL fills
+- Reader should feel ROLL is inevitable by the end of the chapter
+- NP section belongs at the END — it is the narrowest prior work and bridges directly to Ch. 3
+
+** Section Structure
+
+| # | Title | Label | Placement | Length | Status |
+|---|-------|-------|-----------|--------|--------|
+| 2.1 | Imbalanced Classification and TPR/FPR Objective | sec:imbalanced-tpr-fpr | START | ~3–4 pp | Moved from Ch. 3; needs rewrite (see TODO comments in file) |
+| 2.2 | ROC Curves and AUC Optimization | TBD | middle | ~3–4 pp | Not yet written; cover AUC surrogates, pairwise ranking |
+| 2.3 | Partial AUC and TPR@FPR Methods | TBD | middle | ~3 pp | Not yet written; key point: no end-to-end backprop |
+| 2.4 | Kernel Density Estimation | sec:kde-background | middle | ~3–4 pp | Not yet written; forward-pointer to sec:roll-kde (Ch. 3) |
+| 2.5 | Learning with Noisy Labels | sec:noisy-labels | after KDE, before NP | ~3–4 pp | Outline in file; not yet written |
+| 2.6 | Connection to Neyman-Pearson | sec:neyman-pearson | END | ~2–3 pp | Moved from Ch. 3; needs writing (currently a stub) |
+
+** Gotchas
+
+- sec:imbalanced-tpr-fpr and sec:neyman-pearson are \Cref-referenced in Ch. 3 (sec:problem-formulation) — do not rename these labels
+- KDE section (sec:kde-background) must end with a forward pointer to sec:roll-kde in Ch. 3 — established convention from project brain root
+- Ch. 3 KDE section opens with a back-reference to sec:kde-background — keep this pairing consistent
+- sec:imbalanced-tpr-fpr arc (per TODO in file): accuracy gameable under imbalance → class weighting standard fix → no direct operating-point control → therefore need TPR@FPR directly
+- AUC section should distinguish whole-curve AUC from operating-point-specific TPR@FPR — this distinction motivates the partial AUC / TPR@FPR section
+- sec:noisy-labels added 2026-07-13; NP moved to 2.6 (was 2.5); sec:neyman-pearson label is UNCHANGED so Ch. 3 \Cref references still work without modification
+- sec:noisy-labels should be forward-referenced from Ch. 4 poisoning experiment; bridge: ROLL's robustness is a structural consequence of the distributional objective, not a design goal
+- Noisy labels bibtex entries staged in impl/research ** Noisy Labels Literature; move to thesis.bib when writing this section
+
+** Noisy Labels Literature (sec:noisy-labels)
+
+*** wu2025preserving — AUC Fairness + Noisy Protected Groups (ICML 2025)
+- cite key: =wu2025preserving= — ADDED to thesis.bib
+- arxiv: 2505.18532
+- *Fairness notion*: gap between group-level AUC and overall AUC (both intra-group and inter-group AUC); constraint is that no group's AUC deviates far from aggregate
+- *Groups*: demographic/sensitive attributes (gender, race) — separate from class labels; noise = mislabeled demographics, not mislabeled predictions
+- *Datasets*: Adult (tabular, socioeconomic), FF++ (image, deepfake detection)
+- *Baselines*: MinimaxFairAUC (Yang et al. 2023), DAW-FDD (Ju et al. 2024), standard AUC fairness methods
+- *What's novel*: first empirical analysis of noisy protected-group label impact on AUC fairness; DRO framework bounding TV distance between clean/noisy distributions; CLIP-based noise detection (no extra training)
+- *Thesis relevance*: cite in sec:noisy-labels; connects to ROLL's robustness argument in Ch. 4
+
+*** xie2024wsauc — Weakly Supervised AUC, Unified Partial AUC (IEEE TPAMI 2024)
+- cite key: =xie2024wsauc= — ADDED to thesis.bib
+- arxiv: 2305.14258 / doi: 10.1109/TPAMI.2024.3357814
+- Authors: Zheng Xie, Yu Liu, Hao-Yuan He, Ming Li, Zhi-Hua Zhou
+- *Not group fairness* — about robustness of AUC under weak/imperfect supervision
+- *Covers four weak supervision regimes*: noisy label learning, positive-unlabeled (PU) learning, multi-instance learning, semi-supervised learning
+- *Key insight*: all four regimes can be unified as "minimizing AUC risk on contaminated sets"
+- *Novel metric*: rpAUC (reversed partial AUC) — used as a robust surrogate training objective when labels are contaminated; theoretical consistency with true AUC proven
+- *Framework*: WSAUC — universal solution via empirical rpAUC maximization
+- *Datasets/baselines*: PDF is image-based (scanned), could not extract; likely standard benchmarks (CIFAR/MNIST class)
+- *Thesis relevance*: cite in sec:noisy-labels; rpAUC as a robust surrogate is conceptually adjacent to ROLL's KDE-based surrogate — both address the "how do you optimize AUC reliably" question
+
+*** zhang2023doubly — Doubly Robust AUC vs Noisy + Adversarial Samples (KDD 2023)
+- cite key: =zhang2023doubly= — *NOT YET ADDED to thesis.bib; need unpaywalled copy first*
+- doi: 10.1145/3580305.3599316 — ACM paywalled, no arXiv preprint found
+- *Not about group fairness* — robustness of AUC to (1) noisy class labels and (2) adversarial perturbations
+- *Method (DRAUC)*: self-paced learning + adversarial training within AUC optimization; theoretical upper bound on adversarial AUC risk; alternating stochastic descent
+- *Datasets/baselines*: unknown — full text inaccessible
+- *Thesis relevance*: cite in sec:noisy-labels alongside wu2025preserving — brackets noisy-label problem from two sides (noisy demographics vs. noisy class labels)
+- *ACTION NEEDED*: obtain unpaywalled PDF to verify datasets/baselines and add bibtex entry

thesistodos.org

@@ -0,0 +1,9 @@
+:PROPERTIES:
+:ID: 151159e8-1105-4a84-af9c-72d31c5e0660
+:END:
+
+#+title: thesis/todos
+#+filetags: :project: :todo:
+
+* TODO Verify alpha notation w.r.t. FPR — eq:fpr-cdf writes FPR(τ) = F̂₀(τ) and threshold as τ = F̂₀⁻¹(α), but standard FPR = P(score > τ | y=0) = 1−F₀(τ) requires τ = F̂₀⁻¹(1−α). Check for sign/notation inconsistency throughout method.tex and appendix_gauss_grad.tex.
+* TODO Add graphs of CDF tracking for gaussian, KDE, roll.