backup: 2026-07-25 14:39

implwork-historyrecent.org

@@ -0,0 +1,19 @@
+:PROPERTIES:
+:ID: 863ca60e-6a3c-45d2-86fd-fb4ed386e31b
+:END:
+
+#+title: impl/work-history/recent
+#+filetags: :project: :work-history:
+
+* 2026-07-24 Thu
+
+- Investigated stalled cifar10n remote run: 5 workers from 6:45AM were still alive after 15.5h with 0 results; previous session's ps check had missed them, making the run appear dead
+- Diagnosed OOM: memory at ~46MB free at 7h mark with 5 concurrent MPS workers on 16GB; new run with 3 workers also OOM'd after 8.5h (26GB compressed pages, 5B decompressions)
+- Fixed root cause: added =sequential_episodes= flag to =run_configurations= and =_perform_multiple_episodes= in =src/experiment.py= — when True, bypasses the subprocess-parallel path and runs episodes one at a time in-process on MPS; cifar10n now uses this
+- Killed 3-worker subprocess run on remote; restarted cifar10n with =sequential_episodes=True= — now running single-process, one episode at a time
+- Added =kaggle= to =flake.nix= buildInputs; set up =~/.config/kaggle/kaggle.json= (user: anerzakobar); downloaded creditcard.csv (144MB) to =~/.data/creditcard/=
+- Rewrote =experiments/large/experiment-creditcard.py=: replaced old ad-hoc FPR configs with full baseline suite matching KEEL/_base.py — roll-aoc, roll-tpr90, bce-weighted, libauc-auroc, gce-0.7, mae, focal-loss, asymmetric-loss; StepLR on all configs; N_EPISODES=5
+- Launched creditcard experiment locally in background (5 episodes × 8 configs, CPU multiprocessing)
+- Reviewed KEEL AUC results table: libauc-auroc wins 6/12 datasets (mean 0.890); asymmetric-loss strong second (0.887); roll-aoc 6th (0.872); roll-tpr90 last (0.842, dragged by new-thyroid1 instability)
+
+** Archives