** Architecture
Simlish-style procedural speech synthesizer for trmn. Output: ONNX model loaded by JS in trmn.
Pipeline: text → prosody (JS) → phoneme sequence → ONNX flow matching → mel → Griffin-Lim → audio.
Deterministic: seeded per character via trmn's masterSeed PRNG.
See [[just-talking/simlish-project]] for full scoping notes and research version design.

Two-stage inference:
1. Acoustic model (flow matching, ONNX): (phone_ids, durations, pitch, energy, emotion) → mel[80×T]
2. Vocoder (Griffin-Lim, pure JS): mel → waveform. No weights. Swappable later.

** Decided: Acoustic Model
Adapted Matcha-TTS (ICASSP 2024, arXiv:2309.03199). ConvNeXt-based OT-CFM flow decoder.
- Phoneme encoder: embedding table (N_PHONEMES=16) + 2 Conv1d layers → hidden[128, P]
- Duration predictor: DROPPED — durations come from prosody.mjs at inference time
- Length regulator: kept — repeat_interleave phoneme hidden states by duration counts
- Flow decoder: 5 ConvNeXt blocks, 128 channels. FiLM emotion conditioning per block.
- Pitch + energy: linear-embedded per frame, concatenated with phoneme hidden before decoder
- ~2–5M params. Trained with OT-CFM (MSE on predicted velocity x1-x0).

** Decided: Phoneme Inventory (16 total)
0=<pad>  1=<sil>
2=p  3=t  4=k  5=b  6=d  7=m  8=n  9=s  10=l
11=a  12=e  13=i  14=o  15=u
No <bos>/<eos> — not needed for non-autoregressive model.

** Decided: Training Data — LUT Approach
English words → letter LUT → mapped text → Piper TTS → audio → mel extraction.
LUT: letters in inventory map to themselves; others map to nearest class (r→l, f→s, g→d, etc.)
Phone IDs come from re-applying LUT at extract time — no forced alignment.
Duration heuristic: vowels get 1.5× frames, silences 0.4×, consonants 1.0×, proportional to total T.
Length-scale varied 0.8–1.3× per sample for duration variety.

Piper voice: en_US-lessac-medium (fetched via Nix, both .onnx and .onnx.json pinned by sha256).
~10–50h synthetic audio expected to suffice.

** Decided: Nix Targets
Separate derivations with strict dep isolation (mirrors trmn pattern):
  packages.onnx-model — ONNX artifact trmn pulls in; placeholder until checkpoint pinned
  packages.piper-voice — fetchurl'd lessac-medium model + config, cached independently
  apps.gen-data       — Piper + librosa + numpy; no torch, no JS
  apps.train          — torchWithCuda + onnx; no Piper, no MFA, no JS
  apps.test           — nodejs only; no Python
  devShells.data      — same deps as gen-data, PIPER_MODEL/PIPER_CONFIG set via env
  devShells.training  — torchWithCuda + onnx; for LAN laptop (RTX 3050)
  devShells.default   — nixfmt + nodejs; lightweight

** Decided: Training Environment
LAN laptop: aner@192.168.1.200, no password. WSL2 on Windows. RTX 3050 6GB VRAM, CUDA 13.2.
CUDA accessible via /dev/dxg (WSL2 WDDM). nvidia-smi at /usr/lib/wsl/lib/nvidia-smi.
Disk: 899GB available (plenty). RAM: 7.6GB WSL2 limit — may need to raise via C:\Users\<you>\.wslconfig.
PyTorch: use torchWithCuda (not torch) in Nix — default torch is CPU-only.

** Key Files
- flake.nix               — all Nix targets; Piper voice fetched here
- data/gen_corpus.py      — English words → LUT → .txt (Piper input) + .ids (phone IDs)
- data/synthesize.py      — Piper TTS → .wav per sentence (length-scale varied)
- data/extract.py         — audio + .ids → mel/pitch/energy/durs → .npz dataset
- model/acoustic.py       — AcousticModel: PhonemeEncoder + LengthRegulator + FlowDecoder
- model/train.py          — OT-CFM training loop
- model/export.py         — PyTorch checkpoint → ONNX FP32 → INT8
- inference/prosody.mjs   — text + emotion → phone_ids/durations/pitch/energy (pure JS)
- inference/griffin_lim.mjs — mel → waveform (pure JS, replaceable vocoder)
- test/infer.test.mjs     — ONNX load + forward pass + shape checks (node:test)
- test/prosody.test.mjs   — prosody extraction unit tests: determinism, emotion, shapes

** Conventions
- No runtime server calls at inference
- Same seed must always produce same voice (deterministic)
- Vocoder is a swappable box: mel[80×T] in, waveform out
- Nix: no npm/pip lock files — Nix is the build orchestrator
- Only git-tracked files included in builds (src = ./. in derivations)
- node:test for JS tests (no framework, follows trmn no-bundler pattern)
- Piper CLI: echo "text" | piper -m model.onnx -c config.json -f out.wav

** Gotchas
- trmn is static Nix-packaged — ONNX artifact must be added to trmn's flake.nix (not this one)
- SVG mouth sprites in trmn are ready to animate but animation API not yet defined
- ODE integration loop (Euler, N steps) runs in JS — only single-step velocity exported to ONNX
- extract.py reads .ids files from corpus/ subdir, audio from audio/ subdir — path is hardcoded relative
- dataPython defined twice in flake.nix (packages and apps sections) — intentional, avoids cross-section ref
- torchWithCuda (not torch) required in Nix for GPU training — default torch is CPU-only
- WSL2: nvidia-smi not on PATH, use /usr/lib/wsl/lib/nvidia-smi; CUDA works via /dev/dxg

** Subnodes
(none yet)