backup: 2026-10-14 09:41
This commit is contained in:
@@ -10,9 +10,13 @@
|
||||
- =import-tree= auto-discovers features but only sees tracked files — same constraint
|
||||
- The machine name is =lauretta=; machine config is at =_machines/lauretta.nix=
|
||||
- =postPatch= in Nix derivations: alejandra reformats indentation of multiline strings, which can change effective shell script content
|
||||
- A systemd (user) unit's =Service.Environment = "PATH=...";= REPLACES the default PATH rather than appending to it — any bare command (e.g. =sleep=) in the associated script will 127 under that restricted PATH even though it works fine run manually from an interactive shell. Every command needs an absolute =${pkgs.X}/bin/...= reference, not just the ones the override was written for. Hit this in =features/nextcloud/default.nix= (see [[id:f4196b35-5438-4da3-9d34-511291f482b5][azos/nextcloud]]).
|
||||
- Same =PATH= override gotcha has a second trap: several NixOS tools (=fusermount3=, =sudo=, etc.) only work from their *setuid-root* wrapper at =/run/wrappers/bin/=, not from the plain package's =/nix/store/.../bin/= (not setuid, lacks the needed capability). A restricted =PATH=... override must include =/run/wrappers/bin= or any FUSE/setuid-dependent command will fail with "Operation not permitted" (not 127 — a different failure mode, easy to misdiagnose as unrelated/environmental). See [[id:f4196b35-5438-4da3-9d34-511291f482b5][azos/nextcloud]] for the =fusermount3= instance of this.
|
||||
|
||||
** Subnodes
|
||||
- [[id:b8094a02-528a-4eeb-a021-4ffc7396f0f3][azos/emacs]] — Emacs-specific gotchas (async-shell-command/pty, org-babel lexical binding)
|
||||
- [[id:da416dd3-d7bb-4a81-83fb-a5543817b462][azos/mail]] — mu4e/mbsync/msmtp accounts, contexts, bookmarks
|
||||
- [[id:b72d2db1-f539-40b9-ade4-1253415049e2][azos/hardware-auth]] — YubiKey U2F login/sudo setup; lauretta's fingerprint reader is unsupported, why, and what would unblock it
|
||||
- [[id:939a9fa3-ddc8-4cc1-8130-771cc5de000f][azos/todos]] — open project TODOs
|
||||
- [[id:1196aa79-76ba-4090-a896-6e4938338bd1][azos/dav]] — DAV (phone reminders) todo sync via vdirsyncer/todoman; recurring/alarm limitations
|
||||
- [[id:f4196b35-5438-4da3-9d34-511291f482b5][azos/nextcloud]] — Nextcloud webdav mount via rclone, YubiKey-gated; the PATH-override crash-loop fix
|
||||
+31
@@ -0,0 +1,31 @@
|
||||
:PROPERTIES:
|
||||
:ID: 1196aa79-76ba-4090-a896-6e4938338bd1
|
||||
:END:
|
||||
|
||||
#+title: azos/dav
|
||||
#+filetags: :project: :knowledge: :dav:
|
||||
|
||||
Back to [[id:bf4dde56-0967-4374-94ed-771301e47c66][azos]].
|
||||
|
||||
* Architecture
|
||||
- Phone (Apple Reminders) ↔ Nextcloud CalDAV ↔ vdirsyncer ↔ local vdir, with todoman (=todo=) as the CLI. Generic module: =azos-core/suites/base/dav= (=azos.dav.*=); lauretta's account in =features/dav= via home-manager's own =accounts.calendar.accounts= (same split as mail).
|
||||
- DAV todos are deliberately separate from org-roam project todos ("local vs dav", not "phone"). Moving a project todo to DAV = create on DAV with =(project)= prefix, delete locally. The =todo= / =workplate= skills handle DAV when =azos-dav-sync= is on PATH — no separate skill.
|
||||
- =azos-dav-sync=: YubiKey presence gate (exit 75 + last-sync time if absent) → discover → metasync → sync → regenerate a read-only org view (=~/.local/share/azos-dav/todos.org=, added to org-agenda via =~/.emacs.d/extra/azos-dav.el=). Conflicts: server/phone wins.
|
||||
- Live since 2026-10-14 on lauretta: syncs Nextcloud lists =aner= and =zakobar= (todoman shows them as =Aner= / =Zakobar=); collections still ="from a"= (everything). Committed azos-core =ba95691=, azos =b5c7d7d=.
|
||||
- Calendar (events) sync + workplate reading today's schedule is the next step — tracked in [[id:939a9fa3-ddc8-4cc1-8130-771cc5de000f][azos/todos]].
|
||||
|
||||
* Gotchas
|
||||
- todoman 4.7: completing a *recurring* VTODO forks a new UID and drops DTSTART + VALARM (phone stops notifying). plann 1.2 / caldav-lib "safe" mode keeps UID but drifts the next occurrence to completion time and leaves absolute alarm triggers stale. Decision: recurring items are completed on the phone; skill forbids =todo done= on them. Stretch: inspect how Apple writes a completed recurrence, maybe fix todoman upstream (issues #304, #249).
|
||||
- todoman has no alarm support at all — timed tasks created from the computer won't ring on the phone.
|
||||
- todoman =--porcelain= gives epoch seconds; all-day vs local midnight is indistinguishable (org view treats midnight as date-only).
|
||||
- HM vdirsyncer names pairs =calendar_<account>=, a = remote; ="remote wins"= → ="a wins"=.
|
||||
- =vdirsyncer discover= prompts per missing collection and aborts on closed stdin. Module pre-creates explicitly listed local collections and answers "n" (never creates lists on the server; typos fail loudly); answers "y" only for ="from a"=. Runs every sync so new lists appear automatically.
|
||||
- =ical2orgpy= ignores VTODO (events only); no existing VTODO→org tool found, hence the small =org-view.jq=.
|
||||
- Test bed that worked: local Radicale (=nix shell nixpkgs#radicale=, =auth type = none=) + MKCALENDAR via curl.
|
||||
|
||||
* Future work (not scheduled — deliberately not in azos/todos)
|
||||
- Emacs command to mark a DAV item done from the org agenda/view (run =todo done= + =azos-dav-sync=); the org view is read-only by design until then.
|
||||
- See how Apple writes a completed *recurring* reminder (inspect real .ics after ticking one on the phone); if feasible, fix todoman's recurring completion upstream (#304, #249) so recurring items can be completed from the computer.
|
||||
- Pin =collections= in =features/dav= to the wanted lists instead of ="from a"= (currently only =aner= and =zakobar= exist, so no practical difference yet).
|
||||
- Check whether Reminders *flags* survive CalDAV (likely iCloud-only); priority (!/!!/!!!) is the known-working importance marker.
|
||||
- Alarms: todoman can't create them, so computer-created timed tasks don't ring — only fixable with custom code or an upstream todoman feature.
|
||||
@@ -9,5 +9,6 @@
|
||||
- =async-shell-command= spawns subprocesses with =TERM=dumb= and no real controlling tty (confirmed: no =INSIDE_EMACS=, pty exists but curses can't use it). Any gpg/pinentry prompt triggered from such a process (e.g. signed git commits) fails with "Inappropriate ioctl for device" once the pinentry PIN cache expires — pinentry-emacs mode silently isn't engaged either, since gpg only requests it when =INSIDE_EMACS= is set, which =async-shell-command= doesn't set (unlike =M-x shell=/comint, which does). Fix: run the command in a real pty via =vterm= instead (confirmed working — =vterm-term-environment-variable= is set to =xterm-256color=). Fixed in =azos/roam/backup= (=features/lauretta/emacs/config.org=) 2026-10-04.
|
||||
- A nix rebuild (=azos-lauretta-update=) only regenerates files on disk (e.g. =config.org=) — it never reloads anything into an already-running Emacs session. =setq=/=defvar= forms evaluated at startup (e.g. =mu4e-contexts=) stay stale until that exact top-level form is re-evaluated, e.g. via =emacsclient -e '(load-file "/tmp/x.el")'= on a file containing just that extracted sexp (match parens exactly — an off-by-one line range truncates the form and errors "End of file during parsing" instead of warning about staleness). Discovered via mu4e context drift, see [[id:da416dd3-d7bb-4a81-83fb-a5543817b462][azos/mail]] Gotchas for the full incident.
|
||||
- =org-babel= =emacs-lisp= source blocks evaluate with *dynamic* binding by default (=ob-emacs-lisp.el= defaults =:lexical= to ="no"=), and =config.org= sets no =:lexical= header anywhere (file-wide or per-block). This silently breaks closures in async code: a pattern like =(let ((output "")) (make-process :filter (lambda (_p chunk) (setq output ...)) :sentinel (lambda (_p _e) ... output ...)))= fails because the let-bound =output= unwinds before the async filter/sentinel callbacks run — they don't lexically close over it. The failure is silent: errors like "Symbol's value as variable is void: output" only appear in =*Messages*=, not as visible UI errors, and whatever the sentinel was supposed to set (e.g. =elfeed-feeds=) just stays =nil=. This hit the elfeed-protocol Nextcloud News auth block (fetches a =pass(1)= password asynchronously via =make-process= to avoid blocking Emacs's command loop and deadlocking with pinentry-emacs, then sets =elfeed-feeds= from the sentinel) — =elfeed-update= appeared to "do nothing" because of this exact bug. Fix/convention: for any async Emacs Lisp in this repo's org-babel blocks (process filters/sentinels, timers, callbacks) that needs to carry state across the async boundary, don't rely on closures over =let=-bound locals unless the block explicitly sets =:lexical yes=. Instead store state on the process object itself via =process-put=/=process-get= (works regardless of lexical/dynamic binding) — this is the pattern now used in the elfeed-protocol block. Fixed in =features/lauretta/emacs/config.org= (Elfeed Nextcloud News Sync section) 2026-10-05.
|
||||
- Flake inputs that ship a prebuilt Emacs package (e.g. =evil-hl-line='s =packages.default=) are built against the *input's own* pinned nixpkgs, so they propagate deps (=evil=) byte-compiled by a different Emacs version. Both copies end up in =emacs-packages-deps= and the stale one can shadow ours on the load-path. After the Emacs 30.2 → 31.1 bump (2026-10-13) this gave =void-variable evil-local-mode--set-explicitly= on every command: old .elc top-level forms (hook adds old =evil-local-mode-set-explicitly=, no new defvar) mixed with JIT native-comp swapping in 31-macro function bodies that reference the new =--set-explicitly= var. Fix/convention: build such inputs against our own =pkgs.emacs.pkgs= (never take their =packages.default=). =evil-hl-line= upstream now exposes =lib.mkEvilHlLine epkgs= and =overlays.default= for this; azos-core uses =lib.mkEvilHlLine pkgs.emacs.pkgs= in =suites/base/emacs/default.nix=. Diagnose with =find -L <emacs-packages-deps>/share/emacs/site-lisp/elpa -name '*.elc' | xargs sed -s -n 3p | sort | uniq -c= — any mixed "in Emacs version" values means a foreign package set leaked in.
|
||||
|
||||
[[id:bf4dde56-0967-4374-94ed-771301e47c66][azos]]
|
||||
@@ -35,6 +35,43 @@ Key custom keys:
|
||||
- =exwm-input-global-keys=: s-f2, s-tab, s-w, s-0..9, s-&, always grabbed regardless
|
||||
of mode (system-wide, via =exwm-input--global-prefix-keys=)
|
||||
|
||||
** Desktop notifications: EDNC replaces dunst
|
||||
|
||||
Under EXWM, =services.dunst.enable = lib.mkForce false;= and EDNC (Emacs
|
||||
Desktop Notification Center, =azos-core/features/exwm/emacs/config.org=) runs
|
||||
instead, owning =org.freedesktop.Notifications= on the session bus directly
|
||||
from the Emacs process (=ednc-mode=). Arbitrary non-Emacs senders (scripts,
|
||||
=notify-send=, GUI apps like zapzap) reach it the normal way — a plain D-Bus
|
||||
=Notify= call — no dunst-specific glue needed. =azos/ednc-popup--present= adds
|
||||
a posframe toast on top of EDNC's built-in =*ednc-log*= buffer logging.
|
||||
|
||||
Confirmed working 2026-10-11: =busctl --user list= showed the Emacs process
|
||||
owning the bus name; a synthetic =Notify= call shaped like zapzap's real one
|
||||
(icon file path, =category=im.received= hint) landed correctly in
|
||||
=*ednc-log*=. See [[id:f4196b35-5438-4da3-9d34-511291f482b5][azos/nextcloud]]
|
||||
for the investigation that prompted this (zapzap WhatsApp notifications
|
||||
reported missing — pipeline turned out fine; likely just WhatsApp Web's
|
||||
normal "don't notify for the focused chat" behavior).
|
||||
|
||||
** Station renamed to desktop; vestigial Emacs dependency removed (2026-10-13)
|
||||
|
||||
The home-manager =station= suite was renamed to =desktop=
|
||||
(=azos-core/suites/desktop/=, module =config.flake.modules.homeManager.desktop=,
|
||||
option =azos.suites.desktop.enable=) rather than split further — its apps
|
||||
(browser, gimp, libreoffice, spotify, whatsapp, zoom) turned out to be
|
||||
wanted on any GUI machine, work included. exwm still depends on it by
|
||||
default (=azos.suites.desktop.enable = lib.mkDefault true=) since exwm's
|
||||
quick-launch keybindings for chromium/qutebrowser/gimp need those
|
||||
binaries. Only =discord= was dropped from azos-core entirely, now living
|
||||
as standalone =azos/features/discord= — personal-only apps live in azos,
|
||||
not azos-core, going forward.
|
||||
|
||||
exwm's =_pkg.nix= and =config.org= no longer depend on
|
||||
=azos-emacs-station= (renamed =azos-emacs-desktop=) at all -- confirmed
|
||||
via grep that none of exwm's own config used its symbols
|
||||
(mu4e/elfeed/wttrin/kubernetes-/pass); the require was vestigial. Full
|
||||
task history in [[id:939a9fa3-ddc8-4cc1-8130-771cc5de000f][azos/todos]].
|
||||
|
||||
* Gotchas
|
||||
|
||||
** Line-mode does NOT mean "all keys reach Emacs unconditionally"
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
:PROPERTIES:
|
||||
:ID: f4196b35-5438-4da3-9d34-511291f482b5
|
||||
:END:
|
||||
|
||||
#+title: azos/nextcloud
|
||||
#+filetags: :project: :knowledge: :nextcloud:
|
||||
|
||||
Backlink: [[id:bf4dde56-0967-4374-94ed-771301e47c66][azos]]
|
||||
|
||||
* Architecture
|
||||
|
||||
=features/nextcloud/default.nix= (home-manager module, =azos.nextcloud-client= /
|
||||
=azos.nextcloud-mount=) mounts a Nextcloud webdav folder at =~/synced-docs= via
|
||||
=rclone mount= as a systemd user service (=nextcloud-mount.service=,
|
||||
=WantedBy = default.target=, not =graphical-session.target= — the latter never
|
||||
activates under EXWM since nothing explicitly starts it).
|
||||
|
||||
Credentials come from =pass= at service start (not baked into the Nix store):
|
||||
password from =pass zakobar.com=, Nextcloud *user ID* (distinct from login name
|
||||
on this server) from =pass nextcloud-username/aner=. See [[id:09ae970c-2995-43d9-a597-47bb904c98de][azos/pass]].
|
||||
|
||||
Gated on YubiKey smartcard presence via the shared =azos-card-present= overlay
|
||||
(=overlays/default.nix=, wraps =gpg --card-status=) — same silent presence
|
||||
check used by the mbsync mail-sync timer (=features/mail/default.nix=). No card
|
||||
-> loop quietly until plugged in, no pinentry prompt either way.
|
||||
=Restart = always= means unplugging/replugging the key remounts naturally.
|
||||
|
||||
Added in commit 04427dd "Nextcloud sync, also zapzap" (2026-10-11), same commit
|
||||
that bumped azos-core to add the zapzap WhatsApp client
|
||||
(=azos-core/features/station/whatsapp.nix=) — unrelated features landed together.
|
||||
|
||||
* Gotchas
|
||||
|
||||
** PATH-override crash loop (fixed 2026-10-11)
|
||||
|
||||
=nextcloud-mount.service= crash-looped (exit code 127, every 10s) because
|
||||
=Service.Environment = "PATH=${pkgs.fuse3}/bin:${pkgs.fuse}/bin";= in
|
||||
=features/nextcloud/default.nix= *replaces* the service's PATH rather than
|
||||
appending. Every command in the mount script used absolute =${pkgs.X}/bin/...=
|
||||
paths except the YubiKey-poll loop's bare =sleep 10= — which couldn't be found
|
||||
under the restricted PATH, so bash exited 127. Ran fine manually (inherits full
|
||||
interactive PATH), which is why it looked like it should work.
|
||||
|
||||
Fix: =${pkgs.coreutils}/bin/sleep 10= instead of bare =sleep 10=. General
|
||||
lesson logged on the [[id:bf4dde56-0967-4374-94ed-771301e47c66][azos]] root —
|
||||
any systemd unit that restricts PATH this way needs every bare command in its
|
||||
script converted to an absolute nix-store reference, not just the ones the
|
||||
override was added for.
|
||||
|
||||
** Same crash loop recurred with a different command (2026-10-12)
|
||||
|
||||
=nextcloud-mount.service= was 127-looping again, but this time the culprit was
|
||||
=head=, not =sleep=: =password=$(pass zakobar.com | head -1)= and the matching
|
||||
=uid=$(pass nextcloud-username/aner | head -1)= line both piped into bare
|
||||
=head -1=, which isn't on the restricted PATH either. Under =set -eu=, the
|
||||
pipeline's exit status is =head='s (127), which aborts the script. Fixed by
|
||||
changing both to =${pkgs.coreutils}/bin/head -1=.
|
||||
|
||||
Confirms the general lesson above needs to be applied exhaustively: fixing the
|
||||
one bare command that happened to be caught isn't enough — *every* token in
|
||||
the script that isn't an absolute =/nix/store/...= path is a latent 127.
|
||||
Next time this crash-loops, visually scan the whole script rather than
|
||||
assuming the previously-fixed command is the only one.
|
||||
|
||||
Still 127'd after deploying the =head= fix — turned out =mkdir -p
|
||||
"$mountpoint"= (the very first line of the script, before the YubiKey loop)
|
||||
was *also* bare and had been bare the whole time, just masked because it
|
||||
never got reached/tested in isolation before. Fixed to
|
||||
=${pkgs.coreutils}/bin/mkdir -p=. Verified by extracting the deployed
|
||||
=writeShellScript= output, manually running it under the exact restricted
|
||||
=PATH=/<fuse paths>= from =systemctl --user show nextcloud-mount.service -p
|
||||
Environment=, with =bash -x= to see exactly which line 127'd — this is the
|
||||
reliable way to debug this script going forward rather than guessing from
|
||||
=journalctl= alone (which only reports the exit code, not which line).
|
||||
|
||||
** A third, different bug after that: non-setuid fusermount3 (2026-10-12)
|
||||
|
||||
After the =mkdir= fix the script got all the way to =rclone mount=, which
|
||||
then failed with "fusermount3: mount failed: Operation not permitted" (exit
|
||||
1, not 127 — a new failure mode). Root cause: NixOS's real, usable
|
||||
=fusermount3= is the *setuid-root* one at =/run/wrappers/bin/fusermount3=
|
||||
(installed via =security.wrappers=); the plain =${pkgs.fuse3}/bin/fusermount3=
|
||||
in the nix store has no setuid bit and can't perform the mount syscall
|
||||
(lacks CAP_SYS_ADMIN). =rclone mount= looks up =fusermount3= via =PATH=
|
||||
internally rather than an absolute path, and the service's
|
||||
=Service.Environment = "PATH=...";= override pointed only at the raw
|
||||
=pkgs.fuse3=/=pkgs.fuse= bin dirs, never =/run/wrappers/bin= — so it always
|
||||
resolved to the broken non-setuid binary. =ExecStop= had the identical bug
|
||||
(hardcoded to the non-setuid =${pkgs.fuse3}/bin/fusermount3 -u ...=).
|
||||
|
||||
Fix: prepend =/run/wrappers/bin= to the =PATH= override, and point
|
||||
=ExecStop= at =/run/wrappers/bin/fusermount3= directly. Verified by manually
|
||||
exporting the rclone env vars and PATH and running =rclone mount= by hand —
|
||||
failed identically without =/run/wrappers/bin=, succeeded (real files
|
||||
visible) with it.
|
||||
|
||||
General lesson for *any* systemd unit that overrides =PATH= and touches
|
||||
FUSE: =/run/wrappers/bin= must be included, not just the raw package's
|
||||
=/bin= — several NixOS tools (fusermount, sudo, etc.) only work from their
|
||||
setuid wrapper, which lives there, not in the nix store path itself. I
|
||||
initially misdiagnosed this same "Operation not permitted" error as an
|
||||
artifact of my own sandboxed test shell lacking mount capabilities — it
|
||||
was not; always re-test with the real setuid wrapper on PATH before
|
||||
concluding "not reproducible" for FUSE-related permission errors.
|
||||
|
||||
Notably, the *symptom* this time didn't look like a crash-looping systemd
|
||||
service at first glance — it looked like the Nextcloud desktop GUI client
|
||||
(the separate =services.nextcloud-client=, webflow login) showing only the
|
||||
Activity/notifications view with no file browser. That GUI client authenticates
|
||||
fine on its own (confirmed via =~/.config/Nextcloud/logs=: AccountState
|
||||
=CredentialsNotReady -> Connected=) — it's unrelated to file access. Actual
|
||||
file access goes through the rclone webdav mount at =~/synced-docs=, which
|
||||
was empty because this crash loop kept it unmounted. When the GUI client
|
||||
looks "connected but empty", check =systemctl --user status
|
||||
nextcloud-mount.service= rather than assuming an auth/account problem.
|
||||
|
||||
* Related: zapzap notification delivery (confirmed working, no bug)
|
||||
|
||||
Same session also investigated zapzap (WhatsApp client, added alongside this
|
||||
feature) not showing desktop notifications. Confirmed the whole pipeline is
|
||||
correctly wired — see [[id:c52036a9-25ec-4b8b-8b72-90fcd9ea65dd][azos/exwm]]
|
||||
for the EDNC/D-Bus notification architecture note. No bug found; likely
|
||||
explanation if it recurs is WhatsApp Web's standard behavior of suppressing
|
||||
notifications for a chat that's already focused/open in the window.
|
||||
@@ -11,6 +11,7 @@
|
||||
- Database: =~/.emacs.d/org-roam.db= (sqlite-builtin connector)
|
||||
- MCP server: =org-roam-mcp= registered in =~/.claude.json= via home-manager activation
|
||||
- org-agenda includes =~/roam/= so TODOs in roam files appear in agenda
|
||||
- Roam + Capture Emacs config lives in the *base* suite (=azos-core/suites/base/emacs/config.org=, Org section), not editor — moved there so base-suite features (e.g. DAV org view) don't depend on the editor suite. Roam =add-to-list='s =org-agenda-files= rather than =setq= so other modules can append agenda files without load-order issues.
|
||||
|
||||
* Gotchas
|
||||
|
||||
|
||||
@@ -36,6 +36,8 @@ login: user@example.com
|
||||
| BGU mbsync | =mbsync/anerz@post.bgu.ac.il= | =features/mail/default.nix= |
|
||||
| Zakobar mail (privateemail.com) | =privateemail.com= | =features/mail/default.nix= |
|
||||
| Nextcloud / caldav | =zakobar.com= | =features/lauretta/emacs/config.org= |
|
||||
| Nextcloud webdav mount (password, reuses caldav entry) | =zakobar.com= | =features/nextcloud/default.nix= |
|
||||
| Nextcloud webdav mount (uid, differs from login name) | =nextcloud-username/aner= | =features/nextcloud/default.nix= |
|
||||
| Git credentials (zakobar.com) | =zakobar.com= | =features/git-config/pass-git-mapping.ini= |
|
||||
| Lichess API token | =lichess.org= (via auth-source-pass) | =azos-core/features/base/emacs/config.org= |
|
||||
|
||||
|
||||
+422
-32
@@ -7,42 +7,432 @@
|
||||
|
||||
Backlink: [[id:bf4dde56-0967-4374-94ed-771301e47c66][azos]]
|
||||
|
||||
* TODO Decouple the exwm and station suites in azos-core
|
||||
Ordered plan for the new Fedora/EXWM work machine effort. Work top to
|
||||
bottom; check items off with the todo skill as they land. Motivation
|
||||
throughout: a new work computer (Fedora + nix) wants the =exwm= suite,
|
||||
which depends on the (renamed-from-station) =desktop= suite, and many
|
||||
=azos/features/*= modules should become reusable azos-core suites for
|
||||
it — except genuinely personal-only ones, which stay in azos/features
|
||||
per the discord precedent.
|
||||
|
||||
=azos-core/features/exwm/default.nix= sets =azos.suites.station.enable =
|
||||
lib.mkDefault true;= — already overridable at the suite level with a plain
|
||||
=azos.suites.station.enable = false;= in a machine config, no =mkForce=
|
||||
needed. But =features/exwm/emacs/_pkg.nix= hardcodes
|
||||
=pkgs.azos-emacs-station= in =packageRequires=, and
|
||||
=features/exwm/emacs/config.org= line 8 has an unconditional
|
||||
=(require 'azos-emacs-station)= — this forces =azos-emacs-station= (and its
|
||||
transitive package deps: mu4e, elfeed, elfeed-tube, pass, scad-mode,
|
||||
kubernetes, wttrin) to load regardless of the station suite toggle.
|
||||
* DONE Rename azos-core/features to azos-core/suites; enforce suite grouping
|
||||
|
||||
Grepped exwm's =config.org= for use of station-defined symbols
|
||||
(=azos/mail=, =mu4e=, =elfeed=, =wttrin=, =kubernetes-=, =azos/pass=) and
|
||||
found none — the require looks vestigial, not load-bearing.
|
||||
Done 2026-10-13. Renamed =azos-core/features/= -> =azos-core/suites/=,
|
||||
updated both =(inputs.import-tree ./suites)= references in
|
||||
=azos-core/flake.nix=. Folded =claude-memory= and =claude-skills= into
|
||||
=azos-core/suites/base/= (both now register
|
||||
=config.flake.modules.homeManager.base=, same as git/gpg-agent/python) —
|
||||
dropped their now-redundant standalone =suiteModules.homeManager.*=
|
||||
entries from =home-manager/home.nix=. Updated AGENTS.md throughout
|
||||
(directory tree, "where to put a feature", Claude integration table, key
|
||||
files table) plus two stray in-repo path comments. Validated with
|
||||
=nix eval ...nixosConfigurations.lauretta.config.system.build.toplevel=
|
||||
— full derivation path, no errors. Convention going forward: no bare
|
||||
top-level file directly under =suites/=, every azos-core feature belongs
|
||||
inside a suite directory.
|
||||
|
||||
Fix: remove the =require= line and the =packageRequires= entry.
|
||||
* DONE Decouple the exwm/station coupling — ended up as a station rename, not a split
|
||||
|
||||
Softer coupling to also resolve: exwm's =config.org= has quick-launch /
|
||||
window-swallow logic for chromium, qutebrowser, and gimp
|
||||
(=azos/exwm/chromium-p=, =azos/exwm/start-chromium=,
|
||||
=azos/exwm/start-qutebrowser=, the gimp instance-name check), but those
|
||||
*binaries* are installed by station's =browser.nix=/=gimp.nix=, not by
|
||||
exwm. Decide whether to pull those packages into exwm directly or accept
|
||||
them as dead keybindings when station is off.
|
||||
Done 2026-10-13. Re-discussed scope with the user: station's apps
|
||||
(browser, gimp, libreoffice, spotify, whatsapp, zoom) turned out to be
|
||||
generically useful on *any* GUI machine, work included — so instead of
|
||||
splitting station into desktop-plumbing + apps + a separate
|
||||
personal-apps suite, =station= was simply **renamed to =desktop=**, kept
|
||||
intact. The one genuinely personal-only app, =discord=, was dropped
|
||||
entirely from azos-core and now lives as its own standalone
|
||||
=azos/features/discord= (own =azos.discord.enable=, default true) — the
|
||||
reasoning being any machine that's actually "personal" already uses the
|
||||
=azos= project rather than a different downstream repo, so there's no
|
||||
need to carry personal-only apps inside the shared azos-core suite at
|
||||
all. =steam= needed no change — it was already sitting correctly as a
|
||||
standalone =azos/features/steam= feature, never part of station.
|
||||
|
||||
Motivation: new Fedora machine will enable the exwm suite but not station.
|
||||
Full architecture notes already saved to [[id:c52036a9-25ec-4b8b-8b72-90fcd9ea65dd][azos/exwm]].
|
||||
Also fixed the vestigial Emacs-level coupling found earlier (full
|
||||
investigation notes in
|
||||
[[id:c52036a9-25ec-4b8b-8b72-90fcd9ea65dd][azos/exwm]]): removed
|
||||
=pkgs.azos-emacs-station= from exwm's =_pkg.nix= packageRequires and the
|
||||
=(require 'azos-emacs-station)= line from exwm's config.org — grepped and
|
||||
confirmed exwm's own config never used any station symbols
|
||||
(mu4e/elfeed/wttrin/kubernetes-/pass). exwm still defaults to depending
|
||||
on the renamed =desktop= suite itself (=azos.suites.desktop.enable =
|
||||
lib.mkDefault true=, unchanged in kind from the old station dependency)
|
||||
— renamed, not removed, since the chromium/qutebrowser/gimp quick-launch
|
||||
keybindings in exwm's config.org need those binaries from desktop's
|
||||
browser.nix/gimp.nix and desktop is wanted on the work machine anyway.
|
||||
Also renamed the paired elisp package =azos-emacs-station= ->
|
||||
=azos-emacs-desktop= everywhere (desktop suite itself, and the real
|
||||
consumer at =azos/features/lauretta/emacs=, which legitimately wants
|
||||
mu4e/elfeed/kubernetes/wttrin). Dropped a duplicate bare =reaper= package
|
||||
from desktop's plumbing list — =azos/features/reaper= already provides it
|
||||
with plugins, so desktop's copy was a vestigial duplicate.
|
||||
|
||||
* TODO Audit non-azos-core features for portability to azos-core
|
||||
Per-app enable toggles (mirroring =browser.nix='s pattern) turned out
|
||||
unnecessary for gimp/libreoffice/spotify/whatsapp/zoom — they're staying
|
||||
in desktop unconditionally now since the user wants them everywhere, work
|
||||
machine included. Validated the whole thing with targeted =nix eval=
|
||||
checks against lauretta (confirmed =azos.suites.desktop.enable=,
|
||||
=azos.discord.enable=, and =discord= present in the resolved
|
||||
=home.packages= list, plus a full successful toplevel build) — no
|
||||
regressions.
|
||||
|
||||
Go over everything under =/home/aner/azos/features/= (attic, audio,
|
||||
binfmt, bluetooth, chess, claude, encryption, git-config, hardware-auth,
|
||||
headphones, hfsprogs, homey-alert, hyprland, kubernetes, lauretta,
|
||||
leetcode, mail, mpris-proxy, nextcloud, opencode, printing, qutebrowser,
|
||||
reaper, roam-backup, snx-rs, steam, virtualization, ytdl) and assess which
|
||||
are generic/reusable enough to port into =azos-core/features/= for use by
|
||||
the new Fedora machine project, versus which are genuinely
|
||||
lauretta/NixOS-machine-specific and should stay put.
|
||||
* DONE Audit azos/features/* for portability into existing azos-core suites
|
||||
|
||||
Standing principle established by the discord move above: a feature only
|
||||
belongs in azos-core if it's something you'd want on *any* machine
|
||||
including a work one. Personal-taste/consumer apps stay in azos/features
|
||||
even if they're technically generic/reusable code-wise — e.g. =steam= and
|
||||
=reaper= (audio production) are left out of azos-core for this reason,
|
||||
not because they're hard to move.
|
||||
|
||||
DONE so far (2026-10-13), moved into azos-core with behavior preserved
|
||||
(own toggle kept, no new suite-enable AND-gate added, validated via
|
||||
targeted =nix eval= against lauretta — all resolved identically to
|
||||
before the move):
|
||||
- =audio=, =binfmt=, =bluetooth= -> =suites/desktop/= (home-manager +
|
||||
new =nixos.desktop= module)
|
||||
- =headphones= -> =suites/exwm/= (not desktop — user's call)
|
||||
- =chess= -> =suites/editor/chess/= (brought its =emacs/= subdir along
|
||||
too — =_pkg.nix=/=config.org=/=default.nix= defining the
|
||||
=azos-emacs-chess= package; almost missed this on the first pass since
|
||||
it wasn't visible in a maxdepth-1 =default.nix= read, only in the
|
||||
directory listing)
|
||||
- =kubernetes= -> =suites/dev/kubernetes/= (home-manager)
|
||||
- =virtualization= -> =suites/dev/virtualization/=, the **first**
|
||||
=nixos.dev= module (dev suite previously had no NixOS side)
|
||||
- =hfsprogs= -> =suites/desktop/hfsprogs/=, as a *NixOS* module
|
||||
(=environment.systemPackages=, not =home.packages= — deliberately
|
||||
system-wide now, per request, unlike exfatprogs/e2fsprogs which stayed
|
||||
home-manager-side in desktop's plumbing)
|
||||
|
||||
Decided to stay in azos, NOT moved (personal-taste/unused, even though
|
||||
technically generic — see standing principle above): =leetcode=,
|
||||
=qutebrowser= (currently broken/unused anyway), =ytdl=, =opencode=
|
||||
(unused currently), =mpris-proxy= (narrowly tied to one university VPN
|
||||
connection), =nextcloud= (keep personal, not genericizing).
|
||||
|
||||
DONE 2026-10-13, genericization batch — all four moved into
|
||||
=suites/base/=, personal/secret values turned into mandatory options (no
|
||||
=default=, so enabling the feature without supplying them is a hard eval
|
||||
error by design, per explicit instruction: no real default values ever,
|
||||
ever since azos-core shouldn't ship anyone's personal data even as a
|
||||
fallback). Validated end-to-end via targeted =nix eval= against lauretta
|
||||
— every resolved value matches what was hardcoded before:
|
||||
- =git-config= -> =suites/base/git-config/=: =userName=/=userEmail=
|
||||
mandatory; signing and the pass-git-helper credential helper
|
||||
(=passCredentialHelper=) each independently toggleable, mapping content
|
||||
also now an option instead of a committed =.ini= file.
|
||||
- =encryption= -> =suites/base/encryption/=: split into independently
|
||||
toggleable pieces — base gpg-agent setup (always on when
|
||||
=encryption.enable=), =passwordStore= (pass + GPG key id),
|
||||
=yubikey.enable=, =browserpass.enable=, =gpgSsh= (expose a GPG subkey
|
||||
as an SSH key — keygrip/pubkey both mandatory options now, previously
|
||||
committed files). Reason for all the independent toggles: unsure a work
|
||||
machine will have GPG keys at all, let alone a YubiKey.
|
||||
- =homey-alert= -> renamed to =azos-send-alert= (the command — "homey"
|
||||
was personal selfhosted-infra naming), landing in
|
||||
=suites/base/azos-send-alert/=, next to claude-memory/claude-skills.
|
||||
The remote-notification half (previously a hardcoded ntfy curl call)
|
||||
is now =azos.send-alert.remoteCommand=, an arbitrary shell command run
|
||||
with TITLE/PRIORITY/TAGS/MESSAGE in its env; local notify-send fallback
|
||||
unchanged. =--print-remote= still works: sets
|
||||
=AZOS_SEND_ALERT_PRINT_REMOTE=1= and captures the command's stdout, by
|
||||
convention the command resolves local secrets and prints a
|
||||
self-contained invocation. =azos/features/homey-alert= now just hosts
|
||||
the concrete ntfy-backed =azos-homey-ntfy-alert= overlay package,
|
||||
plugged in via =azos.send-alert.remoteCommand= in home.nix.
|
||||
- =printing= -> =suites/base/printing/= (nixos side): hardcoded "aner"
|
||||
username is now =azos.printing.username= (mandatory).
|
||||
|
||||
Decided NOT to genericize, stay in azos as personal: =snx-rs= (BGU VPN
|
||||
client — might drop it soon anyway), =hardware-auth= (any machine this
|
||||
gets used on is personal anyway, per user).
|
||||
|
||||
Stays in azos, not generalized: attic (also flagged separately: it
|
||||
embeds a plaintext JWT token directly in a git-tracked nix file — worth
|
||||
a dedicated look regardless of this migration), lauretta (machine-specific
|
||||
by definition), roam-backup (trivial personal workflow, low value to
|
||||
move), steam, reaper (personal-taste apps, see standing principle above),
|
||||
nextcloud (keep personal).
|
||||
|
||||
Noticed mid-audit but unrelated to this task, left untouched: a HORI
|
||||
fight-stick udev rule uncommitted in =features/steam/default.nix= — the
|
||||
user is actively editing it elsewhere, hands off.
|
||||
|
||||
hyprland suite move explicitly deferred — not used currently, a possible
|
||||
future project in a few months, not part of this effort.
|
||||
|
||||
* DONE Make mail accounts a configurable attrset option
|
||||
|
||||
Done 2026-10-13 — turned out to be less work than expected. Discussed
|
||||
first, without touching anything: =accounts.email.accounts= is already
|
||||
home-manager's own generic attrset-of-accounts option, so there was no
|
||||
need to invent a new =azos.mail.accounts= wrapper around it — azos-core
|
||||
just had to stop pre-populating that option with real addresses.
|
||||
|
||||
- =suites/desktop/mail/= (destination: desktop, not base — user's
|
||||
call) now owns only the mechanical glue: mu/mbsync/msmtp toggles, the
|
||||
=mu init --my-address=...= list *derived* from whatever
|
||||
=accounts.email.accounts= actually contains (via
|
||||
=lib.mapAttrsToList=) instead of hardcoded, and =mbsyncSafe= (new-mail
|
||||
count diff + notify-send, already fully generic).
|
||||
- The hardcoded =azos-card-present= smartcard gate inside =mbsyncSafe=
|
||||
became =azos.mail.presenceCheckCommand= -- an optional pluggable shell
|
||||
command (null by default = always sync unconditionally), since a
|
||||
given machine might not back mail passwords with a GPG smartcard at
|
||||
all. =azos/features/mail= wires it to
|
||||
=${pkgs.azos-card-present}/bin/azos-card-present= but only
|
||||
conditionally, when =azos.encryption.yubikey.enable= is true --
|
||||
directly addresses the concern that whether this gate makes sense at
|
||||
all depends on whether a YubiKey/smartcard is even in the picture.
|
||||
- =azos/features/mail/default.nix= kept the =default_account_params=/
|
||||
=default_gmail_params=/=default_smtp= convenience merge-helpers and
|
||||
the three real accounts (personal boilerplate, not mechanism) and
|
||||
just sets =accounts.email.accounts= + =azos.mail.enable= + the
|
||||
presence-check wiring directly.
|
||||
- Validated via targeted =nix eval= against lauretta: all three
|
||||
addresses present, =mu init= has the derived =--my-address= flags,
|
||||
presence check resolves to the real =azos-card-present= binary path.
|
||||
|
||||
* DONE (stretch) Make claude-memory/claude-skills backend-pluggable
|
||||
|
||||
Done 2026-10-13 — went further than "LLM-agnostic" framing suggested:
|
||||
the real axis is the *agent harness* (Claude Code CLI vs. agent-shell
|
||||
vs. others), not the model.
|
||||
|
||||
** Findings (grounded by cloning and reading actual source, not docs)
|
||||
|
||||
- =agent-shell= (xenodium, Emacs ACP client) supports ~19 backends
|
||||
(Claude, Gemini CLI, Codex, Goose, OpenCode, etc.) via per-backend
|
||||
"agent configs". =agent-shell-mcp-servers= (plain Elisp defcustom) is
|
||||
passed to *every* session's ACP =session/new= regardless of backend —
|
||||
genuinely the "memory regardless of model" mechanism.
|
||||
- =agent-shell-subscribe-to= exposes a =file-write= event (=:path=,
|
||||
=:content=) — a cleaner, backend-agnostic equivalent to Claude Code's
|
||||
=PostToolUse= hook for the org-roam-resync-on-edit behavior.
|
||||
- Initially proposed isolating Claude-via-agent-shell with
|
||||
=CLAUDE_CONFIG_DIR= pointed at an empty dir to avoid double-firing —
|
||||
user correctly pushed back: slash commands
|
||||
(=~/.claude/commands/*.md=) already work today inside agent-shell
|
||||
sessions, and isolating the whole config dir would have broken that.
|
||||
- Cloned =claude-agent-acp= (zed-industries, the ACP shim
|
||||
=azos/features/claude= installs) and confirmed directly in source:
|
||||
it is built on the real Claude Agent SDK and merges
|
||||
=~/.claude/settings.json='s hooks into every session it drives
|
||||
(=resolveSettings=, same merge engine the CLI itself uses; has a
|
||||
dedicated =session-hooks.test.ts= proving this is intentional) — so
|
||||
the double-fire risk for the *two memory-specific hooks* was real.
|
||||
But it **never reads =~/.claude.json=\= at all** (zero references in
|
||||
source) — so the MCP-server registration was never actually at risk
|
||||
of colliding; nothing needed fixing there.
|
||||
|
||||
** What shipped
|
||||
|
||||
- =azos.claude-memory.enable=/=azos.claude-skills.enable= renamed to
|
||||
=azos.agent-memory=/=azos.agent-skills=, both reshaped as
|
||||
=backends.<name>.enable= attrsets instead of a single flag.
|
||||
- =agent-skills= has one backend (=claude=) — no cross-backend
|
||||
mechanism exists for slash-commands/CLAUDE.md, so nothing to add
|
||||
alongside it yet.
|
||||
- =agent-memory= gets a second backend, =agent-shell=: drops
|
||||
=~/.emacs.d/extra/azos-agent-memory.el= (auto-loaded via the existing
|
||||
=load-dir= setup in =suites/base/emacs=, not a compiled
|
||||
=azos-emacs-*= package, since it needs real Nix-interpolated paths)
|
||||
registering =org-roam-mcp= in =agent-shell-mcp-servers= and
|
||||
subscribing to =file-write= for resync.
|
||||
- Double-fire fix: the =agent-shell= backend appends
|
||||
=AZOS_AGENT_SHELL_SESSION=1= to
|
||||
=agent-shell-anthropic-claude-environment=; the =claude= backend's
|
||||
two memory-specific hooks (=PostToolUse= resync, =SessionStart=
|
||||
context injection) check for that var and no-op if present. Nothing
|
||||
else needed touching (permissions, the unrelated =azos-send-alert=
|
||||
=PreToolUse= hook) since those have no agent-shell-side equivalent to
|
||||
conflict with.
|
||||
- Both backends bundled where their tool gets installed, not separately
|
||||
remembered: =azos.claude.enable= (=azos/features/claude=) turns on
|
||||
the claude backend for both memory and skills; installing
|
||||
=agent-shell= (=suites/editor/emacs=, via =azos-emacs-editor=) turns
|
||||
on the agent-shell memory backend.
|
||||
- Validated via =nix eval= against lauretta: both backends resolve
|
||||
=true=, generated =.el= file has real paths substituted correctly,
|
||||
generated Claude hook scripts have the guard clause in the right
|
||||
place.
|
||||
|
||||
** Not verified — needs real interactive testing, not just nix eval
|
||||
|
||||
The Emacs Lisp itself (=agent-shell-mode-hook=, =agent-shell-subscribe-to=,
|
||||
=map-elt=/=string-prefix-p= usage, whether =AZOS_AGENT_SHELL_SESSION=
|
||||
actually propagates from agent-shell's subprocess env down into the
|
||||
hook-command's own env) is a best-effort first pass written from reading
|
||||
=agent-shell='s source, not from running it. =nix eval= can prove the
|
||||
Nix side is wired correctly (confirmed) but can't prove the Elisp
|
||||
actually behaves as designed once Emacs loads it. Needs a real rebuild +
|
||||
interactive check: open an agent-shell session (any backend) and an
|
||||
agent-shell Claude session specifically, confirm org-roam tools show up
|
||||
in both, confirm editing a roam file resyncs without double-firing.
|
||||
|
||||
* DONE Live-test today's azos-core changes after rebuild
|
||||
|
||||
Everything below was validated with =nix eval= against lauretta (option
|
||||
values resolve, derivations build, no eval errors) but =nix eval= can't
|
||||
prove runtime behavior -- actual service startup, Emacs loading real
|
||||
Elisp, hooks firing at the right time. Rebuild first
|
||||
(=azos-lauretta-update=, run by the user -- needs sudo), then work
|
||||
through this top to bottom. Mark each TODO/DONE as it's checked; where
|
||||
something fails, reopen the matching azostodos item that produced it
|
||||
rather than re-diagnosing from scratch (each one is linked below).
|
||||
|
||||
** DONE agent-shell / agent-memory backends (highest risk, least testable from Nix alone)
|
||||
|
||||
Live-tested 2026-10-13 against a *real* agent-shell Claude session: this
|
||||
very Claude Code conversation turned out to be running *through*
|
||||
agent-shell the whole time (confirmed via env — CLAUDECODE=1,
|
||||
CLAUDE_AGENT_SDK_VERSION set, CLAUDE_CODE_ENTRYPOINT=sdk-ts), which made
|
||||
this an unusually direct test rather than a separate exercise.
|
||||
|
||||
Found and fixed a real bug in the process (see the =tool-call-update=
|
||||
fix commit): subscribed to *every* agent-shell event on this session's
|
||||
buffer and made a real Write-tool edit. =file-write= never fired —
|
||||
Claude's native Write/Edit tools write directly via the SDK, never
|
||||
through agent-shell's client-side filesystem round-trip, which is the
|
||||
only thing that event reports on. =tool-call-update= fired repeatedly
|
||||
instead (pending -> pending-with-diff -> completed) and *is*
|
||||
backend-agnostic by construction (every ACP agent must report tool
|
||||
calls for the client's diff/permission UI). Switched the subscription to
|
||||
that, filtered to completed edit calls, verified live it fires exactly
|
||||
once for a real write under =~/roam=.
|
||||
|
||||
Confirmed, with evidence:
|
||||
- =agent-shell-mcp-servers= populated correctly (real =/nix/store/...=
|
||||
paths, correct =ORG_ROAM_DIR=/=ORG_ROAM_DB_PATH=).
|
||||
- =AZOS_AGENT_SHELL_SESSION=1= genuinely present in this session's own
|
||||
process environment — not just configured in Emacs, but actually
|
||||
propagated through the full chain (agent-shell -> claude-agent-acp
|
||||
subprocess -> SDK -> this session -> its own Bash tool calls).
|
||||
- org-roam MCP tools (=search_nodes= etc.) work in this session — and
|
||||
since =claude-agent-acp= never reads =~/.claude.json= (confirmed
|
||||
earlier via source), this is proof the =agent-shell-mcp-servers= path
|
||||
is what's actually serving them, not the Claude-specific registration.
|
||||
- The corrected =tool-call-update= handler fires exactly once per
|
||||
completed edit under =~/roam= (verified via live-loaded test handler
|
||||
before writing the permanent fix).
|
||||
- No =*Warnings*= buffer, clean =load-dir= pickup of
|
||||
=~/.emacs.d/extra/azos-agent-memory.el=.
|
||||
|
||||
Not independently tested, noted as residual gaps rather than assumed:
|
||||
- Session-start context injection specifically for a *non-Claude*
|
||||
agent-shell session (no non-Claude equivalent exists yet, so nothing
|
||||
to test there one way or the other).
|
||||
- A second ACP backend end-to-end: =opencode= is actually installed
|
||||
(=which opencode= resolves, =agent-shell-opencode= feature loads) and
|
||||
would be the natural candidate, but starting a second interactive
|
||||
agent-shell session non-interactively via =emacsclient= wasn't
|
||||
attempted — diminishing returns given =agent-shell-mcp-servers= is
|
||||
confirmed generic by construction (same function serves every backend,
|
||||
verified in source) and already proven live for one backend.
|
||||
- Plain =claude= standalone in a terminal (outside agent-shell) wasn't
|
||||
separately re-run live in this pass, since this whole session *was*
|
||||
the agent-shell path. Lower risk anyway: =~/.claude.json= mcpServers
|
||||
and =~/.claude/settings.json= hooks (checked earlier, both present
|
||||
and correct) are read identically regardless of agent-shell.
|
||||
- Run plain =claude= standalone in a terminal (NOT via agent-shell).
|
||||
Confirm: slash commands still work (=/todo=, =/project-brain=,
|
||||
=/workplate=, =/work-history=, =/notify=), org-roam MCP tools are
|
||||
available, session-start context injection still happens -- all
|
||||
should be completely unaffected by any of today's changes.
|
||||
|
||||
Source: [[id:c52036a9-25ec-4b8b-8b72-90fcd9ea65dd][azos/exwm]] has the
|
||||
EXWM-specific keybinding/architecture notes if agent-shell's own
|
||||
keybindings seem off after the suite rename.
|
||||
|
||||
** DONE Notification system (azos-send-alert, renamed from homey-alert)
|
||||
|
||||
Confirmed 2026-10-13 live: =which azos-send-alert= resolves,
|
||||
=which homey-alert= doesn't; =azos-bash-notify-hook='s script content
|
||||
calls =azos-send-alert= (not =homey-alert=); =azos-send-alert -t "Test"
|
||||
"..."= produced both a local desktop notification and a remote ntfy
|
||||
push (first attempt failed on a PIN entry mistake, not a regression --
|
||||
retry worked clean); =-r/--print-remote= printed a well-formed,
|
||||
already-token-resolved =curl= command. Long-command-threshold live fire
|
||||
not separately tested but the underlying mechanism is confirmed working
|
||||
end to end.
|
||||
|
||||
- =which azos-send-alert= should resolve; =which homey-alert= should
|
||||
**not** (confirms the rename is clean, nothing still refers to the
|
||||
old binary name).
|
||||
- =azos-send-alert -t "Test" "rebuild test"= -- should produce both a
|
||||
local desktop notification (notify-send) and a remote ntfy push
|
||||
(check the ntfy app/web client for ~alerts~).
|
||||
- =azos-send-alert -r -t "Test" "remote test"= -- should print a
|
||||
well-formed, already-token-resolved =curl= command (not an error, not
|
||||
an unresolved =$(pass ...)= call).
|
||||
- Check =~/.claude/settings.json='s =PreToolUse= hook (the long-command
|
||||
notifier) references =azos-send-alert=, not =homey-alert=, in its
|
||||
wrapped command text.
|
||||
- Let a Claude Code Bash command run past the threshold (default 300s,
|
||||
=azos.send-alert.longCommandThresholdSeconds=) and confirm the
|
||||
long-command notification still fires correctly with the new name.
|
||||
|
||||
** DONE Broad regression sweep of everything else moved/genericized today
|
||||
|
||||
Live-tested 2026-10-13, directly on lauretta post-rebuild. Two of the
|
||||
test plan's own binary-name guesses were wrong (not real gaps, just my
|
||||
mistake writing the plan) -- corrected below for next time.
|
||||
|
||||
- *desktop suite (renamed from station)*: =which chromium qutebrowser
|
||||
gimp libreoffice spotify zapzap zoom-us flameshot discord= -- all
|
||||
present (package is =gimp3=, binary is =gimp=, not =gimp3= -- the
|
||||
test plan had the wrong binary name). Discord confirmed present via
|
||||
its new standalone =azos/features/discord=, not the desktop suite.
|
||||
- *audio/binfmt/bluetooth -> desktop (new =nixos.desktop= module)*:
|
||||
=bluetooth.service= active; pipewire is a *user* service
|
||||
(=systemctl --user status pipewire/pipewire.socket=, both active) --
|
||||
the system-level unit shows masked/inactive by design, not a
|
||||
regression.
|
||||
- *headphones -> exwm suite*: =azos-connect-headphones --list= shows
|
||||
all 3 devices correctly.
|
||||
- *chess -> editor suite*: =(featurep 'azos-emacs-chess)= -> t.
|
||||
- *kubernetes/virtualization -> dev suite*: =libvirtd.service= active,
|
||||
=virt-manager= present, =kubectl= correctly absent (default-false,
|
||||
unchanged by the move).
|
||||
- *hfsprogs -> desktop, NixOS-level/system-wide*: =mkfs.hfsplus=/
|
||||
=fsck.hfsplus= resolve from =/run/current-system/sw/bin= -- system-wide
|
||||
as intended, not just the user profile.
|
||||
- *git-config genericized*: =git config user.name/user.email/
|
||||
user.signingkey/credential.helper= all show the real values.
|
||||
Stronger proof than planned: two real commits made *during this very
|
||||
testing session* (the =tool-call-update= fix, both azos-core and
|
||||
outer repo) were actually GPG-signed successfully end to end --
|
||||
pinentry-emacs needed a restart + the user re-entering their PIN once
|
||||
(unrelated pre-existing hiccup, not caused by today's changes), but
|
||||
signing itself worked correctly with the genericized config.
|
||||
- *encryption genericized*: =~/.ssh/gpg-as-ssh.pub= and
|
||||
=~/.gnupg/sshcontrol= both show the real configured values;
|
||||
=yubikey-personalization='s actual binaries (=ykinfo=/=ykpersonalize=
|
||||
-- not =ykman=, that's a different package, =yubikey-manager= --
|
||||
another test-plan naming mistake) resolve correctly.
|
||||
- *printing genericized*: =aner= correctly in the =lp= group; =cups= and
|
||||
=avahi-daemon= both active.
|
||||
- *mail genericized*: =mbsync-timer.service= shows a real recent
|
||||
successful run (=status=0/SUCCESS=) -- the presence-check-gated sync
|
||||
(tied to =azos.encryption.yubikey.enable=) is working end to end on a
|
||||
live timer fire, not just evaluating correctly.
|
||||
|
||||
Not independently exercised: browserpass actually autofilling in a live
|
||||
chromium session (declarative config confirmed correct, GUI behavior
|
||||
not opened and checked); a real print job (no printer currently
|
||||
reachable to test against).
|
||||
- *encryption genericized*: =pass <some-entry>= decrypts fine via
|
||||
pinentry-emacs; =cat ~/.ssh/gpg-as-ssh.pub ~/.gnupg/sshcontrol= show
|
||||
the real key; browserpass still autofills in chromium; =which
|
||||
ykman= (or whatever yubikey-personalization provides) resolves.
|
||||
- *printing genericized*: =groups aner | grep lp=; =systemctl status
|
||||
cups avahi-daemon=; print something if a printer's actually reachable.
|
||||
- *mail genericized*: =mu info= lists all three addresses; =systemctl
|
||||
--user status mbsync-timer= is active; pull the YubiKey and confirm
|
||||
=journalctl --user -u mbsync-timer= shows a silent skip (presence
|
||||
check now tied to =azos.encryption.yubikey.enable= rather than
|
||||
unconditional); replug and confirm sync resumes.
|
||||
|
||||
* TODO DAV calendar sync + workplate reading today's calendar
|
||||
|
||||
Extend the DAV setup ([[id:1196aa79-76ba-4090-a896-6e4938338bd1][azos/dav]]) to sync calendars (events) too — same vdirsyncer account machinery, khal and/or =ical2orgpy= for reading and the org view — and teach the =workplate= skill to take today's schedule into account when planning.
|
||||
|
||||
@@ -29,5 +29,11 @@ Not yet covered: IR/forensics (queued), general frameworks/compliance beyond NER
|
||||
** Protocol Deep-Dive Session (2026-10-06)
|
||||
Completed the priority-redirect protocol session: SNMP (new — manager/agent, MIB/OID, GET/SET/TRAP, v1/v2c plaintext community strings vs. SNMPv3); Modbus (deeper — function codes, four data tables, Modbus Security/TLS retrofit, FrostyGoop case study); DNP3 (deeper — object-based types, unsolicited reporting, DNP3-SA HMAC challenge-response mechanics, Aggressive Mode replay flaw per Cremers et al.); GOOSE (deeper — pub-sub over raw Ethernet L2/multicast MAC, stNum/sqNum and how Industroyer forged them, SCD engineering files, IEC 62351-6, PRP/HSR, process-bus/station-bus segmentation risk); IEC 104 (deeper — ASDU/Information Object typing, port 2404, regional dominance vs. DNP3, IEC 62351-3/5); OPC-UA (deeper — object-oriented address space, X.509/Security Policies, horizontal cross-vendor-interop origin vs. vertical IT/cloud bridging vs. emerging field-level/TSN use). New terms (compensating controls, brownfield/greenfield, MES, etc.) and all new reading resources logged to [[ot-ics-security-ramp/glossary]] and [[ot-ics-security-ramp/reading-list]] as covered, per standing convention. Produced [[ot-ics-security-ramp/protocol-comparison]] as a standalone side-by-side reference across transport, data model, security posture, Purdue level, and known attacks.
|
||||
|
||||
** OPC UA Protocol Mechanics Session (2026-10-12)
|
||||
Condensed pass through the Claroty Team82 10-part OPC UA attack-surface deep dive (see [[ot-ics-security-ramp/reading-list]]), followed by conversational deep-dive on protocol mechanics: full session lifecycle (HEL/ACK → OPN/SecureChannel asymmetric bootstrap → CreateSession/ActivateSession identity layer → Read/Write/Browse/Subscribe → CLO teardown); SecureChannel-vs-Session decoupling and why it matters (renewal without losing session state); GetEndpoints as unauthenticated-by-design discovery and why that's not a secrecy boundary; UserName/Password token encryption mechanics and exactly how/why it degrades to plaintext in practice; Subscription/MonitoredItem pub-sub-style efficiency layer vs. the separate OPC UA PubSub transport profile; Alarms & Conditions model (ConditionType state machine, EventMonitoredItem, ConditionRefresh) as the realistic shape of something like a HighLevelAlarm; Node/AddressSpace meta-model (8 NodeClasses, References) as mandatory grammar vs. vendor-proprietary or Companion-Spec vocabulary, with a worked Pump/Tank example; same Pump/Tank example re-modeled in Modbus/Modbus-TCP to contrast self-describing address space against flat typeless register tables + external register-map documentation. Also covered vulnerability-class vocabulary from the reading (chunk flooding, zip-slip, WerFault abuse, Pwn2Own ICS) and clarified OPC UA client/server roles don't map 1:1 to Purdue levels — it's a chained relay (PLC→SCADA→MES→ERP), each hop its own independent client-server pair. New terms and the reading-list entry status logged to [[ot-ics-security-ramp/glossary]] and [[ot-ics-security-ramp/reading-list]] per standing convention; also added a missing ERP glossary entry.
|
||||
|
||||
** IR/Forensics Deep-Dive Session (2026-10-12)
|
||||
First full pass through IR/forensics, structured as 7 sections, conversational deep-dive with discussion between each: (1) IR lifecycle — NIST SP 800-61 vs. SANS PICERL; (2) Detection & Analysis — SIEM/EDR/IDS-IPS, triage, scoping, and the OT wrinkle that EDR-style endpoint agents can't run on PLCs/RTUs, pushing OT detection toward passive network monitoring; (3) Containment — short/long-term, isolate-vs-monitor-in-place, DNS sinkholing, and the OT-specific constraint that touching a live process needs joint sign-off with plant operations/safety (TRITON/TRISIS as the case where the SIS itself was the target); (4) Digital forensics fundamentals — chain of custody, forensic imaging/write-blockers, RFC 3227 order of volatility, live vs. dead-box; (5) Memory & disk forensics — acquisition mechanics (WinPmem/LiME/avml, hypervisor-level snapshotting, why modern hardening like HVCI/VBS and Lockdown LSM blocks live acquisition and why real IR relies on pre-positioned signed tooling), Volatility, timeline/MACB analysis, file carving, slack space, persistence artifacts — plus a hands-on exercise reading /proc/kcore on the live machine and decoding its ELF64 header (e_type=ET_CORE, e_phoff/e_phnum pointing to the real PT_LOAD segments inside the sparse ~128TiB address space); (6) Network forensics — PCAP vs. NetFlow/IPFIX tradeoff, Zeek as the practical middle ground, beaconing/DGA/DNS-tunneling, JA3/JA3S + SNI as the answer to TLS opacity, and why OT network forensics is actually richer than IT's (unencrypted ICS protocols give full semantic visibility in plaintext); (7) OT-specific forensics gaps — PLCs as forensic dead ends (no audit log) vs. the engineering workstation as where real evidence lives, control-logic verification via vendor Upload functions (S7comm/CIP) as the realistic way to extract a PLC's running program, the Stuxnet PLC-rootkit caveat that defeats naive upload-and-compare, Historian data as the physical-process timeline substitute, SCD file integrity as a distinct attack target, and anti-forensics in OT (corrected mid-session: the Dec-2025 Poland CHP partition-table wipe was remote/digital, not physical — physical anti-forensics is a separate, real but distinct OT consideration). All terms logged to [[ot-ics-security-ramp/glossary]] under a new "IR / Forensics" section; also added a missing standalone-callout note for RTU (already defined, bundled with PLC) and a missing ERP entry.
|
||||
|
||||
** Status
|
||||
Protocol deep-dive session complete (2026-10-06). Next up per original queue: IR/forensics, then frameworks/compliance (NIST CSF, ISO 27001, SOC 2, CIS Controls, STIX/TAXII), remaining defensive-stack acronyms, cloud security beyond k8s.
|
||||
IR/forensics first pass complete (2026-10-12). Next up per original queue: frameworks/compliance (NIST CSF, ISO 27001, SOC 2, CIS Controls, STIX/TAXII), remaining defensive-stack acronyms, cloud security beyond k8s.
|
||||
|
||||
@@ -74,7 +74,16 @@ Running, quiz-ready glossary built up during Aner's cybersecurity terminology ra
|
||||
- Process bus / station bus — the dedicated, VLAN-segmented (sometimes physically separate) Ethernet network a substation runs GOOSE/Sampled Values on, kept apart from general SCADA/IT traffic for two reasons: (1) latency contention — ordinary traffic queuing ahead of a time-critical trip message, (2) security — Ethernet doesn't distinguish IP packets from GOOSE frames on the wire, so any compromised host sharing that L2 segment can craft and inject raw GOOSE frames directly (exactly what Industroyer did), with no "GOOSE-specific access" required. Segmentation here is the control meant to keep an IP-side compromise from ever reaching that segment — in budget-constrained or poorly-designed substations this isolation is sometimes incomplete, a real rather than theoretical risk.
|
||||
- IEC 60870-5-104 ("IEC 104") — DNP3's rough equivalent dominant in Europe/Middle East/Asia/Africa (DNP3 dominates North America); if you see TCP port 2404 in a capture, it's almost certainly non-US utility telemetry. The IP-networked version of the older serial IEC 60870-5-101 (104-to-101 is roughly what Modbus-TCP is to Modbus-serial). Master-slave (controlling/controlled station), not pub-sub. Data carried as ASDUs (Application Service Data Units) containing typed Information Objects (Type Identification field marks e.g. single-point info, measured value, command) — gives semantic typing Modbus's flat registers lack, similar in spirit to DNP3's object model. No authentication/encryption/integrity in the base spec, same as Modbus/DNP3-without-SA; IEC 62351-3/5 defines a TLS security retrofit, same low-adoption brownfield story as everything else.
|
||||
- OPC-UA — platform-independent successor to the original OPC (built on Windows-only COM/DCOM, a firewall nightmare); own binary protocol on TCP port 4840 or an HTTPS binding. Unlike field protocols' flat/typed-but-static registers, exposes a genuine object-oriented, self-describing address space (Nodes: variables, methods, types, relationships) a client can browse/discover at runtime. Real built-in security, not retrofitted: X.509 certs for mutual client/server auth, configurable Security Policies (None/Sign/SignAndEncrypt) with a defined cipher-suite-like profile stack (Basic256Sha256 etc.) — though "Security Policy: None" is a valid, sometimes-used setting that defeats all of it. Primary/original use case is horizontal: cross-vendor interoperability between different Level 2/3 systems (SCADA-to-historian, historian-to-MES, HMI-to-SCADA) — solving the "every vendor has a proprietary driver" problem OPC was founded to fix. Increasingly also vertical: Level 2/3 (historian/SCADA) up to Level 4/5 IT/cloud/analytics, crossing the DMZ. At the leading edge, pushed down to the field level too — modern PLCs (Siemens S7-1500, Beckhoff, Rockwell) often ship a native OPC-UA server directly on the controller, and OPC-UA over TSN (Time-Sensitive Networking) is an emerging effort toward real-time field-level use, a long-horizon candidate to someday displace protocols like Modbus or GOOSE. Doesn't retroactively secure the field-level protocols underneath it — it secures the bridge/interop layer it operates at, nothing lower.
|
||||
- OPC UA message types (HEL/ACK/OPN/MSG/CLO) — the four transport-level frames every OPC UA conversation is built from: client sends HEL (Hello, endpoint URL + buffer limits), server replies ACK; OPN (OpenSecureChannel) bootstraps the encrypted channel via asymmetric cert-based crypto (client/server certs + nonces → derived symmetric keys), periodically renewed with a fresh OPN before expiry; all actual application traffic then rides inside MSG chunks under the symmetric keys; CLO tears the channel down. Large MSG payloads split into multiple chunks (Intermediate/Final/Abort) — never sending the Final chunk is the basis of chunk-flooding DoS (CVE-2023-32787, unbounded server-side buffering).
|
||||
- SecureChannel vs. Session (OPC UA) — two deliberately decoupled layers: the SecureChannel (established via OPN) is a transport artifact providing encryption/integrity between two certificate-validated endpoints, with no concept of user identity; the Session (CreateSession + ActivateSession, carrying a UserIdentityToken — Anonymous/UserName-Password/X.509 cert) is the application-level, identity-bound conversation, including its Subscriptions/MonitoredItems state. A Session can outlive a SecureChannel renewal (new keys, same logical conversation) — same relationship as a TLS session ticket outliving an individual TCP connection.
|
||||
- GetEndpoints — unauthenticated-by-design OPC UA discovery call (has to be, since a client can't know what SecurityPolicy to speak until it asks) returning the server's available endpoint URLs, each with its own SecurityPolicy/SecurityMode combination, plus the server's certificate and ApplicationName/ProductURI metadata. Not a secrecy boundary — the real exposure is a server offering/accepting SecurityPolicy "None" or Anonymous auth among the discoverable options, the same pattern behind "over half of internet-exposed OPC UA servers allow unauthenticated access."
|
||||
- Reverse Connect (OPC UA) — role-inversion at the TCP-dial step only: a server behind NAT/no inbound firewall hole initiates the TCP connection outward and sends a ReverseHello, then hands control back so the client still drives the rest of the HEL/OPN/session handshake. Added in UA spec 1.04 for field devices that can't be reached directly.
|
||||
- Subscription / MonitoredItem (OPC UA) — client-server efficiency mechanism layered on Sessions: client registers interest in specific Nodes at a publishing interval; server proactively pushes change Notifications inside Publish responses instead of the client polling Read repeatedly. Distinct from the separate "OPC UA PubSub" transport profile (connectionless UDP/MQTT/AMQP multicast, no Session concept at all).
|
||||
- Zip-slip — a path-traversal vulnerability class in archive-extraction code: a crafted archive entry name containing `../` sequences writes outside the intended extraction directory when naively joined to a base path. Used by Team82 chained with client-side XSS against Softing edgeAggregator to drop a malicious shared-object file into a system library path for root RCE.
|
||||
- WerFault abuse — using Windows' built-in crash-reporting handler (WerFault.exe) as an unwitting code-execution trigger: after planting a malicious DLL (e.g. in System32), deliberately crashing the target process causes WerFault to run in a context that loads the planted DLL. Seen in Team82's Softing Secure Integration Server exploit chain.
|
||||
- Pwn2Own ICS — the ICS-specific track of the Pwn2Own exploit competition (vendor products, cash prizes for working exploit chains); Team82 used their OPC UA exploit framework across three Pwn2Own ICS competitions.
|
||||
- MES (Manufacturing Execution System) — Purdue Level 3 software managing/tracking production on the plant floor in real time (work orders, batch tracking, quality, OEE) — sits between the Historian/SCADA (Level 2/3, process data) and ERP (Level 4, business/planning data). A common OPC-UA peer: SCADA/historian-to-MES and MES-to-ERP are typical OPC-UA interoperability links.
|
||||
- ERP (Enterprise Resource Planning) — Purdue Level 4/5 corporate business software (SAP, Oracle, etc.) covering finance, procurement, supply chain, and production planning; consumes aggregated/summarized production data from MES rather than real-time process data, and does not connect directly down to field-level devices — that hop-by-hop chain (PLC→SCADA→MES→ERP) is what keeps the Level 3.5 DMZ boundary from being bypassed in a single connection.
|
||||
- S7 — Siemens' proprietary ICS protocol for communicating with SIMATIC PLCs (S7-300/1200/1500 etc.), port 102/TCP; like Modbus/DNP3, a frequent target for ICS network scanning.
|
||||
- CODESYS — the IEC 61131-3 engineering/runtime environment (port 11740/TCP) underlying many different vendors' PLCs, including WAGO's — a shared software base means a CODESYS vulnerability can cross vendor lines.
|
||||
- VFD (Variable Frequency Drive) — a device that controls motor speed by varying the frequency/voltage of its power supply; common ICS target (pumps, fans, compressors) alongside PLCs.
|
||||
@@ -146,3 +155,59 @@ Covered 2026-10-05 in response to "how do I build a fake OT network to play with
|
||||
- Client isolation — a network configuration that prevents devices on the same subnet/segment (e.g. the same private APN or Wi-Fi AP) from communicating directly with each other, typically restricting them to talk only to a gateway. Its absence was the root misconfiguration that let an attacker pivot across devices sharing a DSO private APN in the CERT Polska December 2025 incident.
|
||||
- VPN concentrator — a dedicated device (often combined with a firewall) built to terminate and manage many simultaneous VPN tunnels at once, from many remote clients/sites, as opposed to being a single tunnel's endpoint like client VPN software. A single compromised concentrator can expose every site/segment it serves — as with the FortiGate device in the CERT Polska December 2025 incident.
|
||||
- ICT (Information and Communication Technology) — a broader-scope synonym for "IT," more common in European/telecom/utility usage, covering computing and communications infrastructure together. In utility contexts, "the DSO's ICT network" means the DSO's own IT/corporate-side systems, as distinct from the OT side (RTUs, field devices) — functionally the same split the Purdue Level 3.5 DMZ formalizes.
|
||||
|
||||
** IR / Forensics
|
||||
Covered 2026-10-12 — see [[ot-ics-security-ramp/curriculum]] for session notes.
|
||||
- NIST SP 800-61 IR lifecycle — 4 phases: Preparation, Detection & Analysis, Containment/Eradication/Recovery, Post-Incident Activity.
|
||||
- SANS PICERL — 6-phase equivalent of the same lifecycle, finer-grained: Preparation, Identification, Containment, Eradication, Recovery, Lessons Learned. Same substance as NIST 800-61, just splits phases differently for reporting granularity.
|
||||
- Triage — first-pass judgment on a security alert: real or noise, and how severe. Precedes scoping.
|
||||
- Scoping — determining how far an incident has actually spread (which hosts/accounts/segments are touched) by reconstructing a timeline against the kill chain.
|
||||
- SIEM (Security Information and Event Management) — centralized platform correlating log events from many sources against rules to generate alerts; the main source of IT detection output feeding into triage.
|
||||
- EDR (Endpoint Detection and Response) — agent-based endpoint monitoring that flags suspicious process behavior on a host; the backbone of IT detection, but generally can't be installed on OT field devices (PLC/RTU) since they have no general-purpose OS or spare resources for a third-party agent — a core reason OT detection leans on passive network monitoring instead (see Dragos/Claroty/Nozomi in OT Vendor Landscape).
|
||||
- IDS / IPS (Intrusion Detection / Prevention System) — network-traffic monitoring matching against known-bad signatures; IDS alerts only, IPS can actively block.
|
||||
- Containment: short-term vs. long-term — short-term is an immediate stopgap (firewall block, disable an account, VLAN move) to stop spread right now; long-term is the more thorough fix done once stable (rebuild, patch, rotate every touched credential).
|
||||
- Containment: isolate vs. monitor-in-place — isolating a compromised host immediately stops further damage but tips off the attacker (who may escalate or destroy evidence); monitoring quietly gathers more intel on the attacker's full objective at the cost of letting them keep acting.
|
||||
- DNS sinkholing — redirecting a known-malicious domain (e.g. a C2 domain) to a controlled server instead of its real destination, as a containment action that can be done covertly without alerting the attacker.
|
||||
- OT containment constraint — abruptly isolating/shutting down a live industrial process can itself cause physical harm (overpressure, overheating, mechanical damage), so touching the actual process requires joint sign-off from plant operations/safety engineers, not unilateral cyber-IR action; network-layer containment (segmenting at the Level 3.5 DMZ) is preferred over touching the field device directly. TRITON/TRISIS is the cautionary case: it targeted the SIS itself, meaning you can't always assume your own safety-shutdown layer is trustworthy during an incident.
|
||||
- Chain of custody — the documented, unbroken record of who handled a piece of evidence, when, and how; required for legal admissibility. A gap lets anyone challenge the evidence as tampered.
|
||||
- Forensic imaging — a bit-for-bit copy of storage media (captures deleted/unallocated space, not just visible files), taken via a write blocker so the original is never touched; a cryptographic hash (MD5/SHA) before and after proves the original wasn't altered.
|
||||
- Order of volatility (RFC 3227) — the sequence for what to collect first because it disappears fastest: CPU registers/cache → RAM/process state → network connections/routing tables → running processes → disk → remote logs → physical config/topology → archival backups.
|
||||
- Live vs. dead-box forensics — live forensics captures volatile state (memory, connections, processes) before touching power, at the risk of your own tooling altering system state; dead-box forensics images an already-powered-off disk cleanly, but everything volatile is already gone unless captured first.
|
||||
- Memory forensics / acquisition — dumping RAM to capture a snapshot of everything active (processes, connections, loaded modules, injected code, sometimes credentials/keys); the main way to catch fileless malware or living-off-the-land activity that leaves little/no disk trace, since it still has to execute somewhere. Acquisition needs either a privileged kernel driver on the box (WinPmem/Magnet RAM Capture/FTK Imager on Windows; LiME or avml as a loadable kernel module on Linux) or access from outside the OS entirely (hypervisor-level snapshot of a VM, which sidesteps guest-OS hardening completely). Modern hardening (Windows HVCI/VBS + driver-signature enforcement, Linux Lockdown LSM + CONFIG_STRICT_DEVMEM) specifically blocks this same primitive since it's also what a kernel-level rootkit wants — real IR practice relies on pre-positioned, already-signed/trusted acquisition tooling deployed before an incident (Preparation phase), not improvising mid-incident on a hardened box.
|
||||
- /proc/kcore — a Linux virtual file exposing the kernel's view of physical memory as a (sparse) ELF core file, root-only (mode 0400) by default, readable without loading a new kernel module (lower footprint than LiME). Reported file size (~128 TiB) is the full kernel virtual address space, not real RAM — the ELF header's e_phoff/e_phnum fields point to a Program Header Table listing the actual PT_LOAD segments (contiguous ranges backed by real RAM); real tools read only those ranges rather than the sparse whole. Separate from kptr_restrict (hides raw kernel pointer values from /proc/kallsyms for non-root — a narrower, different protection) and the Lockdown LSM (/sys/kernel/security/lockdown; a stronger, kernel-wide restriction that can block kcore access even for root when active).
|
||||
- Volatility — the standard open-source framework for analyzing a raw memory dump offline (process lists, network connections, injected/unbacked code) without needing the original OS running.
|
||||
- Timeline analysis — correlating MACB timestamps (Modified/Accessed/Changed/Born) across many files and logs to reconstruct a single chronological narrative of what happened when.
|
||||
- File carving — recovering files from unallocated/raw disk space by recognizing file-type signatures (magic bytes) directly in the byte stream, with no file-system metadata to rely on.
|
||||
- Slack space — leftover data in the unused tail of a disk block allocated to a smaller file; can retain fragments of older, otherwise-deleted content.
|
||||
- Persistence artifacts — disk-side evidence of how an attacker maintains access across reboots: registry run keys, scheduled tasks, services, startup folders. The concrete artifacts behind the IOC/IOA concepts.
|
||||
- PCAP (full packet capture) — complete, byte-for-byte network traffic capture; highest fidelity (can reconstruct exact payloads/streams) but too expensive to retain everywhere, always, at scale.
|
||||
- NetFlow / IPFIX — metadata-only flow records (five-tuple + byte/packet counts + timestamps), orders of magnitude cheaper than PCAP; the usual continuous baseline, with full PCAP triggered selectively once something looks suspicious.
|
||||
- Zeek (formerly Bro) — network security monitor sitting between NetFlow and full PCAP: real protocol parsing emitting structured, searchable per-protocol logs (conn.log, dns.log, http.log, ssl.log, files.log, weird.log) without retaining raw payloads. Where most practical SOC/IR network forensics actually happens.
|
||||
- Beaconing — a C2-hunting signal: malware checking in with its C2 infrastructure at suspiciously regular intervals, unlike the bursty/irregular pattern of normal human/application traffic. Detected via inter-arrival-time analysis; modern C2 deliberately jitters timing to evade this.
|
||||
- DGA (Domain Generation Algorithm) — malware algorithmically computing new C2 domains on a schedule so no single domain can be blocklisted; shows up in DNS logs as high-entropy/random-looking subdomains.
|
||||
- DNS tunneling — smuggling arbitrary data through DNS queries/responses (e.g. oversized TXT records) as a covert C2/exfiltration channel, abusing DNS's dual legitimate/exploitable nature.
|
||||
- JA3 / JA3S fingerprinting — hashing specific fields/ordering in a TLS ClientHello (JA3) / ServerHello (JA3S) to fingerprint a specific client or server TLS stack (often a malware family's) even though the encrypted payload itself stays opaque.
|
||||
- SNI (Server Name Indication) — the destination hostname sent in cleartext during the TLS handshake itself, before encryption kicks in; lets network forensics identify which domain was contacted even with zero decryption.
|
||||
- OT network forensics advantage — since ICS protocols (Modbus/DNP3/GOOSE/IEC 104) are largely unencrypted by design, full PCAP against OT traffic gives complete semantic visibility (exact function codes, register values, commands) in plaintext — the literal product core of Dragos/Claroty/Nozomi's deep protocol parsing, and a genuine advantage over IT network forensics where TLS increasingly limits visibility to metadata tricks (JA3/SNI/beaconing) alone.
|
||||
- PLC forensic dead end — most PLCs keep no audit log at all (no record of who wrote what register when, often unreliable local timestamps), a consequence of limited storage/compute and decades-old firmware never designed for forensic retention, not vendor oversight. Real forensic value in an OT incident usually lives on the engineering workstation (normal Windows box programming the PLCs) instead — project files, USB history, SCADA/HMI logs — which is why Stuxnet's forensic trail was reconstructed from engineering workstations and recovered malware samples, not from the Siemens PLCs themselves.
|
||||
- Control logic verification — the OT equivalent of file-integrity monitoring: extracting/uploading a PLC's current program and comparing it against an engineering golden image (last known-legitimate configuration) to spot unauthorized modifications, since there's no log entry saying "logic changed." Realistically done via the vendor engineering software's Upload function (inverse of Download) over the native programming protocol (S7comm/S7+ for Siemens, CIP/EtherNet-IP for Rockwell, not the real-time I/O protocol) — a read-only operation safe to run against a live PLC, one of few OT forensic actions that doesn't collide with the "can't take it offline" containment constraint. Caveat, not hypothetical: Stuxnet defeated this exact technique with a rootkit-like component that intercepted the PLC's own block-read responses, showing clean logic to a normal Upload/monitor query while different logic actually ran (injected into OB1) — the first known PLC rootkit, and proof that upload-and-compare only works if the PLC's self-reporting can be trusted. Rigorous verification against a sophisticated implant needs independent corroboration below that layer (JTAG/firmware extraction, or comparing actual I/O behavior against claimed program logic) — far more invasive, reserved for dedicated malware-research labs rather than routine IR.
|
||||
- Historian as forensic evidence — since PLCs keep no audit trail, the process Historian (continuous timestamped sensor/setpoint/alarm log, Purdue Level 3) is often the only record of actual physical process behavior during an incident; correlating historian trends against whatever host/network timeline is available (engineering workstation, PCAP) is frequently how "what physically happened, and when" gets reconstructed.
|
||||
- SCD file integrity as a forensic target — GOOSE's Substation Configuration Description file (static multicast/dataset-to-relay mapping) carries no built-in runtime integrity check, so a tampered SCD file is a distinct attack vector from a live network injection; detecting tampering has to happen out-of-band (version control, hashing the file itself) since the protocol gives nothing to lean on.
|
||||
- Anti-forensics in OT — can be purely remote/digital (e.g. the Dec-2025 Poland CHP incident: the attacker wiped the WAGO PFC200's partition table as an anti-forensics step, executed remotely through the SSH-tunnel/private-APN access chain already established — not a physical act). Physical anti-forensics (an insider or facility intruder pulling a drive, degaussing storage, cutting a cable) is a separate, real consideration distinct to OT's physically-reachable hardware, but not what happened in the Poland case — worth keeping "destructive" and "physical" as separate axes rather than conflating them.
|
||||
- OT forensics scheduling constraint — the same physical-safety logic from OT containment applies to forensics itself: a live PLC often can't be pulled for imaging, so analysis frequently happens against a still-running system or gets deferred to a planned maintenance window, a real constraint largely absent from IT incident response.
|
||||
|
||||
** VPN / Remote-Access & Tunneling
|
||||
Covered 2026-10-13 — triggered by an interview-process note that the startup is considering a custom VPN product for OT infrastructure at a client's request. Quick vocabulary pass, not a deep dive; see [[ot-ics-security-ramp/curriculum]].
|
||||
- IPsec — network-layer (L3) VPN suite: IKE for key exchange, ESP/AH for packet protection. Transport mode (host-to-host, payload only) vs. tunnel mode (gateway-to-gateway, whole packet wrapped — the one used for site-to-site VPNs).
|
||||
- OpenVPN — SSL/TLS-based VPN, userspace, TUN (IP-layer) or TAP (Ethernet-layer) virtual interfaces. Flexible and widely supported, slower than kernel-space alternatives like WireGuard.
|
||||
- WireGuard — modern, minimal-codebase VPN built on the Noise protocol framework; UDP-only, kernel-integrated on Linux, no IKE-style negotiation overhead. The current default choice for new deployments.
|
||||
- SSL/TLS VPN ("clientless") — browser-portal-based remote access (an HTTPS gateway proxying into specific internal apps) rather than a full network tunnel; common for vendor/contractor access since it doesn't hand out full network reachability like a traditional VPN does.
|
||||
- GRE (Generic Routing Encapsulation) — encapsulation only, not encryption; often paired as "GRE over IPsec" to carry routing protocols/multicast through an encrypted tunnel.
|
||||
- MPLS VPN — carrier-provided L2/L3 VPN service, not something you run yourself; common for utility WAN backhaul (substation ↔ control center).
|
||||
- DMVPN (Dynamic Multipoint VPN) — Cisco spoke-to-spoke tunneling negotiated on demand over a hub, avoiding full-mesh static IPsec between many distributed sites (e.g. substations).
|
||||
- Jump host / bastion — a hardened, logged single chokepoint remote users must pass through to reach OT assets, layered on top of a VPN rather than allowing direct VPN-to-asset reachability.
|
||||
- ZTNA (Zero Trust Network Access) — identity- and session-based access brokering one specific asset/app per session, instead of a VPN's network-level trust (once on a VPN, you can typically reach anything on that segment).
|
||||
- OT Secure Remote Access (SRA) platforms — vendor category purpose-built to replace flat VPN access to OT with session-scoped, often agentless, fully-recorded access: Claroty SRA, Cyolo, Dispel, Xona Systems, Cisco Secure Equipment Access. Likely the competitive category behind the "custom OT VPN" interview note.
|
||||
- Data diode — hardware-enforced unidirectional (OT→IT only) data flow; solves one-way monitoring/log export, not interactive remote access — a different problem than "OT VPN" despite surface-level association.
|
||||
- IEC 62443 zone/conduit model — architectural frame remote access is supposed to respect: assets grouped into trust zones, cross-zone traffic forced through a defined conduit (firewall/jump host) rather than bypassed by a VPN tunnel.
|
||||
- Engineering workstation (EWS) vendor access — the classic high-risk OT remote-access case: an OEM/integrator needing occasional PLC-programming access, historically given via always-on VPN + shared credentials — the main case SRA/ZTNA tooling targets (scope access to one device, one maintenance window).
|
||||
|
||||
@@ -15,4 +15,4 @@ Curated, high-quality cybersecurity resources Aner has found worth reading (and
|
||||
- [[https://media.defense.gov/2026/Jul/09/2003959459/-1/-1/0/CSI_REDUCING_RISK_OF_SNMP_ABUSE.PDF][NSA/CISA CSI: Reducing the Risk of SNMP Abuse]] (2026-07-09) — joint US government cybersecurity advisory on SNMP abuse specifically in critical-infrastructure contexts. Covers v1/v2c plaintext community-string exposure, GET-based network/device-inventory reconnaissance, SET-based config tampering, and cites the Russian FSB Center 16 campaign that exfiltrated router configs (VPN configs, SCADA-network routing, credentials) from critical-infrastructure perimeter routers via SNMP + Cisco Smart Install — a real-world OT-relevant case study. Read 2026-10-09.
|
||||
- [[https://redbotsecurity.com/examining-the-modbus-protocol/][RedBot Security: Examining the Modbus Protocol Through an Offensive-Security Lens]] — offensive/pentester-oriented deep-dive on Modbus TCP: function-code abuse, lack of auth/encryption/integrity, replay/MITM/DoS/command-injection patterns. Complements the FrostyGoop (2024) case study — malware that injected Modbus TCP commands to disable heating in Ukrainian apartment buildings, the modern successor to VPNFilter's 2018 Modbus traffic sniffing. Queued for review 2026-10-06.
|
||||
- [[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7926645/][Vulnerability and Impact Analysis of the IEC 61850 GOOSE Protocol in the Smart Grid]] (open-access journal paper) — experimental analysis of GOOSE pub-sub weaknesses, showing an attacker with substation-network familiarity can falsify messages affecting real physical grid operation; covers why many vendor IEDs still ship without IEC 62351-6 encryption/signing since the overhead can exceed the end-to-end latency budget. Queued for review 2026-10-06.
|
||||
- [[https://claroty.com/team82/research/opc-ua-deep-dive-a-complete-guide-to-the-opc-ua-attack-surface][Claroty Team82: OPC UA Deep Dive — A Complete Guide to the OPC UA Attack Surface]] — comprehensive attack-surface writeup from an OT security vendor's research team; relevant given current data (2026) showing over half of internet-exposed OPC-UA servers allow unauthenticated access and ~80% transmit in plaintext despite the protocol's built-in security options existing — "Security Policy: None" being valid and widely left set. Queued for review 2026-10-06.
|
||||
- [[https://claroty.com/team82/research/opc-ua-deep-dive-a-complete-guide-to-the-opc-ua-attack-surface][Claroty Team82: OPC UA Deep Dive — A Complete Guide to the OPC UA Attack Surface]] — comprehensive attack-surface writeup from an OT security vendor's research team; relevant given current data (2026) showing over half of internet-exposed OPC-UA servers allow unauthenticated access and ~80% transmit in plaintext despite the protocol's built-in security options existing — "Security Policy: None" being valid and widely left set. Read 2026-10-12 — 10-part series covering protocol history, data model, HEL/OPN/MSG/CLO mechanics + SecureChannel/Session security model, supply-chain risk (529 products on a handful of shared SDKs), the ~50-vuln exploit framework, DoS/client-RCE/integration-server case studies, and asset-owner mitigations. See [[ot-ics-security-ramp/glossary]] for terms pulled from it.
|
||||
|
||||
@@ -7,7 +7,8 @@
|
||||
|
||||
Standing TODOs for [[ot-ics-security-ramp]].
|
||||
|
||||
* TODO Continue cyber curriculum per [[ot-ics-security-ramp/curriculum]] — next up: IR/forensics, then frameworks/compliance (NIST CSF, ISO 27001, SOC 2, CIS Controls, STIX/TAXII), remaining defensive-stack acronyms, cloud security
|
||||
* TODO Continue cyber curriculum per [[ot-ics-security-ramp/curriculum]] — next up: frameworks/compliance (NIST CSF, ISO 27001, SOC 2, CIS Controls, STIX/TAXII), remaining defensive-stack acronyms, cloud security (IR/forensics completed 2026-10-12)
|
||||
* DONE Quick session on VPNs and similar remote-access/tunneling tech for OT — done 2026-10-13. Covered standard VPN tech (IPsec, OpenVPN, WireGuard, SSL/TLS VPN, GRE, MPLS VPN, DMVPN) plus OT-specific remote-access patterns (jump host, ZTNA, SRA platforms, data diode, IEC 62443 zone/conduit, EWS vendor access). Terms saved to [[ot-ics-security-ramp/glossary]].
|
||||
* DONE Learn ICS/OT protocols in depth — SNMP, Modbus, and others (DNP3, IEC 61850/GOOSE, IEC 104, OPC-UA already covered per [[ot-ics-security-ramp/curriculum]])
|
||||
* TODO Build a network simulator project (targeted for next week, not this week) — foundation decided 2026-10-06: MiniCPS (Mininet-based, chosen for scale over ICSSIM/ICS-SimLab's Docker-per-device isolation). Full comparison and open questions in [[ot-ics-security-ramp/network-simulator-blueprint]].
|
||||
* DONE Make a standalone ICS/OT protocol comparison reference — done 2026-10-06 as an org table node, [[ot-ics-security-ramp/protocol-comparison]]. If a rendered PDF/HTML version is wanted later, revisit format then.
|
||||
|
||||
+10
-6
@@ -5,16 +5,20 @@
|
||||
#+title: workplate
|
||||
#+filetags: :workplate: :planning:
|
||||
|
||||
* Today (2026-10-07 Wed)
|
||||
** TODO [quick] (ot-ics-security-ramp) Review GOOSE vulnerability/impact-analysis paper (see reading-list)
|
||||
** TODO [quick] (ot-ics-security-ramp) Review Claroty Team82 OPC UA attack-surface deep dive (see reading-list)
|
||||
** TODO [quick] (ot-ics-security-ramp) Curriculum refresher before IR/forensics — quick re-read of curriculum/glossary status
|
||||
** TODO [half-day] (ot-ics-security-ramp) Continue cyber curriculum — IR/forensics
|
||||
* Today (2026-10-13 Tue)
|
||||
** DONE [few-hours] (azos) Rename azos-core/features to azos-core/suites; fold claude-memory/claude-skills into base suite
|
||||
** DONE [few-hours] (azos) Decouple exwm and station suites (Nix + Emacs level) — landed as a station-to-desktop rename + discord moved to azos/features
|
||||
** DONE [half-day] (azos) Audit azos/features/* for portability; mail accounts; generalize claude-memory/skills into backend-pluggable agent-memory/agent-skills (agent-shell backend added) — see todos for full detail
|
||||
** DONE [half-day] (azos) Rebuild lauretta and live-test today's changes — found and fixed a real bug live (agent-shell resync used the wrong event, file-write never fires for Claude's native edit tools); everything else confirmed working, see todos
|
||||
** DONE [quick] (ot-ics-security-ramp) VPN / OT tunneling-tech quick session — triggered by interview note about a custom OT VPN project (see todos)
|
||||
|
||||
* This Week (2026-10-05 to 2026-10-11)
|
||||
* This Week (2026-10-12 to 2026-10-18)
|
||||
** TODO [few-hours] (just-talking) Review LangGraph (per manager's instructions)
|
||||
** TODO [few-hours] (just-talking) Review Node.js — refresher, rusty on it (per manager's instructions)
|
||||
** TODO [half-day] (ot-ics-security-ramp) Build network simulator project (MiniCPS) — stretch goal, may slip to next week
|
||||
|
||||
[[id:3a365596-7744-43a1-9a2b-53d0dcc087cd][ot-ics-security-ramp/todos]]
|
||||
|
||||
[[id:4c4ae633-94c3-4a34-bf81-cf8aa89c034b][just-talking/todos]]
|
||||
|
||||
[[id:939a9fa3-ddc8-4cc1-8130-771cc5de000f][azos/todos]]
|
||||
Reference in New Issue
Block a user