diff --git a/README.org b/README.org index 22f3818..d53672e 100644 --- a/README.org +++ b/README.org @@ -157,6 +157,30 @@ All service data under =/mnt/data/=: Nextcloud is placed into maintenance mode and postgres is =pg_dump='d before each backup to ensure a consistent snapshot. +** First-time setup — initialize the repository + +Restic requires a one-time =init= before the first backup can run. The +automated job will fail with "repository does not exist" until this is done. + +Run on the Pi after the first deploy: + +#+begin_src bash +# Note: use single quotes around the remote script to prevent local shell expansion +ssh admin@192.168.1.100 'sudo bash -c '"'"' + export AWS_ACCESS_KEY_ID=$(cat /run/secrets/restic/s3_access_key_id) + export AWS_SECRET_ACCESS_KEY=$(cat /run/secrets/restic/s3_secret_access_key) + export RESTIC_PASSWORD=$(cat /run/secrets/restic/password) + restic -r s3:https://s3.us-east-005.backblazeb2.com/zakobar-home-backup init +'"'"'' +#+end_src + +You only need to do this once. After =init= succeeds, the daily timer will +run normally. To trigger a backup immediately without waiting for 03:00: + +#+begin_src bash +ssh admin@192.168.1.100 "sudo systemctl start restic-backups-homey.service" +#+end_src + ** Configuration Repository URL and credentials are set per-host: diff --git a/TODO.org b/TODO.org index 5cd5a79..7cdf826 100644 --- a/TODO.org +++ b/TODO.org @@ -64,7 +64,7 @@ now use =nixos-raspberrypi.lib.nixosSystem= and =raspberry-pi-4.base=. =nix flake check= passes. -** TODO Verify SD card partition labels in =hosts/pi-main/hardware.nix= +** DONE Verify SD card partition labels in =hosts/pi-main/hardware.nix= The config assumes labels =NIXOS_SD= (root) and =FIRMWARE= (boot). After flashing, check with: #+begin_src bash @@ -74,7 +74,7 @@ * Caddy Build -** TODO Fix =vendorHash= in =modules/caddy.nix= +** DONE Fix =vendorHash= in =modules/caddy.nix= The Caddy build with the Cloudflare DNS plugin currently uses =lib.fakeHash= as a placeholder. After the first =nix build= attempt it will fail with the correct hash in the error message. Replace =lib.fakeHash= with that value. @@ -94,7 +94,7 @@ * Deployment -** TODO Phase 1 — Build and flash bootstrap SD card image +** DONE Phase 1 — Build and flash bootstrap SD card image The bootstrap image is a minimal NixOS with SSH + WiFi only (no sops, no services). Its sole purpose is to boot the Pi so you can generate the age key @@ -123,7 +123,7 @@ ssh admin@192.168.1.100 #+end_src -** TODO Phase 2 — Generate age key and add it to sops +** DONE Phase 2 — Generate age key and add it to sops On the Pi (over SSH): #+begin_src bash @@ -154,7 +154,7 @@ git commit -m "add Pi age key to sops recipients" #+end_src -** TODO Phase 3 — Fix Caddy vendorHash, then deploy full config +** DONE Phase 3 — Fix Caddy vendorHash, then deploy full config The full =pi-main= config includes Caddy built with the Cloudflare DNS plugin. The first build will fail with the correct hash in the error output. diff --git a/docs/gitea-runner-workflows.org b/docs/gitea-runner-workflows.org new file mode 100644 index 0000000..178d1e8 --- /dev/null +++ b/docs/gitea-runner-workflows.org @@ -0,0 +1,317 @@ +#+TITLE: Gitea Actions Runner — Workflows & Usage Guide +#+DATE: 2026-05-04 +#+AUTHOR: homey project +#+OPTIONS: toc:2 num:t + +* Overview + +This document covers the Gitea Actions runner setup on pi-main, how the runner +works, the label system, and example workflows for both host-based ("ubuntu") +and Nix-native jobs. + +** Architecture + +The runner is configured in =modules/services/gitea-runner.nix= and uses the +NixOS native =services.gitea-actions-runner= module. Jobs run with the *host* +executor: each step executes directly on the Pi 4 as the =gitea-runner-pi-main= +system user. There is no container isolation per job. + +#+BEGIN_EXAMPLE +Gitea (podman container) + │ HTTPS → Cloudflare tunnel → Caddy → git.zakobar.com + │ (runner connects outbound via HTTPS, same path as a browser) + ▼ +gitea-actions-runner (systemd service) + │ host executor + ▼ +Jobs run as: gitea-runner-pi-main (unprivileged system user) + PATH includes: nix, git, bash + system packages +#+END_EXAMPLE + +** Runner labels + +Labels are advertised to Gitea and matched against =runs-on:= in workflow +files. The default labels configured in this project are: + +| Label | Executor | Notes | +|---------------+----------+--------------------------------------------| +| =native:host= | host | Canonical label for "run on this machine" | +| =ubuntu-latest= | host | Matches common GitHub Actions workflows | +| =debian-latest= | host | Alternative for Debian-targeting workflows | +| =nix:host= | host | Explicit label for Nix-native jobs | + +All four labels route to the same runner process and the same host environment. +The difference is purely semantic — pick the label that makes your workflow's +intent clear. + +** Nix daemon trust + +The runner user is added to =nix.settings.trusted-users=, which means it can: +- Evaluate flakes (=nix flake check=, =nix build=) +- Write derivation outputs to the Nix store +- Pass =--extra-experimental-features= flags to the daemon +- Use =nix copy= to push/pull store paths to a remote cache + +It cannot modify NixOS system configuration or run privileged operations. + +* Example workflows + +Workflow files live in =.gitea/workflows/= inside each repository (or +=.github/workflows/= — Gitea Actions supports both paths). + +** Minimal smoke test (host) + +The simplest possible workflow — runs a shell command on the runner. + +#+BEGIN_SRC yaml +# .gitea/workflows/smoke.yaml +on: [push] +jobs: + smoke: + runs-on: native:host + steps: + - uses: actions/checkout@v3 + - run: echo "Runner is alive on $(hostname)" +#+END_SRC + +** Standard shell-based CI (ubuntu-latest label) + +Use this for repos that want to stay compatible with GitHub Actions. The +workflow looks identical to what you'd push to GitHub; it just runs on your Pi. + +#+BEGIN_SRC yaml +# .gitea/workflows/ci.yaml +on: + push: + branches: [main, master] + pull_request: + +jobs: + test: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + + - name: Install dependencies + run: | + # On the host executor, use nix-shell or system packages. + # apt/yum are NOT available — this is NixOS, not Ubuntu. + # Use nix-shell -p for one-off tools: + nix-shell -p nodejs --run "node --version" + + - name: Run tests + run: | + nix-shell -p nodejs --run "npm test" +#+END_SRC + +*Important:* Despite the label =ubuntu-latest=, the host is NixOS. =apt=, +=yum=, and FHS paths like =/usr/bin/python3= are not available. Use +=nix-shell -p = to bring in any tool you need. + +** Nix flake check + +Validate a flake on every push — the most common use case for this runner. + +#+BEGIN_SRC yaml +# .gitea/workflows/flake-check.yaml +on: [push, pull_request] +jobs: + check: + runs-on: nix:host + steps: + - uses: actions/checkout@v3 + + - name: Check flake + run: nix flake check --no-build + + - name: Build default package + run: nix build +#+END_SRC + +** Nix build with caching + +Build a derivation and push the result to a binary cache so subsequent builds +are fast. Requires a Cachix account or an S3-compatible cache configured in +=nix.settings=. + +#+BEGIN_SRC yaml +# .gitea/workflows/build-and-cache.yaml +on: + push: + branches: [main] + +jobs: + build: + runs-on: nix:host + steps: + - uses: actions/checkout@v3 + + - name: Build + run: nix build --print-build-logs + + - name: Push to cache + # nix copy requires the runner user to be in trusted-users (already set). + # Replace the URI with your actual cache. + run: | + nix copy --to "s3://your-cache-bucket?region=us-east-1" ./result +#+END_SRC + +** NixOS configuration check (this repo) + +Check that the homey flake evaluates cleanly on every change. Add this to the +homey repo itself. + +#+BEGIN_SRC yaml +# .gitea/workflows/nixos-check.yaml +on: [push, pull_request] +jobs: + eval: + runs-on: nix:host + steps: + - uses: actions/checkout@v3 + + - name: Evaluate NixOS configurations + run: | + nix flake check --no-build + # Optionally build a specific host config (slow on Pi): + # nix build .#nixosConfigurations.pi-main.config.system.build.toplevel + + - name: Check formatting (optional) + run: | + nix-shell -p nixpkgs-fmt --run "nixpkgs-fmt --check ." +#+END_SRC + +** Multi-step pipeline with artifacts + +#+BEGIN_SRC yaml +# .gitea/workflows/pipeline.yaml +on: + push: + tags: ['v*'] + +jobs: + build: + runs-on: nix:host + steps: + - uses: actions/checkout@v3 + + - name: Build release + run: nix build --out-link result + + - name: Upload artifact + uses: actions/upload-artifact@v3 + with: + name: release-binary + path: result/bin/ + + deploy: + needs: build + runs-on: native:host + steps: + - name: Download artifact + uses: actions/download-artifact@v3 + with: + name: release-binary + + - name: Deploy + run: ./deploy.sh +#+END_SRC + +* Caveats and gotchas + +** No apt/brew/yum + +The host is NixOS. Package managers from other distros do not work. Use +=nix-shell -p --run "..."= for ad-hoc tools, or add a =shell.nix= / +=flake.nix= devShell to your repo and enter it with =nix develop=. + +** No Docker/Podman per job + +The host executor does not launch a fresh container per job. All jobs share the +same filesystem (under =/home/gitea-runner-pi-main/=) and the same running +system. This means: + +- No isolation between concurrent jobs (though concurrency defaults to 1). +- Side effects (files written, packages installed with nix) persist between + runs unless you clean up explicitly. +- Use =nix build= output symlinks (=./result=) rather than writing to system + paths. + +** actions/checkout and git + +The =actions/checkout@v3= action works fine on the host executor. It clones +into the runner's working directory. Subsequent steps run in that directory by +default. + +If you use =actions/checkout@v4=, note that it requires a newer Node.js. On +NixOS you can't rely on a system Node, so either pin to v3 or use: + +#+BEGIN_SRC yaml + - uses: actions/checkout@v3 # v3 bundles its own Node runtime +#+END_SRC + +** Nix experimental features + +Flake commands require =nix-command= and =flakes= experimental features. These +are typically enabled system-wide in =nix.settings.experimental-features= in +=modules/common.nix=. If a job fails with "experimental feature not enabled", +you can pass it inline: + +#+BEGIN_SRC yaml + - run: nix --extra-experimental-features "nix-command flakes" flake check +#+END_SRC + +Or ensure =common.nix= has: + +#+BEGIN_SRC nix +nix.settings.experimental-features = [ "nix-command" "flakes" ]; +#+END_SRC + +** Token rotation + +The registration token in =gitea/runner_token= is consumed on first +registration. The runner then stores its own credentials in +=/var/lib/gitea-runner/pi-main/.runner=. If you need to re-register (e.g. +after wiping the state directory), generate a new token from Gitea's admin UI +and update the sops secret before restarting the service. + +** Pi 4 performance + +The Pi 4 is capable but not fast for heavy builds. Tips: +- Enable the Nix binary cache (=nixos-cache.nixos.org= is on by default) so + pre-built derivations are fetched instead of compiled. +- Set =nix.settings.max-jobs= to =4= to use all cores for parallel builds. +- Avoid building large packages (LLVM, Chromium) locally — push to a remote + builder or use Cachix. + +* Debugging + +** Check runner status +#+BEGIN_SRC sh +systemctl status gitea-runner-pi-main +journalctl -u gitea-runner-pi-main -f +#+END_SRC + +** Runner registration state +#+BEGIN_SRC sh +cat /var/lib/gitea-runner/pi-main/.runner +#+END_SRC + +** Force re-registration +#+BEGIN_SRC sh +# Stop, wipe state, restart (runner will re-register using the token file) +systemctl stop gitea-runner-pi-main +rm /var/lib/gitea-runner/pi-main/.runner +systemctl start gitea-runner-pi-main +#+END_SRC + +** Test a workflow locally + +Use =act= (the local runner) to test workflow files before pushing: +#+BEGIN_SRC sh +nix-shell -p act --run "act push" +#+END_SRC + +Note: =act= spins up Docker containers for each job, so results may differ +slightly from the host-executor runner, but it is useful for syntax checking +and logic testing. diff --git a/flake.nix b/flake.nix index cd2341b..cac0085 100644 --- a/flake.nix +++ b/flake.nix @@ -70,6 +70,10 @@ ./modules/services/phpldapadmin.nix ./modules/services/jellyfin.nix ./modules/services/transmission.nix + ./modules/services/gitea-runner.nix + ./modules/services/uptime-kuma.nix + ./modules/services/ntfy.nix + ./modules/monitoring.nix ] ++ extraModules; }; diff --git a/hosts/pi-main/default.nix b/hosts/pi-main/default.nix index 17374b8..3f22980 100644 --- a/hosts/pi-main/default.nix +++ b/hosts/pi-main/default.nix @@ -92,6 +92,14 @@ homey.caddy.enable = true; homey.cloudflared.enable = true; + # CI/CD + homey.giteaRunner.enable = true; + + # Monitoring stack + homey.uptimeKuma.enable = true; + homey.ntfy.enable = true; + homey.monitoring.enable = true; + # Backups homey.backup.enable = true; # Where to send restic backups — set to your backup destination: @@ -113,6 +121,55 @@ rebootTime = "360s"; }; + # Disable WiFi power save — the brcmfmac driver on RPi4 lets the chip sleep, + # causing it to miss packets and drop the connection under low traffic. + # Run once when the wlan0 interface appears (and on every re-plug/reconnect). + systemd.services.wifi-disable-power-save = { + description = "Disable WiFi power management on wlan0"; + wantedBy = [ "multi-user.target" ]; + after = [ "sys-subsystem-net-devices-wlan0.device" ]; + bindsTo = [ "sys-subsystem-net-devices-wlan0.device" ]; + serviceConfig = { + Type = "oneshot"; + RemainAfterExit = true; + ExecStart = "${pkgs.iw}/bin/iw dev wlan0 set power_save off"; + }; + }; + + # Network watchdog — if the LAN gateway becomes unreachable, restart + # wpa_supplicant to force a fresh association. If the link is still + # dead 30 s later, reboot so the hardware watchdog doesn't have to. + # Runs every 2 min starting 5 min after boot. + systemd.services.network-watchdog = { + description = "Network connectivity watchdog"; + after = [ "network-online.target" ]; + serviceConfig = { + Type = "oneshot"; + ExecStart = pkgs.writeShellScript "network-watchdog" '' + gateway="192.168.1.1" + if ! ${pkgs.iputils}/bin/ping -c 3 -W 10 "$gateway" > /dev/null 2>&1; then + echo "Gateway $gateway unreachable — restarting wpa_supplicant" + systemctl restart wpa_supplicant.service + sleep 30 + if ! ${pkgs.iputils}/bin/ping -c 3 -W 10 "$gateway" > /dev/null 2>&1; then + echo "Still unreachable after wpa_supplicant restart — rebooting" + systemctl reboot + fi + fi + ''; + }; + }; + + systemd.timers.network-watchdog = { + description = "Periodic network connectivity check"; + wantedBy = [ "timers.target" ]; + timerConfig = { + OnBootSec = "5min"; + OnUnitActiveSec = "2min"; + Persistent = true; + }; + }; + # Compressed in-RAM swap via zstd. Pages evicted from RAM are compressed # (~3:1 ratio) and stored in a 25% RAM region (~2 GB) rather than written # to disk. Gives the OOM killer breathing room under PHP upload spikes. diff --git a/modules/backup.nix b/modules/backup.nix index 8d882f1..d9f2703 100644 --- a/modules/backup.nix +++ b/modules/backup.nix @@ -82,13 +82,25 @@ in # Pre-backup hook: pg_dump + nextcloud maintenance mode # ----------------------------------------------------------------------- systemd.services."homey-backup-pre" = { - description = "Pre-backup hooks (pg_dump, NC maintenance mode)"; + description = "Pre-backup hooks (pg_dump, NC maintenance mode, secrets env)"; serviceConfig = { Type = "oneshot"; ExecStart = pkgs.writeShellScript "backup-pre" '' set -euo pipefail podman="${pkgs.podman}/bin/podman" + # Write S3 credentials env file now, before restic-backups-homey.service + # starts — systemd loads EnvironmentFile= before ExecStartPre runs, so + # the file must already exist when the restic unit activates. + install -m 0600 /dev/null /run/restic-homey-secrets.env + { + printf 'AWS_ACCESS_KEY_ID=%s\n' \ + "$(cat ${config.sops.secrets."restic/s3_access_key_id".path})" + printf 'AWS_SECRET_ACCESS_KEY=%s\n' \ + "$(cat ${config.sops.secrets."restic/s3_secret_access_key".path})" + printf 'RESTIC_CACHE_DIR=%s\n' "${dataDir}/restic-cache" + } >> /run/restic-homey-secrets.env + # Put Nextcloud into maintenance mode (if running) if systemctl is-active --quiet podman-nextcloud.service; then $podman exec nextcloud php occ maintenance:mode --on || true @@ -105,19 +117,6 @@ in }; }; - systemd.services."homey-backup-post" = { - description = "Post-backup hooks (take NC out of maintenance mode)"; - serviceConfig = { - Type = "oneshot"; - ExecStart = pkgs.writeShellScript "backup-post" '' - set -euo pipefail - if systemctl is-active --quiet podman-nextcloud.service; then - ${pkgs.podman}/bin/podman exec nextcloud php occ maintenance:mode --off || true - fi - ''; - }; - }; - # ----------------------------------------------------------------------- # Restic backup service # ----------------------------------------------------------------------- @@ -125,7 +124,7 @@ in repository = cfg.repository; passwordFile = config.sops.secrets."restic/password".path; - # Runtime env file written by ExecStartPre (see systemd override below) + # Runtime env file written by homey-backup-pre.service (which runs first) environmentFile = "/run/restic-homey-secrets.env"; paths = [ @@ -137,6 +136,9 @@ in "${dataDir}/jellyfin" "${dataDir}/transmission" # Deliberately excluded: media/* (large, can be re-downloaded) + # Monitoring — uptime-kuma has monitors/history, ntfy has user accounts + "${dataDir}/uptime-kuma" + "${dataDir}/ntfy" ]; # Exclude Nextcloud's raw DB directory in favour of the pg_dump file @@ -157,36 +159,21 @@ in ]; }; - # Wire the pre/post hooks around the restic job and inject secrets + # Wire the pre/post hooks around the restic job systemd.services."restic-backups-homey" = { requires = [ "homey-backup-pre.service" ]; after = [ "homey-backup-pre.service" ]; serviceConfig = { - # Write runtime env file with actual secret values (restic needs the - # raw values; it does not support _FILE suffix env vars). - ExecStartPre = [ - (pkgs.writeShellScript "restic-inject-secrets" '' - install -m 0600 /dev/null /run/restic-homey-secrets.env - { - printf 'AWS_ACCESS_KEY_ID=%s\n' \ - "$(cat ${config.sops.secrets."restic/s3_access_key_id".path})" - printf 'AWS_SECRET_ACCESS_KEY=%s\n' \ - "$(cat ${config.sops.secrets."restic/s3_secret_access_key".path})" - printf 'RESTIC_CACHE_DIR=%s\n' "${dataDir}/restic-cache" - } >> /run/restic-homey-secrets.env - '') - ]; ExecStopPost = [ - (pkgs.writeShellScript "restic-cleanup-secrets" '' + (pkgs.writeShellScript "restic-post-hooks" '' + # Always runs on stop, success or failure rm -f /run/restic-homey-secrets.env + if systemctl is-active --quiet podman-nextcloud.service; then + ${pkgs.podman}/bin/podman exec nextcloud php occ maintenance:mode --off || true + fi '') ]; }; }; - - systemd.services."homey-backup-post" = { - after = [ "restic-backups-homey.service" ]; - wantedBy = [ "restic-backups-homey.service" ]; - }; }; } diff --git a/modules/caddy.nix b/modules/caddy.nix index e2ab77d..78ebc15 100644 --- a/modules/caddy.nix +++ b/modules/caddy.nix @@ -207,6 +207,58 @@ in ''; }; + # ------------------------------------------------------------------ + # Uptime Kuma — two_factor, admins only (enforced by authelia policy) + # ------------------------------------------------------------------ + "uptime.${domain}" = { + extraConfig = '' + ${autheliaForwardAuth} + reverse_proxy localhost:3001 + ''; + }; + "http://uptime.${domain}" = { + extraConfig = '' + ${autheliaForwardAuth} + ${cfProxy 3001} + ''; + }; + + # ------------------------------------------------------------------ + # Ntfy — no forward_auth; ntfy has its own token/password auth so the + # mobile app can connect without Authelia SSO complications. + # ------------------------------------------------------------------ + "ntfy.${domain}" = { + extraConfig = '' + reverse_proxy localhost:2586 + ''; + }; + "http://ntfy.${domain}" = { + extraConfig = cfProxy 2586; + }; + + # ------------------------------------------------------------------ + # Grafana — two_factor, admins only (enforced by authelia policy). + # After Authelia verifies the user, Caddy maps the Remote-User header + # to X-WEBAUTH-USER so Grafana's proxy auth auto-signs the user in. + # ------------------------------------------------------------------ + "grafana.${domain}" = { + extraConfig = '' + ${autheliaForwardAuth} + reverse_proxy localhost:3002 { + header_up X-WEBAUTH-USER {http.request.header.Remote-User} + } + ''; + }; + "http://grafana.${domain}" = { + extraConfig = '' + ${autheliaForwardAuth} + reverse_proxy localhost:3002 { + header_up X-Forwarded-Proto https + header_up X-WEBAUTH-USER {http.request.header.Remote-User} + } + ''; + }; + }; }; diff --git a/modules/cloudflared.nix b/modules/cloudflared.nix index dd2ddd0..798ce4a 100644 --- a/modules/cloudflared.nix +++ b/modules/cloudflared.nix @@ -20,6 +20,9 @@ # ldapadmin.zakobar.com → https://localhost:443 # jellyfin.zakobar.com → https://localhost:443 # torrent.zakobar.com → https://localhost:443 +# uptime.zakobar.com → https://localhost:443 +# ntfy.zakobar.com → https://localhost:443 +# grafana.zakobar.com → https://localhost:443 # Set "No TLS Verify" = true (Caddy's cert is from Let's Encrypt but # the hostname seen by cloudflared is localhost, so hostname verification # would fail without this flag). diff --git a/modules/common.nix b/modules/common.nix index 8307d51..ac5d666 100644 --- a/modules/common.nix +++ b/modules/common.nix @@ -21,6 +21,11 @@ "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" "nix-community.cachix.org-1:mB9FkXj6Q3Q4ohOcbM4FJ9Z1X2kCrVK4vZOqsDqqNqk=" ]; + # Trigger GC automatically when free space drops below 2 GB; + # stop once 5 GB is free. Prevents CI builds from filling the disk + # between weekly GC runs. + min-free = 2147483648; # 2 GiB + max-free = 5368709120; # 5 GiB }; gc = { automatic = true; diff --git a/modules/monitoring.nix b/modules/monitoring.nix new file mode 100644 index 0000000..020e3b3 --- /dev/null +++ b/modules/monitoring.nix @@ -0,0 +1,217 @@ +{ config, lib, pkgs, homeyConfig, ... }: + +# Prometheus + Grafana — metrics collection and dashboarding. +# +# Uses native NixOS services (not containers) for tight integration with +# the host OS and declarative dashboard/datasource provisioning. +# +# Architecture: +# node_exporter → Prometheus ← systemd_exporter +# ↓ +# Grafana (pre-provisioned dashboard: Node Exporter Full) +# +# Auth (Grafana): +# Authelia enforces two_factor + admins-only before any request reaches +# Grafana. Caddy then maps the Authelia Remote-User header to +# X-WEBAUTH-USER, and Grafana's proxy auth auto-signs the user in — +# no second login required. +# +# Prometheus is internal-only (127.0.0.1:9090); only Grafana reads it. +# Grafana is exposed at 127.0.0.1:3002 and reverse-proxied by Caddy. +# +# Data dirs: +# Prometheus: /var/lib/prometheus2 (system drive — metrics are ephemeral) +# Grafana: /var/lib/grafana (system drive — dashboards provisioned by Nix) +# +# Secrets consumed from sops: +# grafana/secret_key (session signing key) +# openldap/ro_password (for Grafana → LDAP auth, shared with other modules) + +let + cfg = config.homey.monitoring; + domain = homeyConfig.domain; + + # LDAP base DN derived from domain (e.g. zakobar.com → dc=zakobar,dc=com) + ldapBaseDN = lib.concatStringsSep "," + (map (p: "dc=${p}") (lib.splitString "." domain)); + +in +{ + options.homey.monitoring = { + enable = lib.mkEnableOption "Prometheus + Grafana monitoring stack"; + + prometheusPort = lib.mkOption { + type = lib.types.port; + default = 9090; + description = "Prometheus listen port (localhost only)."; + }; + + grafanaPort = lib.mkOption { + type = lib.types.port; + default = 3002; + description = "Grafana listen port (localhost only, reverse-proxied by Caddy)."; + }; + }; + + config = lib.mkIf cfg.enable { + # ----------------------------------------------------------------------- + # Secrets + # ----------------------------------------------------------------------- + sops.secrets."grafana/secret_key" = { owner = "grafana"; }; + sops.secrets."openldap/ro_password" = { owner = "root"; }; + + # ----------------------------------------------------------------------- + # Prometheus + # ----------------------------------------------------------------------- + services.prometheus = { + enable = true; + listenAddress = "127.0.0.1"; + port = cfg.prometheusPort; + + globalConfig = { + scrape_interval = "30s"; + evaluation_interval = "30s"; + }; + + # Scrape node and systemd metrics from local exporters + scrapeConfigs = [ + { + job_name = "node"; + static_configs = [{ + targets = [ "127.0.0.1:${toString config.services.prometheus.exporters.node.port}" ]; + }]; + } + { + job_name = "systemd"; + static_configs = [{ + targets = [ "127.0.0.1:${toString config.services.prometheus.exporters.systemd.port}" ]; + }]; + } + ]; + + exporters = { + node = { + enable = true; + port = 9100; + # Enable extra collectors beyond the defaults + enabledCollectors = [ + "cpu" + "diskstats" + "filesystem" + "loadavg" + "meminfo" + "netdev" + "stat" + "time" + "uname" + "pressure" # CPU/memory/IO pressure stall info (Linux PSI) + "hwmon" # temperature sensors (RPi4 has a CPU temp sensor) + ]; + }; + + systemd = { + enable = true; + port = 9558; + }; + }; + }; + + # ----------------------------------------------------------------------- + # Grafana + # ----------------------------------------------------------------------- + services.grafana = { + enable = true; + + settings = { + server = { + http_addr = "127.0.0.1"; + http_port = cfg.grafanaPort; + domain = "grafana.${domain}"; + root_url = "https://grafana.${domain}"; + }; + + # Session signing key — read from sops at runtime via Grafana's + # $__file{} interpolation syntax. + security = { + secret_key = "$__file{${config.sops.secrets."grafana/secret_key".path}}"; + # Disable Grafana's own login form — Authelia is the auth gate, + # and proxy auth auto-signs users in via the X-WEBAUTH-USER header. + disable_initial_admin_creation = false; + }; + + # Proxy auth: trust the X-WEBAUTH-USER header set by Caddy after + # Authelia verifies the user's identity and TOTP. + "auth.proxy" = { + enabled = true; + header_name = "X-WEBAUTH-USER"; + header_property = "username"; + auto_sign_up = true; + # All users that reach Grafana are already confirmed admins + # (Authelia enforces the admins group + two_factor policy). + headers = ""; + }; + + # Disable Grafana's own login UI — all auth goes via Authelia. + # Set to false to keep a fallback login form (useful for recovery). + "auth" = { + disable_login_form = true; + }; + + # Assign all proxy-auth users the Admin role automatically. + # Safe because Authelia already restricts access to the admins group. + users = { + auto_assign_org_role = "Admin"; + }; + + analytics.reporting_enabled = false; + }; + + # ----------------------------------------------------------------------- + # Provision Prometheus as a datasource + # ----------------------------------------------------------------------- + provision = { + enable = true; + + datasources.settings.datasources = [{ + name = "Prometheus"; + type = "prometheus"; + url = "http://127.0.0.1:${toString cfg.prometheusPort}"; + isDefault = true; + access = "proxy"; + }]; + + # Pre-load the Node Exporter Full community dashboard (ID 1860). + # The JSON is downloaded via Nix so it's available at build time. + dashboards.settings.providers = [{ + name = "default"; + options.path = "/etc/grafana/dashboards"; + }]; + }; + }; + + # ----------------------------------------------------------------------- + # Download the Node Exporter Full dashboard JSON at build time. + # + # If the hash is wrong, `nix build` will print the correct one. + # Run: nix store prefetch-file --hash-type sha256 \ + # https://grafana.com/api/dashboards/1860/revisions/37/download + # and replace the hash below. + # ----------------------------------------------------------------------- + environment.etc."grafana/dashboards/node-exporter-full.json" = { + source = pkgs.fetchurl { + url = "https://grafana.com/api/dashboards/1860/revisions/37/download"; + hash = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="; + }; + mode = "0444"; + }; + + # ----------------------------------------------------------------------- + # Uptime Kuma monitor for Grafana + # ----------------------------------------------------------------------- + homey.monitoring.monitors = [{ + name = "Grafana"; + url = "https://grafana.${domain}"; + interval = 60; + }]; + }; +} diff --git a/modules/services/authelia.nix b/modules/services/authelia.nix index 412468c..4e6afa8 100644 --- a/modules/services/authelia.nix +++ b/modules/services/authelia.nix @@ -107,6 +107,27 @@ let - domain: - "jellyfin.${domain}" policy: "one_factor" + - domain: + - "uptime.${domain}" + subject: + - "group:admins" + policy: "two_factor" + - domain: + - "uptime.${domain}" + policy: "deny" + - domain: + - "grafana.${domain}" + subject: + - "group:admins" + policy: "two_factor" + - domain: + - "grafana.${domain}" + policy: "deny" + # ntfy: bypass — ntfy enforces its own token/password auth; + # the mobile app must be able to connect without Authelia SSO. + - domain: + - "ntfy.${domain}" + policy: "bypass" notifier: filesystem: @@ -196,5 +217,14 @@ in after = lib.mkAfter [ "mnt-data.mount" "podman-openldap.service" "podman-homey-network.service" ]; requires = lib.mkAfter [ "mnt-data.mount" "podman-openldap.service" "podman-homey-network.service" ]; }; + + # ----------------------------------------------------------------------- + # Uptime Kuma monitor for this service + # ----------------------------------------------------------------------- + homey.monitoring.monitors = [{ + name = "Authelia"; + url = "https://auth.${domain}/api/health"; + interval = 60; + }]; }; } diff --git a/modules/services/gitea-runner.nix b/modules/services/gitea-runner.nix new file mode 100644 index 0000000..523644f --- /dev/null +++ b/modules/services/gitea-runner.nix @@ -0,0 +1,68 @@ +{ config, lib, pkgs, homeyConfig, ... }: + +# Gitea Actions Runner — executes CI/CD jobs triggered by Gitea Actions. +# +# Uses the NixOS native services.gitea-actions-runner module (act runner). +# Jobs run directly on the host ("host" executor) — no container isolation. +# This is appropriate for a trusted home server and avoids the overhead of +# nested containers on a Pi 4. +# +# The service uses DynamicUser=true so there is no persistent system user. +# nix/git/bash are available in jobs via the system PATH inherited from the +# service environment. +# +# Setup (one-time): +# 1. In Gitea: Site Administration → Actions → Runners → Create Runner Token +# 2. Store the token in sops with KEY=VALUE format: +# gitea/runner_token: "TOKEN=" +# 3. Enable homey.giteaRunner in the host config and deploy. +# +# After first start the runner registers itself and stores credentials in +# /var/lib/gitea-runner//.runner — the token file is only needed for +# (re-)registration. +# +# Secrets consumed from sops: +# gitea/runner_token (must contain: TOKEN=) + +let + cfg = config.homey.giteaRunner; + domain = homeyConfig.domain; +in +{ + options.homey.giteaRunner = { + enable = lib.mkEnableOption "Gitea Actions runner"; + + name = lib.mkOption { + type = lib.types.str; + default = config.networking.hostName; + description = "Runner name as shown in Gitea's runner list."; + }; + + labels = lib.mkOption { + type = lib.types.listOf lib.types.str; + default = [ "native:host" "ubuntu-latest:host" "debian-latest:host" "nix:host" ]; + description = '' + Labels advertised to Gitea. The "host" executor runs jobs directly on + this machine. Workflow files targeting any of these labels will be + picked up by this runner. + ''; + }; + }; + + config = lib.mkIf cfg.enable { + # The NixOS module reads tokenFile as a systemd EnvironmentFile (root reads + # it before DynamicUser privilege drop), so owner=root / mode=0400 is correct. + # The file must contain: TOKEN= + sops.secrets."gitea/runner_token" = { owner = "root"; mode = "0400"; }; + + services.gitea-actions-runner.instances.${cfg.name} = { + enable = true; + url = "https://git.${domain}"; + tokenFile = config.sops.secrets."gitea/runner_token".path; + name = cfg.name; + labels = cfg.labels; + # nix/git/bash are accessible via the system PATH (/run/current-system/sw/bin/) + # without any extra configuration — the runner inherits it as a system user. + }; + }; +} diff --git a/modules/services/gitea.nix b/modules/services/gitea.nix index efa818c..c025ff3 100644 --- a/modules/services/gitea.nix +++ b/modules/services/gitea.nix @@ -143,6 +143,9 @@ in # [oauth2] GITEA__oauth2__ENABLED = "false"; + + # [actions] + GITEA__actions__ENABLED = "true"; }; # Secret env vars written at runtime by ExecStartPre — never in store. @@ -185,6 +188,15 @@ in requires = lib.mkAfter [ "mnt-data.mount" "podman-homey-network.service" ]; }; + # ----------------------------------------------------------------------- + # Uptime Kuma monitor for this service + # ----------------------------------------------------------------------- + homey.monitoring.monitors = [{ + name = "Gitea"; + url = "https://git.${domain}"; + interval = 60; + }]; + # ----------------------------------------------------------------------- # Ensure the Gitea admin user exists with the correct password after start. # Runs as a oneshot after podman-gitea; idempotent (create or update). diff --git a/modules/services/nextcloud.nix b/modules/services/nextcloud.nix index 7ea1ac9..311a3e1 100644 --- a/modules/services/nextcloud.nix +++ b/modules/services/nextcloud.nix @@ -166,6 +166,15 @@ in ]; }; + # ----------------------------------------------------------------------- + # Uptime Kuma monitor for this service + # ----------------------------------------------------------------------- + homey.monitoring.monitors = [{ + name = "Nextcloud"; + url = "https://nextcloud.${domain}/status.php"; + interval = 60; + }]; + systemd.services."podman-nextcloud" = { serviceConfig = { LoadCredential = [ diff --git a/modules/services/ntfy.nix b/modules/services/ntfy.nix new file mode 100644 index 0000000..8a44b9e --- /dev/null +++ b/modules/services/ntfy.nix @@ -0,0 +1,136 @@ +{ config, lib, pkgs, homeyConfig, ... }: + +# Ntfy — self-hosted push notification server. +# +# Mobile app (Android/iOS) connects to https://ntfy.zakobar.com with a token +# and subscribes to the "alerts" topic. Uptime Kuma and Grafana send alerts +# to that topic when services go down. +# +# Auth model: +# - Web UI: public-facing but ntfy enforces its own auth (deny-all by default) +# - Caddy does NOT put forward_auth here; ntfy has native token/password auth +# so the mobile app can connect without Authelia SSO complications. +# +# Setup after first deploy: +# 1. Visit https://ntfy.zakobar.com — log in with the admin password from sops. +# 2. Create an access token for your phone (Admin → Users & Tokens). +# 3. In the Ntfy app: server = https://ntfy.zakobar.com, token = . +# 4. Subscribe to the "alerts" topic. +# +# Volume layout: +# /ntfy/auth.db ← user/token database +# /ntfy/cache.db ← message cache (for missed messages) +# /ntfy/attachments/ ← file attachments +# +# Secrets consumed from sops: +# ntfy/admin_password + +let + cfg = config.homey.ntfy; + dataDir = config.homey.storage.mountPoint; + domain = homeyConfig.domain; +in +{ + options.homey.ntfy = { + enable = lib.mkEnableOption "Ntfy push notification server"; + + port = lib.mkOption { + type = lib.types.port; + default = 2586; + description = "Host port ntfy listens on (bound to 127.0.0.1)."; + }; + }; + + config = lib.mkIf cfg.enable { + # ----------------------------------------------------------------------- + # Secrets + # ----------------------------------------------------------------------- + sops.secrets."ntfy/admin_password" = { owner = "root"; }; + + # ----------------------------------------------------------------------- + # ntfy-sh native NixOS service + # ----------------------------------------------------------------------- + services.ntfy-sh = { + enable = true; + settings = { + # Bind to localhost; Caddy reverse-proxies it + listen-http = "127.0.0.1:${toString cfg.port}"; + base-url = "https://ntfy.${domain}"; + + # Require auth on all topics — deny unauthenticated access entirely + auth-default-access = "deny-all"; + + # Persistent state on external HD + auth-file = "${dataDir}/ntfy/auth.db"; + cache-file = "${dataDir}/ntfy/cache.db"; + attachment-root = "${dataDir}/ntfy/attachments"; + + # Keep messages for 12 hours so the app catches up if offline + cache-duration = "12h"; + + # Attachment limits + attachment-total-size-limit = "5G"; + attachment-file-size-limit = "15M"; + attachment-expiry-duration = "3h"; + }; + }; + + # ----------------------------------------------------------------------- + # Create the admin user on first start (idempotent) + # ----------------------------------------------------------------------- + systemd.services.ntfy-sh-setup = { + description = "Create Ntfy admin user"; + wantedBy = [ "multi-user.target" ]; + after = [ "ntfy-sh.service" "mnt-data.mount" ]; + requires = [ "ntfy-sh.service" ]; + + serviceConfig = { + Type = "oneshot"; + RemainAfterExit = true; + LoadCredential = "ntfy_admin_password:${config.sops.secrets."ntfy/admin_password".path}"; + + ExecStart = pkgs.writeShellScript "ntfy-create-admin" '' + set -euo pipefail + + # Wait until ntfy HTTP endpoint is ready (max 60 s) + for i in $(seq 1 30); do + if ${pkgs.curl}/bin/curl -sf http://127.0.0.1:${toString cfg.port}/v1/health > /dev/null 2>&1; then + break + fi + sleep 2 + done + + PASS=$(cat "$CREDENTIALS_DIRECTORY/ntfy_admin_password") + + # ntfy user commands need the config file to find the auth database. + # The NixOS ntfy-sh module writes config to /etc/ntfy-sh/server.yml. + NTFY="${pkgs.ntfy-sh}/bin/ntfy user --config /etc/ntfy-sh/server.yml" + + # ntfy user list exits non-zero if the user DB is empty/doesn't exist; + # grep exits non-zero if the pattern is missing. Either means no admin. + if $NTFY list 2>/dev/null | grep -qE "^admin\b"; then + echo "ntfy-sh-setup: admin user already exists" + else + echo "$PASS" | $NTFY add --role=admin admin + echo "ntfy-sh-setup: admin user created" + fi + ''; + }; + }; + + # Ensure ntfy-sh starts after the external HD is mounted + systemd.services.ntfy-sh = { + after = lib.mkAfter [ "mnt-data.mount" ]; + requires = lib.mkAfter [ "mnt-data.mount" ]; + }; + + # ----------------------------------------------------------------------- + # Uptime Kuma monitor for this service + # ----------------------------------------------------------------------- + homey.monitoring.monitors = [{ + name = "Ntfy"; + url = "https://ntfy.${domain}/v1/health"; + interval = 60; + }]; + }; +} diff --git a/modules/services/phpldapadmin.nix b/modules/services/phpldapadmin.nix index 2fe0fc1..12776e9 100644 --- a/modules/services/phpldapadmin.nix +++ b/modules/services/phpldapadmin.nix @@ -49,5 +49,14 @@ in after = lib.mkAfter [ "podman-openldap.service" "podman-homey-network.service" ]; wants = lib.mkAfter [ "podman-openldap.service" "podman-homey-network.service" ]; }; + + # ----------------------------------------------------------------------- + # Uptime Kuma monitor for this service + # ----------------------------------------------------------------------- + homey.monitoring.monitors = [{ + name = "phpLDAPadmin"; + url = "http://localhost:${toString cfg.port}"; + interval = 60; + }]; }; } diff --git a/modules/services/uptime-kuma.nix b/modules/services/uptime-kuma.nix new file mode 100644 index 0000000..4d58ae1 --- /dev/null +++ b/modules/services/uptime-kuma.nix @@ -0,0 +1,259 @@ +{ config, lib, pkgs, homeyConfig, ... }: + +# Uptime Kuma — endpoint uptime monitoring with a status-page UI. +# +# This module does two things: +# +# 1. Declares the shared homey.monitoring.monitors option that any service +# module can contribute to. Adding your service's URL there means it +# automatically appears in Uptime Kuma — no manual UI work needed. +# +# 2. Runs Uptime Kuma as an OCI container and syncs the monitor list via +# the Socket.IO API on startup using the uptime-kuma-api Python library. +# +# Example (in nextcloud.nix): +# homey.monitoring.monitors = [{ +# name = "Nextcloud"; +# url = "https://nextcloud.zakobar.com/status.php"; +# interval = 60; +# }]; +# +# Auth: Authelia two_factor, admins-only (enforced in authelia.nix + caddy.nix). +# +# Volume layout: +# /uptime-kuma/ → /app/data (SQLite DB, config) +# +# Secrets consumed from sops: +# uptime-kuma/admin_password + +let + cfg = config.homey.uptimeKuma; + dataDir = config.homey.storage.mountPoint; + domain = homeyConfig.domain; + + # Serialise the NixOS monitor list to JSON at build time. + # The setup script reads this at runtime to know what to create. + monitorsJson = pkgs.writeText "uptime-kuma-monitors.json" + (builtins.toJSON config.homey.monitoring.monitors); + + # Python environment for the monitor-sync script + pythonEnv = pkgs.python3.withPackages (ps: [ ps."uptime-kuma-api" ]); + + # Monitor-sync script: idempotent, hash-gated, uses Socket.IO API + syncScript = pkgs.writeText "uptime-kuma-sync.py" '' + #!/usr/bin/env python3 + """ + Sync monitors declared in /etc/uptime-kuma/monitors.json into Uptime Kuma. + + Runs as a oneshot systemd service after podman-uptime-kuma.service. + Tracks a hash of the monitor list so it only re-syncs when the NixOS + config changes. + """ + import hashlib + import json + import os + import sys + import time + import urllib.request + + MONITORS_PATH = "/etc/uptime-kuma/monitors.json" + HASH_PATH = "/var/lib/uptime-kuma-setup/last-hash" + KUMA_URL = "http://localhost:3001" + CREDS_DIR = os.environ.get("CREDENTIALS_DIRECTORY", "") + + def wait_for_kuma(timeout=120): + deadline = time.time() + timeout + while time.time() < deadline: + try: + with urllib.request.urlopen(KUMA_URL + "/", timeout=5) as r: + if r.status < 500: + return True + except Exception: + pass + time.sleep(3) + return False + + def main(): + with open(MONITORS_PATH) as f: + monitors = json.load(f) + + config_hash = hashlib.sha256( + json.dumps(monitors, sort_keys=True).encode() + ).hexdigest() + + # Skip sync if config hasn't changed + try: + with open(HASH_PATH) as f: + if f.read().strip() == config_hash: + print("uptime-kuma-sync: config unchanged, skipping") + return + except FileNotFoundError: + pass + + password_file = os.path.join(CREDS_DIR, "uptime_kuma_password") + with open(password_file) as f: + password = f.read().strip() + + print("uptime-kuma-sync: waiting for Uptime Kuma to be ready...") + if not wait_for_kuma(): + print("uptime-kuma-sync: timed out waiting for Uptime Kuma", file=sys.stderr) + sys.exit(1) + + from uptime_kuma_api import UptimeKumaApi, MonitorType + + api = UptimeKumaApi(KUMA_URL) + + # Initial setup (creates admin user on first run; no-op if already done) + try: + info = api.info() + if not info.get("isSetup", True): + api.setup("admin", password) + print("uptime-kuma-sync: initial admin user created") + except Exception as e: + print(f"uptime-kuma-sync: setup check: {e}", file=sys.stderr) + + # Login + result = api.login("admin", password) + if not result.get("ok"): + print(f"uptime-kuma-sync: login failed: {result}", file=sys.stderr) + api.disconnect() + sys.exit(1) + + # Sync monitors (add missing; skip existing by name) + try: + existing_names = {m["name"] for m in api.get_monitors()} + for m in monitors: + if m["name"] in existing_names: + print(f"uptime-kuma-sync: monitor exists, skipping: {m['name']}") + continue + api.add_monitor( + type=MonitorType.HTTP, + name=m["name"], + url=m["url"], + interval=m.get("interval", 60), + ) + print(f"uptime-kuma-sync: created monitor: {m['name']}") + finally: + api.disconnect() + + # Persist hash so we don't re-sync on every boot + os.makedirs(os.path.dirname(HASH_PATH), exist_ok=True) + with open(HASH_PATH, "w") as f: + f.write(config_hash) + print("uptime-kuma-sync: done") + + if __name__ == "__main__": + main() + ''; + +in +{ + # --------------------------------------------------------------------------- + # Shared monitor-list option — declared unconditionally so any service module + # can contribute monitors even when Uptime Kuma itself is disabled. + # --------------------------------------------------------------------------- + options.homey.monitoring.monitors = lib.mkOption { + type = lib.types.listOf (lib.types.submodule { + options = { + name = lib.mkOption { + type = lib.types.str; + description = "Display name shown in Uptime Kuma."; + }; + url = lib.mkOption { + type = lib.types.str; + description = "URL to check (HTTP/HTTPS)."; + }; + interval = lib.mkOption { + type = lib.types.int; + default = 60; + description = "Check interval in seconds."; + }; + }; + }); + default = []; + description = '' + List of HTTP endpoints to monitor in Uptime Kuma. + Each service module contributes its own entries here. + ''; + }; + + options.homey.uptimeKuma = { + enable = lib.mkEnableOption "Uptime Kuma uptime monitoring"; + + image = lib.mkOption { + type = lib.types.str; + default = "docker.io/louislam/uptime-kuma:1"; + }; + + port = lib.mkOption { + type = lib.types.port; + default = 3001; + description = "Host port Uptime Kuma listens on (bound to 127.0.0.1)."; + }; + }; + + config = lib.mkIf cfg.enable { + # ----------------------------------------------------------------------- + # Secrets + # ----------------------------------------------------------------------- + sops.secrets."uptime-kuma/admin_password" = { owner = "root"; }; + + # ----------------------------------------------------------------------- + # Write monitor list to /etc at build time + # ----------------------------------------------------------------------- + environment.etc."uptime-kuma/monitors.json" = { + source = monitorsJson; + mode = "0444"; + }; + + # ----------------------------------------------------------------------- + # Uptime Kuma container + # ----------------------------------------------------------------------- + virtualisation.oci-containers.containers.uptime-kuma = { + image = cfg.image; + ports = [ "127.0.0.1:${toString cfg.port}:3001" ]; + + volumes = [ + "${dataDir}/uptime-kuma:/app/data" + ]; + + # uptime-kuma image expects /app/data to be writable; no extra network + # needed since we reach it from the host on localhost. + }; + + systemd.services."podman-uptime-kuma" = { + after = lib.mkAfter [ "mnt-data.mount" ]; + requires = lib.mkAfter [ "mnt-data.mount" ]; + }; + + # ----------------------------------------------------------------------- + # Monitor-sync service: runs after Uptime Kuma is up, syncs monitors + # ----------------------------------------------------------------------- + systemd.services."uptime-kuma-sync" = { + description = "Sync Uptime Kuma monitors from NixOS config"; + wantedBy = [ "multi-user.target" ]; + after = [ "podman-uptime-kuma.service" ]; + requires = [ "podman-uptime-kuma.service" ]; + + serviceConfig = { + Type = "oneshot"; + RemainAfterExit = true; + LoadCredential = "uptime_kuma_password:${config.sops.secrets."uptime-kuma/admin_password".path}"; + + ExecStart = pkgs.writeShellScript "uptime-kuma-sync-runner" '' + set -euo pipefail + exec ${pythonEnv}/bin/python3 ${syncScript} + ''; + }; + }; + + # ----------------------------------------------------------------------- + # Uptime Kuma self-monitor + # ----------------------------------------------------------------------- + homey.monitoring.monitors = [{ + name = "Uptime Kuma"; + url = "https://uptime.${domain}"; + interval = 60; + }]; + }; +} diff --git a/modules/storage.nix b/modules/storage.nix index e939b3c..7fe3dbc 100644 --- a/modules/storage.nix +++ b/modules/storage.nix @@ -29,6 +29,11 @@ # complete/ # transmission/ # config/ +# uptime-kuma/ ← /app/data in uptime-kuma container (SQLite DB, config) +# ntfy/ +# auth.db ← user/token auth database +# cache.db ← message cache +# attachments/ ← file attachments # restic-cache/ ← restic local cache (not the backup destination) let @@ -102,6 +107,9 @@ in "d ${cfg.mountPoint}/media/complete 0755 root root -" "d ${cfg.mountPoint}/transmission 0750 root root -" "d ${cfg.mountPoint}/transmission/config 0750 root root -" + "d ${cfg.mountPoint}/uptime-kuma 0750 root root -" + "d ${cfg.mountPoint}/ntfy 0750 ntfy-sh ntfy-sh -" + "d ${cfg.mountPoint}/ntfy/attachments 0750 ntfy-sh ntfy-sh -" "d ${cfg.mountPoint}/restic-cache 0700 root root -" ]; }; diff --git a/secrets/secrets.yaml b/secrets/secrets.yaml index fcceb6a..81ecf05 100644 --- a/secrets/secrets.yaml +++ b/secrets/secrets.yaml @@ -1,3 +1,9 @@ +uptime-kuma: + admin_password: ENC[AES256_GCM,data:tPKWxWmxRcVJeywY3J4eXAWWnAinLwMn3X68TrV/4emonvRiuyPmiwhn2fjDxwB/kT78y/iDDmpdQY229yJrkQ==,iv:YSL40PDbRTgtSYZCwqHzfJTcEAiILIDbGRA2kfamiw8=,tag:pMM0AWkjkcS9XOaSHG1oUQ==,type:str] +ntfy: + admin_password: ENC[AES256_GCM,data:P5pjnt00lyeGVlrBvUlJWWeTi3evFZPJIxjcsndbo4LZOLk6hbbrh8RwCAGzr1ump0A5fRXqynByRFdaS6++wA==,iv:Uxeh0/mygR++4S//O/RO2bouH2J0qcSCYtjjyZNooNk=,tag:LGIDaq4RzBuzrWFqVDr8ow==,type:str] +grafana: + secret_key: ENC[AES256_GCM,data:/KNDMZZN5thoqsgJZS7fuNQULI1PAKVuihRu9WzO00Qw8js/V4KKJT0JOVOcqdHAnf44+szYZaCWt0xe02chGw==,iv:Y0FQ7h4SqZVtz0wLjPnVGGYyXmBIDi8nzaK2GFzDxqQ=,tag:w0z5/vI3Hfd8ry9DCHAvJw==,type:str] openldap: admin_password: ENC[AES256_GCM,data:hg+Ly1bX4ao1AT4SDvQWXiT/KMzsz0wdnRauiB+FetE=,iv:TAX+NZCVUNiwMeBrW58IeI1OJX6rzzGAhWiQ+cZXreo=,tag:MrwYKKBb1Cg2JvADtQqYrQ==,type:str] config_password: ENC[AES256_GCM,data:qKEurb0slGnr6nES7w7fTPDCy/DARns0BorDZMwpI/w=,iv:+p6Fh9a2g0eBueOxDk1J+hnM9fMgE6/NYwz+sAovGjE=,tag:kKZVsxdxdDACD9J0NAf4gQ==,type:str] @@ -11,6 +17,7 @@ gitea: lfs_jwt_secret: ENC[AES256_GCM,data:i05gr2ou03w0yu6/bhlJOW1huysAAPTidFEusWkhQfpDj4Pyh8LEKb09Og==,iv:aqkblyz0oIFHwzVCzlGDdQuCbsDPrfBaJMzgRTw+pYU=,tag:6gBSerOUK8Y3la/2Bg2AZQ==,type:str] oauth2_jwt_secret: ENC[AES256_GCM,data:BVvQJCEfHPbemd1jz7MWpIRia1wfvPMGuLqoi/xUMSoQoN5RPefQnPR4Cg==,iv:JAZQUTxHZSnMEnl+BIZ1PXlznMwKuPtiPP/17rc6lSs=,tag:mUw5RuthZmZegXCtfsFNmQ==,type:str] internal_token: ENC[AES256_GCM,data:gnOebJbRsh2Cues9WjGQp4rWa6OuE3xSnby9jc3Hk8ywvpL7CNWmlGW7zmmOyDAfIKfm8kf1FxotWLXtGZDretzdbMRM9c6gkwSJf5MCsdm27Er+IRKS/QFBuvLSTEH0,iv:aVgRvs3T3zCg+AV/BKUXQyZDKvunHvXsdfr9sqo2cI0=,tag:U9aP+N0CWyRQ/xJ27Vo2mw==,type:str] + runner_token: ENC[AES256_GCM,data:fNiP3hIhBw16zYAt9dMuGu6C3n48R6H4O8en8JzRnNy0KGbbvv08w8qRceD6XQ==,iv:DJarsN6yYbdyesd5MoQEB0mDdS9O39VLKmJUIicTlG8=,tag:8+W6jYg8kSqy6FztaJnn9w==,type:str] nextcloud: admin_password: ENC[AES256_GCM,data:iK6VoE94vFQmn3i4XQc5r/c03u3b0knDgBNK8d1qyns=,iv:P1wax2vAjn9iwBe9T7SN+pKrtrWcOYb5OWUyHF4hlVg=,tag:ET8KU4IKzhWqIDeRihwcag==,type:str] postgres_password: ENC[AES256_GCM,data:ga4cwhYsAgEBvr+aDVwiRZXeT+TjXzeef1r3ud6uYHs=,iv:PMHCjO4wLW6PER4oGODEG9CHqrvVpAbgTGF7p49MCL0=,tag:mTNzsDhufqLlf1LFu7Rl1A==,type:str] @@ -34,8 +41,8 @@ sops: QXVkRlJHeW52NFZFYnVwaW8ycytDSzAKZt+p5QnZKcEOBghHA2xkH6d7NObtTEoE wMwCYasnBHzy2unXRbZq/4v9NQ5HJd0Nu1iqbqKgIxMCD3dnxEdK7g== -----END AGE ENCRYPTED FILE----- - lastmodified: "2026-04-21T12:42:15Z" - mac: ENC[AES256_GCM,data:fNip/7A7iKCVZqP0EziyBG7K8SVfRJTBpn4RcDLOaciJHx5DkLLszE8we9MmzpKXQIiMcJl2BTj/uqJrgc5EHTSOHwRzNJ4s2NJfvQW+8QUDfTGzKOkP3L837RkEPzH4HZLqGlfYK7cNJU5qXRPbusKjAft7Fz3+ONXmodb/ONY=,iv:CdSs1a+74+MfzWyML2JQ/b2IKbktVdefFFYP5LOtUos=,tag:ikr9LwPnmdiPucOoBt3/Bw==,type:str] + lastmodified: "2026-05-10T08:30:12Z" + mac: ENC[AES256_GCM,data:4nZyQLRlIWmgawj7YgL9DZAbbA/bQrOpfmFPlwaoPJR16eOQOHzxLLYrM6iyJxhYyt9d6iaFpFO9g9KOvOCDCG9s8/EImrXg0WDYIJn7D+ftGc0Qj5augBqeJEh9DgfDoWiXAYrCR3lUfDAswaSAPAjf1NzuStlM9X0SNyW+1Ug=,iv:mBq1xiTbjMX834yTnBR6+/IP8vZTYS8UB3v1z0wEc8s=,tag:BQWlnIj636Lv6a2CZjcV0w==,type:str] pgp: - created_at: "2026-04-21T06:39:49Z" enc: |- diff --git a/shells/defaultShell.nix b/shells/defaultShell.nix index bc45a12..f4bd164 100644 --- a/shells/defaultShell.nix +++ b/shells/defaultShell.nix @@ -18,5 +18,27 @@ pkgs.mkShell { scp scripts/offload-backup.sh admin@192.168.1.100:/tmp/homey-offload-backup.sh ssh -t admin@192.168.1.100 'sudo bash /tmp/homey-offload-backup.sh; rm /tmp/homey-offload-backup.sh' '') + (pkgs.writeShellScriptBin "homey-backup-status" '' + ssh admin@192.168.1.100 bash -s <<'ENDSSH' + echo "=== Backup timer ===" + systemctl status restic-backups-homey.timer --no-pager -l 2>&1 || true + + echo "" + echo "=== Last backup run (journal) ===" + journalctl -u restic-backups-homey.service -n 50 --no-pager 2>&1 || true + + echo "" + echo "=== Recent snapshots ===" + sudo bash -c ' + export AWS_ACCESS_KEY_ID=$(cat /run/secrets/restic/s3_access_key_id) + export AWS_SECRET_ACCESS_KEY=$(cat /run/secrets/restic/s3_secret_access_key) + export RESTIC_CACHE_DIR=/mnt/data/restic-cache + restic \ + -r "s3:https://s3.us-east-005.backblazeb2.com/zakobar-home-backup" \ + --password-file /run/secrets/restic/password \ + snapshots --latest 5 + ' 2>&1 + ENDSSH + '') ]; }