Port to NixOS: replace Helm chart with flake-based NixOS config
Replaces the Helm/k3s setup with a declarative NixOS configuration targeting
a Raspberry Pi 4. Services run as podman containers under systemd, with data
on an external HD at /mnt/data. Key components:
- flake.nix: multi-host flake with pi-main (aarch64) and a placeholder for a
second machine
- modules/common.nix: shared system config (nix, podman, sops, SSH)
- modules/storage.nix: external HD mount with per-service subdirs
- modules/caddy.nix: Caddy with cloudflare DNS-01 ACME + authelia forward_auth
- modules/cloudflared.nix: Cloudflare tunnel for remote access
- modules/backup.nix: restic daily backups with NC maintenance mode pre-hook
- modules/services/{openldap,authelia,gitea,nextcloud,phpldapadmin}.nix: core services
- modules/services/{jellyfin,transmission}.nix: media services (disabled by default)
- secrets/: sops-nix scaffold with .sops.yaml age key config
- hosts/pi-main/: hardware config + service selection for the Pi
- PORTING.md: step-by-step migration guide (SD card → data restore → verify)
This commit is contained in:
Aner Zakobar
+400
@@ -0,0 +1,400 @@
|
||||
# Porting Guide — Helm/k3s → NixOS
|
||||
|
||||
This document walks through setting up the new NixOS-based home server from
|
||||
scratch on a Raspberry Pi 4, restoring data from old Longhorn volumes, and
|
||||
verifying each service.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites (on your workstation)
|
||||
|
||||
- `nix` with flakes enabled (`~/.config/nix/nix.conf`: `experimental-features = nix-command flakes`)
|
||||
- `sops` + `age` CLI tools (`nix-shell -p sops age`)
|
||||
- An SSH key pair
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — Secrets
|
||||
|
||||
### 0.1 Generate your age key (workstation)
|
||||
|
||||
```bash
|
||||
age-keygen -o ~/.config/sops/age/keys.txt
|
||||
age-keygen -y ~/.config/sops/age/keys.txt # print public key
|
||||
```
|
||||
|
||||
You will add the Pi's public key in step 2.2; for now add your workstation
|
||||
public key so you can edit the secrets file offline.
|
||||
|
||||
Edit `secrets/.sops.yaml`, replace the placeholder with your workstation pubkey:
|
||||
|
||||
```yaml
|
||||
- age:
|
||||
- AGE-PUBLIC-KEY-YOUR-WORKSTATION # ← paste here
|
||||
```
|
||||
|
||||
### 0.2 Fill in secrets.yaml
|
||||
|
||||
`secrets/secrets.yaml` is a **plaintext template** — do not commit it until
|
||||
encrypted. Fill in all values:
|
||||
|
||||
| Key | Source |
|
||||
|-----|--------|
|
||||
| `openldap/admin_password` | From old k8s secret `openldap-admin` |
|
||||
| `openldap/config_password` | From old k8s secret `openldap-config` |
|
||||
| `openldap/ro_password` | From old k8s secret `openldap-ro` |
|
||||
| `gitea/admin_password` | From old k8s secret `gitea-admin-pass` |
|
||||
| `nextcloud/admin_password` | From old k8s secret `nextcloud-admin-pass` |
|
||||
| `nextcloud/postgres_password` | From old k8s secret `nextcloud-postgres-pass` |
|
||||
| `authelia/jwt_secret` | Generate: `openssl rand -hex 64` |
|
||||
| `authelia/session_secret` | Generate: `openssl rand -hex 64` |
|
||||
| `authelia/storage_encryption_key` | Generate: `openssl rand -hex 64` |
|
||||
| `gitea/lfs_jwt_secret` | Generate: `openssl rand -base64 32 \| tr -d '='` |
|
||||
| `gitea/oauth2_jwt_secret` | Generate: `openssl rand -base64 32 \| tr -d '='` |
|
||||
| `gitea/internal_token` | Generate: `openssl rand -base64 75 \| tr -d '\n='` |
|
||||
| `cloudflare/api_token` | Cloudflare dashboard → API Tokens → DNS:Edit |
|
||||
| `cloudflare/tunnel_token` | Created in Phase 3 (Cloudflare setup) |
|
||||
| `restic/password` | Generate: `openssl rand -base64 32` |
|
||||
|
||||
To get old k8s secrets (if the cluster is still running):
|
||||
|
||||
```bash
|
||||
kubectl get secret openldap-admin -n homey -o jsonpath='{.data.password}' | base64 -d
|
||||
kubectl get secret openldap-config -n homey -o jsonpath='{.data.password}' | base64 -d
|
||||
kubectl get secret openldap-ro -n homey -o jsonpath='{.data.password}' | base64 -d
|
||||
kubectl get secret gitea-admin-pass -n homey -o jsonpath='{.data.password}' | base64 -d
|
||||
kubectl get secret nextcloud-admin-pass -n homey -o jsonpath='{.data.password}' | base64 -d
|
||||
kubectl get secret nextcloud-postgres-pass -n homey -o jsonpath='{.data.password}' | base64 -d
|
||||
```
|
||||
|
||||
### 0.3 Encrypt secrets.yaml (workstation, before committing)
|
||||
|
||||
```bash
|
||||
sops --encrypt --in-place secrets/secrets.yaml
|
||||
git add secrets/secrets.yaml secrets/.sops.yaml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Install NixOS on the Raspberry Pi 4
|
||||
|
||||
### 1.1 Flash the SD card
|
||||
|
||||
Download the NixOS aarch64 SD card image:
|
||||
|
||||
```
|
||||
https://nixos.org/download#nixos-iso
|
||||
→ "Raspberry Pi (aarch64) SD card image"
|
||||
```
|
||||
|
||||
Flash with:
|
||||
|
||||
```bash
|
||||
# macOS
|
||||
sudo dd if=nixos-*-aarch64-linux.img of=/dev/rdiskN bs=4m status=progress
|
||||
|
||||
# Linux
|
||||
sudo dd if=nixos-*-aarch64-linux.img of=/dev/sdX bs=4M status=progress conv=fsync
|
||||
```
|
||||
|
||||
Label the partitions to match `hardware.nix`:
|
||||
|
||||
```bash
|
||||
# After flashing, mount the root partition and relabel if needed:
|
||||
sudo e2label /dev/sdX2 NIXOS_SD
|
||||
sudo fatlabel /dev/sdX1 FIRMWARE
|
||||
```
|
||||
|
||||
Boot the Pi from the SD card. You should get a serial console or HDMI output.
|
||||
|
||||
### 1.2 Initial network setup
|
||||
|
||||
On the Pi (serial or HDMI):
|
||||
|
||||
```bash
|
||||
# Find your IP
|
||||
ip addr
|
||||
|
||||
# Set a temporary password for nixos user to SSH in
|
||||
passwd nixos
|
||||
```
|
||||
|
||||
From your workstation:
|
||||
|
||||
```bash
|
||||
ssh nixos@<pi-ip>
|
||||
```
|
||||
|
||||
### 1.3 Copy the flake to the Pi
|
||||
|
||||
```bash
|
||||
# From your workstation (repo root)
|
||||
rsync -avz --exclude='.git' . nixos@<pi-ip>:/tmp/homey/
|
||||
```
|
||||
|
||||
### 1.4 Generate the Pi's age key
|
||||
|
||||
```bash
|
||||
# On the Pi
|
||||
nix-shell -p age --run 'age-keygen -o /tmp/pi-age-key.txt'
|
||||
age-keygen -y /tmp/pi-age-key.txt # print public key
|
||||
```
|
||||
|
||||
Copy the public key back to your workstation. Add it to `secrets/.sops.yaml`:
|
||||
|
||||
```yaml
|
||||
- age:
|
||||
- AGE-PUBLIC-KEY-YOUR-WORKSTATION
|
||||
- AGE-PUBLIC-KEY-PI-MAIN-REPLACE-ME # ← paste Pi's public key here
|
||||
```
|
||||
|
||||
Re-encrypt secrets so the Pi can decrypt them:
|
||||
|
||||
```bash
|
||||
# On workstation
|
||||
sops updatekeys secrets/secrets.yaml
|
||||
git add secrets/.sops.yaml secrets/secrets.yaml
|
||||
git commit -m "Add Pi age key"
|
||||
|
||||
# Copy updated files to Pi
|
||||
rsync -avz secrets/ nixos@<pi-ip>:/tmp/homey/secrets/
|
||||
```
|
||||
|
||||
Place the Pi's private key where sops-nix expects it:
|
||||
|
||||
```bash
|
||||
# On the Pi
|
||||
sudo mkdir -p /var/lib/sops-nix
|
||||
sudo cp /tmp/pi-age-key.txt /var/lib/sops-nix/key.txt
|
||||
sudo chmod 600 /var/lib/sops-nix/key.txt
|
||||
```
|
||||
|
||||
### 1.5 Configure host-specific settings
|
||||
|
||||
Edit `hosts/pi-main/default.nix` on the Pi (or on workstation first):
|
||||
|
||||
1. Set your SSH public key in `users.users.admin.openssh.authorizedKeys.keys`
|
||||
2. Set `homey.storage.device` to your USB drive:
|
||||
```bash
|
||||
ls -la /dev/disk/by-id/ | grep -v part
|
||||
```
|
||||
3. Set `homey.backup.repository` to your backup destination
|
||||
|
||||
Edit `hosts/pi-main/hardware.nix` if the disk labels differ from defaults.
|
||||
|
||||
### 1.6 Install NixOS
|
||||
|
||||
```bash
|
||||
# On the Pi
|
||||
sudo nixos-install --flake /tmp/homey#pi-main --no-root-passwd
|
||||
|
||||
# Reboot
|
||||
sudo reboot
|
||||
```
|
||||
|
||||
After reboot, SSH in with your admin key:
|
||||
|
||||
```bash
|
||||
ssh admin@<pi-ip>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Restore Data from Old Volumes
|
||||
|
||||
Mount the external HD (if not auto-mounted):
|
||||
|
||||
```bash
|
||||
sudo mount /dev/disk/by-id/<your-drive-id> /mnt/data
|
||||
```
|
||||
|
||||
Copy data from the old Longhorn volume backups into the new layout:
|
||||
|
||||
```bash
|
||||
# Adjust source paths to wherever your Longhorn volume dumps are
|
||||
BACKUP_SRC=/path/to/longhorn/backups
|
||||
|
||||
# OpenLDAP
|
||||
sudo rsync -av $BACKUP_SRC/openldap/etc-ldap-slapd.d/ /mnt/data/openldap/etc-ldap-slapd.d/
|
||||
sudo rsync -av $BACKUP_SRC/openldap/var-lib-ldap/ /mnt/data/openldap/var-lib-ldap/
|
||||
|
||||
# Gitea
|
||||
sudo rsync -av $BACKUP_SRC/gitea/data/ /mnt/data/gitea/data/
|
||||
|
||||
# Nextcloud
|
||||
sudo rsync -av $BACKUP_SRC/nextcloud/html/ /mnt/data/nextcloud/html/
|
||||
# Restore postgres from pg_dump if available, otherwise restore the data dir:
|
||||
sudo rsync -av $BACKUP_SRC/nextcloud/db/ /mnt/data/nextcloud/db/
|
||||
```
|
||||
|
||||
Fix ownership (containers run as UID 1000 or root depending on image):
|
||||
|
||||
```bash
|
||||
# openldap runs as root inside the container
|
||||
sudo chown -R root:root /mnt/data/openldap/
|
||||
|
||||
# gitea runs as git (UID 1000)
|
||||
sudo chown -R 1000:1000 /mnt/data/gitea/
|
||||
|
||||
# nextcloud runs as www-data (UID 33)
|
||||
sudo chown -R 33:33 /mnt/data/nextcloud/html/
|
||||
# postgres data owned by postgres (UID 999 in the postgres image)
|
||||
sudo chown -R 999:999 /mnt/data/nextcloud/db/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Cloudflare Tunnel Setup
|
||||
|
||||
### 3.1 Create the tunnel in Cloudflare Zero Trust
|
||||
|
||||
1. Go to [https://one.dash.cloudflare.com](https://one.dash.cloudflare.com) → Networks → Tunnels
|
||||
2. Click "Create a tunnel" → Cloudflared → Name it `pi-main`
|
||||
3. Copy the tunnel token (long string starting with `eyJ...`)
|
||||
4. Add it to `secrets/secrets.yaml` under `cloudflare/tunnel_token`
|
||||
5. Re-encrypt: `sops secrets/secrets.yaml` (the file opens in `$EDITOR`)
|
||||
|
||||
### 3.2 Configure public hostnames in the Cloudflare dashboard
|
||||
|
||||
In the tunnel's "Public Hostnames" tab, add:
|
||||
|
||||
| Subdomain | Domain | Service |
|
||||
|-----------|--------|---------|
|
||||
| `auth` | `home.zakobar.com` | `https://localhost:443` |
|
||||
| `git` | `home.zakobar.com` | `https://localhost:443` |
|
||||
| `nextcloud` | `home.zakobar.com` | `https://localhost:443` |
|
||||
| `ldapadmin` | `home.zakobar.com` | `https://localhost:443` |
|
||||
| `jellyfin` | `home.zakobar.com` | `https://localhost:443` |
|
||||
| `torrent` | `home.zakobar.com` | `https://localhost:443` |
|
||||
|
||||
For each entry, under "Additional settings" → TLS → **No TLS Verify: ON**
|
||||
(because cloudflared connects to `localhost` but the cert is for the real hostname).
|
||||
|
||||
### 3.3 Update DNS in Cloudflare
|
||||
|
||||
Add a CNAME for `home.zakobar.com` pointing to your tunnel's UUID (Cloudflare
|
||||
creates this automatically when you add hostnames). You do not need to add
|
||||
`home.zakobar.com` to your domain's A records — Cloudflare handles it.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — Rebuild and Verify
|
||||
|
||||
After restoring data and completing Cloudflare setup, apply the final config:
|
||||
|
||||
```bash
|
||||
# On the Pi
|
||||
sudo nixos-rebuild switch --flake /path/to/homey#pi-main
|
||||
```
|
||||
|
||||
### Verification checklist
|
||||
|
||||
```bash
|
||||
# All container services running?
|
||||
systemctl list-units 'podman-*' --state=active
|
||||
|
||||
# OpenLDAP responding?
|
||||
ldapsearch -x -H ldap://127.0.0.1:389 -b dc=home,dc=zakobar,dc=com -D "cn=admin,dc=home,dc=zakobar,dc=com" -W
|
||||
|
||||
# Authelia health?
|
||||
curl -s http://localhost:9091/api/health | python3 -m json.tool
|
||||
|
||||
# Caddy serving TLS?
|
||||
curl -I https://auth.home.zakobar.com
|
||||
|
||||
# Gitea login?
|
||||
# Visit https://git.home.zakobar.com — should redirect to authelia if not logged in
|
||||
|
||||
# Nextcloud?
|
||||
# Visit https://nextcloud.home.zakobar.com
|
||||
|
||||
# Cloudflare tunnel connected?
|
||||
systemctl status cloudflared-tunnel-pi-main
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — Local DNS (optional but recommended)
|
||||
|
||||
To access services without going through Cloudflare on the LAN, add these
|
||||
records to your router's DNS or Pi-hole:
|
||||
|
||||
```
|
||||
192.168.1.100 home.zakobar.com
|
||||
192.168.1.100 auth.home.zakobar.com
|
||||
192.168.1.100 git.home.zakobar.com
|
||||
192.168.1.100 nextcloud.home.zakobar.com
|
||||
192.168.1.100 ldapadmin.home.zakobar.com
|
||||
192.168.1.100 jellyfin.home.zakobar.com
|
||||
192.168.1.100 torrent.home.zakobar.com
|
||||
```
|
||||
|
||||
Replace `192.168.1.100` with your Pi's actual LAN IP.
|
||||
|
||||
---
|
||||
|
||||
## Day-to-day Operations
|
||||
|
||||
### Apply config changes
|
||||
|
||||
```bash
|
||||
sudo nixos-rebuild switch --flake /path/to/homey#pi-main
|
||||
```
|
||||
|
||||
### Edit secrets
|
||||
|
||||
```bash
|
||||
sops secrets/secrets.yaml
|
||||
# Save and exit — sops re-encrypts automatically
|
||||
# Then copy to Pi and rebuild
|
||||
```
|
||||
|
||||
### Browse service data on disk
|
||||
|
||||
```bash
|
||||
ls /mnt/data/
|
||||
ls /mnt/data/gitea/data/
|
||||
# No special tools needed — plain filesystem
|
||||
```
|
||||
|
||||
### Trigger a manual backup
|
||||
|
||||
```bash
|
||||
sudo systemctl start restic-backups-homey.service
|
||||
```
|
||||
|
||||
### List backup snapshots
|
||||
|
||||
```bash
|
||||
sudo restic -r <your-repo-url> \
|
||||
--password-file /run/secrets/restic_password \
|
||||
snapshots
|
||||
```
|
||||
|
||||
### Restore a single service from backup
|
||||
|
||||
```bash
|
||||
sudo systemctl stop podman-gitea.service
|
||||
sudo restic -r <repo> restore latest \
|
||||
--target / \
|
||||
--include /mnt/data/gitea
|
||||
sudo systemctl start podman-gitea.service
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Adding a Second Machine (future)
|
||||
|
||||
1. Create `hosts/pi-secondary/default.nix` and `hardware.nix`
|
||||
2. Enable the services you want on that machine
|
||||
3. Services communicating cross-machine: reference the primary Pi's LAN IP or
|
||||
hostname directly in environment variables (e.g. point gitea's LDAP config
|
||||
at `192.168.1.100:389` rather than `127.0.0.1:389`).
|
||||
4. Add the new host to `flake.nix`:
|
||||
```nix
|
||||
pi-secondary = mkHost {
|
||||
system = "x86_64-linux";
|
||||
hostPath = ./hosts/pi-secondary/default.nix;
|
||||
};
|
||||
```
|
||||
5. Generate an age key on the new machine and add it to `.sops.yaml`.
|
||||
Reference in New Issue
Block a user