TMP COMMIT BEFORE TRASHING
This commit is contained in:
Aner Zakobar
@@ -1,2 +1,3 @@
|
||||
charts
|
||||
*.lock
|
||||
.agent-shell
|
||||
|
||||
@@ -0,0 +1,255 @@
|
||||
# AGENTS.md
|
||||
|
||||
This is a Helm chart for deploying a self-hosted home environment (Homey) on Kubernetes.
|
||||
|
||||
## Project Overview
|
||||
|
||||
- **Type**: Helm Chart (Kubernetes package manager)
|
||||
- **Language**: YAML + Go template syntax (Helm templating)
|
||||
- **Key Files**:
|
||||
- `Chart.yaml` - Chart metadata
|
||||
- `values.yaml` - Default configuration values
|
||||
- `templates/` - Kubernetes manifest templates (auth.yaml, media.yaml, phpldapadmin.yaml, _definitions.yaml)
|
||||
- `files/` - Configuration file templates (processed by Helm with `tpl` function)
|
||||
|
||||
## Build/Lint/Test Commands
|
||||
|
||||
### Helm Validation
|
||||
```bash
|
||||
# Lint the Helm chart for errors
|
||||
helm lint .
|
||||
|
||||
# Template rendering (dry-run install)
|
||||
helm template test-release . --debug
|
||||
|
||||
# Install/upgrade in cluster
|
||||
helm upgrade --install homey . -n homey
|
||||
|
||||
# Verify chart against Kubernetes API
|
||||
helm kubeval .
|
||||
|
||||
# Check schema validation of values.yaml
|
||||
helm schema generate
|
||||
```
|
||||
|
||||
### Manual Template Testing
|
||||
```bash
|
||||
# Render templates locally with custom values
|
||||
helm template homey . -f values.yaml --set homey.url=example.com
|
||||
|
||||
# Template with debug output
|
||||
helm template homey . --debug 2>&1 | less
|
||||
```
|
||||
|
||||
### Kubectl Validation
|
||||
```bash
|
||||
# Dry-run apply to validate manifests
|
||||
kubectl apply -f templates/auth.yaml --dry-run=server
|
||||
|
||||
# Get rendered template directly
|
||||
helm template homey . | kubectl apply --dry-run=server -f -
|
||||
```
|
||||
|
||||
## Code Style Guidelines
|
||||
|
||||
### YAML Structure
|
||||
|
||||
1. **Document Separators**: Use `---` at the start of each YAML document
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
```
|
||||
|
||||
2. **Indentation**: Use 2 spaces (not tabs)
|
||||
```yaml
|
||||
spec:
|
||||
containers:
|
||||
- name: app
|
||||
image: nginx
|
||||
```
|
||||
|
||||
3. **Trailing Commas**: Optional but preferred for multi-line lists
|
||||
```yaml
|
||||
accessModes:
|
||||
- ReadWriteMany
|
||||
- ReadOnlyMany
|
||||
```
|
||||
|
||||
4. **Quotes**: Use quotes for strings that might be interpreted as other types
|
||||
- Always quote: `.Values.homey.url | quote`
|
||||
- Optional for simple strings like names
|
||||
|
||||
### Kubernetes Resources
|
||||
|
||||
1. **Labels**: Use Kubernetes recommended labels
|
||||
```yaml
|
||||
labels:
|
||||
app.kubernetes.io/name: openldap
|
||||
app.kubernetes.io/component: auth
|
||||
```
|
||||
|
||||
2. **Naming**: Use kebab-case for resource names
|
||||
```yaml
|
||||
name: openldap-admin
|
||||
name: nextcloud-postgres
|
||||
```
|
||||
|
||||
3. **Storage**: Always specify `storageClassName: longhorn`
|
||||
```yaml
|
||||
spec:
|
||||
storageClassName: longhorn
|
||||
```
|
||||
|
||||
### Helm Template Syntax
|
||||
|
||||
1. **Variable Assignment**: Use `$_ := set` for complex assignments
|
||||
```yaml
|
||||
{{- $_ := set $ "varname" (include "homey.lookuporgensecret" (merge (dict "secretname" "secret-name") $)) }}
|
||||
```
|
||||
|
||||
2. **Include with Merge**: Always pass `$` as the last argument
|
||||
```yaml
|
||||
{{ include "homey.randomsecret" (merge (dict "secretname" "secret-name" "secretval" $secretval) $) }}
|
||||
```
|
||||
|
||||
3. **Quote Values from .Values**: Use `quote` filter
|
||||
```yaml
|
||||
value: {{ .Values.homey.url | quote }}
|
||||
```
|
||||
|
||||
4. **Template Definitions**: Define reusable templates in `_definitions.yaml`
|
||||
- `homey.lookuporgensecret` - Look up existing secrets or generate random
|
||||
- `homey.randomsecret` - Generate a random secret
|
||||
- `homey.randHex` - Generate random hex string
|
||||
|
||||
5. **Template Spacing**: Use whitespace control to avoid extra newlines
|
||||
```yaml
|
||||
{{- "leading minus" -}} # No newline before
|
||||
{{ "trailing minus" -}} # No newline after
|
||||
```
|
||||
|
||||
### Secret Management
|
||||
|
||||
1. **Annotations**: Always annotate managed secrets to prevent deletion
|
||||
```yaml
|
||||
annotations:
|
||||
"helm.sh/resource-policy": "keep"
|
||||
```
|
||||
|
||||
2. **Secret Generation Pattern**:
|
||||
```yaml
|
||||
# Check for existing secret, create if not exists
|
||||
{{- $secretObj := (lookup "v1" "Secret" .Release.Namespace "secret-name") | default dict -}}
|
||||
{{- $secretData := (get $secretObj "data") | default dict -}}
|
||||
{{- $pass := (get $secretData "password") | default (randAlphaNum 32 | b64enc) -}}
|
||||
```
|
||||
|
||||
3. **Never hardcode secrets** - Use the secret lookup pattern above
|
||||
|
||||
### Config Files (files/ directory)
|
||||
|
||||
1. **Go Templates in Configs**: Use `tpl` function to process config files
|
||||
```yaml
|
||||
data:
|
||||
config.yml: |-
|
||||
{{ tpl (.Files.Get "files/authelia-config.yaml" | indent 4) . }}
|
||||
```
|
||||
|
||||
2. **Accessing Variables**: Config files can access `.Values.*` and custom variables set in templates
|
||||
|
||||
### Ingress Configuration
|
||||
|
||||
1. **TLS**: Always specify TLS with proper hosts and secret
|
||||
```yaml
|
||||
spec:
|
||||
ingressClassName: {{ .Values.homey.ingress_class }}
|
||||
tls:
|
||||
- hosts:
|
||||
- auth.{{ .Values.homey.url }}
|
||||
secretName: {{ .Values.homey.certname }}
|
||||
```
|
||||
|
||||
2. **Authelia Integration**: Use auth snippets for protected ingresses
|
||||
```yaml
|
||||
annotations:
|
||||
nginx.ingress.kubernetes.io/auth-url: http://authelia.{{ .Release.Namespace }}.svc.cluster.local:9091/api/verify
|
||||
nginx.ingress.kubernetes.io/auth-signin: https://auth.{{ .Values.homey.url }}?rm=$request_method
|
||||
```
|
||||
|
||||
### Resource Organization
|
||||
|
||||
1. **File Structure**:
|
||||
- `templates/_definitions.yaml` - Helper templates (secrets, utilities)
|
||||
- `templates/auth.yaml` - Authentication services (OpenLDAP, Authelia, Gitea, Nextcloud, Radicale)
|
||||
- `templates/media.yaml` - Media services (Jellyfin, Transmission)
|
||||
- `templates/phpldapadmin.yaml` - LDAP admin interface
|
||||
|
||||
2. **Manifest Order** (within a file):
|
||||
- PersistentVolumeClaim
|
||||
- Secrets
|
||||
- ConfigMaps
|
||||
- Deployments
|
||||
- Services
|
||||
- Ingress
|
||||
|
||||
3. **Unused Resources**: Keep deprecated manifests in `unused/` directory
|
||||
|
||||
### Environment Variables
|
||||
|
||||
1. **Naming**: Use uppercase with underscores
|
||||
```yaml
|
||||
- name: LDAP_ORGANISATION
|
||||
value: {{ .Values.homey.organization }}
|
||||
```
|
||||
|
||||
2. **Value Sources**: Prefer `valueFrom.secretKeyRef` over inline values
|
||||
```yaml
|
||||
- name: PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: secret-name
|
||||
key: password
|
||||
```
|
||||
|
||||
### Volume Mounts
|
||||
|
||||
1. **subPath**: Use `subPath` for shared PVCs
|
||||
```yaml
|
||||
volumeMounts:
|
||||
- mountPath: /data
|
||||
subPath: service-name/data
|
||||
```
|
||||
|
||||
2. **Read-only ConfigMaps**: Mark config mounts as read-only
|
||||
```yaml
|
||||
readOnly: true
|
||||
```
|
||||
|
||||
## Common Operations
|
||||
|
||||
### Adding a New Service
|
||||
|
||||
1. Add values to `values.yaml`
|
||||
2. Create/extend template in `templates/`
|
||||
3. Add PVC if persistent storage needed
|
||||
4. Add Ingress with appropriate annotations
|
||||
5. Test with `helm template .`
|
||||
|
||||
### Updating Secrets
|
||||
|
||||
Secrets are generated on first install. To regenerate:
|
||||
```bash
|
||||
kubectl delete secret <secret-name> -n homey
|
||||
helm upgrade --install homey . -n homey
|
||||
```
|
||||
|
||||
### Debugging Templates
|
||||
|
||||
```bash
|
||||
# Show all template variables available
|
||||
helm template . --show-only templates/_helpers.tpl
|
||||
|
||||
# Render single template
|
||||
helm template . --show-only templates/auth.yaml
|
||||
```
|
||||
Executable
+60
@@ -0,0 +1,60 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
BACKUP_DIR="$HOME/homey-backup"
|
||||
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
|
||||
NODE="root@192.168.1.100"
|
||||
|
||||
NC_DATA_PVC="pvc-5c1f48e3-346f-4c35-8e6a-8fc0c4c3a842-96d72815"
|
||||
NC_DB_PVC="pvc-c5b28179-1b9c-462a-be5b-05c4f0bb36ca-5f2dbf4d"
|
||||
|
||||
NC_DATA_DEST="$BACKUP_DIR/longhorn-nextcloud-data/$TIMESTAMP"
|
||||
NC_DB_DEST="$BACKUP_DIR/longhorn-nextcloud-db/$TIMESTAMP"
|
||||
|
||||
echo "=== Longhorn Volume Backup (Emergency) ==="
|
||||
echo "Started: $(date)"
|
||||
echo ""
|
||||
echo "WARNING: Backing up raw Longhorn volume images"
|
||||
echo "These are sparse files - actual data is smaller than file size"
|
||||
echo ""
|
||||
|
||||
mkdir -p "$NC_DATA_DEST"
|
||||
mkdir -p "$NC_DB_DEST"
|
||||
|
||||
echo "--- Backing up Nextcloud data volume ---"
|
||||
echo "Source: $NODE:/hda/replicas/$NC_DATA_PVC/"
|
||||
echo "Dest: $NC_DATA_DEST/"
|
||||
echo ""
|
||||
|
||||
rsync -avzP --no-owner --no-group --sparse \
|
||||
"$NODE:/hda/replicas/$NC_DATA_PVC/" \
|
||||
"$NC_DATA_DEST/"
|
||||
|
||||
echo ""
|
||||
echo "Nextcloud volume backup: $(du -sh "$NC_DATA_DEST" | cut -f1)"
|
||||
|
||||
echo ""
|
||||
echo "--- Backing up PostgreSQL volume ---"
|
||||
echo "Source: $NODE:/hda/replicas/$NC_DB_PVC/"
|
||||
echo "Dest: $NC_DB_DEST/"
|
||||
echo ""
|
||||
|
||||
rsync -avzP --no-owner --no-group --sparse \
|
||||
"$NODE:/hda/replicas/$NC_DB_PVC/" \
|
||||
"$NC_DB_DEST/"
|
||||
|
||||
echo ""
|
||||
echo "PostgreSQL volume backup: $(du -sh "$NC_DB_DEST" | cut -f1)"
|
||||
|
||||
echo ""
|
||||
echo "=== Backup Complete ==="
|
||||
echo "Timestamp: $TIMESTAMP"
|
||||
echo "Finished: $(date)"
|
||||
|
||||
echo ""
|
||||
echo "Total backup size:"
|
||||
du -sh "$BACKUP_DIR/longhorn-nextcloud-data/$TIMESTAMP" "$BACKUP_DIR/longhorn-nextcloud-db/$TIMESTAMP"
|
||||
|
||||
echo ""
|
||||
echo "NOTE: These are raw Longhorn volume images."
|
||||
echo "To restore, copy back to /hda/replicas/ and restart Longhorn."
|
||||
Executable
+120
@@ -0,0 +1,120 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
BACKUP_DIR="$HOME/homey-backup"
|
||||
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
|
||||
NAMESPACE="homey"
|
||||
NODE="root@192.168.1.100"
|
||||
|
||||
NC_DATA_DEST="$BACKUP_DIR/nextcloud-data/backups/$TIMESTAMP"
|
||||
NC_DB_DEST="$BACKUP_DIR/nextcloud-postgres/backups/$TIMESTAMP"
|
||||
|
||||
show_progress() {
|
||||
local file="$1"
|
||||
local label="$2"
|
||||
local total="${3:-0}"
|
||||
|
||||
while [[ ! -f "$file" ]]; do
|
||||
sleep 0.2
|
||||
done
|
||||
|
||||
while kill -0 "$BACKUP_PID" 2>/dev/null; do
|
||||
local size=$(stat -c%s "$file" 2>/dev/null || echo "0")
|
||||
if [[ "$total" -gt 0 ]]; then
|
||||
local pct=$((size * 100 / total))
|
||||
local size_hr=$(numfmt --to=iec-i --suffix=B "$size" 2>/dev/null || echo "${size}B")
|
||||
local total_hr=$(numfmt --to=iec-i --suffix=B "$total" 2>/dev/null || echo "${total}B")
|
||||
printf "\r%s: %s / %s (%d%%) " "$label" "$size_hr" "$total_hr" "$pct"
|
||||
else
|
||||
local size_hr=$(numfmt --to=iec-i --suffix=B "$size" 2>/dev/null || echo "${size}B")
|
||||
printf "\r%s: %s " "$label" "$size_hr"
|
||||
fi
|
||||
sleep 0.5
|
||||
done
|
||||
|
||||
local final=$(stat -c%s "$file" 2>/dev/null || echo "0")
|
||||
local final_hr=$(numfmt --to=iec-i --suffix=B "$final" 2>/dev/null || echo "${final}B")
|
||||
printf "\r%s: %s - done! \n" "$label" "$final_hr"
|
||||
}
|
||||
|
||||
wait_for_cluster() {
|
||||
echo "Waiting for Kubernetes cluster..."
|
||||
local max_wait=300
|
||||
local start=$(date +%s)
|
||||
while true; do
|
||||
if kubectl get nodes &>/dev/null; then
|
||||
echo "Cluster ready!"
|
||||
return 0
|
||||
fi
|
||||
local now=$(date +%s)
|
||||
if [[ $((now - start)) -ge $max_wait ]]; then
|
||||
echo "ERROR: Cluster not available after ${max_wait}s"
|
||||
return 1
|
||||
fi
|
||||
printf "\rWaiting... %ds " $((now - start))
|
||||
sleep 5
|
||||
done
|
||||
}
|
||||
|
||||
echo "=== Nextcloud Backup ==="
|
||||
echo "Started: $(date)"
|
||||
|
||||
mkdir -p "$NC_DATA_DEST"
|
||||
mkdir -p "$NC_DB_DEST"
|
||||
|
||||
if ! wait_for_cluster; then
|
||||
echo "Cluster unavailable. Cannot proceed."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "--- Backing up PostgreSQL ---"
|
||||
DB_POD=$(kubectl get pods -n "$NAMESPACE" -l app=nextcloud-postgres -o jsonpath='{.items[0].metadata.name}')
|
||||
if [[ -z "$DB_POD" ]]; then
|
||||
echo "ERROR: PostgreSQL pod not found"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
DB_SIZE=$(kubectl exec -n "$NAMESPACE" "$DB_POD" -- psql -U postgres -d nextcloud_db -t -c "SELECT pg_database_size('nextcloud_db');" 2>/dev/null | tr -d ' ' || echo "0")
|
||||
echo "Database size: $(numfmt --to=iec-i --suffix=B "$DB_SIZE" 2>/dev/null || echo "$DB_SIZE bytes")"
|
||||
|
||||
echo "Dumping database from $DB_POD..."
|
||||
kubectl exec -n "$NAMESPACE" "$DB_POD" -- sh -c "pg_dump -U postgres nextcloud_db" | gzip > "$NC_DB_DEST/dump.sql.gz" &
|
||||
BACKUP_PID=$!
|
||||
show_progress "$NC_DB_DEST/dump.sql.gz" "Database" "$DB_SIZE"
|
||||
wait $BACKUP_PID 2>/dev/null || true
|
||||
|
||||
echo "Database backup: $(du -sh "$NC_DB_DEST/dump.sql.gz" | cut -f1)"
|
||||
|
||||
echo ""
|
||||
echo "--- Backing up Nextcloud data ---"
|
||||
NC_POD=$(kubectl get pods -n "$NAMESPACE" -l app=nextcloud -o jsonpath='{.items[0].metadata.name}')
|
||||
if [[ -z "$NC_POD" ]]; then
|
||||
echo "ERROR: Nextcloud pod not found"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
NC_SIZE=$(kubectl exec -n "$NAMESPACE" "$NC_POD" -- du -sb /var/www/html 2>/dev/null | awk '{print $1}' || echo "0")
|
||||
echo "Data size: $(numfmt --to=iec-i --suffix=B "$NC_SIZE" 2>/dev/null || echo "$NC_SIZE bytes")"
|
||||
|
||||
echo "Creating tar archive from $NC_POD..."
|
||||
kubectl exec -n "$NAMESPACE" "$NC_POD" -- tar cf - -C /var/www/html . 2>/dev/null | gzip > "$NC_DATA_DEST/data.tar.gz" &
|
||||
BACKUP_PID=$!
|
||||
show_progress "$NC_DATA_DEST/data.tar.gz" "Data" "$NC_SIZE"
|
||||
wait $BACKUP_PID 2>/dev/null || true
|
||||
|
||||
echo "Data backup: $(du -sh "$NC_DATA_DEST/data.tar.gz" | cut -f1)"
|
||||
|
||||
echo ""
|
||||
echo "--- Creating latest symlinks ---"
|
||||
rm -f "$BACKUP_DIR/nextcloud-data/latest" "$BACKUP_DIR/nextcloud-postgres/latest"
|
||||
ln -sf "backups/$TIMESTAMP" "$BACKUP_DIR/nextcloud-data/latest"
|
||||
ln -sf "backups/$TIMESTAMP" "$BACKUP_DIR/nextcloud-postgres/latest"
|
||||
|
||||
echo ""
|
||||
echo "=== Backup Complete ==="
|
||||
echo "Timestamp: $TIMESTAMP"
|
||||
echo "Finished: $(date)"
|
||||
|
||||
echo ""
|
||||
du -sh "$NC_DATA_DEST" "$NC_DB_DEST"
|
||||
Executable
+41
@@ -0,0 +1,41 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SRC="/mnt/replicas"
|
||||
DEST="$HOME/homey-backup/longhorn-volumes"
|
||||
SKIP="pvc-dfe2aa08-bbb8-423b-9001-fb6aea181597-baf06a7f"
|
||||
|
||||
mkdir -p "$DEST"
|
||||
|
||||
echo "=== Copying Longhorn volumes from HD ==="
|
||||
echo "Source: $SRC"
|
||||
echo "Dest: $DEST"
|
||||
echo "Skip: $SKIP (Jellyfin)"
|
||||
echo ""
|
||||
|
||||
for pvc in "$SRC"/*/; do
|
||||
name=$(basename "$pvc")
|
||||
|
||||
if [[ "$name" == "$SKIP" ]]; then
|
||||
echo "Skipping: $name"
|
||||
continue
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Copying: $name"
|
||||
|
||||
src_size=$(sudo du -sb "$pvc" 2>/dev/null | awk '{print $1}' || echo "0")
|
||||
src_size_hr=$(numfmt --to=iec-i --suffix=B "$src_size" 2>/dev/null || echo "${src_size}B")
|
||||
echo "Size: $src_size_hr"
|
||||
|
||||
sudo rsync -a --no-owner --no-group --info=progress2 "${pvc%/}" "$DEST/"
|
||||
sudo chown -R "$USER" "$DEST/$name"
|
||||
|
||||
size=$(du -sh "$DEST/$name" | cut -f1)
|
||||
echo "Done: $size"
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "=== Copy Complete ==="
|
||||
echo "Total size:"
|
||||
sudo du -sh "$DEST"
|
||||
Executable
+184
@@ -0,0 +1,184 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SRC="${SRC:-/mnt/replicas}"
|
||||
DEST="${DEST:-/mnt2/homey-backup}"
|
||||
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
|
||||
MANIFEST="$DEST/manifest.json"
|
||||
|
||||
PVC_MAPPING=(
|
||||
"pvc-0310a337-9642-464b-a458-fcb3439328e7-fbc07d5a:ldap-pvc"
|
||||
"pvc-1cdc51ee-b965-4cab-baf7-077cc6df6f11-0fcfb9cd:authelia-pvc"
|
||||
"pvc-4888bf84-62c8-4340-adbc-cb31073d8fd2-d065d20b:gitea-pvc"
|
||||
"pvc-5c1f48e3-346f-4c35-8e6a-8fc0c4c3a842-96d72815:nextcloud-data-pvc"
|
||||
"pvc-c5b28179-1b9c-462a-be5b-05c4f0bb36ca-5f2dbf4d:nextcloud-postgres-pvc"
|
||||
"pvc-7f73ee94-5583-4e4a-9788-cba054214b1c-f767850a:radicale-pvc"
|
||||
"pvc-9e75f35a-27c3-4251-b25a-1a876f82f6c7-c9c8b185:jellyfin-config-pvc"
|
||||
"pvc-dfe2aa08-bbb8-423b-9001-fb6aea181597-baf06a7f:jellyfin-data-pvc"
|
||||
"pvc-dd4a069a-a638-49c0-8c95-f954510816e5-7e81a6f6:transmission-config-pvc"
|
||||
"pvc-e4ba414d-d9c2-4927-b0ae-f6bfb90ce311-a0963101:unknown-pvc-1"
|
||||
"pvc-ec6afe10-aca3-42ce-9d89-32fc4ac77f9a-8d6baa34:unknown-pvc-2"
|
||||
)
|
||||
|
||||
progress_bar() {
|
||||
local current=$1
|
||||
local total=$2
|
||||
local width=40
|
||||
local percent=$((current * 100 / total))
|
||||
local filled=$((current * width / total))
|
||||
local empty=$((width - filled))
|
||||
printf "\r["
|
||||
printf "%${filled}s" | tr ' ' '='
|
||||
printf "%${empty}s" | tr ' ' ' '
|
||||
printf "] %3d%% (%d/%d)" "$percent" "$current" "$total"
|
||||
}
|
||||
|
||||
get_pvc_name() {
|
||||
local pvc_id="$1"
|
||||
for mapping in "${PVC_MAPPING[@]}"; do
|
||||
if [[ "$mapping" == "$pvc_id:"* ]]; then
|
||||
echo "${mapping#*:}"
|
||||
return
|
||||
fi
|
||||
done
|
||||
echo "unknown"
|
||||
}
|
||||
|
||||
echo "========================================"
|
||||
echo " Longhorn Volume Backup Tool"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
echo "Source: $SRC"
|
||||
echo "Destination: $DEST"
|
||||
echo "Timestamp: $TIMESTAMP"
|
||||
echo ""
|
||||
|
||||
mkdir -p "$DEST/volumes"
|
||||
mkdir -p "$DEST/metadata"
|
||||
|
||||
VOLUMES=()
|
||||
TOTAL_SIZE=0
|
||||
|
||||
echo "Scanning volumes..."
|
||||
for pvc_dir in "$SRC"/*/; do
|
||||
pvc_name=$(basename "$pvc_dir")
|
||||
friendly_name=$(get_pvc_name "$pvc_name")
|
||||
VOLUMES+=("$pvc_name:$friendly_name")
|
||||
|
||||
size=$(sudo du -sb "$pvc_dir" 2>/dev/null | awk '{print $1}' || echo "0")
|
||||
TOTAL_SIZE=$((TOTAL_SIZE + size))
|
||||
|
||||
printf " %-50s %s\n" "$friendly_name" "$(numfmt --to=iec-i --suffix=B "$size" 2>/dev/null || echo "${size}B")"
|
||||
done
|
||||
|
||||
TOTAL_VOLUMES=${#VOLUMES[@]}
|
||||
echo ""
|
||||
echo "Found $TOTAL_VOLUMES volumes, total size: $(numfmt --to=iec-i --suffix=B "$TOTAL_SIZE")"
|
||||
echo ""
|
||||
read -p "Continue with backup? [y/N] " -n 1 -r
|
||||
echo ""
|
||||
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
|
||||
echo "Aborted."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Starting backup..."
|
||||
echo ""
|
||||
|
||||
COPIED_SIZE=0
|
||||
START_TIME=$(date +%s)
|
||||
|
||||
for i in "${!VOLUMES[@]}"; do
|
||||
volume="${VOLUMES[$i]}"
|
||||
pvc_name="${volume%%:*}"
|
||||
friendly_name="${volume#*:}"
|
||||
|
||||
CURRENT=$((i + 1))
|
||||
progress_bar "$CURRENT" "$TOTAL_VOLUMES"
|
||||
echo " - $friendly_name"
|
||||
|
||||
sudo rsync -a --no-owner --no-group --info=progress2 \
|
||||
"$SRC/$pvc_name/" \
|
||||
"$DEST/volumes/$pvc_name/" 2>&1 | while read -r line; do
|
||||
if [[ "$line" =~ to-chk=*([0-9]+)/([0-9]+) ]]; then
|
||||
printf "\r %s" "$line"
|
||||
fi
|
||||
done
|
||||
|
||||
sudo chown -R "$USER:$USER" "$DEST/volumes/$pvc_name" 2>/dev/null || true
|
||||
|
||||
if [[ -f "$SRC/$pvc_name/volume.meta" ]]; then
|
||||
sudo cp "$SRC/$pvc_name/volume.meta" "$DEST/metadata/${pvc_name}.meta" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
echo ""
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "Generating manifest..."
|
||||
|
||||
cat > "$MANIFEST" << EOF
|
||||
{
|
||||
"backup_timestamp": "$TIMESTAMP",
|
||||
"source_path": "$SRC",
|
||||
"destination_path": "$DEST",
|
||||
"total_volumes": $TOTAL_VOLUMES,
|
||||
"total_size_bytes": $TOTAL_SIZE,
|
||||
"volumes": [
|
||||
EOF
|
||||
|
||||
FIRST=true
|
||||
for volume in "${VOLUMES[@]}"; do
|
||||
pvc_name="${volume%%:*}"
|
||||
friendly_name="${volume#*:}"
|
||||
|
||||
vol_size=$(sudo du -sb "$SRC/$pvc_name" 2>/dev/null | awk '{print $1}' || echo "0")
|
||||
vol_size_hr=$(numfmt --to=iec-i --suffix=B "$vol_size" 2>/dev/null || echo "${vol_size}B")
|
||||
|
||||
head_file=$(sudo find "$DEST/volumes/$pvc_name" -name "volume-head-*.img" 2>/dev/null | head -1)
|
||||
head_file=$(basename "$head_file" 2>/dev/null || echo "")
|
||||
|
||||
if [[ "$FIRST" == "true" ]]; then
|
||||
FIRST=false
|
||||
else
|
||||
echo "," >> "$MANIFEST"
|
||||
fi
|
||||
|
||||
cat >> "$MANIFEST" << EOF
|
||||
{
|
||||
"pvc_id": "$pvc_name",
|
||||
"friendly_name": "$friendly_name",
|
||||
"size_bytes": $vol_size,
|
||||
"size_human": "$vol_size_hr",
|
||||
"volume_head": "$head_file",
|
||||
"backup_path": "volumes/$pvc_name"
|
||||
}
|
||||
EOF
|
||||
done
|
||||
|
||||
cat >> "$MANIFEST" << EOF
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
END_TIME=$(date +%s)
|
||||
DURATION=$((END_TIME - START_TIME))
|
||||
|
||||
echo ""
|
||||
echo "========================================"
|
||||
echo " Backup Complete!"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
echo "Duration: $((DURATION / 60))m $((DURATION % 60))s"
|
||||
echo "Location: $DEST"
|
||||
echo "Manifest: $MANIFEST"
|
||||
echo ""
|
||||
echo "Backup size:"
|
||||
sudo du -sh "$DEST/volumes"
|
||||
echo ""
|
||||
echo "To mount a volume, run:"
|
||||
echo " ./scripts/mount-longhorn-volume.sh <pvc-name-or-friendly-name>"
|
||||
echo ""
|
||||
echo "To restore a volume, run:"
|
||||
echo " ./scripts/restore-longhorn-volume.sh <pvc-name-or-friendly-name>"
|
||||
@@ -0,0 +1,16 @@
|
||||
for dir in /mnt/replicas/pvc-*/; do
|
||||
name=$(basename "$dir")
|
||||
head=$(sudo find "$dir" -name "volume-head-*.img" | head -1)
|
||||
|
||||
sudo mkdir -p /tmp/inspect
|
||||
loop=$(sudo losetup -fP --show "$head")
|
||||
|
||||
echo "=== $name ==="
|
||||
sudo mount "$loop" /tmp/inspect 2>/dev/null && {
|
||||
sudo ls -la /tmp/inspect | head -10
|
||||
sudo umount /tmp/inspect
|
||||
} || echo "(mount failed)"
|
||||
|
||||
sudo losetup -d "$loop"
|
||||
echo ""
|
||||
done
|
||||
Executable
+37
@@ -0,0 +1,37 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
BACKUP_DIR="${BACKUP_DIR:-/mnt2/homey-backup}"
|
||||
MANIFEST="$BACKUP_DIR/manifest.json"
|
||||
|
||||
echo "========================================"
|
||||
echo " Longhorn Volume Backup List"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
|
||||
if [[ ! -f "$MANIFEST" ]]; then
|
||||
echo "No manifest found at $MANIFEST"
|
||||
echo "Run backup-longhorn-to-disk.sh first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Backup timestamp: $(grep -oP '"backup_timestamp":\s*"\K[^"]+' "$MANIFEST")"
|
||||
echo "Source: $(grep -oP '"source_path":\s*"\K[^"]+' "$MANIFEST")"
|
||||
echo "Total volumes: $(grep -oP '"total_volumes":\s*\K[0-9]+' "$MANIFEST")"
|
||||
echo "Total size: $(grep -oP '"total_size_bytes":\s*\K[0-9]+' "$MANIFEST" | numfmt --to=iec-i --suffix=B)"
|
||||
echo ""
|
||||
echo "Volumes:"
|
||||
echo "----------------------------------------"
|
||||
|
||||
grep -A5 '"volumes"' "$MANIFEST" | grep -E '"friendly_name"|"size_human"' | \
|
||||
while read -r name_line; read -r size_line; do
|
||||
name=$(echo "$name_line" | grep -oP '"friendly_name":\s*"\K[^"]+')
|
||||
size=$(echo "$size_line" | grep -oP '"size_human":\s*"\K[^"]+')
|
||||
pvc=$(grep -B1 "$name_line" "$MANIFEST" | grep -oP '"pvc_id":\s*"\K[^"]+' || echo "")
|
||||
printf " %-30s %10s %s\n" "$name" "$size" "$pvc"
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "Commands:"
|
||||
echo " Mount: ./scripts/mount-longhorn-volume.sh <name>"
|
||||
echo " Restore: ./scripts/restore-longhorn-volume.sh <name>"
|
||||
@@ -0,0 +1,70 @@
|
||||
#!/usr/bin/env python3
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from fuse import FUSE, FuseOSError, Operations
|
||||
|
||||
class LonghornBackupFS(Operations):
|
||||
def __init__(self, backup_dir):
|
||||
self.backup_dir = backup_dir
|
||||
self.blocks_dir = f"{backup_dir}/blocks"
|
||||
|
||||
backup_cfg = f"{backup_dir}/backups/backup_backup-eac0221d1cab4a9c.cfg"
|
||||
with open(backup_cfg) as f:
|
||||
data = json.load(f)
|
||||
|
||||
self.size = int(data['Size'])
|
||||
self.block_map = {b['Offset']: b['BlockChecksum'] for b in data['Blocks']}
|
||||
self.block_size = 2097152 # 2MB
|
||||
|
||||
print(f"Volume size: {self.size}")
|
||||
print(f"Blocks: {len(self.block_map)}")
|
||||
|
||||
def getattr(self, path, fh=None):
|
||||
return {'st_size': self.size, 'st_mode': 0o100644, 'st_nlink': 1}
|
||||
|
||||
def read(self, path, size, offset, fh):
|
||||
result = bytearray()
|
||||
remaining = size
|
||||
current_offset = offset
|
||||
|
||||
while remaining > 0:
|
||||
block_start = (current_offset // self.block_size) * self.block_size
|
||||
block_offset = current_offset - block_start
|
||||
read_size = min(remaining, self.block_size - block_offset)
|
||||
|
||||
if block_start in self.block_map:
|
||||
checksum = self.block_map[block_start]
|
||||
block_path = f"{self.blocks_dir}/{checksum[:2]}/{checksum[2:4]}/{checksum}.blk"
|
||||
|
||||
if os.path.exists(block_path):
|
||||
with open(block_path, 'rb') as f:
|
||||
f.seek(block_offset)
|
||||
result.extend(f.read(read_size))
|
||||
else:
|
||||
result.extend(b'\x00' * read_size)
|
||||
else:
|
||||
result.extend(b'\x00' * read_size)
|
||||
|
||||
current_offset += read_size
|
||||
remaining -= read_size
|
||||
|
||||
return bytes(result)
|
||||
|
||||
if __name__ == '__main__':
|
||||
if len(sys.argv) < 3:
|
||||
print(f"Usage: {sys.argv[0]} <backup_dir> <mount_point>")
|
||||
print(f"Example: {sys.argv[0]} /mnt2/backed-up-drive/backupstore/volumes/2c/df/pvc-5c1f48e3-346f-4c35-8e6a-8fc0c4c3a842 /tmp/longhorn-fuse")
|
||||
sys.exit(1)
|
||||
|
||||
backup_dir = sys.argv[1]
|
||||
mount_point = sys.argv[2]
|
||||
|
||||
os.makedirs(mount_point, exist_ok=True)
|
||||
|
||||
print(f"Mounting {backup_dir} at {mount_point}")
|
||||
print("This creates a virtual block device file at the mount point")
|
||||
print("Then run: sudo losetup -fP {mount_point}/volume.img && sudo mount /dev/loopX /mnt/point")
|
||||
|
||||
fs = LonghornBackupFS(backup_dir)
|
||||
fuse = FUSE(fs, mount_point, nothreads=True, foreground=True, allow_other=True)
|
||||
@@ -0,0 +1,49 @@
|
||||
#!/usr/bin/env python3
|
||||
import nbdkit
|
||||
import json
|
||||
import os
|
||||
|
||||
backup_dir = "/mnt2/backed-up-drive/backupstore/volumes/2c/df/pvc-5c1f48e3-346f-4c35-8e6a-8fc0c4c3a842"
|
||||
blocks_dir = f"{backup_dir}/blocks"
|
||||
backup_cfg = f"{backup_dir}/backups/backup_backup-eac0221d1cab4a9c.cfg"
|
||||
|
||||
with open(backup_cfg) as f:
|
||||
data = json.load(f)
|
||||
|
||||
size = int(data['Size'])
|
||||
block_map = {b['Offset']: b['BlockChecksum'] for b in data['Blocks']}
|
||||
block_size = 2097152
|
||||
|
||||
def thread_model():
|
||||
return nbdkit.THREAD_MODEL_SERIALIZE_ALL_REQUESTS
|
||||
|
||||
def get_size():
|
||||
return size
|
||||
|
||||
def pread(h, count, offset, flags):
|
||||
result = bytearray()
|
||||
remaining = count
|
||||
current_offset = offset
|
||||
|
||||
while remaining > 0:
|
||||
block_start = (current_offset // block_size) * block_size
|
||||
block_offset = current_offset - block_start
|
||||
read_size = min(remaining, block_size - block_offset)
|
||||
|
||||
if block_start in block_map:
|
||||
checksum = block_map[block_start]
|
||||
block_path = f"{blocks_dir}/{checksum[:2]}/{checksum[2:4]}/{checksum}.blk"
|
||||
|
||||
if os.path.exists(block_path):
|
||||
with open(block_path, 'rb') as f:
|
||||
f.seek(block_offset)
|
||||
result.extend(f.read(read_size))
|
||||
else:
|
||||
result.extend(b'\x00' * read_size)
|
||||
else:
|
||||
result.extend(b'\x00' * read_size)
|
||||
|
||||
current_offset += read_size
|
||||
remaining -= read_size
|
||||
|
||||
return bytes(result)
|
||||
Executable
+146
@@ -0,0 +1,146 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
BACKUP_DIR="${BACKUP_DIR:-/mnt2/homey-backup}"
|
||||
MOUNT_BASE="${MOUNT_BASE:-/mnt/longhorn-volumes}"
|
||||
|
||||
usage() {
|
||||
echo "Usage: $0 <pvc-name-or-friendly-name> [mount-point]"
|
||||
echo ""
|
||||
echo "Mounts a Longhorn volume backup for exploration."
|
||||
echo ""
|
||||
echo "Arguments:"
|
||||
echo " pvc-name-or-friendly-name The PVC ID or friendly name (e.g., 'nextcloud-data-pvc')"
|
||||
echo " mount-point Optional custom mount point (default: $MOUNT_BASE/<name>)"
|
||||
echo ""
|
||||
echo "Examples:"
|
||||
echo " $0 nextcloud-data-pvc"
|
||||
echo " $0 pvc-5c1f48e3-346f-4c35-8e6a-8fc0c4c3a842-96d72815"
|
||||
echo " $0 nextcloud-data-pvc /mnt/my-mount"
|
||||
echo ""
|
||||
echo "To unmount, run:"
|
||||
echo " sudo umount <mount-point>"
|
||||
echo " sudo losetup -d /dev/loopX"
|
||||
exit 1
|
||||
}
|
||||
|
||||
if [[ $# -lt 1 ]]; then
|
||||
usage
|
||||
fi
|
||||
|
||||
SEARCH_NAME="$1"
|
||||
CUSTOM_MOUNT="${2:-}"
|
||||
|
||||
MANIFEST="$BACKUP_DIR/manifest.json"
|
||||
|
||||
if [[ ! -f "$MANIFEST" ]]; then
|
||||
echo "Error: Manifest not found at $MANIFEST"
|
||||
echo "Make sure you've run the backup script first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
find_volume() {
|
||||
local search="$1"
|
||||
local found=""
|
||||
|
||||
while IFS= read -r line; do
|
||||
pvc_id=$(echo "$line" | grep -oP '"pvc_id":\s*"\K[^"]+')
|
||||
friendly=$(echo "$line" | grep -oP '"friendly_name":\s*"\K[^"]+')
|
||||
|
||||
if [[ "$pvc_id" == "$search" ]] || [[ "$friendly" == "$search" ]]; then
|
||||
echo "$pvc_id:$friendly"
|
||||
return 0
|
||||
fi
|
||||
done < <(grep -A6 '"volumes"' "$MANIFEST" | grep -E '"pvc_id"|"friendly_name"')
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
VOLUME_INFO=$(find_volume "$SEARCH_NAME")
|
||||
|
||||
if [[ -z "$VOLUME_INFO" ]]; then
|
||||
echo "Error: Volume '$SEARCH_NAME' not found in manifest."
|
||||
echo ""
|
||||
echo "Available volumes:"
|
||||
grep -oP '"friendly_name":\s*"\K[^"]+' "$MANIFEST" | while read -r name; do
|
||||
echo " - $name"
|
||||
done
|
||||
exit 1
|
||||
fi
|
||||
|
||||
PVC_ID="${VOLUME_INFO%%:*}"
|
||||
FRIENDLY_NAME="${VOLUME_INFO#*:}"
|
||||
|
||||
VOLUME_DIR="$BACKUP_DIR/volumes/$PVC_ID"
|
||||
|
||||
if [[ ! -d "$VOLUME_DIR" ]]; then
|
||||
echo "Error: Volume directory not found: $VOLUME_DIR"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
VOLUME_HEAD=$(find "$VOLUME_DIR" -name "volume-head-*.img" | head -1)
|
||||
|
||||
if [[ -z "$VOLUME_HEAD" ]]; then
|
||||
echo "Error: No volume-head-*.img file found in $VOLUME_DIR"
|
||||
echo "Contents:"
|
||||
ls -la "$VOLUME_DIR"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ -n "$CUSTOM_MOUNT" ]]; then
|
||||
MOUNT_POINT="$CUSTOM_MOUNT"
|
||||
else
|
||||
MOUNT_POINT="$MOUNT_BASE/$FRIENDLY_NAME"
|
||||
fi
|
||||
|
||||
echo "========================================"
|
||||
echo " Mount Longhorn Volume"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
echo "PVC ID: $PVC_ID"
|
||||
echo "Name: $FRIENDLY_NAME"
|
||||
echo "Volume file: $(basename "$VOLUME_HEAD")"
|
||||
echo "Mount point: $MOUNT_POINT"
|
||||
echo ""
|
||||
|
||||
LOOP_DEV=$(sudo losetup -fP --show "$VOLUME_HEAD")
|
||||
echo "Attached to: $LOOP_DEV"
|
||||
|
||||
sudo mkdir -p "$MOUNT_POINT"
|
||||
|
||||
echo ""
|
||||
echo "Mounting..."
|
||||
if sudo mount "$LOOP_DEV" "$MOUNT_POINT" 2>/dev/null; then
|
||||
echo ""
|
||||
echo "========================================"
|
||||
echo " Mounted Successfully!"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
echo "Mount point: $MOUNT_POINT"
|
||||
echo "Loop device: $LOOP_DEV"
|
||||
echo ""
|
||||
echo "Contents:"
|
||||
ls -la "$MOUNT_POINT" 2>/dev/null | head -20
|
||||
echo ""
|
||||
echo "To unmount:"
|
||||
echo " sudo umount $MOUNT_POINT"
|
||||
echo " sudo losetup -d $LOOP_DEV"
|
||||
else
|
||||
echo "Mount failed. Trying with filesystem detection..."
|
||||
FS_TYPE=$(sudo blkid -o value -s TYPE "$LOOP_DEV" 2>/dev/null || echo "")
|
||||
|
||||
if [[ -n "$FS_TYPE" ]]; then
|
||||
echo "Detected filesystem: $FS_TYPE"
|
||||
sudo mount -t "$FS_TYPE" "$LOOP_DEV" "$MOUNT_POINT"
|
||||
echo ""
|
||||
echo "Mounted successfully at $MOUNT_POINT"
|
||||
else
|
||||
echo "Could not detect filesystem. Volume may be empty or corrupted."
|
||||
echo ""
|
||||
echo "Loop device: $LOOP_DEV"
|
||||
echo "Run 'sudo blkid $LOOP_DEV' to inspect."
|
||||
echo ""
|
||||
echo "To detach:"
|
||||
echo " sudo losetup -d $LOOP_DEV"
|
||||
fi
|
||||
fi
|
||||
@@ -0,0 +1,48 @@
|
||||
#!/usr/bin/env python3
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import gzip
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
|
||||
backup_dir = "/mnt2/backed-up-drive/backupstore/volumes/2c/df/pvc-5c1f48e3-346f-4c35-8e6a-8fc0c4c3a842"
|
||||
output_img = "/mnt/nextcloud-restored.img"
|
||||
|
||||
backup_cfg = f"{backup_dir}/backups/backup_backup-eac0221d1cab4a9c.cfg"
|
||||
blocks_dir = f"{backup_dir}/blocks"
|
||||
|
||||
with open(backup_cfg) as f:
|
||||
data = json.load(f)
|
||||
|
||||
blocks = data['Blocks']
|
||||
total = len(blocks)
|
||||
size = int(data['Size'])
|
||||
|
||||
print(f"Volume size: {size // 1024 // 1024 // 1024} GB")
|
||||
print(f"Block count: {total}")
|
||||
|
||||
os.makedirs(os.path.dirname(output_img) if os.path.dirname(output_img) else '.', exist_ok=True)
|
||||
|
||||
if not os.path.exists(output_img):
|
||||
import subprocess
|
||||
subprocess.run(['truncate', '-s', str(size), output_img], check=True)
|
||||
|
||||
with open(output_img, 'r+b') as img:
|
||||
for i, block in enumerate(blocks):
|
||||
offset = block['Offset']
|
||||
checksum = block['BlockChecksum']
|
||||
|
||||
block_path = f"{blocks_dir}/{checksum[:2]}/{checksum[2:4]}/{checksum}.blk"
|
||||
|
||||
if os.path.exists(block_path):
|
||||
with gzip.open(block_path, 'rb') as bf:
|
||||
img.seek(offset)
|
||||
img.write(bf.read())
|
||||
|
||||
if (i + 1) % 500 == 0:
|
||||
percent = (i + 1) * 100 // total
|
||||
bar = '=' * (percent // 2) + ' ' * (50 - percent // 2)
|
||||
sys.stdout.write(f"\r[{bar}] {percent}% ({i + 1}/{total})")
|
||||
sys.stdout.flush()
|
||||
|
||||
print(f"\nDone! Image: {output_img}")
|
||||
Executable
+94
@@ -0,0 +1,94 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
BACKUP_DIR="${1:-/mnt2/backed-up-drive/backupstore/volumes/2c/df/pvc-5c1f48e3-346f-4c35-8e6a-8fc0c4c3a842}"
|
||||
OUTPUT_IMG="${2:-./nextcloud-data-restored.img}"
|
||||
|
||||
BACKUP_CFG="$BACKUP_DIR/backups/backup_backup-eac0221d1cab4a9c.cfg"
|
||||
BLOCKS_DIR="$BACKUP_DIR/blocks"
|
||||
|
||||
if [[ ! -f "$BACKUP_CFG" ]]; then
|
||||
echo "Error: Backup config not found at $BACKUP_CFG"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "========================================"
|
||||
echo " Longhorn Backup Restore Tool"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
echo "Backup: $BACKUP_DIR"
|
||||
echo "Output: $OUTPUT_IMG"
|
||||
echo ""
|
||||
|
||||
SIZE=$(python3 -c "import json; print(json.load(open('$BACKUP_CFG'))['Size'])")
|
||||
BLOCK_COUNT=$(python3 -c "import json; print(len(json.load(open('$BACKUP_CFG'))['Blocks']))")
|
||||
|
||||
echo "Volume size: $((SIZE / 1024 / 1024 / 1024)) GB ($SIZE bytes)"
|
||||
echo "Block count: $BLOCK_COUNT"
|
||||
echo ""
|
||||
|
||||
read -p "Continue? [y/N] " -n 1 -r
|
||||
echo ""
|
||||
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
|
||||
echo "Aborted."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Creating sparse image..."
|
||||
truncate -s "$SIZE" "$OUTPUT_IMG"
|
||||
|
||||
echo "Restoring blocks..."
|
||||
python3 << 'PYEOF'
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
backup_cfg = os.environ['BACKUP_CFG']
|
||||
blocks_dir = os.environ['BLOCKS_DIR']
|
||||
output_img = os.environ['OUTPUT_IMG']
|
||||
|
||||
with open(backup_cfg) as f:
|
||||
data = json.load(f)
|
||||
|
||||
blocks = data['Blocks']
|
||||
total = len(blocks)
|
||||
|
||||
with open(output_img, 'r+b') as img:
|
||||
for i, block in enumerate(blocks):
|
||||
offset = block['Offset']
|
||||
checksum = block['BlockChecksum']
|
||||
|
||||
block_path = f"{blocks_dir}/{checksum[:2]}/{checksum[2:4]}/{checksum}.blk"
|
||||
|
||||
if not os.path.exists(block_path):
|
||||
print(f"Warning: Block not found: {checksum}")
|
||||
continue
|
||||
|
||||
with open(block_path, 'rb') as bf:
|
||||
img.seek(offset)
|
||||
img.write(bf.read())
|
||||
|
||||
if (i + 1) % 1000 == 0 or i + 1 == total:
|
||||
percent = (i + 1) * 100 // total
|
||||
bar = '=' * (percent // 2) + ' ' * (50 - percent // 2)
|
||||
sys.stdout.write(f"\r[{bar}] {percent}% ({i + 1}/{total})")
|
||||
sys.stdout.flush()
|
||||
|
||||
print()
|
||||
PYEOF
|
||||
|
||||
echo ""
|
||||
echo "========================================"
|
||||
echo " Restore Complete!"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
echo "Image: $OUTPUT_IMG"
|
||||
echo "Size: $(du -sh "$OUTPUT_IMG" | cut -f1)"
|
||||
echo ""
|
||||
echo "To mount:"
|
||||
echo " sudo losetup -fP $OUTPUT_IMG"
|
||||
echo " sudo mount /dev/loopX /mnt/point"
|
||||
echo ""
|
||||
echo "Or directly:"
|
||||
echo " sudo mount -o loop $OUTPUT_IMG /mnt/point"
|
||||
Executable
+135
@@ -0,0 +1,135 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
BACKUP_DIR="${BACKUP_DIR:-/mnt2/homey-backup}"
|
||||
RESTORE_BASE="${RESTORE_BASE:-/mnt/replicas}"
|
||||
|
||||
usage() {
|
||||
echo "Usage: $0 <pvc-name-or-friendly-name> [--dry-run]"
|
||||
echo ""
|
||||
echo "Restores a Longhorn volume backup to the replicas directory."
|
||||
echo ""
|
||||
echo "Arguments:"
|
||||
echo " pvc-name-or-friendly-name The PVC ID or friendly name"
|
||||
echo " --dry-run Show what would be done without copying"
|
||||
echo ""
|
||||
echo "Examples:"
|
||||
echo " $0 nextcloud-data-pvc"
|
||||
echo " $0 nextcloud-data-pvc --dry-run"
|
||||
echo ""
|
||||
echo "WARNING: This will overwrite existing data in $RESTORE_BASE"
|
||||
exit 1
|
||||
}
|
||||
|
||||
if [[ $# -lt 1 ]]; then
|
||||
usage
|
||||
fi
|
||||
|
||||
SEARCH_NAME="$1"
|
||||
DRY_RUN=false
|
||||
|
||||
if [[ "${2:-}" == "--dry-run" ]]; then
|
||||
DRY_RUN=true
|
||||
fi
|
||||
|
||||
MANIFEST="$BACKUP_DIR/manifest.json"
|
||||
|
||||
if [[ ! -f "$MANIFEST" ]]; then
|
||||
echo "Error: Manifest not found at $MANIFEST_DIR"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
find_volume() {
|
||||
local search="$1"
|
||||
|
||||
while IFS= read -r line; do
|
||||
pvc_id=$(echo "$line" | grep -oP '"pvc_id":\s*"\K[^"]+')
|
||||
friendly=$(echo "$line" | grep -oP '"friendly_name":\s*"\K[^"]+')
|
||||
|
||||
if [[ "$pvc_id" == "$search" ]] || [[ "$friendly" == "$search" ]]; then
|
||||
echo "$pvc_id:$friendly"
|
||||
return 0
|
||||
fi
|
||||
done < <(grep -A6 '"volumes"' "$MANIFEST" | grep -E '"pvc_id"|"friendly_name"')
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
VOLUME_INFO=$(find_volume "$SEARCH_NAME")
|
||||
|
||||
if [[ -z "$VOLUME_INFO" ]]; then
|
||||
echo "Error: Volume '$SEARCH_NAME' not found in manifest."
|
||||
echo ""
|
||||
echo "Available volumes:"
|
||||
grep -oP '"friendly_name":\s*"\K[^"]+' "$MANIFEST" | while read -r name; do
|
||||
echo " - $name"
|
||||
done
|
||||
exit 1
|
||||
fi
|
||||
|
||||
PVC_ID="${VOLUME_INFO%%:*}"
|
||||
FRIENDLY_NAME="${VOLUME_INFO#*:}"
|
||||
|
||||
BACKUP_VOLUME_DIR="$BACKUP_DIR/volumes/$PVC_ID"
|
||||
RESTORE_VOLUME_DIR="$RESTORE_BASE/$PVC_ID"
|
||||
|
||||
echo "========================================"
|
||||
echo " Restore Longhorn Volume"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
echo "PVC ID: $PVC_ID"
|
||||
echo "Name: $FRIENDLY_NAME"
|
||||
echo "Source: $BACKUP_VOLUME_DIR"
|
||||
echo "Destination: $RESTORE_VOLUME_DIR"
|
||||
echo "Dry run: $DRY_RUN"
|
||||
echo ""
|
||||
|
||||
if [[ "$DRY_RUN" == "true" ]]; then
|
||||
echo "[DRY RUN] Would copy:"
|
||||
du -sh "$BACKUP_VOLUME_DIR" 2>/dev/null || echo " (size unknown)"
|
||||
echo ""
|
||||
echo "Files to copy:"
|
||||
find "$BACKUP_VOLUME_DIR" -type f | head -20
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [[ -d "$RESTORE_VOLUME_DIR" ]]; then
|
||||
echo "WARNING: Destination already exists!"
|
||||
echo ""
|
||||
read -p "Overwrite existing data? [y/N] " -n 1 -r
|
||||
echo ""
|
||||
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
|
||||
echo "Aborted."
|
||||
exit 1
|
||||
fi
|
||||
echo ""
|
||||
echo "Removing existing data..."
|
||||
sudo rm -rf "$RESTORE_VOLUME_DIR"
|
||||
fi
|
||||
|
||||
echo "Creating destination directory..."
|
||||
sudo mkdir -p "$RESTORE_VOLUME_DIR"
|
||||
|
||||
echo "Copying volume data..."
|
||||
sudo rsync -a --no-owner --no-group --info=progress2 \
|
||||
"$BACKUP_VOLUME_DIR/" \
|
||||
"$RESTORE_VOLUME_DIR/"
|
||||
|
||||
echo ""
|
||||
echo "Setting permissions..."
|
||||
sudo chmod 700 "$RESTORE_VOLUME_DIR"
|
||||
|
||||
echo ""
|
||||
echo "========================================"
|
||||
echo " Restore Complete!"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
echo "Restored to: $RESTORE_VOLUME_DIR"
|
||||
echo ""
|
||||
echo "Size:"
|
||||
sudo du -sh "$RESTORE_VOLUME_DIR"
|
||||
echo ""
|
||||
echo "Next steps:"
|
||||
echo "1. Ensure Longhorn is configured to use $RESTORE_BASE"
|
||||
echo "2. Restart Longhorn or the affected pod"
|
||||
echo "3. Verify data integrity"
|
||||
+119
-1
@@ -5,7 +5,7 @@ metadata:
|
||||
name: ldap-pvc
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteMany
|
||||
- ReadWriteMany
|
||||
resources:
|
||||
requests:
|
||||
storage: 100Mi
|
||||
@@ -548,3 +548,121 @@ spec:
|
||||
port:
|
||||
number: 80
|
||||
---
|
||||
#START RADICALE
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: radicale-pvc
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteMany
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
storageClassName: longhorn
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: radicale-conf
|
||||
labels:
|
||||
app: radicale
|
||||
data:
|
||||
config: |-
|
||||
{{ tpl (.Files.Get "files/radicale-configmap.ini" | indent 4) . }}
|
||||
---
|
||||
{{- $_ := set $ "homey_radicale_basic_auth" (include "homey.lookuporgensecret" (merge (dict "secretname" "radicale-basic-auth") $))}}
|
||||
{{ include "homey.randomsecret" (merge (dict "secretname" "radicale-basic-auth" "secretval" .homey_radicale_basic_auth) $) }}
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: radicale
|
||||
labels:
|
||||
app.kubernetes.io/name: radicale
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: radicale
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: radicale
|
||||
spec:
|
||||
containers:
|
||||
- name: radicale
|
||||
image: tomsquest/docker-radicale
|
||||
imagePullPolicy: IfNotPresent
|
||||
ports:
|
||||
- name: dav
|
||||
containerPort: 5232
|
||||
protocol: TCP
|
||||
volumeMounts:
|
||||
- name: collections
|
||||
mountPath: /data/collections
|
||||
- name: config
|
||||
mountPath: /config/config
|
||||
subPath: config
|
||||
readOnly: true
|
||||
restartPolicy: Always
|
||||
volumes:
|
||||
- name: collections
|
||||
persistentVolumeClaim:
|
||||
claimName: radicale-pvc
|
||||
- name: config
|
||||
configMap:
|
||||
name: radicale-conf
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: radicale
|
||||
labels:
|
||||
app.kubernetes.io/name: radicale
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- name: dav
|
||||
port: 5232
|
||||
targetPort: dav
|
||||
selector:
|
||||
app.kubernetes.io/name: radicale
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: radicale-dav
|
||||
annotations:
|
||||
kubernetes.io/ingress.allow-http: "false"
|
||||
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
|
||||
nginx.ingress.kubernetes.io/auth-method: GET
|
||||
nginx.ingress.kubernetes.io/auth-url: http://authelia.{{ .Release.Namespace }}.svc.cluster.local:9091/api/verify
|
||||
nginx.ingress.kubernetes.io/auth-signin: https://auth.{{ .Values.homey.url }}?rm=$request_method
|
||||
nginx.ingress.kubernetes.io/auth-response-headers: Remote-User,Remote-Name,Remote-Groups,Remote-Email
|
||||
nginx.ingress.kubernetes.io/auth-snippet: |
|
||||
proxy_set_header X-Forwarded-Method $request_method;
|
||||
auth_request_set $user $upstream_http_remote_user;
|
||||
auth_request_set $groups $upstream_http_remote_groups;
|
||||
auth_request_set $name $upstream_http_remote_name;
|
||||
auth_request_set $email $upstream_http_remote_email;
|
||||
proxy_set_header X-Remote-User $user;
|
||||
proxy_set_header X-Remote-Fullname $name;
|
||||
proxy_set_header X-Remote-Email $email;
|
||||
spec:
|
||||
ingressClassName: {{ .Values.homey.ingress_class }}
|
||||
tls:
|
||||
- hosts:
|
||||
- dav.{{ .Values.homey.url }}
|
||||
secretName: {{ .Values.homey.certname }}
|
||||
rules:
|
||||
- host: dav.{{ .Values.homey.url }}
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: radicale
|
||||
port:
|
||||
number: 5232
|
||||
|
||||
+12
-5
@@ -51,8 +51,18 @@ spec:
|
||||
spec:
|
||||
containers:
|
||||
- name: jellyfin
|
||||
image: docker.io/jellyfin/jellyfin
|
||||
imagePullPolicy: "IfNotPresent"
|
||||
image: jellyfin/jellyfin:10.11.6
|
||||
ports:
|
||||
- containerPort: 8096
|
||||
name: http
|
||||
env:
|
||||
- name: PGUID
|
||||
value: "1000"
|
||||
- name: PUID
|
||||
value: "1000"
|
||||
- name: JELLYFIN_PublishedServerUrl
|
||||
value: jellyfin.{{ .Values.homey.url }}
|
||||
imagePullPolicy: "Always"
|
||||
volumeMounts:
|
||||
- name: jellyfin-volume-config
|
||||
mountPath: "/config"
|
||||
@@ -63,9 +73,6 @@ spec:
|
||||
- name: jellyfin-volume-data
|
||||
mountPath: "/data/tvshows"
|
||||
subPath: downloads/tvshows
|
||||
env:
|
||||
- name: JELLYFIN_PublishedServerUrl
|
||||
value: jellyfin.{{ .Values.homey.url }}
|
||||
volumes:
|
||||
- name: jellyfin-volume-config
|
||||
persistentVolumeClaim:
|
||||
|
||||
@@ -5,7 +5,7 @@ metadata:
|
||||
name: radicale-pvc
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteMany
|
||||
- ReadWriteMany
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
|
||||
+1
-1
@@ -59,7 +59,7 @@ affinity: {}
|
||||
homey:
|
||||
organization: "Zakobar Home Server"
|
||||
url: zakobar.com
|
||||
ip: 10.0.0.100
|
||||
ip: 192.168.1.100
|
||||
certname: zakobarcert
|
||||
ingress_class: nginx
|
||||
|
||||
|
||||
Reference in New Issue
Block a user