From e0013bc077a746f01f0e0e680f67312f846aec80 Mon Sep 17 00:00:00 2001 From: Isaias Tavares Date: Mon, 2 Nov 2026 17:23:09 -0300 Subject: [PATCH] docs(db): document running TREK on Postgres, with compose and Helm support - wiki/Database-Postgres.md: requirements (PostgreSQL 14 to 17, an empty database of its own, a role that is not a superuser, ICU), configuration and SSL, first boot, upgrades and the .pgdump copy, backups and restore, limitations (no SQLite to Postgres move yet, one replica), notes for Supabase, and the manual checklist still to run against it. - docker-compose.postgres.yml: TREK beside a postgres:17 service. - Helm: database.url (written into the chart's Secret) or database.existingSecret supplies TREK_DB_URL; replicas stay at one, now for the state TREK keeps in memory. - Environment variables, backups, updating, troubleshooting, the Docker Compose and Helm install pages, the sidebar and the README point at it. --- README.md | 4 + charts/README.md | 1 + charts/trek/templates/deployment.yaml | 21 +++- charts/trek/templates/secret.yaml | 6 ++ charts/trek/values.yaml | 24 ++++- docker-compose.postgres.yml | 87 +++++++++++++++ server/.env.example | 2 +- wiki/Backups.md | 10 +- wiki/Database-Postgres.md | 147 ++++++++++++++++++++++++++ wiki/Environment-Variables.md | 9 +- wiki/Install-Docker-Compose.md | 17 ++- wiki/Install-Helm.md | 29 ++++- wiki/Troubleshooting.md | 56 ++++++++++ wiki/Updating.md | 19 +++- wiki/_Sidebar.md | 1 + 15 files changed, 414 insertions(+), 19 deletions(-) create mode 100644 docker-compose.postgres.yml create mode 100644 wiki/Database-Postgres.md diff --git a/README.md b/README.md index 84e184bee..421f5face 100644 --- a/README.md +++ b/README.md @@ -319,6 +319,10 @@ docker compose up -d See [Install with Docker Compose](https://github.com/liketrek/TREK/wiki/Install-Docker-Compose) for the full walkthrough. +TREK stores its data in SQLite by default. Set `TREK_DB_URL` to run it on a PostgreSQL 14 to 17 database instead. The +[`docker-compose.postgres.yml`](docker-compose.postgres.yml) example starts one beside TREK, and the +[Postgres guide](https://github.com/liketrek/TREK/wiki/Database-Postgres) covers the requirements and the backups. +

Helm (Kubernetes)

diff --git a/charts/README.md b/charts/README.md index c935c87d5..56cace6ac 100644 --- a/charts/README.md +++ b/charts/README.md @@ -45,6 +45,7 @@ See `values.yaml` for more options. - PVCs use the cluster's default StorageClass. Set `persistence.data.storageClassName` and/or `persistence.uploads.storageClassName` to bind a specific class. - To use your own PVCs, set `persistence.data.existingClaim` and/or `persistence.uploads.existingClaim`. The other values for that volume (size, storageClassName, annotations) are then ignored. - With `persistence.enabled: false`, the data and uploads volumes use an `emptyDir` — storage is ephemeral and lost on pod restart. Intended for testing only. +- `database.url` runs TREK on a PostgreSQL database instead of the SQLite file. The chart writes it into its Secret as `TREK_DB_URL`; `database.existingSecret` and `database.existingSecretKey` read it from a Secret you manage. Leave both empty for SQLite. Keep `replicas: 1` either way. See the wiki page `Database-Postgres`. - `JWT_SECRET` is managed entirely by the server — auto-generated into the data PVC on first start and rotatable via the admin panel (Settings → Danger Zone). No Helm configuration needed. - `ENCRYPTION_KEY` encrypts stored secrets (API keys, MFA, SMTP, OIDC) at rest. Recommended: set via `secretEnv.ENCRYPTION_KEY` or `existingSecret`. If left empty, the server falls back automatically: existing installs use `data/.jwt_secret` (no action needed on upgrade); fresh installs auto-generate a key persisted to the data PVC. - Liveness and readiness probes are configurable under `probes`. Overrides are merged over the defaults, so `--set probes.liveness.initialDelaySeconds=60` keeps the `httpGet` handler. When switching to `exec` or `tcpSocket`, clear `httpGet` in the same override. Set `probes.liveness: null` to remove a probe. diff --git a/charts/trek/templates/deployment.yaml b/charts/trek/templates/deployment.yaml index 7e426593f..fe844e908 100644 --- a/charts/trek/templates/deployment.yaml +++ b/charts/trek/templates/deployment.yaml @@ -1,3 +1,6 @@ +{{- if and .Values.database.url (not .Values.database.existingSecret) .Values.existingSecret }} +{{- fail "database.url is written into the chart's own Secret, which existingSecret replaces. Put TREK_DB_URL in your Secret and set database.existingSecret instead." }} +{{- end }} apiVersion: apps/v1 kind: Deployment metadata: @@ -5,11 +8,13 @@ metadata: labels: app: {{ include "trek.name" . }} spec: + # One replica. Its rate limits, pending codes, permission cache and WebSocket + # connections live in the process's memory, so a second pod would not share them. That + # holds with database.url (PostgreSQL) too; see updateStrategy in values.yaml. replicas: 1 - # TREK is a single-writer SQLite app on a ReadWriteOnce PVC, so the default - # RollingUpdate would start a second pod holding the same volume before the old one - # exits — a Multi-Attach deadlock, or two processes on one travel.db. Recreate tears - # the old pod down first. Keep it, see updateStrategy in values.yaml. + # The default RollingUpdate would start a second pod on the same ReadWriteOnce PVC + # before the old one exits: a Multi-Attach deadlock, or two processes on one travel.db. + # Recreate tears the old pod down first. Keep it, see updateStrategy in values.yaml. strategy: type: {{ .Values.updateStrategy | default "Recreate" }} selector: @@ -61,6 +66,14 @@ spec: name: {{ default (printf "%s-secret" (include "trek.fullname" .)) .Values.existingSecret }} key: {{ .Values.existingSecretKey | default "ENCRYPTION_KEY" }} optional: true + {{- if or .Values.database.url .Values.database.existingSecret }} + # Required when set: a missing key stops the pod rather than starting it on SQLite. + - name: TREK_DB_URL + valueFrom: + secretKeyRef: + name: {{ default (printf "%s-secret" (include "trek.fullname" .)) .Values.database.existingSecret }} + key: {{ if .Values.database.existingSecret }}{{ .Values.database.existingSecretKey | default "TREK_DB_URL" }}{{ else }}TREK_DB_URL{{ end }} + {{- end }} - name: ADMIN_EMAIL valueFrom: secretKeyRef: diff --git a/charts/trek/templates/secret.yaml b/charts/trek/templates/secret.yaml index eacc4da4e..f9c1440e2 100644 --- a/charts/trek/templates/secret.yaml +++ b/charts/trek/templates/secret.yaml @@ -8,6 +8,9 @@ metadata: type: Opaque data: {{ .Values.existingSecretKey | default "ENCRYPTION_KEY" }}: {{ .Values.secretEnv.ENCRYPTION_KEY | b64enc | quote }} + {{- if .Values.database.url }} + TREK_DB_URL: {{ .Values.database.url | b64enc | quote }} + {{- end }} {{- if .Values.secretEnv.ADMIN_EMAIL }} ADMIN_EMAIL: {{ .Values.secretEnv.ADMIN_EMAIL | b64enc | quote }} {{- end }} @@ -65,6 +68,9 @@ stringData: {{- else }} {{ .Values.existingSecretKey | default "ENCRYPTION_KEY" }}: {{ randAlphaNum 32 | quote }} {{- end }} + {{- if .Values.database.url }} + TREK_DB_URL: {{ .Values.database.url | quote }} + {{- end }} {{- if .Values.secretEnv.ADMIN_EMAIL }} ADMIN_EMAIL: {{ .Values.secretEnv.ADMIN_EMAIL | quote }} {{- end }} diff --git a/charts/trek/values.yaml b/charts/trek/values.yaml index 278cd05d7..91fd20566 100644 --- a/charts/trek/values.yaml +++ b/charts/trek/values.yaml @@ -4,10 +4,13 @@ image: # tag: latest pullPolicy: IfNotPresent -# Deployment update strategy. Keep Recreate: TREK is a single-writer SQLite app, and -# during a RollingUpdate the new pod migrates the database while the old one still -# writes to it. A ReadWriteMany volume does not make that safe; SQLite's WAL needs both -# processes on one host, which network filesystems do not provide. +# Deployment update strategy. Keep Recreate. The new pod migrates the database when it +# starts, and during a RollingUpdate it would do that while the old pod still writes to +# it. Recreate stops the old pod first. On SQLite two processes on one travel.db are +# unsafe, and a ReadWriteOnce volume cannot be attached to both pods at once. +# With database.url the database is PostgreSQL, but the reason for one replica stays: TREK +# keeps rate limits, pending codes, the permission cache and WebSocket connections in +# memory, so a second pod would not share them. Postgres does not make more replicas safe. updateStrategy: Recreate # Pod-level security context. fsGroup gives the node user (uid/gid 1000) group access @@ -174,6 +177,19 @@ env: # address is sent unless you set one here. +# Database. Empty (the default) keeps TREK on its SQLite file in the data volume. Set url to +# run TREK on a PostgreSQL database instead (see the wiki page Database-Postgres). The URL +# carries the password, so it is never a plain env value: with url, the chart writes it into +# its own Secret as TREK_DB_URL; with existingSecret, TREK reads TREK_DB_URL from that Secret +# under existingSecretKey and the chart writes nothing. existingSecret wins over url. The +# variable is required when either is set: a missing Secret key stops the pod instead of +# starting it on SQLite. +# url: "postgres://trek:@postgres.example.internal:5432/trek?sslmode=require" +database: + url: "" + existingSecret: "" + existingSecretKey: TREK_DB_URL + # Secret environment variables stored in a Kubernetes Secret. # JWT_SECRET is managed entirely by the server (auto-generated into the data PVC, # rotatable via the admin panel) — it is not configured here. diff --git a/docker-compose.postgres.yml b/docker-compose.postgres.yml new file mode 100644 index 000000000..dc5aaa17f --- /dev/null +++ b/docker-compose.postgres.yml @@ -0,0 +1,87 @@ +# TREK on a Postgres database instead of the SQLite file. See the wiki page +# Database-Postgres for what this changes, the requirements and the backups. +# +# Set the database password in a .env file beside this file (or in the +# shell) before the first start: +# POSTGRES_PASSWORD= +# The password goes into TREK_DB_URL below, so keep it to URL-safe characters +# or percent-encode it. Without POSTGRES_PASSWORD the postgres image refuses to +# initialize its database, and TREK does not start. +services: + db: + image: postgres:17 + container_name: trek-db + restart: unless-stopped + environment: + - POSTGRES_DB=${POSTGRES_DB:-trek} + - POSTGRES_USER=${POSTGRES_USER:-trek} + - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-} + security_opt: + - no-new-privileges:true + # No ports are published: only the trek service on the compose network reaches the database. + volumes: + - trek-pgdata:/var/lib/postgresql/data + healthcheck: + test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"] + interval: 10s + timeout: 5s + retries: 5 + start_period: 10s + logging: + driver: json-file + options: + max-size: 10m + max-file: "3" + + app: + image: mauriceboe/trek:latest + container_name: trek + depends_on: + db: + condition: service_healthy + read_only: true + security_opt: + - no-new-privileges:true + cap_drop: + - ALL + cap_add: + - CHOWN + - SETUID + - SETGID + # dumb-init runs as root and forwards SIGTERM to node, which runs as the + # node user; signalling another user's process needs KILL. See docker-compose.yml. + - KILL + tmpfs: + - /tmp:noexec,nosuid,size=128m + ports: + - "3000:3000" + environment: + - NODE_ENV=production + - PORT=3000 + - ENCRYPTION_KEY=${ENCRYPTION_KEY:-} # Recommended. Generate with: openssl rand -hex 32. + - TZ=${TZ:-UTC} + - LOG_LEVEL=${LOG_LEVEL:-info} + - ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-} + # The database URL. Its host is the service name above; sslmode=disable is + # right on the compose network. Use sslmode=verify-full with sslrootcert= + # for a database reached over a network you do not control. + - TREK_DB_URL=postgres://${POSTGRES_USER:-trek}:${POSTGRES_PASSWORD:-}@db:5432/${POSTGRES_DB:-trek}?sslmode=disable + # Scale-out is not supported: keep one trek container (see Database-Postgres). + volumes: + - ./data:/app/data + - ./uploads:/app/uploads + restart: unless-stopped + logging: + driver: json-file + options: + max-size: 10m + max-file: "3" + healthcheck: + test: ["CMD-SHELL", "wget -qO- http://localhost:$${PORT:-3000}/api/health || exit 1"] + interval: 30s + timeout: 10s + retries: 3 + start_period: 15s + +volumes: + trek-pgdata: diff --git a/server/.env.example b/server/.env.example index 68b86d0af..902b89f05 100644 --- a/server/.env.example +++ b/server/.env.example @@ -56,7 +56,7 @@ DEMO_MODE=false # Demo mode: resets data hourly # TREK_DB_PG_BIN_DIR=/usr/lib/postgresql/17/bin # The directory holding pg_dump, pg_restore and psql, which back up and restore a Postgres database. Unset, TREK looks in /usr/lib/postgresql/<17..14>/bin, /usr/local/bin and /usr/bin (the Docker image installs the version 17 tools, which cannot dump a newer server). Unused on SQLite. # TREK_DB_POOL_MAX=10 # Most connections TREK opens to the Postgres database (1 to 100). Unused on SQLite. (default: 10) # TREK_DB_JOURNAL_MODE=WAL # SQLite journal mode (DELETE, TRUNCATE, PERSIST, MEMORY, WAL, OFF). WAL is fastest and the right choice on local disk. Set DELETE if the data directory lives on network storage (Azure App Service, an SMB/NFS volume on a NAS) — WAL coordinates writers through a shared-memory file that those filesystems do not implement reliably. The mode is stored inside the database file, so it survives a restart; changing this applies on the next boot. -# TREK_DB_PRE_MIGRATE_SNAPSHOT=true # Copy the database to data/pre-migrate--' > .env +docker compose -f docker-compose.postgres.yml up -d +``` + +The password goes into `TREK_DB_URL`, so use URL-safe characters or percent-encode it. Without `POSTGRES_PASSWORD` the database does not initialize and TREK does not start. To run it against a database that already exists elsewhere, use the plain `docker-compose.yml` and set `TREK_DB_URL` yourself. + +Read [Database: Postgres](Database-Postgres) before you use it. It covers the requirements, SSL, backups and restores, and the limits of a Postgres install. + ## Next Steps - [Environment-Variables](Environment-Variables) — full variable reference diff --git a/wiki/Install-Helm.md b/wiki/Install-Helm.md index 9c139880a..d6e92bba9 100644 --- a/wiki/Install-Helm.md +++ b/wiki/Install-Helm.md @@ -177,6 +177,33 @@ Every value the chart writes into its Secret is quoted, so a password such as `1 > **Note:** Without `UNSPLASH_ACCESS_KEY` the server queries Unsplash's unauthenticated endpoint, which many datacenter and VPS IP ranges — including a lot of Kubernetes clusters — are blocked or rate-limited on. Trip-cover and place-image search then fail with **"Unsplash search unavailable"**. A free Access Key switches the server to Unsplash's authenticated API (`api.unsplash.com`), which is not subject to that block. The key can also be set in **Admin → Settings → API Keys**; this value takes priority over that one. See [Environment-Variables](Environment-Variables#image-search-unsplash). +### Running on Postgres + +To run TREK on a PostgreSQL database, set `database.url`. The connection URL carries the password, so the chart writes it into its own Secret as `TREK_DB_URL` rather than into the ConfigMap: + +```yaml +database: + url: "postgres://trek:@postgres.example.internal:5432/trek?sslmode=require" +``` + +To keep the URL out of your values file, create a Secret yourself and point the chart at it. The key holds the same URL: + +```bash +kubectl create secret generic trek-db --from-literal=TREK_DB_URL='postgres://trek:@postgres.example.internal:5432/trek?sslmode=require' +``` + +```yaml +database: + existingSecret: trek-db + existingSecretKey: TREK_DB_URL # the default +``` + +The chart then reads `TREK_DB_URL` from that Secret. If the key is missing, the pod does not start. It does not fall back to SQLite. The chart refuses to render when `database.url` and the top-level `existingSecret` are both set, because the chart's own Secret, which holds the URL, is not created in that case. Use `database.existingSecret` instead. + +The database must be empty and dedicated to TREK, with a role that is not a superuser. The data and uploads volumes are still needed: they hold the data directory, the uploads and the copies TREK takes before migrating. The PostgreSQL requirements, the SSL settings (`sslmode`, `sslrootcert` with a CA file mounted into the pod) and the backups are on [Database: Postgres](Database-Postgres). + +Keep `replicas: 1` on Postgres too. TREK keeps rate limits, pending codes, the permission cache and WebSocket connections in memory, so a second pod would not share them. See [Limitations](Database-Postgres#limitations). + ### Persistent Storage The data PVC holds the database, the plugins you install and the automatic `pre-migrate--