diff --git a/wiki/Admin-Panel-Overview.md b/wiki/Admin-Panel-Overview.md index 9fddadb3c..880bd674b 100644 --- a/wiki/Admin-Panel-Overview.md +++ b/wiki/Admin-Panel-Overview.md @@ -19,6 +19,7 @@ The Admin Panel is divided into tabs. Most tabs are always visible; a few appear | **User Defaults** | Default settings applied to new users | No | | **Addons** | Enable or disable optional features instance-wide | No | | **Plugins** | Install, update, and manage plugins; rescan the plugins folder; review each plugin's capability audit. See [Admin-Plugins](Admin-Plugins) | No | +| **Storage** | Storage backends, category assignment, replication, health | Hidden on managed instances | | **Settings** | Authentication methods, MFA, allowed file types, API keys, OIDC/SSO configuration, and JWT secret rotation | No | | **Notifications** | SMTP, webhook, ntfy, and push notification channel configuration; trip reminder toggle; admin notification preferences | No | | **Backup** | Manual and scheduled database backups | No | @@ -45,5 +46,6 @@ See [Audit-Log](Audit-Log) for details on the hash chain and how the two logs di - [Admin-Categories](Admin-Categories) - [Admin-Packing-Templates](Admin-Packing-Templates) - [Admin-Permissions](Admin-Permissions) +- [[Admin: Storage|Admin-Storage]] - [Admin-MCP-Tokens](Admin-MCP-Tokens) - [Admin-GitHub-Releases](Admin-GitHub-Releases) diff --git a/wiki/Admin-Storage.md b/wiki/Admin-Storage.md new file mode 100644 index 000000000..29b6d628f --- /dev/null +++ b/wiki/Admin-Storage.md @@ -0,0 +1,130 @@ +# Admin: Storage + +The **Storage** tab (Admin Panel → Storage) controls where TREK stores each +kind of content: which storage backends exist, which content category writes +to which backend, and whether writes are replicated to additional backends. +On managed/hosted instances this tab is hidden — storage is the operator's +concern. + +## Backends + +A backend is a named place bytes can live. Two exist out of the box: + +| Backend | What it is | +|---------|------------| +| `uploads-local` | The local `uploads/` directory (the `/app/uploads` volume in Docker) | +| `backups-local` | The local `data/backups` directory | + +You can **edit** a built-in to relocate its root directory, and **add** +backends of type: + +- **Local** — a directory on a disk or mount reachable by the server. +- **S3** — any S3-compatible service (AWS S3, Cloudflare R2, Backblaze B2, + Garage, MinIO/AIStor). Endpoint URL, bucket, credentials, and optionally + region, key prefix, retries, and timeout. For self-hosted endpoints, use an + IP address or `localhost` in the URL unless your server is configured for + virtual-hosted bucket addressing. + +If `TREK_PLACE_PHOTO_DIR` is set, a read-only `place-photos-local` backend +appears too — it is defined by that environment variable, not by this panel. + +### Replication (Mirror targets) + +Edit any backend and tick one or more **Mirror targets**: every write to that +backend is then also copied to each selected target. Removing all targets +turns replication off again. + +- Writes go to the backend itself first (it stays the source of truth), then + to each target in order. A slow or unreachable target slows every upload of + every category on that backend — ideal for backups, worth weighing before + replicating hot categories like trip documents. +- Replica failures **never fail the original request**. They are recorded and + shown in the **Health** strip at the top of the tab, with the backend, the + object, the operation, and how long ago it happened. An empty strip means + every replicated write landed. +- Replication starts with the next write — objects that existed before you + added a target are not copied to it. + +### Test + +Every backend row has a **Test** button: the server writes, checks, and +deletes a small probe object (`trek-probe/…`) on the backend — for a +replicated backend, on the backend itself and each target individually — and +reports per-target results. "Connection OK" means credentials, bucket, and +reachability all check out. Targets must be **saved** before Test can probe +them; a just-added, unsaved backend used as a target reports an error until +you save. + +## Categories + +Every kind of content TREK stores belongs to one of eight categories, each +assigned to exactly one backend: + +| Category | Id | What it stores | +|----------|----|----------------| +| Trip documents | `files` | File attachments uploaded to trips — tickets, PDFs, booking confirmations, and files shared in trip chat | +| Journey photos | `journey` | Photos and thumbnails attached to journey entries | +| Cover images | `covers` | Trip and collection cover images, including covers fetched from Unsplash | +| Profile pictures | `avatars` | User account profile pictures | +| Place images | `places` | Images attached to places and collection places — uploaded or imported | +| Google photo cache | `photos-google` | Cached copies of Google Places photos — re-fetchable, safe to lose | +| TREK photo cache | `photos-trek` | Cached photos from the TREK photo service used by Memories — re-fetchable, safe to lose | +| Backups | `backups` | Server backup archives created by the Backup panel or schedule | + +Reassigning a category changes where **new** objects go — existing objects do +not move and keep being served from wherever they already are. The panel +shows this warning inline when you change an assignment. Replicating the two +re-fetchable caches is possible but flagged as not recommended. + +The legacy `/uploads/photos` directory written by older TREK versions is not +a category: its files are still served and included in backups, but nothing +writes there anymore and it cannot be reassigned. + +## Secrets and encryption + +Backend credentials (the S3 secret access key) are stored encrypted. Saving a +credentialed backend requires the `ENCRYPTION_KEY` environment variable to be +set explicitly — without it the panel shows a notice instead of the save +button. See [[Encryption Key Rotation|Encryption-Key-Rotation]] for changing +the key later. Saved secrets display as a masked placeholder; leaving the mask untouched +when editing keeps the stored value. + +## Provisioning at first boot (seed file) + +For scripted deployments, place a `storage-config.json` in the data directory +(in Docker: mount it at `/app/data/storage-config.json`). It is imported +exactly once — on the first boot that has no stored storage configuration — +then ignored (a log line says so). An invalid file aborts boot with the exact +validation error. Example and the compose mount line: see the +[README Storage section](https://github.com/liketrek/TREK#storage). + +To reset storage configuration (or re-import a seed file): stop the server, +`sqlite3 data/travel.db "DELETE FROM app_settings WHERE key LIKE 'storage.%';"`, +start it again — it boots on the built-in defaults, or re-imports the seed +file if present. + +**Restore chicken-and-egg:** backups include the storage configuration. If +your only backup lives on S3 and the credentials are inside it: start a fresh +instance, enter the S3 credentials here (or mount a seed file), then restore +from the Backup panel. + +## Removed environment variables + +Storage used to be configured through `TREK_S3_*` and `TREK_UPLOADS_DIR`. +These were **removed in v4** and a server started with any of them set +refuses to boot, naming each offending variable: + +``` + - TREK_S3_ENDPOINT was removed — configure storage in the admin UI or data/storage-config.json. +``` + +Remove the variables and configure the equivalent here. `TREK_PLACE_PHOTO_DIR` +is unaffected. + +## Related pages + +- [[Backups|Backups]] — what a backup contains, restore procedure +- [[Environment Variables|Environment-Variables]] +- [[Encryption Key Rotation|Encryption-Key-Rotation]] +- [[Admin Panel Overview|Admin-Panel-Overview]] +- [[Troubleshooting|Troubleshooting]] diff --git a/wiki/Backups.md b/wiki/Backups.md index 71c368f06..f9a78ea3b 100644 --- a/wiki/Backups.md +++ b/wiki/Backups.md @@ -1,6 +1,6 @@ # Backups -TREK stores all data in a single SQLite database (`travel.db`) plus an `uploads/` directory of attachments, cover photos, and avatars. The Backup panel lets you create, download, restore, and schedule backups of both. +TREK stores all data in a single SQLite database (`travel.db`) plus an `uploads/` directory of attachments, cover photos, and avatars — by default; storage backends and replication are configured in [[Admin: Storage|Admin-Storage]]. The Backup panel lets you create, download, restore, and schedule backups of both. ## Where to find it @@ -15,7 +15,7 @@ A backup is a ZIP archive with these entries: | Entry | Contents | |---|---| | `travel.db` | The full SQLite database | -| `uploads/` | All uploaded attachments, covers, and avatars | +| `uploads/` | All uploaded attachments, covers, and avatars (default location — see [[Admin: Storage|Admin-Storage]]) | | `plugins-data/` | Each installed plugin's own database + files (present only if plugins are installed) | | `plugins-code/` | The installed plugin code, so a restore is self-contained (dev-linked plugins are skipped) | @@ -84,6 +84,7 @@ The following actions are recorded in the [Audit-Log](Audit-Log): ## See also +- [[Admin: Storage|Admin-Storage]] — replicate backups to S3-compatible storage - [Encryption-Key-Rotation](Encryption-Key-Rotation) - [Admin-Panel-Overview](Admin-Panel-Overview) - [Security-Hardening](Security-Hardening) diff --git a/wiki/Environment-Variables.md b/wiki/Environment-Variables.md index c01addc4e..7bdf15e45 100644 --- a/wiki/Environment-Variables.md +++ b/wiki/Environment-Variables.md @@ -275,6 +275,13 @@ To get a key: create a free account at [unsplash.com/developers](https://unsplas ## Storage & Paths +Storage backends and category assignment are configured in +[[Admin: Storage|Admin-Storage]] (or a seed-once `storage-config.json`), not +through environment variables. The former `TREK_S3_*` and `TREK_UPLOADS_DIR` +variables were removed in v4 — a server started with any of them set refuses +to boot and names the offending variable. `TREK_PLACE_PHOTO_DIR` below is +unaffected. + | Variable | Description | Default | |--------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------| | `TREK_PLACE_PHOTO_DIR` | Directory where cached Google place photos are stored. Created recursively on boot. Set this to point photo storage at a dedicated mounted volume. | `uploads/photos/google` | diff --git a/wiki/Troubleshooting.md b/wiki/Troubleshooting.md index 28d6ef815..09f65781f 100644 --- a/wiki/Troubleshooting.md +++ b/wiki/Troubleshooting.md @@ -145,6 +145,16 @@ sudo chown -R 1000:1000 ./data ./uploads --- +## Server refuses to start after updating: "TREK_S3_ENDPOINT was removed" + +Storage is no longer configured through environment variables (v4). Remove +every `TREK_S3_*` variable and `TREK_UPLOADS_DIR` from your run command or +compose file, start the server, and configure the equivalent in +[[Admin: Storage|Admin-Storage]] or via a seed-once `storage-config.json`. +The boot report lists each offending variable by name. + +--- + ## Container won't start: "exec /usr/bin/dumb-init: operation not permitted" **Symptoms:** The container restarts in a loop and the logs show nothing but: diff --git a/wiki/_Sidebar.md b/wiki/_Sidebar.md index 8be0791db..332544376 100644 --- a/wiki/_Sidebar.md +++ b/wiki/_Sidebar.md @@ -94,6 +94,7 @@ - [[Admin: Packing Templates|Admin-Packing-Templates]] - [[Admin: Permissions|Admin-Permissions]] - [[Admin: Plugins|Admin-Plugins]] +- [[Admin: Storage|Admin-Storage]] - [[Admin: MCP Tokens|Admin-MCP-Tokens]] - [[Admin: GitHub Releases|Admin-GitHub-Releases]]