mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-10-18 21:16:47 +00:00
305b113d5f
* feat(storage): storage key validation, driver contract suite, LocalDriver * feat(storage): MirrorDriver composite backend * feat(server): add TREK_UPLOADS_DIR env var for the uploads root * feat(storage): settings-backed registry, StorageService facade, module wiring * test(storage): pin src/nest/storage coverage thresholds * feat(storage): multer upload options factory * refactor(files): spool trip-file uploads through storage * refactor(collab): spool note attachments through storage; normalize files/ prefix * refactor(journey): spool journey media uploads through storage * refactor(trips): spool trip cover upload through storage * refactor(places): spool place-image and collection uploads through storage * refactor(auth): spool avatar upload through storage * fix(server): consolidate allowed_file_types onto AllowedFileTypesService * fix(storage): default defParamCharset to utf8 in the upload factory * test(db): re-point the trailing-migration guard at the trip_files prefix strip * fix(storage): age-gate the boot spool sweep * test(files): key FILE-P02 residue checks on upload size * test(storage): raise src/nest/storage coverage pin * refactor(platform): serve /uploads routes through storage * refactor(files): serve trip-file downloads through storage * refactor(memories): serve photo bytes through storage * refactor(maps): stream place-photo bytes through storage * refactor(share): stream shared place-photo bytes through storage * refactor(journey): serve public journey assets through storage * fix(share): answer place-photo cache miss with 204 like the maps proxy * test(server): close serving-swap gaps surfaced by the full tier * test(storage): raise src/nest/storage branches pin * refactor(place-photos): route place-photo cache through storage * refactor(places): reclaim place images and collection covers through storage * refactor(trips): trip covers through storage * refactor(auth): avatar reclaim through storage * refactor(collab): note-file deletion through storage * refactor(files): trash deletion through storage * refactor(memories): trek photo cache internals through storage * refactor(memories): thumbnail service category-addressed * refactor(memories): immich upload reads through storage; drop uploads-root * refactor(journey): photo deletion through storage * fix(memories): reclaim orphaned trek photo cache objects in the ttl sweep * feat(journey): journey-thumbs orphan sweep and delete-site derived-artifact reclaim * test(server): close slice-4 gaps surfaced by the full tier * test(coverage): raise slice-4 domain pins * refactor(backup): list, existence and delete through storage * refactor(backup): download through storage sendToResponse * refactor(backup): create backup through per-category storage enumeration * refactor(backup): restore and upload-restore through storage * refactor(backup): auto-backup retention through storage * refactor(server): boot dir-creation to driver init * test(server): close slice-5 gaps surfaced by the coverage tier * test(coverage): raise slice-5 backup pin * test(server): resolve async backupFileExists mocks for the tests typecheck * test(server): pin ranged-read stat and zero-byte roundtrip in the storage driver contract * feat(server): TREK_S3_* env wiring — schema, s3 namespace, all-or-nothing boot rule * feat(server): S3 driver core — aws-lite client seam, stat/delete, driver-owned deadlines * refactor(server): type the aws-lite S3 client via s3-types reference * feat(server): S3 driver getStream and list — ranged full-object stat, iterator pagination * feat(server): S3 driver put — bounded-peek routing, Upload inactivity deadline * fix(server): S3 driver Upload monitoring — Transform interposition, stream cleanup * feat(server): env-declared s3-main storage backend; settings-declared s3 rejected * test(server): env-gated S3 driver contract suite against MinIO * fix(server): S3 driver zero-byte put — work around aws4 duplicate Content-Length signing defect * fix(server): scope the zero-byte put workaround to buffered bodies; bind test MinIO to localhost * ci(server): MinIO-backed S3 driver contract job * ci(server): align s3-contract shim comment and MinIO bind with references * docs: S3 storage backend configuration and off-box backup mirroring * docs: restore README formatting, keep S3 additions only * ci(server): make live-MinIO contract runs manual-only (AIStor licensing) MinIO's community line is discontinued entirely; the maintained successor, AIStor, requires a license key that can't safely live in a shared CI runner. The s3-contract CI job is removed; live-provider contract runs against MinIO/AIStor are now a manual, license-gated procedure documented in docker-compose.minio-test.yml's header. * chore(server): final-review polish — stale CI refs, dockerignore key, boundary pins, zero-byte tempfile contract case * expose admin ui * feat(shared): storage backend type registry, wire schemas, category relocation * feat(server): accept settings-declared s3 backends — shared-schema options, decrypt-at-build, registry preview() * feat(server): remove TREK_S3_*/TREK_UPLOADS_DIR — tombstone boot rule, built-in override as the relocation path * feat(server): deriveSecurity.encryptionKeySet and storage secret plumbing (gate, encrypt, unmask, mask, redact) * chore(server): drop storage-secrets import that only Task 8's probe helper will use * feat(server): seed-once storage-config.json boot import — fail closed on any invalid seed * feat(server): registry snapshot() — effective storage config with provenance * feat(server): api/admin/storage GET/PUT — masked secrets, preview-gated saves, wired reload, audits * feat(server): api/admin/storage/test — per-target ephemeral probes, mirror targets probed individually * test(server): e2e coverage for api/admin/storage — guards, managed refusal, envelope, probes * test(server): give e2e harness DBs the app_settings table the storage registry reads at boot * test(coverage): raise pins after the storage admin-config slice * fix(server): final-review polish — encryption-gate wording, all-field sentinel guard, README/e2e notes * test(server): storage slice-1 review follow-ups — mirror s3-replica probe decrypt, all-encrypted seed without key, test-route 401 * feat(i18n): storage admin domain + tab label in every locale * feat(client): adminApi storage endpoints — getStorage, updateStorage, testStorageBackend * feat(client): storage admin draft-document helpers + relative-time util * feat(client): schema-driven storage BackendForm — fields by kind, mask echo, encryption banner * feat(client): admin storage panel — backends list, category map, health strip, one-PUT save * feat(client): storage tab on the admin page * feat(client): storage admin panel and section on the phone shell * fix(client): block renaming a storage backend onto an existing name * feat(server): allow mirrors on any storage category — lift the backups-only rule * feat(i18n): replicas-on-primary mirror strings in every locale * feat(client): mirror fold/synthesize model helpers for replicas-on-primary * feat(client): BackendForm mirror-targets composer; raw mirror type hidden * feat(client): desktop storage panel folds mirrors into their primaries * feat(client): phone storage panel folds mirrors into their primaries * fix(client): strip removed backends from remaining mirror replicas — no phantom refs * feat(storage): retire photos as a configurable category — served-only via the ServedCategory seam * feat(i18n): descriptive storage category names and descriptions in every locale * feat(client): descriptive category names, descriptions and id badges in the storage panel * docs(storage): fix stale nine-category comments after the eight-category shrink * docs: storage configuration via the admin UI and seed file — README + compose * docs(wiki): Admin Storage page and storage cross-links * docs(server): storage domain blueprint entry; TREK_PLACE_PHOTO_DIR tombstone clarifier * refactor(server): drop the storage env-var tombstones — the variables never shipped * docs: storage config is simply new in v4 — drop the removed-variables story * feat(shared): backfill-status and usage wire schemas on the storage admin state * feat(server): MirrorDriver.backfill — prefix-scoped, size-compared, cancellable replica catch-up * feat(server): one-at-a-time storage backfill jobs + the storage events bridge * feat(server): storage usage scan — nightly cron, on-demand service, persisted with computedAt * feat(server): replica_failure admin notifications — events bridge, hourly debounce, channel registration * feat(i18n): replica-failure notification text in every locale * feat(server): storage backfill and stats-refresh admin endpoints * feat(client): adminApi backfill and stats-refresh endpoints * feat(i18n): storage sync and usage strings in every locale * feat(client): desktop storage panel — sync now, usage display, backfill polling * feat(client): phone storage panel — sync now, usage display, backfill polling * fix(client): per-category usage line on the phone storage panel * docs: sync now, usage stats and replica-failure notifications * test(server): admin scope now includes replica_failure, not just version_available Commit6db135daadded replica_failure to ADMIN_SCOPED_EVENTS but left NPREF-005b asserting the admin scope contained only version_available. Update the expectation to match. * test(server): cover storage backfill/stats/jobs branches to hold the coverage pin * fix(client): guard storage admin polling against stale responses racing a save A poll GET issued before a save's PUT resolves could land after the save applied its fresher response, overwriting it with the pre-save world and re-deriving the draft from that stale snapshot. useStorageAdmin now tracks a state-application sequence number and drops any refreshState() response whose sequence has moved on, or that resolves while a save is in flight. Also guard the fire-and-forget refreshState() calls in startBackfill/ cancelBackfill against unhandled rejections, matching the poll's style. * fix(server): report 404 before 409 when a backfill start names an unknown mirror startBackfill checked the global busy flag before validating that :name is a mirror routed by some category, so POST .../backends/garbage/backfill answered 409 instead of 404 whenever a sync was already running. Resolve and validate the target first (BackfillTargetError -> 404), then check busy (BackfillBusyError -> 409). * feat(client): two-column category grid on the storage panel * feat(server): plugin files.getContent reads through the storage layer * feat(server): plugin files.create writes through the storage layer * refactor(server): drop FilesService.resolveFilePath, last direct-fs seam in the plugin surface * fix(server): map StorageInvalidKeyError to the plugin file accessibility envelope * feat(storage): allow credentialed backends without an explicit ENCRYPTION_KEY Secrets are still always encrypted at rest; without an explicit key the implicit data-directory key is used. Removes the plaintext-secret refusal from the admin PUT and the seed import, the encryptionReady GET field, the save-blocking banner in both admin shells, and the now-consumerless security.encryptionKeySet env derivation. Docs now recommend (not require) an explicit key, since the implicit key rides inside backup archives. * test(client): pin the stale-poll release to the pre-save GET in FE-ADMIN-STOR-024 The stub deferred EVERY poll GET and rebound the release handle to the newest one, so a tick landing after the save was released with the pre-save payload — a response no real backend can produce, which the seq guard rightly applied. Only the first (genuinely stale) GET is held now; later GETs answer with the world the mock server currently holds. Also drops the encryptionReady fixture field removed from the schema in the previous commit. * fix(storage): backfill puts carry content-type, stat failures report op stat, progress clamps * feat(storage): category migration job — copy, flip, delta sweep, reclaimable tally * feat(storage): migration start/cancel endpoints and client API * test(client): storage admin fixtures carry the new migrations field * fix(storage): migration POST answers 200 and validates its body via the shared schema * fix(storage): cancel running storage jobs when a config save removes their backend * fix(notifications): in-app replica-failure row carries the suppressed count * feat(client): migration prompt, queue, and progress on the storage panel * fix(client): migration queue holds a synchronous single-flight lock * feat(client): migration prompt and progress on the mobile storage panel * fix(server): register category-migration routes in the managed-route allow list StorageAdminController.migrationStart/migrationCancel were added under the controller-level @ManagedForbidden() decorator but never added to MANAGED_ROUTE_ALLOW_LIST, so validateManagedRoutes() threw "Managed surface changed" on every buildApp() call, failing 46 test files (any test booting the real app). * test(server): cover category-migration copy/sweep branches to hold the storage coverage pin Adds MIG-005 (a target object already matching size/key is skipped in the copy phase itself, not just the delta sweep) and MIG-006 (a copy failure during the delta sweep is counted without undoing the flip that already happened). src/nest/storage/**'s branch coverage was 83.91%, just under the 84% pin. * docs(storage): category migration — move existing objects on reassign * fix(storage): close the pre-flip cancel window and cancel re-routed migrations A cancel landing between the last copy iteration and the category flip was silently ignored, and a config save that re-routed the migrating category to a third, still-defined backend never stopped the in-flight migration. Also guards assignCategory against writing an unknown backend name, and fixes a missing `suppressed` notification param being treated as truthy suppression. * fix(client): migrations keep mirrors, queue waits on backfills, sweep failures surface Reassigning a category onto a mirrored primary now posts the migration to the mirror's wire name instead of the bare primary, so replication survives the move. The queued-migration effect also waits out a running backfill (the server's one-job rule spans both), and a done migration with sweep failures now surfaces a doneFailures line in both shells, translated into every locale. * ci(server): run the S3 contract suite against MinIO Resurrects the s3-contract job removed in6fbff835, adapted for AIStor: writes the MINIO_LICENSE_KEY secret to a license file, boots the licensed container, and runs the contract suite hermetically (no --coverage) so the storage coverage pin keeps holding from unit tests alone. Gated to same-repo PRs since fork PRs have no access to the license secret. * fix(server): backup archives uploads from a remote primary createBackup() enumerated uploads through withLocalFile, which downloads a remote object to a temp file, hands it to the callback, then deletes it before archiver ever reads it — archiver reads entries lazily during finalize(), so a remote (S3) primary silently backed up zero upload bytes. Add StorageService.getLocalPathOrNull(category, name): a narrow locality probe that returns a real path only when one exists on disk right now (the fs-existence check is the fail-safe half of the contract — a driver-reported path that vanished between list() and here falls back too, not just a path-less remote driver). The enumeration loop uses it directly when available (zero-copy, unchanged for a local primary) and otherwise streams the object into a per-backup staging dir under the backups spool, cleaned up in the existing finally alongside zipSpool/dbSnap. Also wire archive.on('warning', reject): archiver treats a missing/unreadable entry as a warning, not an error, and skips it by default — which would silently drop the entry from an otherwise "successful" backup. * fix(server): client aborts stop 500ing and leaking streams storage.service.ts sendToResponse: the local res.sendFile branch now treats ECONNABORTED/ECONNRESET/ERR_STREAM_PREMATURE_CLOSE/syscall=write as success, mirroring res.sendFile's own default callback (the contract storageStaticHandler already implements caller-side). The remote branch replaces the hand-rolled pipe+Promise with pipeline(stream, res), which destroys the source stream on any failure — fixing the leaked S3 keep-alive socket on a client abort — while swallowing the same client-abort error set; a real source error still rejects so callers keep their miss contracts. This centralizes the fix for every caller that streams through sendToResponse: files-download, journey-public, photo-resolver, trek-photo-cache, backup downloads, and /uploads/photos. trek-exception.filter.ts now no-ops (destroying the socket) when headers are already sent, so no filter path ever writes onto a response in flight. maps.controller.ts and share.controller.ts's place-photo bytes proxies switch their direct getStream+pipe to pipeline() with the same abort-tolerant catch, keeping the existing 204-empty fallback for pre-header failures. * fix(storage): mirror read fallback consults replicas on primary miss MirrorDriver.getStream/stat now fall back to replicas on a primary miss (StorageNotFoundError / null stat), not just an infra error — binding decision: a read succeeds if ANY member holds the object. Only a deterministic StorageInvalidKeyError skips fallback. stat() gets a dedicated fallback loop instead of reusing fromReplicas: a replica returning null means "keep looking," which the generic catch-and-continue helper would have short-circuited on. StorageService.sendToResponse/withLocalFile stop treating composite driver.stat() as proof the object is at localPath; both now fs-check the path itself and fall through to the stream branch when it's missing, which is what actually reaches a MirrorDriver's replicas. * feat(storage): remote serving parity — content-type, validators, range Local backends get content type, ETag/Last-Modified, conditional GET, Range and HEAD for free from res.sendFile; the proxying branch every remote backend (and a local primary whose file vanished) falls through to had none of it — every request re-streamed the whole object with no cache validator and no way to seek. sendStreamed() reproduces the contract from ONE pre-flight driver.stat(): a miss throws before any header is set (the caller's 404/204/next() contract), and that same stat then answers 304s, 416s, HEADs and suffix ranges without ever opening a body. The decisions live in a new pure module, http-serving.ts: - validatorsFor: the driver's ETag, else express/send's weak W/"<size hex>-<mtime hex>". mtimeMs 0 (S3 can report it) means no Last-Modified and no derived tag — serving uncached beats a 1970 header that reads as fresh forever. - isNotModified: If-None-Match (weak compare, *, list) wins over If-Modified-Since, per RFC 9110 §13.2.2. - decideRange: single range only, inclusive clamp, suffix computed from the pre-stat size, unsatisfiable → 416, If-Range strong-exact or full entity. A multi-range or malformed spec is answered with the full object rather than multipart/byteranges. If-Match/If-Unmodified-Since stay out of scope. Range math uses the pre-stat size as the single source of truth, the ranged getStream stat is never re-consulted, and the pipeline abort semantics are unchanged. S3Driver now surfaces the bucket's ETag on stat()/getStream() (on a ranged GET it is still the full object's tag). aws-lite strips the tag's surrounding quotes — verified live against AIStor RELEASE.2026-08-07 — and an unquoted entity-tag is not a valid HTTP ETag, so the driver restores exactly the quotes the SDK removed and nothing else. * fix(server): key rotation covers storage.backends The S3 storage backend's secretAccessKey is encrypted at rest inside the app_settings row keyed storage.backends, but scripts/migrate-encryption.ts didn't know about it — after a key rotation that secret stayed encrypted under the old key and silently failed to decrypt on the next read. The script now re-encrypts secretAccessKey for every s3 backend inside the storage.backends JSON array, using a pinned copy of the secret-field list (it deliberately can't import @trek/shared). The parity test pins that copy against storageSecretFields('s3') so a new secret field can't be missed silently. * fix(server): restore reloads the storage registry restoreFromZip closed and reopened the DB but never reloaded the storage registry, so uploads rehydration ran against the STALE pre-restore driver/category map instead of the config the restored archive's own travel.db carries (audit #4, major). StorageService gains reloadConfig(), a narrow passthrough to the (unexported) registry's reload() — same shape as health(). backup.impl's restoreFromZip now calls it right after reinitialize() and right before the uploads wipe/rehydrateUploads/plugin staging, so rehydrated bytes land where the restored config says. When reopen fails (reinitFailed), reload and rehydration are skipped entirely — already reported as "restart required", and there is no live DB handle for the registry to read anyway. * fix(server): category migration rewrites keys across prefix changes Category migration copied objects to the destination backend using the SOURCE key verbatim. A mode-A ('' prefix) source migrating onto a prefixed backend (or vice versa) landed objects at keys the registry could never resolve post-flip (audit #8). Extracts StorageRegistryService.keyPrefixFor(category, backendName) from build()'s inline conditional so startMigration can compute the destination prefix before the flip. runMigration now maps every destination key (destPrefix + key.slice(sourcePrefix.length)) in the copy skip-check, the put, and the delta sweep; the reclaimable tally stays source-keyed. Equal-prefix migrations are unaffected (identity mapping). * fix(plugins): await the now-async place/collection services in RPCs PlacesService.update/remove and CollectionsService.deletePlace became async when they started deleting storage objects, but their plugin RPC handlers still called them synchronously: the null/false refusal check compared against an always-truthy Promise, and places.update broadcast a Promise instead of the resolved place. Both handlers now await the service call; mapCollectionError is async-aware (accepts a sync-or- async thunk) so the sync callers (create/update/savePlace/copyToTrip) keep working through the same path. Verified the RPC host already awaits handler results, so no change was needed there. * fix(storage): backfill Sync now gains a deletion sweep The Sync now backfill copied primary objects to replicas but never removed replica objects the primary no longer has, leaving mirror read-fallback able to serve stale data indefinitely after a failed replica delete. backfill() now sweeps each replica post-copy: any replica key absent from the primary key-set is re-checked against primary.stat (a race gate against the primary regaining the key) before being deleted. Adds a deleted counter end-to-end (driver, jobs service, shared schema, both admin shells, all 23 locales) and a new ReplicaFailure 'list' op for a replica whose listing itself fails during the sweep. * feat(storage): optimistic concurrency for admin config saves A stale admin PUT of storage settings could silently undo a category migration's just-flipped assignment. Adds a version counter (app_settings row storage.config_version) bumped inside the same transaction by both writers — applyConfig and the migration flip (assignCategory). applyConfig compares the submitted version first and throws StorageConflictError, mapped to 409 ahead of the blanket 400. The client carries the draft's loaded version through every local edit and maps a 409 to a distinct "settings changed underneath you" message, refreshing state while preserving the dirty draft for review. * fix(storage): surface a failed config load + redact secretAccessKey Registry load() now records the exact parse/validate failure as lastLoadError() instead of silently falling back to last-good config or built-in defaults; StorageAdminService exposes it as configError on the admin state (storageAdminStateSchema extended additively), and AdminStoragePanel renders it as a warning banner naming the error. Save stays enabled — it's the recovery path — but an admin can no longer save over a broken stored config without knowing. Also adds 'secretaccesskey' to globalMiddleware's SENSITIVE_KEYS request-log redaction list, so storage admin PUT bodies never leak the S3 secret into debug logs. * fix(client): admin storage panel state races Migration prompt now recomputes computeMigrationCandidates(draft, state) on render and again at confirm time, so a stale open-time snapshot never drives stripCategories or the migration queue; adds a Cancel button to the dialog. A failed startMigration POST explicitly clears the remaining queue with a toast naming the dropped categories and refreshes state, instead of leaving it invisibly stranded; a "Queued: X, Y" line renders while the queue is non-empty. Testing a mirrored row now expands the draft primary + replicas client-side and probes each individually, instead of sending the mirror stub to the server (which resolved targets from its own live snapshot, silently testing saved options over unsaved draft edits). A completed/cancelled/errored backfill now renders its terminal status alongside the Sync now button instead of hiding it, so it can be re-run without a page reload. * fix(storage): refuse shared replica backends so the sync sweep cannot cross categories A mirror's "Sync now" sweep makes each replica MATCH the primary: it lists the replica under every prefix the mirror's categories occupy and deletes the keys the primary doesn't hold. The backups category's prefix is '' — the replica's entire root — so making a category-serving backend (say uploads-local) a replica of a backups mirror meant the first sync deleted every user upload. Nothing forbade the config: validateConfig only checked that mirror targets existed and weren't nested, and the admin picker offered every backend. The sweep is correct, so the fix is at the config level: - validateConfig refuses a backend that is both a mirror replica and a category target (directly, or as the primary of a mirror a category routes to), and refuses a backend replicating two mirrors whose swept prefixes overlap — '' overlaps everything. Disjoint prefixes stay legal. - assignCategory (the migration flip, the other writer of the settings rows) refuses routing a category onto a backend that is currently a replica. - The admin mirror-target picker (both shells) only offers backends that serve no category; already-selected targets stay listed so they can be unchecked. A category assigned to a replica still surfaces the server's 400 verbatim. Existing shared-replica fixtures were rewritten onto dedicated replicas. * fix(client): a 409 conflict save is recoverable without a page reload After a 409 the admin storage panel was a dead end. refreshState leaves a dirty draft's version alone on purpose (audit #7: the background poll must never float a mid-edit draft past a migration flip it never carried), setDraft re-attaches that same version to every later edit, and save() resubmits it — so every retry 409s again and only a full page reload got the operator out, while the banner told them to "review and save again". The version block stays. Instead the conflict banner now offers the one action that can clear it: discard the local edits and reload the saved settings — a fresh GET applied through applyState, re-deriving the draft (and its version) and clearing dirty/saveError. Bumping the version in refreshState, or only in the 409 branch, was rejected: either resurrects audit #7, since a full-document save would then revert a flip the draft never carried. storage.saveConflict now points at that action instead of implying a plain re-save works (all 23 locales). The escape lives in the shared hook, so the phone panel renders it too. * fix(client): deflake the queued-migration-waits-out-backfill panel test FE-ADMIN-STOR-037 seeds a running backfill, so the panel polls GET from first render — but the stub served the post-save world (files: off-box) before the PUT had landed, which no real server would do. A poll resolving while the confirm dialog was open made the confirm-time candidate recompute see from === to, so nothing was queued and the migration never POSTed. The GET stub now stays faithful (pre-save world until the PUT lands, usage included so candidates keep their object counts), and the test deterministically forces the once-flaky interleaving: it waits for polls to land while the dialog is open and asserts the candidate survives them. * fix(storage): surface the underlying cause in probe target errors The admin storage Test probe reported only the driver wrapper's outer message ("put failed for '…' on 'probe-s3'"), dropping the cause that says WHY — connection refused vs missing bucket. Flatten the error's cause chain into the probe result, appending an S3-style code when it isn't already in the message. * fix(storage): surface the underlying cause in replica-failure reports Same defect as the probe fix, one layer over: MirrorDriver's reportReplicaFailure captured only the wrapper's message, so replica-failure notifications said "put failed" without the why. Move the probe's cause-chain flattener into storage.types.ts as describeError and use it in both call sites. * fix(storage): strip ContentType from multipart Upload calls (MalformedXML) Every multipart S3 put carrying a contentType — every backup zip and photo over the 10MB threshold — failed with 400 MalformedXML. Upstream @aws-lite/s3's Upload leaks the ContentType param into its final CompleteMultipartUpload headers ({ ...xml, ...paramHeaders }), clobbering application/xml, which flips @aws-lite/client's payload serializer to JSON — so the server receives a JSON parts list where XML belongs. Confirmed live against MinIO AIStor; any S3 refuses the same. uploadMonitored now drops ContentType before s3.Upload. Multipart objects are stored without a content-type, which TREK never reads back (serving derives Content-Type from the key/caller, never object metadata); single-request PutObject paths keep sending it. Also closes the coverage gap that let this ship: the live contract suite never crossed MULTIPART_THRESHOLD, so s3.Upload had never run against a real server — it now round-trips a >10MB put carrying a contentType. --------- Co-authored-by: Maurice <mauriceboe@icloud.com>
615 lines
33 KiB
Markdown
615 lines
33 KiB
Markdown
<div align="center">
|
|
|
|
<picture>
|
|
<source media="(prefers-color-scheme: dark)" srcset="docs/logo-trek-light.svg" />
|
|
<source media="(prefers-color-scheme: light)" srcset="docs/logo-trek-dark.svg" />
|
|
<img src="docs/logo-trek-dark.svg" alt="TREK" height="96" />
|
|
</picture>
|
|
|
|
<br />
|
|
<picture>
|
|
<source media="(prefers-color-scheme: dark)" srcset="docs/subtitle-light.png" />
|
|
<source media="(prefers-color-scheme: light)" srcset="docs/subtitle-dark.png" />
|
|
<img src="docs/subtitle-dark.png" alt="your trip. your plan." height="28" />
|
|
</picture>
|
|
|
|
A self-hosted, real-time collaborative travel planner — with maps, budgets, packing lists, a journal, and AI built in.
|
|
|
|
<br />
|
|
|
|
<a href="https://demo.liketrek.com"><img alt="Demo" src="https://img.shields.io/badge/Demo-try-111827?style=for-the-badge" /></a>
|
|
|
|
<a href="https://hub.docker.com/r/mauriceboe/trek"><img alt="Docker" src="https://img.shields.io/badge/Docker-ready-2496ED?style=for-the-badge" /></a>
|
|
|
|
<a href="https://discord.gg/NhZBDSd4qW"><img alt="Discord" src="https://img.shields.io/badge/Discord-join-5865F2?style=for-the-badge" /></a>
|
|
|
|
<a href="https://kanban.pakulat.org/shared/I4wxF6inOOMB0C6hH6kQm3efyNxFjwyI"><img alt="Roadmap" src="https://img.shields.io/badge/Roadmap-view-0EA5E9?style=for-the-badge" /></a>
|
|
<br />
|
|
<a href="https://ko-fi.com/mauriceboe"><img alt="Ko-fi" src="https://img.shields.io/badge/Ko--fi-support-FF5E5B?style=for-the-badge" /></a>
|
|
|
|
<a href="https://www.buymeacoffee.com/mauriceboe"><img alt="BMAC" src="https://img.shields.io/badge/BMAC-support-FFDD00?style=for-the-badge" /></a>
|
|
<br />
|
|
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-AGPL_v3-6B7280?style=flat-square" /></a>
|
|
<a href="https://github.com/liketrek/TREK/releases"><img alt="Latest Release" src="https://img.shields.io/github/v/release/liketrek/trek?include_prereleases&style=flat-square&color=6B7280" /></a>
|
|
<a href="https://hub.docker.com/r/mauriceboe/trek"><img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/mauriceboe/trek?style=flat-square&color=6B7280" /></a>
|
|
<a href="https://github.com/liketrek/TREK"><img alt="Stars" src="https://img.shields.io/github/stars/liketrek/trek?style=flat-square&color=6B7280" /></a>
|
|
|
|
</div>
|
|
|
|
---
|
|
|
|
<div align="center">
|
|
|
|
<img src="https://github.com/mauriceboe/trek-media/releases/download/readme-assets/TREK1.webp" alt="TREK, a tour of the app" width="100%" />
|
|
|
|
</div>
|
|
|
|
<br />
|
|
|
|
## What you get
|
|
|
|
<picture>
|
|
<source media="(max-width: 700px)" srcset="docs/tiles/grid-mobile.svg" />
|
|
<source media="(prefers-color-scheme: dark)" srcset="docs/tiles/grid-desktop-dark.svg" />
|
|
<img src="docs/tiles/grid-desktop.svg" alt="Plan, track, share" width="100%" />
|
|
</picture>
|
|
|
|
<details>
|
|
<summary><b>See all features</b></summary>
|
|
|
|
<br />
|
|
|
|
Most of what follows is an addon an admin switches on or off. Lists, Costs, Documents, Collab, Vacay and Atlas ship on; Journey, Collections, MCP, AI Parsing and AirTrail ship off and are marked below.
|
|
|
|
<table>
|
|
<tr>
|
|
<td width="50%" valign="top">
|
|
|
|
#### 🧭 Planning
|
|
|
|
- **Day plans**: drag places between days and reorder inside a day, with undo. Notes and bookings drag the same way, and a map marker drops straight onto a day
|
|
- **Maps**: Leaflet, Mapbox GL or MapLibre GL (OpenFreeMap, no token), with clustering, photo markers and route lines. 3D buildings and terrain are Mapbox only
|
|
- **Place search**: Google Places when a key is set (photos, ratings, opening hours), otherwise OpenStreetMap with no key
|
|
- **Place enrichment**: descriptions, facts, hours and photo candidates from OpenStreetMap, Wikipedia, Wikidata and Wikimedia Commons
|
|
- **POI explore**: pull OpenStreetMap POIs by category for the current viewport over Overpass
|
|
- **Import**: shared Google Maps and Naver Maps lists, plus GPX, KML and KMZ files
|
|
- **Export**: GPX of a trip's places and tracks, and an ICS feed per trip or across all of them
|
|
- **Routes**: auto-sort a day (nearest neighbour then 2-opt, locked stops and hotel anchors stay put), driving, walking or cycling profiles over OSRM, then open it in Google Maps or CoMaps
|
|
- **Public transport**: door-to-door itineraries over Transitous
|
|
- **Weather**: 16-day forecast from Open-Meteo, no key. Dates outside that window read the archive for the same date instead
|
|
- **Day notes**: markdown body with an icon and a colour, reordered by drag and drop or moved to another day
|
|
- **Trip dates**: move a trip and the days re-date themselves, either dragging the bookings along or re-anchoring them. Trips also copy and archive
|
|
|
|
</td>
|
|
<td width="50%" valign="top">
|
|
|
|
#### 🧳 Bookings and money
|
|
|
|
- **Reservations**: 16 booking types with status, confirmation code, travellers and attached files
|
|
- **Flights and trains**: several legs with stopovers, per-leg times and endpoint timezones, against 4,045 bundled airports so local times resolve without a key
|
|
- **Accommodation**: a stay spans a range of days with a check-in window, and shows on every night it covers
|
|
- **Booking import**: EML, PDF, PKPass, HTML and TXT confirmations through [KItinerary](https://invent.kde.org/pim/kitinerary). Needs the `kitinerary-extractor` binary, which ships in the Docker image
|
|
- **AirTrail** (off by default): link a self-hosted AirTrail instance to import flights as reservations and keep them in sync
|
|
- **Costs**: split expenses in integer cents with equal or custom shares, several payers per expense, settle-up suggestions, a settlement log, and CSV export
|
|
- **Currencies**: a currency per expense with the rate frozen at entry. Rates come from Frankfurter, no key
|
|
- **Packing lists**: categories, admin-managed templates, assignees, three visibility tiers, and a packed-of-total line. Bags with weight roll-up are a separate admin switch, off by default
|
|
- **To-dos**: assignee, due date, priority, and a reminder before one falls due
|
|
- **Files**: attach to a trip, place, day entry or reservation. 50 MB each, 500 MB for video, with trash and restore
|
|
- **PDF export**: cover page, place photos, day notes, bookings and costs, with an optional page break per day
|
|
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td width="50%" valign="top">
|
|
|
|
#### 👥 Collaboration
|
|
|
|
- **Real-time sync** (WebSocket): edits land live for everyone who has that trip open
|
|
- **Members**: add by email or username, hand ownership to someone else, or add guests who have no login at all
|
|
- **Permissions**: an admin maps each of 16 trip actions to admin, trip owner, trip member or everybody
|
|
- **Invite links**: one reusable link per trip with an optional expiry. Admins can also issue signup invites with a use limit that drop the new account straight into a trip
|
|
- **Public share**: a read-only trip page anyone can open without an account
|
|
- **Collab**: group chat with replies, reactions and link previews, shared notes with attachments, polls, and a What's Next list of upcoming activities. Each of the four switches on its own
|
|
|
|
#### 📔 Journal, Atlas and Vacay
|
|
|
|
- **Journey** (off by default): dated entries with story, mood, weather and tags, photos and video from uploads or a linked [Immich](https://immich.app) or Synology Photos library, map views, co-authors, and a public share link
|
|
- **Atlas**: mark countries and sub-national regions visited on [geoBoundaries](https://www.geoboundaries.org/) outlines, plus a bucket list, travel stats and a consecutive-years-travelled count
|
|
- **Vacay**: leave calendar with half days, public holidays from date.nager.at, school holiday overlays for 16 European countries, carry-over between years, and joint or read-only sharing of a plan
|
|
- **Collections** (off by default): a place library outside any trip, with labels, ratings, a custom image per place, copy-into-a-trip, and sharing by invite
|
|
|
|
</td>
|
|
<td width="50%" valign="top">
|
|
|
|
#### 🧩 Plugins
|
|
|
|
- **Third-party plugins**: install from the TREK registry or sideload a zip, switch on per instance, with their own pages under `/plugins/<id>`
|
|
- **Sandboxed**: one child process per plugin, 63 grantable permissions, an admin-edited outbound host allowlist, memory and RPC caps, and daily caps on AI and notification calls
|
|
- **Trust**: registry downloads are pinned by sha256 and checked against the author's minisign key. A sideloaded zip is marked unverified, and `TREK_PLUGINS_ENABLED=false` turns the whole system off
|
|
- **Extension points**: map markers and layers, place details, day schedules, PDF sections, Atlas layers, journal rows, trip warnings, calendar sources, route profiles and notification channels. Plugin pages run in an opaque-origin iframe
|
|
- **SDK**: [`trek-plugin-sdk`](https://www.npmjs.com/package/trek-plugin-sdk) on npm, with a manifest validator, a mock host, and a dev-link mode that runs a local build against real data
|
|
|
|
#### 🤖 AI and MCP
|
|
|
|
- **MCP server** (off by default): OAuth 2.1 with mandatory PKCE and dynamic client registration. 199 tools, 30 resources, 4 prompts
|
|
- **Scopes**: 29 scopes in 14 groups, each one tickable on the consent screen. Tokens are bound to the `/mcp` resource, and every tool call lands in the audit log
|
|
- **Reach**: create trips and days, edit places, packing lists, to-dos, costs, reservations, collections and journeys, mark countries visited, all inside the scopes the token holds
|
|
- **Prompts**: `trip-summary`, plus `packing-list` and `budget-overview` when those addons are on
|
|
- **Addon-aware**: seven addon gates decide which tools and resources a session sees. Flipping one drops live sessions so the surface re-registers
|
|
- **Booking extraction** (off by default): read a confirmation with a local Ollama server, any OpenAI-compatible endpoint, or Anthropic, configured instance-wide or per user
|
|
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td width="50%" valign="top">
|
|
|
|
#### 📱 Mobile and offline
|
|
|
|
- **Installable**: iOS and Android straight from the browser, no App Store. Runs standalone without browser chrome, with the status bar tinted per theme
|
|
- **Phone layouts**: a separate shell below 768px with its own tokens, a bottom bar and safe-area insets. Which items sit in that bar is up to the user
|
|
- **Offline reads**: the app shell and every route chunk are precached, and trips, places and file blobs live in a per-user IndexedDB store, so a trip opens with no network
|
|
- **Offline writes**: mutations queue and replay with an `X-Idempotency-Key`, so a reconnect cannot double-apply. A stale edit is parked for a keep-mine or keep-theirs choice
|
|
- **Offline maps**: pre-download a trip's raster tiles, and clear them again
|
|
- **API responses are never cached** by the service worker. They vary per session, so they always come from the network
|
|
|
|
</td>
|
|
<td width="50%" valign="top">
|
|
|
|
#### ⚙️ Admin, accounts and security
|
|
|
|
- **Login methods**: password login, password registration, OIDC login, OIDC registration and passkey login each switch on and off on their own. `OIDC_ONLY` leaves SSO as the only way in
|
|
- **SSO (OIDC)**: one provider by discovery, with PKCE and `id_token` verification (Authentik, Keycloak, Google and the like)
|
|
- **2FA**: TOTP with ten single-use backup codes, and an admin can require it instance-wide
|
|
- **Passkeys**: WebAuthn login by fingerprint, face, PIN or security key, off until an admin enables it. A passkey also satisfies the 2FA requirement
|
|
- **Hardening**: per-IP limits on login, password reset and 2FA attempts, a password policy, secrets encrypted at rest and masked on read, and an SSRF guard on every URL you configure
|
|
- **Admin panel**: users and invites, the permission matrix, packing templates, categories, addons, plugins, API keys, MCP tokens and OAuth sessions, backups, storage, audit log, and GitHub releases
|
|
- **Backups**: manual or scheduled hourly, daily, weekly or monthly, with retention in days. The zip carries the at-rest key, so a restore can decrypt its own secrets
|
|
- **Storage**: pluggable storage backends per content category — keep everything on local disk, or add S3-compatible backends and replicate any category to them, configured entirely from the admin panel
|
|
- **Notifications**: a per-user matrix of events against in-app, email (SMTP), webhook and ntfy, plus any channel a plugin registers
|
|
- **Appearance**: light, dark or follow the OS, seven colour schemes plus a custom accent, transparency, compact density, reduce motion, and text size per tier
|
|
- **23 languages**: en, de, es, fr, it, nl, hu, ru, zh, zh-TW, pl, cs, ar (right to left), br, id, tr, ja, ko, uk, gr, sv, vi, ca
|
|
- **In-app help**: the wiki ships inside the image and is served from disk at `/help`, so the docs match the version you are running
|
|
|
|
</td>
|
|
</tr>
|
|
</table>
|
|
|
|
</details>
|
|
|
|
<br />
|
|
|
|
<div align="center">
|
|
<a href="docs/screenshots/showcase-desktop-1.webp"><img src="docs/screenshots/showcase-desktop-1.webp" alt="Atlas · dashboard · trip planner" width="100%" /></a>
|
|
<a href="docs/screenshots/showcase-mobile.webp"><img src="docs/screenshots/showcase-mobile.webp" alt="Mobile PWA · dashboard, day plan, map, costs" width="100%" /></a>
|
|
<a href="docs/screenshots/showcase-desktop-2.webp"><img src="docs/screenshots/showcase-desktop-2.webp" alt="Collections · journey journal · costs" width="100%" /></a>
|
|
</div>
|
|
|
|
<br />
|
|
|
|
## AI usage
|
|
|
|
We use LLM-assisted coding tools across parts of this codebase. Nothing ships that a
|
|
maintainer has not read and understood: every change goes through a pull request, is
|
|
reviewed and tested, and has a human who can answer for it. "The AI wrote that" is not
|
|
an answer any of us would accept from ourselves.
|
|
|
|
See [**How we use AI in TREK**](https://github.com/liketrek/TREK/discussions/1851) for
|
|
the details.
|
|
|
|
<br />
|
|
|
|
## Get started in 30 seconds
|
|
|
|
```bash
|
|
ENCRYPTION_KEY=$(openssl rand -hex 32) docker run -d -p 3000:3000 \
|
|
-e ENCRYPTION_KEY=$ENCRYPTION_KEY \
|
|
-v ./data:/app/data -v ./uploads:/app/uploads mauriceboe/trek
|
|
```
|
|
|
|
Open `http://localhost:3000`. On first boot TREK seeds an admin account — if you set `ADMIN_EMAIL`/`ADMIN_PASSWORD` those are used, otherwise the credentials are printed to the container log (`docker logs trek`).
|
|
|
|
<div align="center">
|
|
|
|
<a href="#docker-compose-production">Docker Compose</a> · <a href="#helm-kubernetes">Helm / Kubernetes</a> · <a href="#install-as-app-pwa">Install as PWA</a> · <a href="#reverse-proxy">Reverse Proxy</a>
|
|
|
|
</div>
|
|
|
|
<br />
|
|
|
|
## Tech stack
|
|
|
|
<div align="center">
|
|
|
|

|
|

|
|

|
|

|
|

|
|

|
|

|
|

|
|

|
|
|
|
</div>
|
|
|
|
Real-time sync via WebSocket (`ws`). Backend on NestJS 11. State with Zustand. Auth via JWT + OAuth 2.1 + OIDC + Passkeys (WebAuthn) + TOTP MFA. Weather via Open-Meteo (no key required). Maps with Leaflet and Mapbox GL.
|
|
|
|
<br />
|
|
|
|
<h2 id="docker-compose-production">Docker Compose (production)</h2>
|
|
|
|
<details>
|
|
<summary>Full compose example with secure defaults</summary>
|
|
|
|
```yaml
|
|
services:
|
|
app:
|
|
image: mauriceboe/trek:latest
|
|
container_name: trek
|
|
read_only: true
|
|
security_opt:
|
|
- no-new-privileges:true
|
|
cap_drop:
|
|
- ALL
|
|
cap_add:
|
|
- CHOWN
|
|
- SETUID
|
|
- SETGID
|
|
tmpfs:
|
|
- /tmp:noexec,nosuid,size=64m
|
|
ports:
|
|
- "3000:3000"
|
|
environment:
|
|
- NODE_ENV=production
|
|
- PORT=3000
|
|
- ENCRYPTION_KEY=${ENCRYPTION_KEY:-} # generate with: openssl rand -hex 32
|
|
- TZ=${TZ:-UTC}
|
|
- LOG_LEVEL=${LOG_LEVEL:-info}
|
|
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-}
|
|
- APP_URL=${APP_URL:-} # required for OIDC + email links
|
|
# - FORCE_HTTPS=true # behind a TLS-terminating proxy
|
|
# - TRUST_PROXY=1
|
|
# - OIDC_ISSUER=https://auth.example.com
|
|
# - OIDC_CLIENT_ID=trek
|
|
# - OIDC_CLIENT_SECRET=supersecret
|
|
# - OIDC_DISPLAY_NAME=SSO
|
|
# - OIDC_ADMIN_CLAIM=groups
|
|
# - OIDC_ADMIN_VALUE=app-trek-admins
|
|
volumes:
|
|
- ./data:/app/data
|
|
- ./uploads:/app/uploads
|
|
restart: unless-stopped
|
|
healthcheck:
|
|
test: ["CMD", "wget", "-qO-", "http://localhost:3000/api/health"]
|
|
interval: 30s
|
|
timeout: 10s
|
|
retries: 3
|
|
start_period: 15s
|
|
```
|
|
|
|
Then:
|
|
|
|
```bash
|
|
docker compose up -d
|
|
```
|
|
|
|
**HTTPS notes:** `FORCE_HTTPS=true` is optional — it adds a 301 redirect, HSTS, CSP upgrade-insecure-requests, and forces the `secure` cookie flag. Only use it behind a TLS-terminating reverse proxy. `TRUST_PROXY=1` tells the server how many proxies sit in front so real client IPs and `X-Forwarded-Proto` work.
|
|
|
|
</details>
|
|
|
|
<br />
|
|
|
|
<h2 id="helm-kubernetes">Helm (Kubernetes)</h2>
|
|
|
|
```bash
|
|
helm repo add trek https://chart.liketrek.com
|
|
helm repo update
|
|
helm install trek trek/trek
|
|
```
|
|
|
|
See [`charts/README.md`](https://github.com/liketrek/TREK/blob/main/charts/README.md) for values.
|
|
|
|
<h2 id="install-as-app-pwa">Install as App (PWA)</h2>
|
|
|
|
TREK works as a Progressive Web App — no App Store needed.
|
|
|
|
1. Open TREK in the browser (HTTPS required)
|
|
2. **iOS**: Share ▸ *Add to Home Screen*
|
|
3. **Android**: Menu ▸ *Install app* (or *Add to Home Screen*)
|
|
|
|
TREK then launches fullscreen with its own icon, just like a native app.
|
|
|
|
<br />
|
|
|
|
## Updating
|
|
|
|
**Docker Compose:**
|
|
|
|
```bash
|
|
docker compose pull && docker compose up -d
|
|
```
|
|
|
|
**Docker run** — reuse the original volume paths:
|
|
|
|
```bash
|
|
docker pull mauriceboe/trek
|
|
docker rm -f trek
|
|
docker run -d --name trek -p 3000:3000 -v ./data:/app/data -v ./uploads:/app/uploads --restart unless-stopped mauriceboe/trek
|
|
```
|
|
|
|
> Not sure which paths you used? `docker inspect trek --format '{{json .Mounts}}'` before removing the container.
|
|
|
|
Your data stays in the mounted `data` and `uploads` volumes — updates never touch it.
|
|
|
|
> [!IMPORTANT]
|
|
> Mount **only** the data and uploads directories — `-v ./data:/app/data -v ./uploads:/app/uploads`. **Never mount a volume at `/app`.** Doing so hides the application code shipped in the image and the container fails to start with `Cannot find module 'tsconfig-paths/register'`. If you previously mounted `/app`, switch to the two mounts above; your data in `data/` and `uploads/` is preserved.
|
|
|
|
<h3>Rotating the Encryption Key</h3>
|
|
|
|
If you need to rotate `ENCRYPTION_KEY` (e.g. upgrading from a version that derived encryption from `JWT_SECRET`):
|
|
|
|
```bash
|
|
docker exec -it trek node --import tsx scripts/migrate-encryption.ts
|
|
```
|
|
|
|
The script creates a timestamped DB backup before making changes and prompts for old + new keys (input is not echoed).
|
|
|
|
<h2 id="reverse-proxy">Reverse Proxy</h2>
|
|
|
|
For production, put TREK behind a TLS-terminating reverse proxy. TREK uses WebSockets for real-time sync, so the proxy **must** support WebSocket upgrades on `/ws`.
|
|
|
|
If you use the MCP addon, the proxy must also pass the `Mcp-Session-Id` header through in both directions on `/mcp` — Nginx and Caddy do this by default, but a proxy that strips it makes every tool call open a new session instead of reusing one. See the [Reverse Proxy wiki page](https://github.com/liketrek/TREK/wiki/Reverse-Proxy) for details.
|
|
|
|
<details>
|
|
<summary>Nginx</summary>
|
|
|
|
```nginx
|
|
server {
|
|
listen 80;
|
|
server_name trek.yourdomain.com;
|
|
return 301 https://$host$request_uri;
|
|
}
|
|
|
|
server {
|
|
listen 443 ssl http2;
|
|
server_name trek.yourdomain.com;
|
|
|
|
ssl_certificate /etc/ssl/fullchain.pem;
|
|
ssl_certificate_key /etc/ssl/privkey.pem;
|
|
|
|
# 500 MB covers backup-restore uploads (capped at 500 MB server-side).
|
|
client_max_body_size 500m;
|
|
|
|
location / {
|
|
proxy_pass http://localhost:3000;
|
|
proxy_http_version 1.1;
|
|
proxy_set_header Host $host;
|
|
proxy_set_header X-Real-IP $remote_addr;
|
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
proxy_set_header X-Forwarded-Proto $scheme;
|
|
}
|
|
|
|
location /ws {
|
|
proxy_pass http://localhost:3000;
|
|
proxy_http_version 1.1;
|
|
proxy_set_header Upgrade $http_upgrade;
|
|
proxy_set_header Connection "upgrade";
|
|
proxy_set_header Host $host;
|
|
proxy_read_timeout 86400;
|
|
}
|
|
|
|
# Only needed if you use the MCP addon. Responses are Server-Sent Events,
|
|
# so buffering must be off or tool results arrive late.
|
|
location /mcp {
|
|
proxy_pass http://localhost:3000;
|
|
proxy_http_version 1.1;
|
|
proxy_set_header Host $host;
|
|
proxy_set_header X-Real-IP $remote_addr;
|
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
proxy_set_header X-Forwarded-Proto $scheme;
|
|
proxy_buffering off;
|
|
proxy_read_timeout 3600s;
|
|
}
|
|
}
|
|
```
|
|
|
|
</details>
|
|
|
|
<details>
|
|
<summary>Caddy</summary>
|
|
|
|
```caddy
|
|
trek.yourdomain.com {
|
|
reverse_proxy localhost:3000
|
|
}
|
|
```
|
|
|
|
Caddy handles TLS and WebSockets automatically.
|
|
|
|
</details>
|
|
|
|
<br />
|
|
|
|
## Environment variables
|
|
|
|
> [!NOTE]
|
|
> Variables are validated at startup (fail-fast). An unset or blank variable
|
|
> always falls back to its default, but a variable set to a malformed value
|
|
> (e.g. `PORT=abc`, `SESSION_DURATION=bogus`, `DEMO_MODE=maybe`) aborts boot
|
|
> with a report listing every offending variable. Boolean switches accept
|
|
> `true`/`false`, `1`/`0`, `on`/`off` and `yes`/`no` (any casing).
|
|
|
|
<details>
|
|
<summary><b>Full reference</b></summary>
|
|
|
|
<br />
|
|
|
|
| Variable | Description | Default |
|
|
|----------|-------------|---------|
|
|
| **Core** | | |
|
|
| `PORT` | Server port | `3000` |
|
|
| `NODE_ENV` | Environment (`production` / `development`) | `production` |
|
|
| `ENCRYPTION_KEY` | At-rest encryption key for stored secrets (API keys, MFA, SMTP, OIDC). Recommended: generate with `openssl rand -hex 32`. If unset, falls back to `data/.jwt_secret` (existing installs) or auto-generates a key (fresh installs). | Auto |
|
|
| `TZ` | Timezone for logs, reminders and cron jobs (e.g. `Europe/Berlin`) | `UTC` |
|
|
| `LOG_LEVEL` | `info` = concise user actions, `debug` = verbose details | `info` |
|
|
| `TREK_WIKI_DIR` | Where the in-app Help pages (`/help`) read their content from. TREK ships its wiki and serves it from disk, so Help always matches the version you are running — you should not need to set this. Point it at your own directory to serve custom docs. If the path does not exist, Help falls back to fetching the public GitHub wiki (needs outbound network, and tracks the latest release). | bundled `wiki/` |
|
|
| `DEFAULT_LANGUAGE` | Default language on the login page for users with no saved preference. Browser/OS language is auto-detected first; this is the fallback. Supported: `de`, `en`, `es`, `fr`, `hu`, `nl`, `br`, `cs`, `pl`, `ru`, `zh`, `zh-TW`, `it`, `ar`, `id`, `tr`, `ja`, `ko`, `uk`, `gr` | `en` |
|
|
| `ALLOWED_ORIGINS` | Comma-separated origins for CORS and email links | same-origin |
|
|
| `FORCE_HTTPS` | Optional. When `true`: 301-redirects HTTP to HTTPS, sends HSTS, adds CSP `upgrade-insecure-requests`, forces the session cookie `secure` flag. Useful behind a TLS-terminating reverse proxy. Requires `TRUST_PROXY`. | `false` |
|
|
| `HSTS_INCLUDE_SUBDOMAINS` | When `true`: adds the `includeSubDomains` directive to the HSTS header, extending HTTPS enforcement to all subdomains. Only effective when HSTS is active (`FORCE_HTTPS=true` or `NODE_ENV=production`). Leave `false` if you run other services on sibling subdomains over plain HTTP. | `false` |
|
|
| `COOKIE_SECURE` | Controls the `secure` flag on the `trek_session` cookie. Auto-derived: on when `NODE_ENV=production` or `FORCE_HTTPS=true`. Escape hatch: set `false` to allow session cookies over plain HTTP. Not recommended in production. | auto |
|
|
| `SESSION_DURATION` | How long a login session stays valid when **"Remember me" is unchecked** (the default): sets the `trek_session` JWT `exp` and issues a browser-session cookie (cleared when the browser closes). Accepts `ms`-style strings: `1h`, `12h`, `7d`, `30d`, `90d`. Invalid values warn at startup and fall back to the default. | `24h` |
|
|
| `SESSION_DURATION_REMEMBER` | Session length when **"Remember me" is ticked** at login: a longer-lived JWT plus a persistent `trek_session` cookie that survives browser restarts. Same format and startup-fallback behaviour as `SESSION_DURATION`. | `30d` |
|
|
| `TRUST_PROXY` | Number of trusted reverse proxies. Tells the server to read client IP from `X-Forwarded-For` and protocol from `X-Forwarded-Proto`. Defaults to `1` in production; off in dev unless set. | `1` |
|
|
| `ALLOW_INTERNAL_NETWORK` | Allow outbound requests to private/RFC-1918 IPs (e.g. Immich on your LAN). Loopback and link-local addresses remain blocked. | `false` |
|
|
| `APP_URL` | Public base URL of this instance (e.g. `https://trek.example.com`). Required when OIDC is enabled; used as base for email notification links. | — |
|
|
| **OIDC / SSO** | | |
|
|
| `OIDC_ISSUER` | OpenID Connect provider URL | — |
|
|
| `OIDC_CLIENT_ID` | OIDC client ID | — |
|
|
| `OIDC_CLIENT_SECRET` | OIDC client secret | — |
|
|
| `OIDC_DISPLAY_NAME` | Label shown on the SSO login button | `SSO` |
|
|
| `OIDC_ONLY` | Force SSO-only mode: disables password login + registration, regardless of Admin > Settings. The first SSO login becomes admin. | `false` |
|
|
| `OIDC_ADMIN_CLAIM` | OIDC claim used to identify admin users | — |
|
|
| `OIDC_ADMIN_VALUE` | Value of the OIDC claim that grants admin role | — |
|
|
| `OIDC_SCOPE` | Space-separated OIDC scopes. **Fully replaces** the default — always include `openid email profile`. | `openid email profile` |
|
|
| `OIDC_DISCOVERY_URL` | Override the auto-constructed OIDC discovery endpoint (e.g. Authentik: `.../application/o/trek/.well-known/openid-configuration`) | — |
|
|
| **Initial setup** | | |
|
|
| `ADMIN_EMAIL` | Email for the first admin on initial boot. Must be set together with `ADMIN_PASSWORD`. If either is omitted a random password is printed to the server log. No effect once a user exists. | `admin@trek.local` |
|
|
| `ADMIN_PASSWORD` | Password for the first admin on initial boot. Pairs with `ADMIN_EMAIL`. | random |
|
|
| **Other** | | |
|
|
| `TREK_MANAGED` | Marks the install as centrally administered: the operator owns the configuration, credentials and upgrades, so the instance admin is not offered settings the operator sets. Leave unset when you run TREK yourself. | `false` |
|
|
| `PLACES_API_KEY` | Places credential supplied by whoever operates the install. When set, it is used for every lookup and the per-user keys in Admin > Settings are ignored. Leave unset to keep the stored keys. | — |
|
|
| `PLACES_API_BASE` | Send the Places calls to this origin instead of `https://places.googleapis.com` (an egress proxy, a cache, a gateway holding the key). Path and query are unchanged, so it has to speak the same API. | — |
|
|
| `DEMO_MODE` | Enable demo mode (hourly data resets) | `false` |
|
|
| `UNSPLASH_ACCESS_KEY` | Optional Unsplash Access Key for trip-cover and place-image search. Without one, TREK uses Unsplash's unauthenticated endpoint, which some datacenter/VPS IPs are blocked from. Get a free key at [unsplash.com/developers](https://unsplash.com/developers). Overrides any per-admin key set in Admin > Settings (where it can also be configured instead). | — |
|
|
| `MCP_RATE_LIMIT` | Max MCP API requests per user per minute | `300` |
|
|
| `MCP_MAX_SESSION_PER_USER` | Max concurrent MCP sessions per user. At the cap, the least-recently-active session is closed to make room | `20` |
|
|
|
|
</details>
|
|
|
|
<br />
|
|
|
|
<h2 id="storage">Storage</h2>
|
|
|
|
TREK separates *what* it stores (eight content categories — trip documents,
|
|
journey photos, cover images, profile pictures, place images, the two photo
|
|
caches, and backups) from *where* it stores it (named backends). Out of the
|
|
box everything lives on local disk (`uploads/` and `data/backups`). From
|
|
**Admin → Storage** you can:
|
|
|
|
- **Add S3-compatible backends** (AWS S3, Cloudflare R2, Backblaze B2, Garage,
|
|
MinIO/AIStor) and assign any category to them.
|
|
- **Replicate a backend**: edit it and pick **Mirror targets** — every write is
|
|
then also copied to each target. The classic off-site backup setup is:
|
|
add an S3 backend, edit `backups-local`, tick the S3 backend as a mirror
|
|
target, save. Replica writes happen one after another during each upload, so
|
|
a slow or unreachable target slows every upload of every category on that
|
|
backend — fine for backups, worth weighing for hot categories. Existing
|
|
objects are not replicated when you add a target — press **Sync now** on
|
|
the backend to copy them (one sync at a time; progress and cancel in the
|
|
panel).
|
|
- **Watch the Health strip**: replica failures never fail the original
|
|
request; they are recorded and shown there instead. All-clear means every
|
|
replicated write landed. Failures also notify admins over the configured
|
|
notification channels (first failure immediately, then hourly summaries) —
|
|
tune it under Admin → Notifications.
|
|
- **Test** any backend from its row (for a replicated backend it probes the
|
|
primary and each target individually). Targets must be saved before Test
|
|
can probe them.
|
|
- **Usage**: per-category and per-backend object counts and sizes, scanned
|
|
nightly and on demand.
|
|
|
|
Backend credentials are encrypted at rest. Without an explicit
|
|
`ENCRYPTION_KEY` the implicit key from the data directory is used — that
|
|
works, but the implicit key rides inside backups, so setting `ENCRYPTION_KEY`
|
|
explicitly is recommended for credentialed backends. On an existing install
|
|
that has been running without an explicit key, do not just set a new one —
|
|
that would orphan already-stored secrets; use the key-rotation procedure (see
|
|
*Rotating the Encryption Key* under Updating) to move to an explicit key.
|
|
|
|
Notes and limits: reassigning a populated category prompts to move its
|
|
existing objects — accepting copies everything to the new backend, flips the
|
|
category, then sweeps any writes that raced the copy; source objects are kept
|
|
(not deleted) and reported as reclaimable (count + size) to reclaim manually;
|
|
declining routes only new writes to the new backend, leaving old objects in
|
|
place as before. Only one storage job (a mirror sync or a category migration)
|
|
runs at a time, cancelling before the flip is always safe, and a failed copy
|
|
never flips the category; media categories on S3 work but every served byte
|
|
proxies through the server — the proxy answers conditional GETs (ETag /
|
|
Last-Modified) and a single `Range` request, while a multi-range request is
|
|
answered with the full object; AWS buckets
|
|
with Object-Lock or checksum-requiring policies
|
|
reject the server's uploads; self-hosted endpoints (MinIO/Garage) should be
|
|
addressed by IP or `localhost` unless configured for virtual-hosted buckets.
|
|
The legacy `/uploads/photos` directory from older TREK versions is still
|
|
served and included in backups, but it is not a configurable category.
|
|
Running the storage contract suite against a live MinIO/AIStor is a
|
|
license-gated procedure — CI runs it automatically for same-repo pull
|
|
requests (fork PRs skip it, since they have no access to the license
|
|
secret); to run it yourself locally, see `docker-compose.minio-test.yml`'s
|
|
header for the manual procedure.
|
|
|
|
### Provisioning at first boot (seed file)
|
|
|
|
For infrastructure-as-code setups, mount a `storage-config.json` into the
|
|
data directory — it is imported **once**, on the first boot that has no
|
|
stored storage configuration, and loudly ignored afterwards:
|
|
|
|
```jsonc
|
|
// storage-config.json — secrets may be plaintext (encrypted on import)
|
|
// or already-encrypted enc:v1: values.
|
|
{
|
|
"backends": [
|
|
{ "name": "off-site", "type": "s3", "options": {
|
|
"endpoint": "https://s3.example.com", "bucket": "trek",
|
|
"accessKeyId": "…", "secretAccessKey": "…" } },
|
|
{ "name": "backups-mirror", "type": "mirror", "options": {
|
|
"primary": "backups-local", "replicas": ["off-site"] } }
|
|
],
|
|
"categories": { "backups": "backups-mirror" }
|
|
}
|
|
```
|
|
|
|
(The panel presents this exact setup as **Mirror targets** on `backups-local`.)
|
|
|
|
```yaml
|
|
# docker-compose: add under the trek service's volumes
|
|
- ./storage-config.json:/app/data/storage-config.json:ro
|
|
```
|
|
|
|
An invalid seed file aborts boot with the exact validation error — an
|
|
actively-provisioning operator sees the problem instead of a silent default.
|
|
|
|
### Recovery and restore
|
|
|
|
To reset storage configuration to the built-in defaults (or to re-import a
|
|
seed file): stop the server, run
|
|
`sqlite3 data/travel.db "DELETE FROM app_settings WHERE key LIKE 'storage.%';"`,
|
|
and start it again.
|
|
|
|
Backups include the storage configuration (it lives in the database). If your
|
|
only backup sits on S3 and the credentials for it sit inside that backup:
|
|
start a fresh instance, enter the S3 credentials in **Admin → Storage** (or
|
|
mount a seed file), then restore the backup from the Backup panel.
|
|
|
|
## Data & Backups
|
|
|
|
- **Database** — SQLite, stored in `./data/travel.db`
|
|
- **Uploads** — stored in `./uploads/` by default; every content category can be reassigned to another backend in **Admin → Storage**
|
|
- **Logs** — `./data/logs/trek.log` (auto-rotated)
|
|
- **Backups** — create and restore via Admin Panel
|
|
- **Auto-Backups** — configurable schedule and retention in Admin Panel
|
|
|
|
<br />
|
|
|
|
## Data sources
|
|
|
|
The Atlas map's country and sub-national (province/county) boundaries come from
|
|
[**geoBoundaries**](https://www.geoboundaries.org/) (Runfola et al., 2020), licensed
|
|
[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). See [NOTICE.md](NOTICE.md)
|
|
for full third-party attributions.
|
|
|
|
## License
|
|
|
|
TREK is [AGPL v3](LICENSE). Self-host freely for personal or internal company use. If you modify and offer TREK as a network service to third parties, your modifications must be open-sourced under the same licence.
|