mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-10-22 15:06:32 +00:00
4346d7f2c1e27cd938d57e57d4c2c4d28b68f3ba
60 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
5e5fe288d8 |
fix(server): release sockets on shutdown so docker stop no longer ends in SIGKILL
Every `docker stop` of a TREK instance with a browser attached exited 137, exactly ten seconds after the stop. `http.Server.close()` stops new connections and then waits for the open ones to end by themselves — and a WebSocket never does. TREK holds one per open tab, so the close callback never ran. `closeDb()` and `process.exit(0)` lived inside that callback, which left the only remaining exit a fallback timer set to 10000ms: exactly when Docker sends SIGKILL. Docker starts counting when it sends the signal and Node only once the handler is dispatched, so that timer started strictly later and could never win. It was unreachable code in every default deployment. Waiting for Nest was not an option either: `WsAdapter.close()` is the one place that terminates clients, and Nest reaches it only after every onModuleDestroy hook has resolved — including the plugin supervisor's SIGTERM grace for its forked children. The sockets holding the server open would be hostage to the slowest hook in the container. So the sockets are now released deliberately: ws clients get 1001 "going away", idle keep-alive sockets are dropped at once, and anything still attached after a 2s drain is destroyed. Nest is closed alongside that, the database is closed on both exit paths rather than only the unreachable one, and the fallback fires at 5s — inside Docker's grace, so a forced exit is still ours. The sequence moved into src/shutdown.ts behind injected dependencies. It was untestable by construction before, which is part of why this went unnoticed; it now has nine tests, including the signal that beats the bootstrap. Closes #2193 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S5VwmiBQ9sPCJwfmpv64r3 |
||
|
|
305b113d5f |
Feat/storage driver (#1949)
* feat(storage): storage key validation, driver contract suite, LocalDriver * feat(storage): MirrorDriver composite backend * feat(server): add TREK_UPLOADS_DIR env var for the uploads root * feat(storage): settings-backed registry, StorageService facade, module wiring * test(storage): pin src/nest/storage coverage thresholds * feat(storage): multer upload options factory * refactor(files): spool trip-file uploads through storage * refactor(collab): spool note attachments through storage; normalize files/ prefix * refactor(journey): spool journey media uploads through storage * refactor(trips): spool trip cover upload through storage * refactor(places): spool place-image and collection uploads through storage * refactor(auth): spool avatar upload through storage * fix(server): consolidate allowed_file_types onto AllowedFileTypesService * fix(storage): default defParamCharset to utf8 in the upload factory * test(db): re-point the trailing-migration guard at the trip_files prefix strip * fix(storage): age-gate the boot spool sweep * test(files): key FILE-P02 residue checks on upload size * test(storage): raise src/nest/storage coverage pin * refactor(platform): serve /uploads routes through storage * refactor(files): serve trip-file downloads through storage * refactor(memories): serve photo bytes through storage * refactor(maps): stream place-photo bytes through storage * refactor(share): stream shared place-photo bytes through storage * refactor(journey): serve public journey assets through storage * fix(share): answer place-photo cache miss with 204 like the maps proxy * test(server): close serving-swap gaps surfaced by the full tier * test(storage): raise src/nest/storage branches pin * refactor(place-photos): route place-photo cache through storage * refactor(places): reclaim place images and collection covers through storage * refactor(trips): trip covers through storage * refactor(auth): avatar reclaim through storage * refactor(collab): note-file deletion through storage * refactor(files): trash deletion through storage * refactor(memories): trek photo cache internals through storage * refactor(memories): thumbnail service category-addressed * refactor(memories): immich upload reads through storage; drop uploads-root * refactor(journey): photo deletion through storage * fix(memories): reclaim orphaned trek photo cache objects in the ttl sweep * feat(journey): journey-thumbs orphan sweep and delete-site derived-artifact reclaim * test(server): close slice-4 gaps surfaced by the full tier * test(coverage): raise slice-4 domain pins * refactor(backup): list, existence and delete through storage * refactor(backup): download through storage sendToResponse * refactor(backup): create backup through per-category storage enumeration * refactor(backup): restore and upload-restore through storage * refactor(backup): auto-backup retention through storage * refactor(server): boot dir-creation to driver init * test(server): close slice-5 gaps surfaced by the coverage tier * test(coverage): raise slice-5 backup pin * test(server): resolve async backupFileExists mocks for the tests typecheck * test(server): pin ranged-read stat and zero-byte roundtrip in the storage driver contract * feat(server): TREK_S3_* env wiring — schema, s3 namespace, all-or-nothing boot rule * feat(server): S3 driver core — aws-lite client seam, stat/delete, driver-owned deadlines * refactor(server): type the aws-lite S3 client via s3-types reference * feat(server): S3 driver getStream and list — ranged full-object stat, iterator pagination * feat(server): S3 driver put — bounded-peek routing, Upload inactivity deadline * fix(server): S3 driver Upload monitoring — Transform interposition, stream cleanup * feat(server): env-declared s3-main storage backend; settings-declared s3 rejected * test(server): env-gated S3 driver contract suite against MinIO * fix(server): S3 driver zero-byte put — work around aws4 duplicate Content-Length signing defect * fix(server): scope the zero-byte put workaround to buffered bodies; bind test MinIO to localhost * ci(server): MinIO-backed S3 driver contract job * ci(server): align s3-contract shim comment and MinIO bind with references * docs: S3 storage backend configuration and off-box backup mirroring * docs: restore README formatting, keep S3 additions only * ci(server): make live-MinIO contract runs manual-only (AIStor licensing) MinIO's community line is discontinued entirely; the maintained successor, AIStor, requires a license key that can't safely live in a shared CI runner. The s3-contract CI job is removed; live-provider contract runs against MinIO/AIStor are now a manual, license-gated procedure documented in docker-compose.minio-test.yml's header. * chore(server): final-review polish — stale CI refs, dockerignore key, boundary pins, zero-byte tempfile contract case * expose admin ui * feat(shared): storage backend type registry, wire schemas, category relocation * feat(server): accept settings-declared s3 backends — shared-schema options, decrypt-at-build, registry preview() * feat(server): remove TREK_S3_*/TREK_UPLOADS_DIR — tombstone boot rule, built-in override as the relocation path * feat(server): deriveSecurity.encryptionKeySet and storage secret plumbing (gate, encrypt, unmask, mask, redact) * chore(server): drop storage-secrets import that only Task 8's probe helper will use * feat(server): seed-once storage-config.json boot import — fail closed on any invalid seed * feat(server): registry snapshot() — effective storage config with provenance * feat(server): api/admin/storage GET/PUT — masked secrets, preview-gated saves, wired reload, audits * feat(server): api/admin/storage/test — per-target ephemeral probes, mirror targets probed individually * test(server): e2e coverage for api/admin/storage — guards, managed refusal, envelope, probes * test(server): give e2e harness DBs the app_settings table the storage registry reads at boot * test(coverage): raise pins after the storage admin-config slice * fix(server): final-review polish — encryption-gate wording, all-field sentinel guard, README/e2e notes * test(server): storage slice-1 review follow-ups — mirror s3-replica probe decrypt, all-encrypted seed without key, test-route 401 * feat(i18n): storage admin domain + tab label in every locale * feat(client): adminApi storage endpoints — getStorage, updateStorage, testStorageBackend * feat(client): storage admin draft-document helpers + relative-time util * feat(client): schema-driven storage BackendForm — fields by kind, mask echo, encryption banner * feat(client): admin storage panel — backends list, category map, health strip, one-PUT save * feat(client): storage tab on the admin page * feat(client): storage admin panel and section on the phone shell * fix(client): block renaming a storage backend onto an existing name * feat(server): allow mirrors on any storage category — lift the backups-only rule * feat(i18n): replicas-on-primary mirror strings in every locale * feat(client): mirror fold/synthesize model helpers for replicas-on-primary * feat(client): BackendForm mirror-targets composer; raw mirror type hidden * feat(client): desktop storage panel folds mirrors into their primaries * feat(client): phone storage panel folds mirrors into their primaries * fix(client): strip removed backends from remaining mirror replicas — no phantom refs * feat(storage): retire photos as a configurable category — served-only via the ServedCategory seam * feat(i18n): descriptive storage category names and descriptions in every locale * feat(client): descriptive category names, descriptions and id badges in the storage panel * docs(storage): fix stale nine-category comments after the eight-category shrink * docs: storage configuration via the admin UI and seed file — README + compose * docs(wiki): Admin Storage page and storage cross-links * docs(server): storage domain blueprint entry; TREK_PLACE_PHOTO_DIR tombstone clarifier * refactor(server): drop the storage env-var tombstones — the variables never shipped * docs: storage config is simply new in v4 — drop the removed-variables story * feat(shared): backfill-status and usage wire schemas on the storage admin state * feat(server): MirrorDriver.backfill — prefix-scoped, size-compared, cancellable replica catch-up * feat(server): one-at-a-time storage backfill jobs + the storage events bridge * feat(server): storage usage scan — nightly cron, on-demand service, persisted with computedAt * feat(server): replica_failure admin notifications — events bridge, hourly debounce, channel registration * feat(i18n): replica-failure notification text in every locale * feat(server): storage backfill and stats-refresh admin endpoints * feat(client): adminApi backfill and stats-refresh endpoints * feat(i18n): storage sync and usage strings in every locale * feat(client): desktop storage panel — sync now, usage display, backfill polling * feat(client): phone storage panel — sync now, usage display, backfill polling * fix(client): per-category usage line on the phone storage panel * docs: sync now, usage stats and replica-failure notifications * test(server): admin scope now includes replica_failure, not just version_available Commit |
||
|
|
3a7c8e1d26 |
Make a bind failure fatal again, and undo four contract tightenings
An adversarial audit over the three merged migration PRs. Sixteen findings
survived triple refutation; these are the ones with a defect behind them.
The serious one is mine, from the gateway move. ws registers
`error: this.emit.bind(this, 'error')` on the http server it attaches to, and
the adapter now runs inside buildApp, before listen(). So from that moment the
http server HAS an error listener and Node stops throwing on one. The adapter's
handler was `() => {}`, which meant EADDRINUSE was forwarded there and dropped:
the process survived, the listen callback never ran, so the banner never
printed and the scheduler never started. A container that looks healthy and
serves nothing. Reproduced, then fixed at both ends: index.ts owns the fatal
decision on the http server, and the adapter logs instead of swallowing.
Four DTOs were tightening contracts rather than describing them, which is the
thing the pipe-before-handler ordering makes easy to do by accident:
- OauthClientCreateDto replaced 'Name is required' with a Zod field report, and
pre-empted the handler's 403 on an instance with the MCP addon off.
- BookingImportConfirmDto demanded per-item fields the service never reads, so
a caller assembling items itself got a 400 about `source`.
- TagCreateDto/TagUpdateDto typed name and color as strings; the legacy route
bound whatever arrived, so a numeric name used to create a tag.
- PluginUserSettingsUpdateDto typed config as a record, where the handler
answers 200 for anything it cannot use.
Two of my own tests were false-green, which the audit demonstrated by mutation
rather than by reading. WSGW-008 asserted on frames that never carry the user
object at all, so it passed with the password_version strip deleted; it reads
the retained identity now. GEO-009 made the same assertion before and after the
write; it distinguishes a cached negative from a miss now, through a hasCached
seam that says what getCached cannot. Both were re-run against the mutation
they exist to catch, and both fail on it.
booking-import geocodes every venue and endpoint in one request loop, up to
thirty sequential calls. Folding maps onto the shared Nominatim cursor put that
loop on the interactive lane, so an import starved every concurrent keystroke
for half a minute. It takes the background lane now, which is what the two-lane
split was built for.
Also: the README's in-container bridge list had outlived one entry and was
missing another, and the journey e2e still mocked a module this branch deleted.
|
||
|
|
b82651bf93 | Merge remote-tracking branch 'origin/dev' into dev | ||
|
|
f43aed762d | refactor(server): retire src/scheduler.ts | ||
|
|
de5d196344 | refactor(server): move the cache sweeps, idempotency purge and airtrail poll in-container | ||
|
|
78b63972d0 | refactor(server): move the version check and demo reset into the admin domain | ||
|
|
1cc46f11b5 | refactor(server): move the reminder crons onto the injected notifications | ||
|
|
d981719941 |
Serve /ws from a Nest gateway with its own adapter
The transport was a hand-rolled setupWebSocket(server) that index.ts kicked
off with a dynamic import after listen(). It is a @WebSocketGateway now, with
DatabaseService and EphemeralTokenService injected instead of imported, and
the heartbeat owned by a lifecycle hook rather than an interval nobody stops.
The wire protocol does not move. Nest's WsAdapter dispatches on
{ event, data } and every deployed client sends { type, tripId }, so
TrekWsAdapter translates rather than asking clients to change. It also builds
the ws server itself instead of delegating: the base's port-0 branch uses
noServer and performs the upgrade from its own registry, which never consults
verifyClient, so delegating would have dropped the origin check and answered
101 to anyone. The per-socket error listener stays attached first, because an
unhandled one kills the process (#1576), and the flood guard counts every
inbound frame before parsing, because malformed frames are exactly the ones
worth limiting.
Four things had to stay where they were, and each is a silent failure if it
does not:
The socket registry stays module state. Seventeen bridges and
notifications.instance build `new RealtimeService()` outside the container;
rooms on a provider instance would leave their broadcasts going to an empty
map, with no error and no log.
The socket id stays a monotonic integer. The client echoes it as X-Socket-Id
and broadcast excludes the originator with Number(excludeSid). A uuid is NaN,
NaN === NaN is false, and every client would receive its own writes back,
which shows up as drag-and-drop that jumps rather than as an error.
buildApp creates the http server and binds the adapter before app.init(), and
callers take it from getHttpServer(). Nest binds gateways during init, so a
later adapter binds nothing; an adapter given the app binds to Nest's own
internal server, which this process never listens on. Both boot green with no
client able to connect.
RealtimeGatewayModule is imported by AppModule alone. RealtimeModule is
@Global, so a gateway there follows into every hand-assembled TestingModule,
and Nest's SocketModule then loads an adapter none of them set: it reaches for
@nestjs/platform-socket.io and calls process.exit(1). Eighteen e2e harnesses
died that way before the split.
src/websocket.ts survives as a re-export of the three fan-out primitives. 115
test files vi.mock that path to assert on broadcasts.
The two transport suites pass unchanged, and 30 new cases pin the pieces
directly: the pv and MFA gates, the numeric socket id, the plugin announcement
firing before the room check, both sides of the origin allowlist, and the
flood guard counting frames no handler wants.
|
||
|
|
c63d6430af | refactor(server): move the auto-backup cron into the backup domain | ||
|
|
19f999329e |
Fold the last legacy service into its domain
src/services/ held one thing: airtrail/, 1305 lines of plain functions over the
better-sqlite3 singleton. They are four providers and two pure helper modules
now, and the directory is empty.
Five hidden edges go with them. The db singleton in three files, the raw
broadcast from src/websocket in two, and the reservations, addons and audit
bridges - all module-level imports that the module graph could not see.
The scheduler stops reaching for it at call time. scheduler.ts ran
require('./services/airtrail/airtrailSync') inside the cron tick, so a wiring
mistake surfaced as "AirTrail sync tick failed" every five minutes rather than
at boot. It takes the dependency through SchedulerDeps now, the same structural
contract backups and placePhotos already used.
One edge stays a seam, deliberately. AirtrailSyncService needs
ReservationsService, because the inbound pull applies remote changes through
the real reservation update path - endpoint restamping and all - and
reimplementing that against raw SQL would be worse than the coupling. The
write-back trigger runs the other way, from reservations.controller, and
injecting there would close
ReservationsModule -> AirtrailModule -> ReservationsModule. airtrail.bridge.ts
holds that one function with the reason in its docblock, rather than a
forwardRef hiding it.
The write-gate suite was module mocks over free functions and is stubs through
a constructor now; it gained four cases on the way (auth failure, remote
delete, disconnected owner). The domain ratchet goes from 19/0/10/19 - set when
integrations was two thin controllers - to 69/52/72/71, which is what the
folded code measures. It is a 1:1 move, so it arrives with the coverage its
tests always gave it.
|
||
|
|
5eab8128bd |
refactor(server): fold the ephemeral-token store and the in-app action registry (#1836)
Two small leftovers, moved to the domains that own them. ephemeralTokens holds the short-lived tokens the WebSocket handshake and the file-download links use. It already had startTokenCleanup/stopTokenCleanup with an unref-ed handle — the right shape. What was missing is anyone calling stop: index.ts kicked the sweep off with a bare require() after listen() and nothing ever ended it. AuthService owns that lifecycle now, through onModuleInit/onModuleDestroy, which is where it belongs — it is the service that issues the tokens. The store itself stays module state. websocket.ts consumes tokens from outside the container, and a token store is only useful if the issuer and the consumer share one. Same reasoning as the weather cache in #1835. inAppNotificationActions is a 22-line handler registry with exactly one consumer; it moves next to it as notifications/in-app-actions.ts. services/ is down to 9 files, from 44. |
||
|
|
c7fed4b1f8 |
refactor(server): make the place-photo cache a provider (#1833)
The last file in be-places-helpers-fold, and the one that had to be a provider rather than a move: it carries three in-memory structures that only do their job if the process has exactly one of them. recentFailures stops a screenful of markers hammering an API that is down inFlight stops a stampede when several requests miss the same placeId knownOnDisk saves an existsSync per cache hit A second instance gives the stampede guard its own inFlight map, so two simultaneous requests for the same uncached place both call the provider. That is why the scheduler takes this one from the container instead of constructing it: SchedulerDeps gains a placePhotos entry alongside backups, filled from app.get() in index.ts — the shape PR #1827 was built for. The module-scope mkdirSync becomes onModuleInit, and GOOGLE_PHOTO_DIR is read per call through RuntimeEnvService rather than frozen at import. That is not cosmetic: TREK_PLACE_PHOTO_DIR is the seam the tests use to keep off the real uploads tree, and its own test used to have to import the module dynamically to set the variable in time. The __dirname fallback moved a level with the file and is corrected accordingly. Consumers: maps, places and share inject it; places.helpers.reclaimPhotoCache takes it as a parameter, since it is a free function. Six test seams follow. Four path mocks become injected stubs with the same mock functions behind them; the collections case keeps the REAL implementation, because what it asserts is that removeIfUnreferenced counts collection_places as a reference (#1081) — a stub there would assert nothing. The share e2e overrides the provider in its container. services/ is down to 14 top-level files, from 44 when this started. |
||
|
|
a634b2a26b |
feat(server): give the scheduler a contract with the container (#1827)
The scheduler runs outside the Nest container but needs four things inside it: backupService, trekPhotoCache, placePhotoCache and airtrailSync. It reaches all four with a require() or await import() at call time. This settles how that should work before four separate folds each invent their own answer, and wires one consumer as proof. It takes the dependencies from the container rather than growing four more bridges, for a reason that showed up while writing the previous PRs: adding one constructor parameter to a service meant hand-editing five bridge files that rebuild their services with `new`. app.get() has no such cost, and hands back the container singleton instead of a second instance. The scheduler declares what it needs structurally — a `backups` with a createBackup — and never names BackupService. That is not fastidiousness: an import of the class would pull in src/nest/backup → src/services/backupService → src/scheduler, which is the exact cycle the lazy `await import()` in runBackup existed to dodge. With the dependency handed in, the scheduler stops importing backupService entirely and the cycle is gone; backupService still imports the scheduler for VALID_INTERVALS, but now only in one direction. index.ts fills the contract from the app buildApp() already initialised, right before the first cron starts, the way bootstrap.ts hands the /mcp handler its registry. Two smaller things fall out. The Nest BackupService was swallowing createBackup's prefix parameter, so it could only ever produce 'backup' names; it forwards it now. And the scheduler can be started without a container at all — its unit tests do exactly that — so runBackup says so in the log instead of throwing, with a case covering it. The other three consumers stay on their require() calls; they move with their own folds. |
||
|
|
1fb6d19f6f |
fix(uploads): pre-create journey and places upload dirs
Both were created lazily on the first upload, which needs write permission on the uploads mount point itself at request time. On hosts whose bind-mounted uploads dir is not writable by the runtime user, every feature writing into an already-existing subdir keeps working while journey photo and place image uploads fail with EACCES (#1762). Creating them in the image and at boot turns that into one loud startup failure instead of a stray 500 on upload. Also drop the recursive chown over all of /app from the final layer: it copied up every inode it touched and duplicated node_modules, dist and the client bundle into a 518 MB layer. The chown now rides the layer that installs node_modules, later copies carry --chown=node:node, and the last layer chowns only the paths it creates. Everything under /app stays node-owned; image size 2.36 GB -> 1.68 GB. |
||
|
|
cafff003a3 | refactor(server): move instance-URL resolution to app-config | ||
|
|
6ff936c613 | chore: move auditLog backend to proper DI | ||
|
|
fb9abdb76c | refactor(server): non-DI islands read env via app-config | ||
|
|
48ba9092c3 | feat(server): add app-config core with fail-fast env validation | ||
|
|
ad893eb1cc |
Release 3.1.0 (#1185)
* Phase 0 — NestJS + Zod foundation harness (F1–F8) (#1050) Co-hosted NestJS app behind the existing Express server via a strangler-fig dispatcher, sharing the same better-sqlite3 connection and JWT httpOnly cookie. Additive and dormant: default routing stays on Express, Nest only serves its own /api/_nest diagnostics until a module opts in. F1 @trek/shared Zod contract package; F2 Nest bootstrap co-hosted (fall-through, single Dockerfile/port); F3 shared better-sqlite3 provider; F4 JWT cookie auth guard (+ @CurrentUser, admin guard); F5 Zod validation pipe + error-envelope parity; F6 Nest test + coverage gates; F7 per-prefix strangler toggle (env, default Express); F8 CI build/typecheck/test/coverage. Remaining F4/F6/F8 checklist items (trip-access + permission levels + MFA policy, e2e harness/seed + 80% gate, Nest↔Express parity test, Playwright PR-comment workflow) are tracked on the first consuming module cards (L1/A1/C1). * feat(weather): migrate /api/weather to the NestJS pilot module (L1) (#1053) First strangler migration (L1): /api/weather is served by a NestJS module. - @trek/shared/weather Zod contract; Nest controller byte-identical to the legacy Express route (paths, query params, status codes, { error } bodies, lang default, ApiError/500 passthrough). Service reuses getWeather/getDetailedWeather (+ shared cache; MCP tools unchanged). - Strangler routes /api/weather to Nest by default; the legacy Express route + its migration-time parity test were decommissioned in this PR. - Frontend (FE2): weatherApi typed against the @trek/shared WeatherResult contract. - Harness: reusable Nest-vs-Express parity harness, e2e harness (temp SQLite + seed/cookie helpers, real JwtAuthGuard), src/nest coverage gate raised to >=80%, src/nest test guide. - Verified end-to-end on a prod mirror (dev1): 401/400/200 via Nest with real Open-Meteo data, Express route gone. * fix(packing): multiply item weight by quantity in bag/total weight calcs (#898) Quantity now counts toward bag and total weights. Generalised to an itemWeight() helper used by every weight sum (bag totals + max, unassigned, grand total; sidebar + bag modal) with unit tests. * feat(i18n): add Korean (ko) translation (#977) Korean translation by @ppuassi, topped up to full en.ts key parity. Language registration follows separately. * feat(i18n): add Japanese (ja) translation (#829) Japanese translation by @soma3978, at full en.ts key parity, registered in supportedLanguages + TranslationContext. * Add Turkish (tr) translation + language registry (#1029) Turkish translation by @SkyLostTR, at full en.ts key parity, registered in supportedLanguages + TranslationContext. * i18n: register Korean + add Ukrainian translation (#1055) Korean translation by @ppuassi (#977) — now registered. Ukrainian by @JeffyOLOLO (#902) — lifted onto a clean branch. Both at full en.ts key parity (2258 keys). * chore: fix monorepo build pipeline and migrate shared to built package (#1056) * chore: fix monorepo build pipeline and migrate shared to built package - Root package.json: add workspace scripts (dev, build, test, test:cov, test:e2e) that delegate to actual scripts in shared/server/client workspaces - shared: add tsup build step (CJS + ESM dual output, .d.ts); consumers now import from the built dist instead of raw TS source via path aliases - server: replace tsc-alias with tsconfig-paths (tsc-alias mangled node_modules paths); fix MCP SDK path aliases to point to root node_modules (../node_modules) - server/scripts/dev.mjs: delay node --watch until tsc -w signals first-pass done, eliminating the spurious restart on every dev startup - client/vite.config.js + vitest.config.ts: remove @trek/shared path alias (no longer needed now that shared is a proper package) - Consolidate package-lock.json at the workspace root; drop per-workspace lock files * chore: fix test script to reflect root package.json * chore: add missing lint and prettier script in root package.json * fix(ci): build shared before tests; fix vitest MCP SDK alias paths vitest.config.ts aliases pointed at ./node_modules/ (server-local) but packages are hoisted to the root node_modules/ in the npm workspace — changed to ../node_modules/. CI jobs now install and build shared before running server/client tests so that @trek/shared's dist/ exists when vitest resolves the package. * fix(docker): update Dockerfile and CI for monorepo workspace structure Dockerfile: - Add shared-builder stage that produces @trek/shared dist before client and server stages need it - Each build stage carries root package.json + package-lock.json so npm can resolve @trek/shared as a workspace dependency - Production stage installs via workspace context (npm ci --workspace=server --omit=dev) so node_modules/@trek/shared symlinks to shared/dist correctly - Copy server/tsconfig.json into the image so tsconfig-paths/register can find the MCP SDK path aliases at runtime - CMD cds into /app/server before starting node so tsconfig-paths baseUrl resolves and ../node_modules points to /app/node_modules - Remove mkdir for /app/server (now a real dir); keep symlinks for uploads/data docker.yml version-bump: - Replace manual per-workspace cd+npm-version calls with single: npm version --workspaces --include-workspace-root --no-git-tag-version (mirrors the version:* scripts in root package.json) - git add now references root package-lock.json; adds shared/package.json .dockerignore: add shared/dist package.json: fix version:prerelease preid (alpha → pre) * fix(tests): use in-memory SQLite per worker in test mode vitest pool:forks spawns parallel worker processes that all called initDb() on the same data/travel.db, causing SQLite "database is locked" and "duplicate column name" races. When NODE_ENV=test each fork now gets an isolated :memory: DB so migrations run independently with no file contention. * chore(ci): add ACT guards to skip DockerHub steps in local act runs act sets ACT=true automatically. Guards added: - docker login: if: ${{ !env.ACT }} - build outputs: type=docker (local load) when ACT, push-by-digest when CI - digest export/upload: if: ${{ !env.ACT }} - merge job: if: ${{ !env.ACT }} - release-helm job (docker.yml): if: ${{ !env.ACT }} - version-bump git push (docker.yml): wrapped in [ -z "$ACT" ] shell guard Run locally with: ./bin/act -j build -W .github/workflows/docker.yml \ -P ubuntu-latest=catthehacker/ubuntu:act-latest * fix(ci): move ACT guards to step level; add guards to security.yml env context is invalid in job-level if conditions — moved all ACT guards down to individual steps. Also guards docker login + scout in security.yml so act can run the build-only part of that workflow. * fix(ci): skip git fetch and tag logic in act (no remote access in local containers) * Revert "fix(ci): skip git fetch and tag logic in act (no remote access in local containers)" This reverts commit |
||
|
|
de6c0fb781 |
fix: prevent Invalid URL crash when APP_URL lacks a protocol (#972)
* fix: prevent Invalid URL crash when APP_URL lacks a protocol (issue #970) - Add getMcpSafeUrl() to notifications.ts: wraps getAppUrl() and guarantees a result that satisfies the MCP SDK's checkIssuerUrl requirement (https:// or http://localhost). Non-HTTPS, non-localhost URLs fall back to http://localhost:{PORT} instead of propagating an "Issuer URL must be HTTPS" error. - Switch app.ts, mcp/index.ts, mcp/oauthProvider.ts, and oauthService.ts to import getMcpSafeUrl instead of getAppUrl for all MCP resource URL construction, so a misconfigured APP_URL never crashes the metadata router initialisation. - Restrict the SDK metadata router middleware to /.well-known/* paths only. Previously it was invoked on every request; in production the lazy getMetaRouter() init ran on GET / and threw "Invalid URL" when APP_URL had no scheme, returning 500 for every page load. - Log a startup warning when APP_URL is set but not usable, and include the resolved App URL in the startup banner so operators can confirm the correct value at a glance. - Update oauth.test.ts mock to target notifications.getMcpSafeUrl. * fix: show getAppUrl in banner and add two separate APP_URL startup checks - Banner now displays getAppUrl() (the resolved app URL) rather than getMcpSafeUrl() so operators see the actual configured value - Two independent startup warnings after the banner when APP_URL is set: 1. whether APP_URL is a valid URL (parseable by new URL()) 2. whether APP_URL is MCP-safe (https:// or http://localhost) - Fix getMcpSafeUrl() fallback port to use Number(PORT) || 3001, consistent with how index.ts parses PORT * fix: update oidc.ts to import getAppUrl from notifications |
||
|
|
22f3bf4bfc |
fix: add APP_VERSION fallback and HOST bind address env var (#952 #953) (#955)
* fix: add APP_VERSION fallback and HOST bind env var (#952 #953) - Read package.json version when APP_VERSION env var is absent so the startup banner shows the correct version for source/Proxmox installs - Add HOST env var to control the HTTP bind address; only applied when set so Docker deployments are unaffected (bind-all-interfaces default) - Parse PORT as Number() so malformed values like '10.0.0.72:3001' fall back to 3001 instead of silently misbehaving - Document HOST in .env.example, Environment-Variables wiki, and Install-Proxmox wiki with explicit warnings against using it in Docker * fix: correct package.json path in APP_VERSION fallback index.ts sits at server/src/ — one level up reaches server/package.json, not two (../../ overshot to the repo root where no package.json exists). |
||
|
|
1f5deeba6c |
Bug fixes - April 27th 2026 (#907)
* fix: clean up dangling FK references before deleting a user Resolves FOREIGN KEY constraint failed (500) on DELETE /api/admin/users/:id and DELETE /api/auth/me when the target user had rows in trip_members.invited_by, share_tokens.created_by, budget_items.paid_by_user_id, journeys.user_id, journey_entries.author_id, journey_contributors.user_id, or journey_share_tokens.created_by — none of which had ON DELETE clauses. Introduces deleteUserCompletely() in userCleanupService.ts which wraps all cleanup and the final DELETE FROM users in a single transaction. Both adminService.deleteUser and authService.deleteAccount now call it instead of the bare DELETE. Tests ADMIN-005b and AUTH-040 cover all reference types including notification sender/recipient and notice dismissals. * test: extend FK deletion tests to cover journeys, files, and photos ADMIN-005b and AUTH-040 now also seed and assert: - owned journey with entries (cascade-deleted via journeys.user_id cleanup) - trip_files.uploaded_by (SET NULL — file survives, attribution cleared) - trek_photos.owner_id (SET NULL — photo record survives, owner cleared) - trip_photos.user_id (CASCADE — photo association removed) * test: extend user deletion tests to cover all FK relationships ADMIN-005b and AUTH-040 now seed and assert every user FK relationship: CASCADE (row deleted): trips, trip_members, tags, mcp_tokens, oauth_tokens, oauth_consents, vacay_plans, vacay_plan_members, bucket_list, visited_countries, visited_regions, packing_templates, invite_tokens, collab_notes, settings, password_reset_tokens, notification_channel_preferences SET NULL (row survives, column nulled): categories, todo_items.assigned_user_id, packing_bags, audit_log Caught and fixed: notification_preferences was dropped in migration 72; correct table is notification_channel_preferences. * fix: preserve URL hash and OIDC redirect target through login flow - Include location.hash in redirect param at all three producer sites (ProtectedRoute, axios 401 interceptor, OAuthAuthorizePage) so hash fragments survive the login bounce - Stash redirectTarget in sessionStorage before any OIDC provider redirect and restore it after the code exchange, since the IdP strips the original ?redirect= param during the roundtrip - Clear sessionStorage on OIDC error to avoid stale state - Add tests covering sessionStorage stash on mount, navigate to saved redirect after OIDC exchange, fallback to /dashboard, and cleanup on error * fix: use day position instead of ID for accommodation date range clamping Math.min/Math.max over raw day IDs breaks the start/end picker when a trip's day IDs are non-monotonic relative to day_number (normal after repeated generateDays extend/shrink cycles). Replaced with findIndex lookups so clamping is always based on positional order. Closes #889 * fix: normalize env var comparisons to be case-insensitive All NODE_ENV, DEMO_MODE, OIDC_ONLY, FORCE_HTTPS, COOKIE_SECURE, and ALLOW_INTERNAL_NETWORK checks now use .toLowerCase() so values like 'Production' or 'True' behave identically to their lowercase forms. Also adds APP_VERSION to the startup banner. * fix: delete surplus days when shortening a trip When shrinking a trip's date range, surplus days are now deleted along with their assignments, notes, and accommodations (cascade). Places remain in the trip pool; reservations keep their day reference nulled by the existing ON DELETE SET NULL constraint (issue #909). Updates TRIP-SVC-011 to reflect the new behaviour; adds TRIP-SVC-016 as a regression test for the empty-day case. * fix: auto-backup retention deletes itself and manual backups on Docker Two bugs in cleanupOldBackups: 1. Filter was .endsWith('.zip') — swept manual backup-*.zip files too. Now restricted to auto-backup-* prefix. 2. Age was derived from stat.birthtimeMs, which is 0 on overlayfs (Docker default), making every backup appear epoch-old and get deleted immediately. Age is now parsed from the filename timestamp and falls back to mtimeMs (reliable on overlayfs). Also converts inline require('./services/auditLog') calls to a static import throughout scheduler.ts, and adds 8 unit tests covering the fixed retention logic including the overlayfs regression case. * test: update TRIP-024 to match delete behavior on trip shrink * feat: add bypass-branch-check label to skip branch enforcement |
||
|
|
d7a71c0572 |
feat(notifications): reminders for todos with upcoming due dates
Todos already support a due_date field but nothing notifies the user when a deadline is approaching — you'd only remember if you happened to look at the Lists tab. This wires a reminder into the existing notification pipeline so due-date todos behave like trip-start reminders. Details: - New `todo_due` event type alongside trip_reminder; all four channels (in-app, email, webhook, ntfy) supported and toggleable per user in Settings > Notifications. - New daily scheduler task (9 AM local TZ) queries unchecked todos whose due_date is within the next 3 days. Each todo gets at most one reminder per 24 hours, tracked via a new todo_items.reminded_at column (migration 116). - If the todo has an assigned user, only that user is reminded; if not, every member of the trip gets the notification. - Strings added in all 15 UI languages and for all notification carriers. - Gated by app_settings.notify_todo_due (default on) so admins can disable it globally. |
||
|
|
b5b1d32b31 |
feat(photos): add 1h disk cache for remote thumbnails and keep tabs mounted
Closes #686 - Add trekPhotoCache service: SHA1-keyed disk cache under uploads/photos/trek/, 1h TTL, in-flight dedup map to prevent stampedes on concurrent requests - Add migration 108: trek_photo_cache_meta table - Hook cache into streamPhoto for Immich/Synology thumbnail path; originals bypass cache - Add fetchImmichThumbnailBytes / fetchSynologyThumbnailBytes returning Buffer instead of piping, used by the cache layer - Add scheduler entry (every 2h + startup sweep) to evict expired disk files and DB rows via sweepExpired() - Client: convert journey tab conditional-mount to hidden-toggle so img elements stay in DOM across tab switches, preventing redundant thumbnail requests on rapid tab changes - Expose invalidateSize() on JourneyMapHandle; call it on map tab activation to fix Leaflet rendering in previously-hidden container |
||
|
|
b194e8317d |
feat(pwa): implement real offline mode with IndexedDB sync
Add genuine offline read/write capability for trips: - Dexie IndexedDB schema (trips, places, packing, todo, budget, reservations, files, mutationQueue, syncMeta, blobCache) - Repo layer for all domains: offline reads from Dexie, writes optimistically to Dexie and enqueue mutations for later replay - Mutation queue with UUID idempotency keys (X-Idempotency-Key), FIFO flush, temp-ID reconciliation on 2xx, fail-and-continue on 4xx - Trip sync manager: caches all trips with end_date >= today or null, auto-evicts 7d after end_date, fetches bundle endpoint in one request - Map tile prefetcher: bbox from place coords, zooms 10-16, 50MB cap, warms SW cache via fetch - Sync triggers: network online → flush + syncAll; WS reconnect → flush only (rate-limiter safe); visibilitychange/30s → flush only - WS remoteEventHandler writes through to Dexie on every event - Server idempotency middleware + idempotency_keys table (migration 100, 24h TTL nightly cleanup) - GET /api/trips/:id/bundle endpoint for efficient single-request sync - OfflineBanner component: amber (offline) / blue (syncing) / hidden - OfflineTab in Settings: cached trip list, re-sync and clear actions - usePendingMutations hook for per-item pending indicators Closes #505 #541 |
||
|
|
fc29c5f7d0 |
feat(notifications): add unified multi-channel notification system
Introduces a fully featured notification system with three delivery channels (in-app, email, webhook), normalized per-user/per-event/ per-channel preferences, admin-scoped notifications, scheduled trip reminders and version update alerts. - New notificationService.send() as the single orchestration entry point - In-app notifications with simple/boolean/navigate types and WebSocket push - Per-user preference matrix with normalized notification_channel_preferences table - Admin notification preferences stored globally in app_settings - Migration 69 normalizes legacy notification_preferences table - Scheduler hooks for daily trip reminders and version checks - DevNotificationsPanel for testing in dev mode - All new tests passing, covering dispatch, preferences, migration, boolean responses, resilience, and full API integration (NSVC, NPREF, INOTIF, MIGR, VNOTIF, NROUTE series) - Previous tests passing |
||
|
|
905c7d460b |
Add comprehensive backend test suite (#339)
* add test suite, mostly covers integration testing, tests are only backend side * workflow runs the correct script * workflow runs the correct script * workflow runs the correct script * unit tests incoming * Fix multer silent rejections and error handler info leak - Revert cb(null, false) to cb(new Error(...)) in auth.ts, collab.ts, and files.ts so invalid uploads return an error instead of silently dropping the file - Error handler in app.ts now always returns 500 / "Internal server error" instead of forwarding err.message to the client * Use statusCode consistently for multer errors and error handler - Error handler in app.ts reads err.statusCode to forward the correct HTTP status while keeping the response body generic |
||
|
|
a4d6348a79 |
fix: add raw.githubusercontent.com to CSP connect-src for Atlas map
The Atlas feature fetches country GeoJSON from GitHub raw content, which was blocked by the Content Security Policy connect-src directive. Closes #285 |
||
|
|
fabf5a7e26 |
fix: remove redundant db import alias in index.ts
db was already imported as addonDb; the extra db named import was unnecessary. Updated the one stray db.prepare call at line 155 to use addonDb consistently. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
e71bd6768e |
fix: show actual backend error messages on login page and add missing db import
- LoginPage now uses getApiErrorMessage() instead of err.message so backend validation errors (e.g. "Password must be at least 8 characters") are displayed instead of the generic "Request failed with status code 400" - Add missing db import in server/src/index.ts Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
43fc4db00e | fix: convert stored HTTP photo URLs to base64 for map markers, add exchangerate-api to CSP | ||
|
|
edafe01387 | Merge branch 'dev' into dev | ||
|
|
16277a3811 |
security: fix missing trip access checks on Immich routes (GHSA-pcr3-6647-jh72)
security: require auth for uploaded photos (GHSA-wxx3-84fc-mrx2) GHSA-pcr3-6647-jh72 (HIGH): - Add canAccessTrip check to all /trips/:tripId/photos and /trips/:tripId/album-links endpoints - Prevents authenticated users from accessing other trips' photos GHSA-wxx3-84fc-mrx2 (LOW): - /uploads/photos now requires JWT auth token or valid share token - Covers and avatars remain public (needed for login/share pages) - Files were already blocked behind auth |
||
|
|
add0b17e04 |
feat(auth): migrate JWT storage from localStorage to httpOnly cookies
Eliminates XSS token theft risk by storing session JWTs in an httpOnly cookie (trek_session) instead of localStorage, making them inaccessible to JavaScript entirely. - Add cookie-parser middleware and setAuthCookie/clearAuthCookie helpers - Set trek_session cookie on login, register, demo-login, MFA verify, OIDC exchange - Auth middleware reads cookie first, falls back to Authorization: Bearer (MCP unchanged) - Add POST /api/auth/logout to clear the cookie server-side - Remove all localStorage auth_token reads/writes from client - Axios uses withCredentials; raw fetch calls use credentials: include - WebSocket ws-token exchange uses credentials: include (no JWT param) - authStore initialises isLoading: true so ProtectedRoute waits for /api/auth/me Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
e10f6bf9af |
fix: remove JWT_SECRET env var — server manages it exclusively
Setting JWT_SECRET via environment variable was broken by design: the admin panel rotation updates the in-memory binding and persists the new value to data/.jwt_secret, but an env var would silently override it on the next restart, reverting the rotation. The server now always loads JWT_SECRET from data/.jwt_secret (auto-generating it on first start), making the file the single source of truth. Rotation is handled exclusively through the admin panel. - config.ts: drop process.env.JWT_SECRET fallback and JWT_SECRET_IS_GENERATED export; always read from / write to data/.jwt_secret - index.ts: remove the now-obsolete JWT_SECRET startup warning - .env.example, docker-compose.yml, README: remove JWT_SECRET entries - Helm chart: remove JWT_SECRET from secretEnv, secret.yaml, and deployment.yaml; rename generateJwtSecret → generateEncryptionKey and update NOTES.txt and README accordingly |
||
|
|
78695b4e03 |
fix: replace JWT tokens in URL query params with short-lived ephemeral tokens
Addresses CWE-598: long-lived JWTs were exposed in WebSocket URLs, file download links, and Immich asset proxy URLs, leaking into server logs, browser history, and Referer headers. - Add ephemeralTokens service: in-memory single-use tokens with per-purpose TTLs (ws=30s, download/immich=60s), max 10k entries, periodic cleanup - Add POST /api/auth/ws-token and POST /api/auth/resource-token endpoints - WebSocket auth now consumes an ephemeral token instead of verifying the JWT directly from the URL; client fetches a fresh token before each connect - File download ?token= query param now accepts ephemeral tokens; Bearer header path continues to accept JWTs for programmatic access - Immich asset proxy replaces authFromQuery JWT injection with ephemeral token consumption - Client: new getAuthUrl() utility, AuthedImg/ImmichImg components, and async onClick handlers replace the synchronous authUrl() pattern throughout FileManager, PlaceInspector, and MemoriesPanel - Add OIDC_DISCOVERY_URL env var and oidc_discovery_url DB setting to allow overriding the auto-constructed discovery endpoint (required for Authentik and similar providers); exposed in the admin UI and .env.example |
||
|
|
fbe3b5b17e |
Merge pull request #225 from andreibrebene/improvements/various-improvements
Improvements/various improvements |
||
|
|
10107ecf31 |
fix: require auth for file downloads, localize atlas search, use flag images
- Block direct access to /uploads/files (401), serve via authenticated /api/trips/:tripId/files/:id/download with JWT verification - Client passes auth token as query parameter for direct links - Atlas country search now uses Intl.DisplayNames (user language) instead of English GeoJSON names - Atlas search results use flagcdn.com flag images instead of emoji |
||
|
|
7522f396e7 |
feat: configurable trip reminders, admin full access, and enhanced audit logging
- Add configurable trip reminder days (1, 3, 9 or custom up to 30) settable by trip owner - Grant administrators full access to edit, archive, delete, view and list all trips - Show trip owner email in audit logs and docker logs when admin edits/deletes another user's trip - Show target user email in audit logs when admin edits or deletes a user account - Use email instead of username in all notifications (Discord/Slack/email) to avoid ambiguity - Grey out notification event toggles when no SMTP/webhook is configured - Grey out trip reminder selector when notifications are disabled - Skip local admin account creation when OIDC_ONLY=true with OIDC configured - Conditional scheduler logging: show disabled reason or active reminder count - Log per-owner reminder creation/update in docker logs - Demote 401/403 HTTP errors to DEBUG log level to reduce noise - Hide edit/archive/delete buttons for non-owner invited users on trip cards - Fix literal "0" rendering on trip cards from SQLite numeric is_owner field - Add missing translation keys across all 14 language files Made-with: Cursor |
||
|
|
9b2f083e4b |
feat: notifications, audit logging, and admin improvements
- Add centralized notification service with webhook (Discord/Slack) and email (SMTP) support, triggered for trip invites, booking changes, collab messages, and trip reminders - Webhook sends one message per event (group channel); email sends individually per trip member, excluding the actor - Discord invite notifications now include the invited user's name - Add LOG_LEVEL env var (info/debug) controlling console and file output - INFO logs show user email, action, and IP for audit events; errors for HTTP requests - DEBUG logs show every request with full body/query (passwords redacted), audit details, notification params, and webhook payloads - Add persistent trek.log file logging with 10MB rotation (5 files) in /app/data/logs/ - Color-coded log levels in Docker console output - Timestamps without timezone name (user sets TZ via Docker) - Add Test Webhook and Save buttons to admin notification settings - Move notification event toggles to admin panel - Add daily trip reminder scheduler (9 AM, timezone-aware) - Wire up booking create/update/delete and collab message notifications - Add i18n keys for notification UI across all 13 languages Made-with: Cursor |
||
|
|
804c2586a9 |
fix: tighten CSP, fix API key exposure, improve error handling
- Remove 'unsafe-inline' from script-src CSP directive - Restrict connectSrc and imgSrc to known external domains - Move Google API key from URL query parameter to X-Goog-Api-Key header - Sanitize error logging in production (no stack traces) - Log file link errors instead of silently swallowing them https://claude.ai/code/session_01SoQKcF5Rz9Y8Nzo4PzkxY8 |
||
|
|
13580ea5fb | Merge branch 'main' into feat/#155 | ||
|
|
faebc62917 |
Merge branch 'pr-125'
# Conflicts: # client/src/api/client.ts # client/src/i18n/translations/ar.ts # client/src/i18n/translations/es.ts # client/src/i18n/translations/fr.ts # client/src/i18n/translations/nl.ts # client/src/i18n/translations/ru.ts # client/src/i18n/translations/zh.ts # client/src/pages/AdminPage.tsx # client/src/pages/SettingsPage.tsx # server/package.json # server/src/db/migrations.ts # server/src/index.ts # server/src/routes/admin.ts |
||
|
|
66f5ea50c5 |
feat(require-mfa): #155 enforce MFA via admin policy toggle across app access
Add an admin-controlled `require_mfa` policy in App Settings and expose it via `/auth/app-config` so the client can enforce it globally. Users without MFA are redirected to Settings after login and blocked from protected API/WebSocket access until setup is completed, while preserving MFA setup endpoints and admin recovery paths. Also prevent enabling the policy unless the acting admin already has MFA enabled, and block MFA disable while the policy is active. Includes UI toggle in Admin > Settings, required-policy notice in Settings, client-side 403 `MFA_REQUIRED` handling, and i18n updates for all supported locales. |
||
|
|
26c1676cdd | revert: remove auth from file uploads — breaks img/pdf rendering in browser | ||
|
|
4ddfa92c14 |
security: require auth for file and photo uploads
/uploads/files/ and /uploads/photos/ now require a valid Bearer token. Covers and avatars remain public (needed for shared pages and profiles). Prevents unauthenticated access to uploaded documents and trip photos. |
||
|
|
de859318fa |
feat: admin audit log — merged PR #118
Audit logging for admin actions, backups, auth events. New AuditLogPanel in Admin tab with pagination. Dockerfile security: run as non-root user. i18n keys for all 9 languages. Thanks @fgbona for the implementation! |
||
|
|
10ebf46a98 |
harden runtime config and automate first-run permissions
Run the container as a non-root user in production to fail fast on insecure deployments. Add DEBUG env-based request/response logging for container diagnostics, and introduce a one-shot init-permissions service in docker-compose so fresh installs automatically fix data/uploads ownership for SQLite write access. |
||
|
|
a314ba2b80 |
feat: public read-only share links with permissions — closes #79
Share links: - Generate a public link in the trip share modal - Choose what to share: Map & Plan, Bookings, Packing, Budget, Chat - Permissions enforced server-side - Delete link to revoke access instantly Shared trip page (/shared/:token): - Read-only view with TREK logo, cover image, trip details - Tabbed navigation with Lucide icons (responsive on mobile) - Interactive map with auto-fit bounds per day - Day plan, Bookings, Packing, Budget, Chat views - Language picker, TREK branding footer Technical: - share_tokens DB table with per-field permissions - Public GET /shared/:token endpoint (no auth) - Two-column share modal (max-w-5xl) |