mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-10-19 05:26:47 +00:00
dev
30 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Maurice
|
9494521ac7 |
feat(tours): plan paddle tours, hand-drawn legs, reverse and return to start, maps per tour type
- Paddle as a plannable tour type: one boat (kayak, canoe, SUP, other), river or lake, a pace that a boat change never overwrites; time is distance over pace - Hand-drawn legs beside routed ones, drawn dotted and labelled as not routed, kept through save, reopen and edits elsewhere - Reverse a tour and close it back to its start, each one undo step - GPX import shows a preview first (line, measures, duplicates, type) and only the confirmation stores the tour - Base maps and overlays per tour type in the planner: hike keeps topo and the via ferrata layer, paddle gets the OpenSeaMap seamark overlay |
||
|
Maurice
|
98cc57942b |
feat(tours): 3D terrain view with flyover and grade colours, imported tours carry no hiking difficulty
The tours map gets a 3D mode on the same map surface between the side panels: a "3D" switch in the layer pill drapes the route over open Terrarium elevation tiles (AWS Open Data), colours it by grade, plays a flyover tied to the elevation profile in the rail, and keeps planning working by clicking and dragging waypoints on the terrain. The corridor around the route is preloaded before the flyover so the landscape never loads in front of the camera. GPX imports no longer get a made-up hiking difficulty: max_hiking_difficulty becomes nullable (new migration, waypoints kept), the import writes null and the client shows no T badge for it. Drawn tours still default to T2. Info hints in the planner show their tooltip only, and the discard confirmation reads "Discard changes". |
||
|
Maurice
|
724c88c201 |
style: format server and shared with prettier
A mechanical prettier --write over server/src, server/tests, server/scripts and shared/src with the workspaces' .prettierrc; nothing but layout and import order changes, and every file keeps the same set of imports. shared is formatted throughout and its format baseline is empty. The server baseline keeps the 35 files that Prettier would grow past their lint:size limit (they need a split first, which is not a formatting change), and the generated entity files stay as their generator writes them. |
||
|
Maurice
|
8c6157aa46 |
test(server): pin the request id through the body parsers
A handler behind express.json() is reached from the request stream's 'end' event, long after the correlation middleware returned, so it looked as if every write lost its id. It does not: raw-body and multer bind their callbacks with AsyncResource at the point they are called, which is inside the request's correlation. Nothing tested that, though, because every REQID case was a GET. REQID-007 posts a JSON body and REQID-008 one that arrives in several chunks, and both expect the handler to see the id the response carries. The middleware comment now says why this holds. |
||
|
Maurice
|
8632f93c37 |
feat(server): correlate log lines with a request id per entry point
A 5xx stack and its access-log line were two unrelated lines, the exception filter printed the stack with console.error and no request context, and the MCP, WebSocket, plugin RPC and cron entry points logged nothing per call. A self-hosted bug report could not be traced from a failure back to the call that caused it. Every HTTP request now runs under an id held in AsyncLocalStorage: a well-formed incoming X-Request-Id is kept, anything else is replaced by a UUID, and the response carries it in X-Request-Id. The logger tags every line written inside a unit of work with `[http <id>]`. The exception filter leaves a server-side failure on res.locals and the access log writes the request line and the stack as one entry under that id; without an access log watching the response it still prints the error itself. MCP tool, resource and prompt calls (through a new `around` seam on the registry), WebSocket messages, plugin RPC dispatches and cron ticks each run through traceEntry(), which gives the call its own id (remembering the outer one as its parent) and logs one line for it: debug on success, like the HTTP access log, and warn or error with the reason on failure. The call's result and timing are unchanged, a failed cron tick is logged once rather than twice, and a WebSocket handler rejection no longer reaches rxjs's asynchronous rethrow. |
||
|
Maurice
|
ba1504d6e9 |
refactor(server): give every environment variable one config owner
Eight registerAs tokens existed while every consumer read the same values through readEnv() instead, so the boot-stable layer the repo rules describe was unused and nothing said which path owned a variable. FORCE_HTTPS, for instance, was frozen by the middleware through the http token and read live by the cookie code at the same time. A variable is now owned by exactly one side. TRUST_PROXY and HSTS_INCLUDE_SUBDOMAINS belong to httpConfig, which buildApp() already injects into the pre-init middleware, and TREK_PLACE_PHOTO_DIR to the new storageConfig, which the storage registry now injects instead of reading it through RuntimeEnvService. Their derive functions moved to boot-derive.ts, which deriveAll() never calls. The six tokens nothing injected are gone: their values are needed before the container exists (config.ts, the database, src/mcp, the SSRF guard) and stay on readEnv(). config-ownership.test.ts records the keys each side reads through a Proxy and fails on a variable read through both, on a token nothing injects and on a variable the schema does not validate. env-reference.test.ts holds server/.env.example to the schema, which had drifted by 34 variables (SMTP, WebAuthn, plugins, MCP session tuning, TREK_MANAGED and others); they are documented now. jwt-verify.ts no longer cites a middleware file that is gone. |
||
|
elmocito
|
9e475f447a | feat(tours): add non-motorized day tour planning | ||
|
Maurice
|
bd5a6cc748 |
fix(cors): the instance's own address no longer trips ALLOWED_ORIGINS
With ALLOWED_ORIGINS set, every request carrying an Origin that was not on the list was failed with an unhandled error, which the exception filter answered with a 500. A browser sends Origin on every POST and WebSocket upgrade, including same-origin ones, so opening TREK under any address the list did not name (the LAN IP, a second hostname, http instead of https) broke the login with "Internal Server Error" while another browser that used the listed address worked. A request whose Origin is the host it was sent to is now treated as same-origin and always passes, for REST and the /ws upgrade alike. A foreign origin is still refused, now with a 403 "Not allowed by CORS" and a warning in the log naming the origin to add. Closes #2543 |
||
|
Maurice
|
650451bdf7 |
feat(notifications): Web Push delivers notifications to every device, even with TREK closed
A fourth built-in channel next to email, webhook and ntfy. Browsers register per device, the server encrypts per RFC 8291 and signs per RFC 8292 with node:crypto and sends only to known push services through the SSRF guard. The event matrix decides per category what arrives by push; admins switch the channel on under Notifications. VAPID keys come from the environment or are generated once, only while neither stored row exists, and kept encrypted. A pair that does not decrypt or match is never overwritten and never swapped for another one: push stays off with an error in the log and every device stays registered until the key is fixed. The subject is VAPID_SUBJECT, then an https APP_URL, then the project URL. Only chat messages collapse into one notification per trip, and a replacement still alerts. A 403 removes a device only after three failures in a row; 404 and 410 remove it at once. Password change, reset and an admin setting a password drop the user's devices with the other credentials. The request log redacts subscriptions and endpoints, boot errors print *** for every secret variable, and logout forgets the device with a plain fetch so an expired session cannot redirect midway. Addresses discussion #894. |
||
|
Maurice
|
141afe708d |
fix(journey): let the browser read a picked clip, so it previews and gets a poster
A clip picked in the entry editor is previewed through a <video> on an object URL, and its poster frame is grabbed the same way before anything is uploaded. The CSP never named media-src, so the browser fell back to default-src, which does not admit blob: at all. The preview stayed empty and the poster capture failed silently, so every clip landed without a poster and its thumbnail route answered 404 (issue #2341, the part that survived 4.3.0). None of it showed in development, where Vite serves the document and helmet never sees it. media-src now carries 'self' and blob:. For the clips that still come up without a poster, because the browser cannot decode them, the entry card and the editor tile draw the play badge on a black tile instead of asking for a thumbnail that is 404 by design, the way the gallery already did. Closes #2341 |
||
|
Maurice
|
77162aa8d5 |
feat(roadtrip): avoid tolls, motorways and ferries, through a second engine
The switch a road tripper keeps asking for has never worked. OSRM's `exclude` is the only
avoidance TREK has, and the two public hosts it ships with answer it HTTP 400 on every
leg: their car profile is built without excludable classes, and their own Lua says why
("no memory for classes on demo server"). So "drive this without the tolls" produced
nothing at all on a default install, and the fallback inside the alternatives bar quietly
gave up after one refusal per host.
FOSSGIS runs a public Valhalla on the whole planet next to that OSRM, keyless, and there
avoidance is a runtime weighting rather than a profile that has to be rebuilt. This adds
it as a second engine asked only the questions the first cannot answer, and puts three
switches under the road trip settings that change the road itself rather than offering an
alternative beside it. Paris to Lyon measured 465.8 km / 297 min with tolls against
492.0 km / 454 min without; Hamburg to Copenhagen comes back ferry-free at 469.8 km,
which OSRM cannot produce at all — where a ferry IS the crossing it answers NoRoute
rather than driving round.
Four things the docs would have got wrong, all measured against the live instance:
- `format=osrm` does not carry `has_toll`. That flag is the entire reason for doing this,
and it exists only in the native answer — which is also 5.7x smaller, because the OSRM
shape drags every intersection along. So the adapter translates instead of borrowing a
familiar envelope.
- Without a `radius` Valhalla refuses a point it cannot bind to a road, where OSRM still
snaps one. Vienna's Stephansdom, in a pedestrian zone, answers error 442 until a search
radius is allowed. TREK is full of such places, so every location carries one.
- Ten locations is a hard cap and a road trip day is bundled into one request. A day over
it is split into runs overlapping by one point, which is exact rather than approximate:
a measured six-stop day came back within two metres and under a second of the same day
asked whole.
- Its drive times run 5.4 % above OSRM's over ten European legs and 36 % above on one, and
those times feed the arrival clocks of a day. So a day is answered by one engine or by
neither, never by both, and a piece that will not answer loses the whole day rather than
half of it — the schedule chains leg times blindly and would otherwise print a complete,
plausible, silently wrong timetable.
Because the weighting can fail, the honest surface is the answer rather than the request.
`use_tolls: 0` still returns a tolled road where there is no untolled crossing, so what
the road actually turned out to be is read back per leg, and a day that could not avoid
what it was asked to avoid says so on its own header instead of wearing a label the road
disproves. The switches disable themselves, with a reason, on an instance pointed at its
own OSRM: that operator chose against public hosts, and a second one they never asked for
would undo the choice.
Two things that would have broken quietly and did not:
- The route cache keys on `profile:unit:coords` and is shared with the day planner, the
studio and the booking geometry. Without the avoidance in the key, flipping a switch
returns the old road out of cache in under a millisecond — and worse, an avoiding answer
stored under the plain key hands those other callers a road nobody drives.
- `snapped` is OSRM-only and carries the dashed kerb-to-door spur from #2273. Valhalla
echoes the coordinates it was given and never says where it bound them, but the ends of
a leg's line are exactly those points: measured against Füssen, its first vertex matches
OSRM's snapped location to the digit.
The alternatives bar gets the same engine, which finally makes its "no motorway" and "no
tolls" offers appear on a default install, plus a ferry-free one. `sameRoad` now compares
geometry rather than numbers there, because two engines' distances for the same road sit
kilometres apart and the old 200 m tolerance called that a discovery.
Desktop only, like the whole road trip mode: nothing under client/src/mobile renders it.
|
||
|
Maurice
|
2f71549413 |
fix(maps): let the imagery through the CSP, so the GL switch does something
The satellite toggle has worked on Leaflet for as long as it has existed and did nothing at all on a GL map, which is the default renderer since the vector basemap landed. Not for want of the layer: connect-src never named server.arcgisonline.com. Leaflet asks for a tile as an <img>, and img-src waves every https: host through, so the gap could sit there unnoticed for as long as Leaflet was the only renderer that showed the imagery. A GL map reads the raster through fetch to hand it to WebGL, and that lands on connect-src instead, where the host is missing. The browser refuses it in the document, which is the worst shape a failure can take here: the switch flips, the layer goes on, the source is there, and no tile ever arrives. Nothing in the app can see it either, because a CSP refusal is not an error the fetch can catch. The prefetcher asks for the same tiles the same way, with mode 'no-cors', which relaxes CORS and not CSP, so the imagery was never cached for offline use either. Same shape as #2180, which added the other two raster presets for exactly this reason and did not reach this host. |
||
|
Maurice
|
e11ae6f056 |
feat(roadtrip): shape the drive — vias, alternatives and stops along the way
The road-trip addon's second half: the parts that let a planned drive be argued with rather than only read. Reshaping. A via records which stop it follows by position, and that position is resolved fresh every time the day is routed — so inserting, reordering or removing a stop moved the ground under every via after it, and the drive silently snapped back to the road it had been steered away from while the handle still sat on the map where it was left. The correction is computed against the day as it stands and written with the same change, in one transaction: a half-applied re-anchoring mixes two numbering schemes in one day. The one judgement that needs geometry — which half of a split leg a via belongs to when a stop lands in the middle of it — is measured on the road actually driven, so a fuel stop added to a bent leg keeps the bend on the side it belongs to. Alternatives. OSRM answers most long legs with exactly one route, which read as "there is no other way" when a perfectly good slower road existed. When it offers nothing, the same drive is asked for again without the motorway, and failing that without the tolls — one question at a time, stopping as soon as one lands. The offer says what it is rather than how much slower. Stops along the way. Everything the corridor search can find is a pause, not a destination: fuel, charging, rest areas, campsites, food and sights all sit inside the leg they were found on, carry their own colour and icon, and are left out of every stop count. That colour is defined once and read by the rail, the corridor list, the map pin and the picker — and it replaced the chains' logos, which turned a corridor full of petrol stations into a row of advertisements and left the brands with no logo on file looking like something else entirely. How long a stop takes was readable in the rail and writable nowhere; new places were given an invented hour that nobody had entered. It is now set from the rail, cleared as a zero because the update statement folds a null into "leave it alone", and new places start with nothing. Both columns are rebuilt on one type scale, anchored to the day plan's own sizes so switching tabs does not change the size of the text beside it. The rail reads as trip totals, day cards and a chain of stops with the drives between them; the corridor column as what to look for, how the search is getting on, and what it found. Six category pills over three lines became one dropdown, which is the space the results wanted. Also: the offered routes are framed on the map when the picker opens (`fitKey` carries no coordinates and each renderer reads it as "the selected day"), road trip mode does not cluster at all so the whole drive stays readable zoomed out, and the drive-time labels finally render in the app's own typeface instead of Leaflet's Helvetica. All of it in both renderers. |
||
|
Maurice
|
2ab2dcc3dc |
fix(offline): the osm-de and stadia prefetch was CSP-blocked and the opening view stayed grey (#2180)
connect-src never listed tile.openstreetmap.de or tiles.stadiamaps.com, so every prefetch fetch was refused before the service worker saw it, while the swallowed error still counted as a cached tile. Both hosts are allowed now and only reached tiles count. The prefetch rectangle is also widened to the screen the fitBounds view fills, on the raster and the vector path alike; the places extent alone left grey bands beside a north-south trip. |
||
|
Maurice
|
1f66ef1d32 |
fix(maps): let users and admins enter a CARTO API key (#2060)
CARTO stamps an "API KEY REQUIRED" watermark onto every basemap tile fetched without a key since 26.08.2026, and both CartoDB presets plus the built-in fallback are CARTO. The key is free and needs no account, so add it as a setting rather than dropping the provider. carto_api_key sits next to mapbox_access_token: per user in Settings/Map, instance-wide under Admin/User Defaults, encrypted at rest, injected from CARTO_API_KEY on managed instances. Unlike Mapbox it never switches the map provider, since a raster key says nothing about the renderer. The key is appended as ?key= at render time instead of being stored in the template, so a saved URL stays portable and survives a rotation. That is also what reaches the four maps that pinned CARTO and ignored the user's tile setting: atlas, collections, the public trip share and the public journey. The two share pages read it from their token-gated payload, the same way baseCurrency already travels. Also lifts the release notice above page chrome. Its overlay sat at z-index 50, under the Vacay mode toolbar at 61, so the dialog was neither on top nor modal. Adds a six-step layer scale to index.css and moves the three notice shells onto it. Closes #2054 Closes #2052 |
||
|
Maurice
|
4eee08e45c |
test(middleware): cover the branches the forced-HTTPS change added
The redirect grew three paths nothing exercised: a malformed APP_URL falling back to the request host, a request the proxy already terminated TLS for, and the health probe, which talks plain HTTP on the loopback and must not be sent to a hostname it cannot resolve. The per-domain coverage ratchet caught the gap - src/middleware sat at 93.44% against its 94% pin. |
||
|
Maurice
|
38f3cb3101 |
fix(server): the second pass over this branch's own fixes
An adversarial review of what had already landed here turned up three that
were wrong, and they are the reason for this commit.
The CSP change removed 'unsafe-eval', which is not leftover: heic-to's
libheif build initialises embind through new Function(), and without it every
iPhone .heic upload fails in the browser. 'wasm-unsafe-eval' alone was tried
once already and was not enough (
|
||
|
Maurice
|
8e5fdaa877 |
fix(server): tighten the guard, audit and token paths
Permissions cached an empty map before it had loaded, so a DB error during the load cached permissive defaults and every later check answered from that cache - fail-open, and it never expired. The rate limiter grew its per-IP map forever; the GC the legacy version had was dropped in the fold. Idempotency did a check-then-insert, so two concurrent duplicates both ran the handler. Audit rows recorded whatever X-Forwarded-For said, which the client writes, and the request-body debug logger redacted by exact key match - it missed refresh tokens, api keys and PKCE verifiers. FORCE_HTTPS built its redirect from the client's Host header. CSP asked for unsafe-eval on top of the wasm-unsafe-eval it actually needs. The OAuth deny path redirected to an unvalidated URI, ws-token minting had no rate limit at all, and PKCE was skipped when code_verifier was absent rather than refused. An MCP session resumed with a token whose scopes had since been reduced kept the capability set it was created with, and the session cap was checked before the async registration that increments it. Plugin frame containment was lexical, so a symlink out of the directory was never noticed, and instance config accepted keys the manifest never declared. |
||
|
jubnl
|
305b113d5f |
Feat/storage driver (#1949)
* feat(storage): storage key validation, driver contract suite, LocalDriver * feat(storage): MirrorDriver composite backend * feat(server): add TREK_UPLOADS_DIR env var for the uploads root * feat(storage): settings-backed registry, StorageService facade, module wiring * test(storage): pin src/nest/storage coverage thresholds * feat(storage): multer upload options factory * refactor(files): spool trip-file uploads through storage * refactor(collab): spool note attachments through storage; normalize files/ prefix * refactor(journey): spool journey media uploads through storage * refactor(trips): spool trip cover upload through storage * refactor(places): spool place-image and collection uploads through storage * refactor(auth): spool avatar upload through storage * fix(server): consolidate allowed_file_types onto AllowedFileTypesService * fix(storage): default defParamCharset to utf8 in the upload factory * test(db): re-point the trailing-migration guard at the trip_files prefix strip * fix(storage): age-gate the boot spool sweep * test(files): key FILE-P02 residue checks on upload size * test(storage): raise src/nest/storage coverage pin * refactor(platform): serve /uploads routes through storage * refactor(files): serve trip-file downloads through storage * refactor(memories): serve photo bytes through storage * refactor(maps): stream place-photo bytes through storage * refactor(share): stream shared place-photo bytes through storage * refactor(journey): serve public journey assets through storage * fix(share): answer place-photo cache miss with 204 like the maps proxy * test(server): close serving-swap gaps surfaced by the full tier * test(storage): raise src/nest/storage branches pin * refactor(place-photos): route place-photo cache through storage * refactor(places): reclaim place images and collection covers through storage * refactor(trips): trip covers through storage * refactor(auth): avatar reclaim through storage * refactor(collab): note-file deletion through storage * refactor(files): trash deletion through storage * refactor(memories): trek photo cache internals through storage * refactor(memories): thumbnail service category-addressed * refactor(memories): immich upload reads through storage; drop uploads-root * refactor(journey): photo deletion through storage * fix(memories): reclaim orphaned trek photo cache objects in the ttl sweep * feat(journey): journey-thumbs orphan sweep and delete-site derived-artifact reclaim * test(server): close slice-4 gaps surfaced by the full tier * test(coverage): raise slice-4 domain pins * refactor(backup): list, existence and delete through storage * refactor(backup): download through storage sendToResponse * refactor(backup): create backup through per-category storage enumeration * refactor(backup): restore and upload-restore through storage * refactor(backup): auto-backup retention through storage * refactor(server): boot dir-creation to driver init * test(server): close slice-5 gaps surfaced by the coverage tier * test(coverage): raise slice-5 backup pin * test(server): resolve async backupFileExists mocks for the tests typecheck * test(server): pin ranged-read stat and zero-byte roundtrip in the storage driver contract * feat(server): TREK_S3_* env wiring — schema, s3 namespace, all-or-nothing boot rule * feat(server): S3 driver core — aws-lite client seam, stat/delete, driver-owned deadlines * refactor(server): type the aws-lite S3 client via s3-types reference * feat(server): S3 driver getStream and list — ranged full-object stat, iterator pagination * feat(server): S3 driver put — bounded-peek routing, Upload inactivity deadline * fix(server): S3 driver Upload monitoring — Transform interposition, stream cleanup * feat(server): env-declared s3-main storage backend; settings-declared s3 rejected * test(server): env-gated S3 driver contract suite against MinIO * fix(server): S3 driver zero-byte put — work around aws4 duplicate Content-Length signing defect * fix(server): scope the zero-byte put workaround to buffered bodies; bind test MinIO to localhost * ci(server): MinIO-backed S3 driver contract job * ci(server): align s3-contract shim comment and MinIO bind with references * docs: S3 storage backend configuration and off-box backup mirroring * docs: restore README formatting, keep S3 additions only * ci(server): make live-MinIO contract runs manual-only (AIStor licensing) MinIO's community line is discontinued entirely; the maintained successor, AIStor, requires a license key that can't safely live in a shared CI runner. The s3-contract CI job is removed; live-provider contract runs against MinIO/AIStor are now a manual, license-gated procedure documented in docker-compose.minio-test.yml's header. * chore(server): final-review polish — stale CI refs, dockerignore key, boundary pins, zero-byte tempfile contract case * expose admin ui * feat(shared): storage backend type registry, wire schemas, category relocation * feat(server): accept settings-declared s3 backends — shared-schema options, decrypt-at-build, registry preview() * feat(server): remove TREK_S3_*/TREK_UPLOADS_DIR — tombstone boot rule, built-in override as the relocation path * feat(server): deriveSecurity.encryptionKeySet and storage secret plumbing (gate, encrypt, unmask, mask, redact) * chore(server): drop storage-secrets import that only Task 8's probe helper will use * feat(server): seed-once storage-config.json boot import — fail closed on any invalid seed * feat(server): registry snapshot() — effective storage config with provenance * feat(server): api/admin/storage GET/PUT — masked secrets, preview-gated saves, wired reload, audits * feat(server): api/admin/storage/test — per-target ephemeral probes, mirror targets probed individually * test(server): e2e coverage for api/admin/storage — guards, managed refusal, envelope, probes * test(server): give e2e harness DBs the app_settings table the storage registry reads at boot * test(coverage): raise pins after the storage admin-config slice * fix(server): final-review polish — encryption-gate wording, all-field sentinel guard, README/e2e notes * test(server): storage slice-1 review follow-ups — mirror s3-replica probe decrypt, all-encrypted seed without key, test-route 401 * feat(i18n): storage admin domain + tab label in every locale * feat(client): adminApi storage endpoints — getStorage, updateStorage, testStorageBackend * feat(client): storage admin draft-document helpers + relative-time util * feat(client): schema-driven storage BackendForm — fields by kind, mask echo, encryption banner * feat(client): admin storage panel — backends list, category map, health strip, one-PUT save * feat(client): storage tab on the admin page * feat(client): storage admin panel and section on the phone shell * fix(client): block renaming a storage backend onto an existing name * feat(server): allow mirrors on any storage category — lift the backups-only rule * feat(i18n): replicas-on-primary mirror strings in every locale * feat(client): mirror fold/synthesize model helpers for replicas-on-primary * feat(client): BackendForm mirror-targets composer; raw mirror type hidden * feat(client): desktop storage panel folds mirrors into their primaries * feat(client): phone storage panel folds mirrors into their primaries * fix(client): strip removed backends from remaining mirror replicas — no phantom refs * feat(storage): retire photos as a configurable category — served-only via the ServedCategory seam * feat(i18n): descriptive storage category names and descriptions in every locale * feat(client): descriptive category names, descriptions and id badges in the storage panel * docs(storage): fix stale nine-category comments after the eight-category shrink * docs: storage configuration via the admin UI and seed file — README + compose * docs(wiki): Admin Storage page and storage cross-links * docs(server): storage domain blueprint entry; TREK_PLACE_PHOTO_DIR tombstone clarifier * refactor(server): drop the storage env-var tombstones — the variables never shipped * docs: storage config is simply new in v4 — drop the removed-variables story * feat(shared): backfill-status and usage wire schemas on the storage admin state * feat(server): MirrorDriver.backfill — prefix-scoped, size-compared, cancellable replica catch-up * feat(server): one-at-a-time storage backfill jobs + the storage events bridge * feat(server): storage usage scan — nightly cron, on-demand service, persisted with computedAt * feat(server): replica_failure admin notifications — events bridge, hourly debounce, channel registration * feat(i18n): replica-failure notification text in every locale * feat(server): storage backfill and stats-refresh admin endpoints * feat(client): adminApi backfill and stats-refresh endpoints * feat(i18n): storage sync and usage strings in every locale * feat(client): desktop storage panel — sync now, usage display, backfill polling * feat(client): phone storage panel — sync now, usage display, backfill polling * fix(client): per-category usage line on the phone storage panel * docs: sync now, usage stats and replica-failure notifications * test(server): admin scope now includes replica_failure, not just version_available Commit |
||
|
Maurice
|
e5c99903ea |
Pin the request body ceiling instead of inheriting it
The Express shell set a 100kb limit and stopped applying it the moment the
Nest instance took over parsing: bootstrap passed bodyParser:false and
nothing replaced it. The number happened to stay the same, because it is also
the framework default - but it lived nowhere anybody would look for it, and
one NestFactory default away from silently becoming something else.
It goes through Nest's own parser rather than a second express.json(),
because that parser carries the verify hook that keeps req.rawBody, and the
plugin webhook routes need the exact payload bytes to check a provider's
HMAC. Adding a parser in front of it would have taken rawBody away without
failing anything.
MISC-009 and MISC-010 pin it from both sides.
The bodyParser flag on applyGlobalMiddleware goes with it: one caller, which
passed false.
Not moved: cookieParser stays on the raw Express layer rather than becoming a
configure(consumer) middleware. The platform routes - the OAuth consent flow
among them - are registered on that instance before Nest routing exists, so a
Nest-slot cookie parser would simply not run for them. helmet, CORS,
compression, the forced-HTTPS 301 and the res.on('finish') logging stay for
the same reason: they have to see requests Nest never routes.
|
||
|
Maurice
|
be8081c3b4 |
Enforce the MFA policy as a guard, and stop 403ing public routes
The last live non-shell middleware verified the JWT and read the users row a second time on every /api request, then threw the loaded user away - two jwt.verify calls and two SELECTs per authenticated request, because it ran before Nest and could not see what the auth guard had already resolved. The guard reads req.user. It also corrects an asymmetry that had grown quietly. The middleware decided on two literal path lists, and every public /api endpoint added after those lists were written - /api/config, /api/help/*, the public journey and share routes, /api/health/features - answered a logged-in user without MFA with a 403 while answering the same request from a stranger perfectly well. Nothing intended that; the lists simply stopped being maintained. Keying on @Public() means the exemption cannot drift from the route again. This is a deliberate behaviour change and the only one here. @MfaExempt carries the reason for the other kind of exemption: the routes that let somebody complete setup. Without them the policy locks out every new account, which is why the middleware had its second list. One ordering trap on the way: global guards run before route guards, so the auth guard standing down on a controller with its own @UseGuards chain left req.user unset when the MFA guard ran behind it, and it waved everything through. AUTH-020 caught it. It resolves the user now without refusing - the declared chain still owns the 401. |
||
|
Maurice
|
9814480a96 |
refactor(server): move the JWT verify into nest/auth, delete four dead middlewares (#1828)
middleware/auth.ts exported seven things. Two of them — extractToken and verifyJwtAndLoadUser — are the canonical session check every auth surface goes through. The other five (authenticate, requireCookieAuth, optionalAuth, adminOnly, demoUploadBlock) are Express middlewares with no production caller left; the guards in nest/auth replaced them and only comments still mention them. The two live functions move to nest/auth/jwt-verify.ts and the file goes, along with middleware/validate.ts (nothing but its own test imported it) and middleware/idempotency.ts (its only caller was `authenticate`, and IdempotencyInterceptor has replaced it). AuthRequest and OptionalAuthRequest go with them — src/typings/express.d.ts is what types req.user now. Eight importers repoint: the three guards, auth/files/plugins-proxy services, platform.routes and the global MFA policy. Two deliberate departures from the plan for this package. JWT_SECRET does not move to app-config. It is not an env value: it is generated into data/.jwt_secret, and the admin panel rotates it at runtime through updateJwtSecret. `export let` is precisely what makes a rotation take effect in-process — a registerAs token would freeze the boot value and silently break rotation. The verify also keeps reading the db singleton rather than DatabaseService. That change is not a one-line swap: it gives the three guards a constructor dependency, which means registering them as providers everywhere they are used. Measured: 38 directories use those guards and 21 of them do not import AuthModule, on top of 34 e2e tests that each build a container around a single domain module. That is its own PR, not a rider on one that deletes dead files. tests/unit/middleware/auth.test.ts covered real behaviour, so it is not simply deleted. Its extractToken cases and the five verify cases it drove through `authenticate` come back as tests/unit/nest/auth/jwt-verify.test.ts, now hitting verifyJwtAndLoadUser directly. Four cases are new: the password_version gate — the whole reason this function exists — had no direct coverage at all. Its three adminOnly cases are dropped because AdminGuard, their successor, is covered by three e2e suites. |
||
|
Maurice
|
939452f9d8 |
refactor(server): consolidate trip access and fold the query helpers into the container (#1824)
* refactor(server): route every trip-access check through DatabaseService
Twelve Nest services carry the same verifyTripAccess/canEdit pair. Six already
asked DatabaseService.canAccessTrip; the other six went through
services/tripAccess, a nine-line re-export of the identical function. Same
query, same result, two ways to reach it.
All twelve now take the injected route, so services/tripAccess is gone.
The bigger duplication was in the types. Thirteen files declared their own
`Trip` for the same return value, in three different shapes:
NonNullable<ReturnType<typeof verifyTripAccess>> 6x
{ user_id: number } 4x
TripAccess 3x
The narrow one is why assignments, days, places and accommodations all wrote
`as Trip | null | undefined` on the call — the cast existed to shrink
TripAccess down to a local type that was never meant to differ from it. All
thirteen now say TripAccess, and the four casts are gone with no other change.
middleware/tripAccess.ts goes too: it was already dead in production — nothing
under src/ imported requireTripAccess or requireTripOwner, only comments in the
services that had absorbed them. Its test went with it.
* refactor(server): fold the query helpers and trip joins into the container
Two more legacy files whose only reason for staying outside the container was
that they predated it.
services/queryHelpers.ts held five exports doing two different jobs. Three are
the batch loaders that keep the list endpoints off N+1 queries — they hit the
database and are now QueryHelpersService over the injected connection, imported
by assignments, days, places and share. The other two, formatAssignmentWithPlace
and ratingAggregate, touch no database at all; they sat next to the loaders only
because the loaders were free functions too. Those stay free functions, in
common/rowShape.ts.
The README note on the assignments service claimed the loaders had to stay in
services/ because the day, share and place services were unmigrated. All three
have since moved, so the reason is gone.
services/tripMembership.ts becomes TripMembershipService in its own module. It
could have folded into auth or trip-invite, but both call it — as do oidc and
the plugin RPC host — and putting it in either would have made AuthModule and
TripInviteModule import each other.
Neither module is @Global, matching PermissionsModule: e2e TestingModules
resolve them transitively through each consumer's explicit import.
Test seams changed shape accordingly. Two path mocks of services/tripMembership
became injected stubs; the trip-invite e2e overrides the provider in the
container instead. The oidc test keeps the real implementation, which is what
OIDC-SVC-045 was always asserting.
ratingAggregate moved out of the ungated services/ tree into the coverage-gated
one, so it gets the direct tests it never had — as does the tags/participants
pass-through, which two long-dead fixture variables in the moved test file had
been waiting for.
Also drops four stale comments that outlived their subject: the requireTripAccess
middleware they pointed at was deleted in the previous commit.
|
||
|
Maurice
|
9ecca404c5 |
fix(map): drop the retired OpenStreetMap tile subdomains (#1733)
The a/b/c shards stopped being necessary in 2022 and d has since lost its DNS record. The map never asked for d — Leaflet defaults to abc — but the offline prefetcher rotated over a to d, which is where the failing requests came from, and it cached most tiles under a host the map never requests. Presets and the prefetcher use the apex host now. A URL already saved in user_settings is rewritten where it is read and written back, so existing instances heal without a migration touching what someone typed themselves. The CSP only allowed https://*.tile.openstreetmap.org, and a wildcard host never matches the apex domain, so the prefetch fetch would have been blocked. The service worker rule had the same gap and would have stopped filling the tile cache. |
||
|
Maurice
|
ad893eb1cc |
Release 3.1.0 (#1185)
* Phase 0 — NestJS + Zod foundation harness (F1–F8) (#1050) Co-hosted NestJS app behind the existing Express server via a strangler-fig dispatcher, sharing the same better-sqlite3 connection and JWT httpOnly cookie. Additive and dormant: default routing stays on Express, Nest only serves its own /api/_nest diagnostics until a module opts in. F1 @trek/shared Zod contract package; F2 Nest bootstrap co-hosted (fall-through, single Dockerfile/port); F3 shared better-sqlite3 provider; F4 JWT cookie auth guard (+ @CurrentUser, admin guard); F5 Zod validation pipe + error-envelope parity; F6 Nest test + coverage gates; F7 per-prefix strangler toggle (env, default Express); F8 CI build/typecheck/test/coverage. Remaining F4/F6/F8 checklist items (trip-access + permission levels + MFA policy, e2e harness/seed + 80% gate, Nest↔Express parity test, Playwright PR-comment workflow) are tracked on the first consuming module cards (L1/A1/C1). * feat(weather): migrate /api/weather to the NestJS pilot module (L1) (#1053) First strangler migration (L1): /api/weather is served by a NestJS module. - @trek/shared/weather Zod contract; Nest controller byte-identical to the legacy Express route (paths, query params, status codes, { error } bodies, lang default, ApiError/500 passthrough). Service reuses getWeather/getDetailedWeather (+ shared cache; MCP tools unchanged). - Strangler routes /api/weather to Nest by default; the legacy Express route + its migration-time parity test were decommissioned in this PR. - Frontend (FE2): weatherApi typed against the @trek/shared WeatherResult contract. - Harness: reusable Nest-vs-Express parity harness, e2e harness (temp SQLite + seed/cookie helpers, real JwtAuthGuard), src/nest coverage gate raised to >=80%, src/nest test guide. - Verified end-to-end on a prod mirror (dev1): 401/400/200 via Nest with real Open-Meteo data, Express route gone. * fix(packing): multiply item weight by quantity in bag/total weight calcs (#898) Quantity now counts toward bag and total weights. Generalised to an itemWeight() helper used by every weight sum (bag totals + max, unassigned, grand total; sidebar + bag modal) with unit tests. * feat(i18n): add Korean (ko) translation (#977) Korean translation by @ppuassi, topped up to full en.ts key parity. Language registration follows separately. * feat(i18n): add Japanese (ja) translation (#829) Japanese translation by @soma3978, at full en.ts key parity, registered in supportedLanguages + TranslationContext. * Add Turkish (tr) translation + language registry (#1029) Turkish translation by @SkyLostTR, at full en.ts key parity, registered in supportedLanguages + TranslationContext. * i18n: register Korean + add Ukrainian translation (#1055) Korean translation by @ppuassi (#977) — now registered. Ukrainian by @JeffyOLOLO (#902) — lifted onto a clean branch. Both at full en.ts key parity (2258 keys). * chore: fix monorepo build pipeline and migrate shared to built package (#1056) * chore: fix monorepo build pipeline and migrate shared to built package - Root package.json: add workspace scripts (dev, build, test, test:cov, test:e2e) that delegate to actual scripts in shared/server/client workspaces - shared: add tsup build step (CJS + ESM dual output, .d.ts); consumers now import from the built dist instead of raw TS source via path aliases - server: replace tsc-alias with tsconfig-paths (tsc-alias mangled node_modules paths); fix MCP SDK path aliases to point to root node_modules (../node_modules) - server/scripts/dev.mjs: delay node --watch until tsc -w signals first-pass done, eliminating the spurious restart on every dev startup - client/vite.config.js + vitest.config.ts: remove @trek/shared path alias (no longer needed now that shared is a proper package) - Consolidate package-lock.json at the workspace root; drop per-workspace lock files * chore: fix test script to reflect root package.json * chore: add missing lint and prettier script in root package.json * fix(ci): build shared before tests; fix vitest MCP SDK alias paths vitest.config.ts aliases pointed at ./node_modules/ (server-local) but packages are hoisted to the root node_modules/ in the npm workspace — changed to ../node_modules/. CI jobs now install and build shared before running server/client tests so that @trek/shared's dist/ exists when vitest resolves the package. * fix(docker): update Dockerfile and CI for monorepo workspace structure Dockerfile: - Add shared-builder stage that produces @trek/shared dist before client and server stages need it - Each build stage carries root package.json + package-lock.json so npm can resolve @trek/shared as a workspace dependency - Production stage installs via workspace context (npm ci --workspace=server --omit=dev) so node_modules/@trek/shared symlinks to shared/dist correctly - Copy server/tsconfig.json into the image so tsconfig-paths/register can find the MCP SDK path aliases at runtime - CMD cds into /app/server before starting node so tsconfig-paths baseUrl resolves and ../node_modules points to /app/node_modules - Remove mkdir for /app/server (now a real dir); keep symlinks for uploads/data docker.yml version-bump: - Replace manual per-workspace cd+npm-version calls with single: npm version --workspaces --include-workspace-root --no-git-tag-version (mirrors the version:* scripts in root package.json) - git add now references root package-lock.json; adds shared/package.json .dockerignore: add shared/dist package.json: fix version:prerelease preid (alpha → pre) * fix(tests): use in-memory SQLite per worker in test mode vitest pool:forks spawns parallel worker processes that all called initDb() on the same data/travel.db, causing SQLite "database is locked" and "duplicate column name" races. When NODE_ENV=test each fork now gets an isolated :memory: DB so migrations run independently with no file contention. * chore(ci): add ACT guards to skip DockerHub steps in local act runs act sets ACT=true automatically. Guards added: - docker login: if: ${{ !env.ACT }} - build outputs: type=docker (local load) when ACT, push-by-digest when CI - digest export/upload: if: ${{ !env.ACT }} - merge job: if: ${{ !env.ACT }} - release-helm job (docker.yml): if: ${{ !env.ACT }} - version-bump git push (docker.yml): wrapped in [ -z "$ACT" ] shell guard Run locally with: ./bin/act -j build -W .github/workflows/docker.yml \ -P ubuntu-latest=catthehacker/ubuntu:act-latest * fix(ci): move ACT guards to step level; add guards to security.yml env context is invalid in job-level if conditions — moved all ACT guards down to individual steps. Also guards docker login + scout in security.yml so act can run the build-only part of that workflow. * fix(ci): skip git fetch and tag logic in act (no remote access in local containers) * Revert "fix(ci): skip git fetch and tag logic in act (no remote access in local containers)" This reverts commit |
||
|
jubnl
|
20bf9c2312 |
security: close SEC-H4/H6 gaps from second-pass review
- SEC-H6: remove conditional audience check in mcp/index.ts — audience is now always enforced against the mcpResource URL. Add migration to revoke pre-existing oauth_tokens with audience=NULL so dead rows don't linger. - SEC-H4: validate doc.issuer against config.issuer inside discover() to prevent a MITM'd discovery doc from supplying a crafted expected issuer. verifyIdToken caller now passes config.issuer as ground truth, not doc.issuer. - tests: cover three new OIDC callback failure paths (no_id_token, id_token_invalid, subject_mismatch) and two idempotency caps (key length >128 chars returns 400, body >256 KiB skips caching). |
||
|
Maurice
|
9f57ab4517 |
security: address second-pass audit findings
- CI-C1 false positive: actions/{checkout,setup-node,upload-artifact}
@v6 do exist (v6.0.0 releases published Oct-Dec 2025). Restore the
@v6 refs — the earlier batch-1 commit downgraded them unnecessarily.
- Widen idempotency_keys primary key to (key, user_id, method, path)
via new migration. Batch 1 widened the middleware lookup but left
the table PK at (key, user_id), so `INSERT OR IGNORE` silently
skipped the second endpoint that reused a key — the cache was
never populated for it and a replay re-ran the handler. The
migration rebuilds the table preserving existing rows (the old
narrower PK guarantees no conflicts against the new looser key).
- HSTS: keep `includeSubDomains` OFF by default. Enabling it for
every NODE_ENV=production install would break apex-domain setups
where siblings still serve HTTP. Operators who want the stricter
policy opt in with HSTS_INCLUDE_SUBDOMAINS=true.
- Extend the idempotency unit tests to cover the (method, path)
dimension — same user+key on different path no longer replays.
|
||
|
jubnl
|
b194e8317d |
feat(pwa): implement real offline mode with IndexedDB sync
Add genuine offline read/write capability for trips: - Dexie IndexedDB schema (trips, places, packing, todo, budget, reservations, files, mutationQueue, syncMeta, blobCache) - Repo layer for all domains: offline reads from Dexie, writes optimistically to Dexie and enqueue mutations for later replay - Mutation queue with UUID idempotency keys (X-Idempotency-Key), FIFO flush, temp-ID reconciliation on 2xx, fail-and-continue on 4xx - Trip sync manager: caches all trips with end_date >= today or null, auto-evicts 7d after end_date, fetches bundle endpoint in one request - Map tile prefetcher: bbox from place coords, zooms 10-16, 50MB cap, warms SW cache via fetch - Sync triggers: network online → flush + syncAll; WS reconnect → flush only (rate-limiter safe); visibilitychange/30s → flush only - WS remoteEventHandler writes through to Dexie on every event - Server idempotency middleware + idempotency_keys table (migration 100, 24h TTL nightly cleanup) - GET /api/trips/:id/bundle endpoint for efficient single-request sync - OfflineBanner component: amber (offline) / blue (syncing) / hidden - OfflineTab in Settings: cached trip list, re-sync and clear actions - usePendingMutations hook for per-item pending indicators Closes #505 #541 |
||
|
jubnl
|
b4922322ae |
test: expand test suite to 87.3% backend coverage
Add new integration test files covering previously untested routes: - categories.test.ts — GET /api/categories - oidc.test.ts — full OIDC login flow (callback, state, errors) - settings.test.ts — GET/PUT /api/settings, bulk save - tags.test.ts — CRUD for trip tags - todo.test.ts — todo items CRUD and reorder Add new unit test files covering service-layer logic: - adminService.test.ts — user/invite management, packing templates, OIDC settings - atlasService.test.ts — atlas search and place enrichment - authServiceDb.test.ts — DB-backed auth helpers (login, register, MFA) - backupService.test.ts — export/import/restore logic - categoryService.test.ts — category CRUD - dayService.test.ts — day management and accommodation helpers - mapsService.test.ts — route/directions helpers - oidcService.test.ts — OIDC state, auth code, role resolution, user upsert - packingService.test.ts — packing item/bag/template operations - placeService.test.ts — place CRUD and tag attachment - settingsService.test.ts — settings get/set/bulk - tagService.test.ts — tag CRUD - todoService.test.ts — todo CRUD and reorder - tripService.test.ts — trip CRUD, member management, archiving - vacayService.test.ts — vacay integration helpers - tripAccess.test.ts (middleware) — requireTripAccess middleware Expand existing integration and unit test files with additional cases across admin, atlas, auth, backup, collab, days, files, maps, memories (Immich/Synology), notifications, places, reservations, share, vacay, weather, auth middleware, ephemeral tokens, notification preferences, permissions, SSRF guard, and WebSocket connection tests. Update test helpers (factories.ts, test-db.ts) with new factory functions and seed data required by the expanded suite. Fix minor issues in server/src/routes/reservations.ts and server/src/services/atlasService.ts surfaced by new test coverage. Update sonar-project.properties to reflect new coverage thresholds. |
||
|
Julien G.
|
905c7d460b |
Add comprehensive backend test suite (#339)
* add test suite, mostly covers integration testing, tests are only backend side * workflow runs the correct script * workflow runs the correct script * workflow runs the correct script * unit tests incoming * Fix multer silent rejections and error handler info leak - Revert cb(null, false) to cb(new Error(...)) in auth.ts, collab.ts, and files.ts so invalid uploads return an error instead of silently dropping the file - Error handler in app.ts now always returns 500 / "Internal server error" instead of forwarding err.message to the client * Use statusCode consistently for multer errors and error handler - Error handler in app.ts reads err.statusCode to forward the correct HTTP status while keeping the response body generic |