mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-10-21 22:46:32 +00:00
aa5002b2ed4b48834b61ba808636ba67692db1cc
12 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Maurice
|
521e52c853 |
fix(collab): harden the link preview fetcher (#2058)
The preview endpoint takes a URL from whoever is typing in a trip chat and fetches it from the server. Trip access was added to it earlier; this is the rest of what that route needed. The SSRF guard had a hole under it. `isAlwaysBlocked` knew `::1` and `0.0.0.0` but not the IPv6 unspecified address: `::` starts with a colon, so the `0.` check never saw it, and connecting to it lands on loopback. `http://[::]:9200/` therefore passed the guard and came back with the target's title and og: tags scraped out of the body — an exfiltrating read of anything listening locally, from any account. `[::0]`, `[0:0:0:0:0:0:0:0]` and `[::ffff:0:0]` are the same address in other spellings and behaved the same way. The two `::ffff:`-prefix regexes had the same shape of gap: they matched the dotted spelling of loopback and link-local, so `::ffff:7f00:1` walked past them. Both checks now decide on the expanded hextets, which covers every spelling at once, and fe80::/10 is read as ten bits rather than as the four characters 'fe80'. This is in ssrfGuard, so every caller gets it, not just collab. The scrape could be turned into a denial of service. Each og: regex had two unbounded `[^>]*` runs separated by a literal, so a document with no `>` in it made the engine rescan the remainder from every `<meta` it passed — quadratic, and the byte cap admits half a megabyte. Measured on this build: 240KB of '<meta ' took 58 seconds, and Node has one thread, so that is the entire server, health check included, on one request. Bounding the runs at 512 characters makes it linear (540KB now takes 1.4s); no real attribute list comes close to that. The five-second timeout only covered the headers — `clearTimeout` ran before the body was read — so a server that answered fast and then dripped one byte every few seconds held the handler, the socket and the per-request undici agent open indefinitely. The byte cap counts bytes and at that rate would never reach one. `AbortSignal.timeout` covers both halves, the agent is closed in a `finally`, and the three bail-outs that used to return without touching the body now discard it, which is what cappedFetch's helper is there for. The rest is smaller. Nothing metered the fetches, so an account with a trip of its own could drive arbitrary outbound requests in a loop; there is a budget of sixty new fetches a minute per user now, charged per outbound fetch rather than per request. Previews are cached for ten minutes and simultaneous askers for one URL share a single fetch, so the client's habit of requesting a preview per rendered message — up to a hundred at once, with no debounce — costs one fetch per new link rather than one per render. Refusals answer with one constant reason instead of the guard's own: it distinguishes "could not resolve" from "private address", which together map out the server's internal DNS for anyone willing to guess hostnames. A body whose declared type is not markup is dropped rather than combed for og: tags, and an og:image that is not an http(s) URL is not passed to the client, which puts it straight into an <img src>. Deliberately no `@RequirePermission` on the route: the client asks for a preview while rendering a message or a note, never while composing one, so it belongs with the GET siblings. Requiring collab_edit would blank every preview for members of a trip whose owner narrowed that right while leaving them able to read the chat, and it would not stop the abuse it looks like it addresses — the permission check passes for a trip's owner at every level, and anyone can create a trip. |
||
|
Maurice
|
38f3cb3101 |
fix(server): the second pass over this branch's own fixes
An adversarial review of what had already landed here turned up three that
were wrong, and they are the reason for this commit.
The CSP change removed 'unsafe-eval', which is not leftover: heic-to's
libheif build initialises embind through new Function(), and without it every
iPhone .heic upload fails in the browser. 'wasm-unsafe-eval' alone was tried
once already and was not enough (
|
||
|
Maurice
|
515b832fa3 |
fix(collab): stop serving the text of deleted chat messages
A deleted message keeps its row so the client can draw the placeholder off the flag, but listMessages still returned the original text - in the list itself and in the reply preview joined onto whatever quoted it. Both REST and the MCP resource read through that one method, so blanking it there covers every surface. Also cap the link-preview read, and delete the spooled upload when addNoteFile refuses the request instead of leaving it on disk. |
||
|
jubnl
|
305b113d5f |
Feat/storage driver (#1949)
* feat(storage): storage key validation, driver contract suite, LocalDriver * feat(storage): MirrorDriver composite backend * feat(server): add TREK_UPLOADS_DIR env var for the uploads root * feat(storage): settings-backed registry, StorageService facade, module wiring * test(storage): pin src/nest/storage coverage thresholds * feat(storage): multer upload options factory * refactor(files): spool trip-file uploads through storage * refactor(collab): spool note attachments through storage; normalize files/ prefix * refactor(journey): spool journey media uploads through storage * refactor(trips): spool trip cover upload through storage * refactor(places): spool place-image and collection uploads through storage * refactor(auth): spool avatar upload through storage * fix(server): consolidate allowed_file_types onto AllowedFileTypesService * fix(storage): default defParamCharset to utf8 in the upload factory * test(db): re-point the trailing-migration guard at the trip_files prefix strip * fix(storage): age-gate the boot spool sweep * test(files): key FILE-P02 residue checks on upload size * test(storage): raise src/nest/storage coverage pin * refactor(platform): serve /uploads routes through storage * refactor(files): serve trip-file downloads through storage * refactor(memories): serve photo bytes through storage * refactor(maps): stream place-photo bytes through storage * refactor(share): stream shared place-photo bytes through storage * refactor(journey): serve public journey assets through storage * fix(share): answer place-photo cache miss with 204 like the maps proxy * test(server): close serving-swap gaps surfaced by the full tier * test(storage): raise src/nest/storage branches pin * refactor(place-photos): route place-photo cache through storage * refactor(places): reclaim place images and collection covers through storage * refactor(trips): trip covers through storage * refactor(auth): avatar reclaim through storage * refactor(collab): note-file deletion through storage * refactor(files): trash deletion through storage * refactor(memories): trek photo cache internals through storage * refactor(memories): thumbnail service category-addressed * refactor(memories): immich upload reads through storage; drop uploads-root * refactor(journey): photo deletion through storage * fix(memories): reclaim orphaned trek photo cache objects in the ttl sweep * feat(journey): journey-thumbs orphan sweep and delete-site derived-artifact reclaim * test(server): close slice-4 gaps surfaced by the full tier * test(coverage): raise slice-4 domain pins * refactor(backup): list, existence and delete through storage * refactor(backup): download through storage sendToResponse * refactor(backup): create backup through per-category storage enumeration * refactor(backup): restore and upload-restore through storage * refactor(backup): auto-backup retention through storage * refactor(server): boot dir-creation to driver init * test(server): close slice-5 gaps surfaced by the coverage tier * test(coverage): raise slice-5 backup pin * test(server): resolve async backupFileExists mocks for the tests typecheck * test(server): pin ranged-read stat and zero-byte roundtrip in the storage driver contract * feat(server): TREK_S3_* env wiring — schema, s3 namespace, all-or-nothing boot rule * feat(server): S3 driver core — aws-lite client seam, stat/delete, driver-owned deadlines * refactor(server): type the aws-lite S3 client via s3-types reference * feat(server): S3 driver getStream and list — ranged full-object stat, iterator pagination * feat(server): S3 driver put — bounded-peek routing, Upload inactivity deadline * fix(server): S3 driver Upload monitoring — Transform interposition, stream cleanup * feat(server): env-declared s3-main storage backend; settings-declared s3 rejected * test(server): env-gated S3 driver contract suite against MinIO * fix(server): S3 driver zero-byte put — work around aws4 duplicate Content-Length signing defect * fix(server): scope the zero-byte put workaround to buffered bodies; bind test MinIO to localhost * ci(server): MinIO-backed S3 driver contract job * ci(server): align s3-contract shim comment and MinIO bind with references * docs: S3 storage backend configuration and off-box backup mirroring * docs: restore README formatting, keep S3 additions only * ci(server): make live-MinIO contract runs manual-only (AIStor licensing) MinIO's community line is discontinued entirely; the maintained successor, AIStor, requires a license key that can't safely live in a shared CI runner. The s3-contract CI job is removed; live-provider contract runs against MinIO/AIStor are now a manual, license-gated procedure documented in docker-compose.minio-test.yml's header. * chore(server): final-review polish — stale CI refs, dockerignore key, boundary pins, zero-byte tempfile contract case * expose admin ui * feat(shared): storage backend type registry, wire schemas, category relocation * feat(server): accept settings-declared s3 backends — shared-schema options, decrypt-at-build, registry preview() * feat(server): remove TREK_S3_*/TREK_UPLOADS_DIR — tombstone boot rule, built-in override as the relocation path * feat(server): deriveSecurity.encryptionKeySet and storage secret plumbing (gate, encrypt, unmask, mask, redact) * chore(server): drop storage-secrets import that only Task 8's probe helper will use * feat(server): seed-once storage-config.json boot import — fail closed on any invalid seed * feat(server): registry snapshot() — effective storage config with provenance * feat(server): api/admin/storage GET/PUT — masked secrets, preview-gated saves, wired reload, audits * feat(server): api/admin/storage/test — per-target ephemeral probes, mirror targets probed individually * test(server): e2e coverage for api/admin/storage — guards, managed refusal, envelope, probes * test(server): give e2e harness DBs the app_settings table the storage registry reads at boot * test(coverage): raise pins after the storage admin-config slice * fix(server): final-review polish — encryption-gate wording, all-field sentinel guard, README/e2e notes * test(server): storage slice-1 review follow-ups — mirror s3-replica probe decrypt, all-encrypted seed without key, test-route 401 * feat(i18n): storage admin domain + tab label in every locale * feat(client): adminApi storage endpoints — getStorage, updateStorage, testStorageBackend * feat(client): storage admin draft-document helpers + relative-time util * feat(client): schema-driven storage BackendForm — fields by kind, mask echo, encryption banner * feat(client): admin storage panel — backends list, category map, health strip, one-PUT save * feat(client): storage tab on the admin page * feat(client): storage admin panel and section on the phone shell * fix(client): block renaming a storage backend onto an existing name * feat(server): allow mirrors on any storage category — lift the backups-only rule * feat(i18n): replicas-on-primary mirror strings in every locale * feat(client): mirror fold/synthesize model helpers for replicas-on-primary * feat(client): BackendForm mirror-targets composer; raw mirror type hidden * feat(client): desktop storage panel folds mirrors into their primaries * feat(client): phone storage panel folds mirrors into their primaries * fix(client): strip removed backends from remaining mirror replicas — no phantom refs * feat(storage): retire photos as a configurable category — served-only via the ServedCategory seam * feat(i18n): descriptive storage category names and descriptions in every locale * feat(client): descriptive category names, descriptions and id badges in the storage panel * docs(storage): fix stale nine-category comments after the eight-category shrink * docs: storage configuration via the admin UI and seed file — README + compose * docs(wiki): Admin Storage page and storage cross-links * docs(server): storage domain blueprint entry; TREK_PLACE_PHOTO_DIR tombstone clarifier * refactor(server): drop the storage env-var tombstones — the variables never shipped * docs: storage config is simply new in v4 — drop the removed-variables story * feat(shared): backfill-status and usage wire schemas on the storage admin state * feat(server): MirrorDriver.backfill — prefix-scoped, size-compared, cancellable replica catch-up * feat(server): one-at-a-time storage backfill jobs + the storage events bridge * feat(server): storage usage scan — nightly cron, on-demand service, persisted with computedAt * feat(server): replica_failure admin notifications — events bridge, hourly debounce, channel registration * feat(i18n): replica-failure notification text in every locale * feat(server): storage backfill and stats-refresh admin endpoints * feat(client): adminApi backfill and stats-refresh endpoints * feat(i18n): storage sync and usage strings in every locale * feat(client): desktop storage panel — sync now, usage display, backfill polling * feat(client): phone storage panel — sync now, usage display, backfill polling * fix(client): per-category usage line on the phone storage panel * docs: sync now, usage stats and replica-failure notifications * test(server): admin scope now includes replica_failure, not just version_available Commit |
||
|
Maurice
|
7078ac7a4d |
Keep the mobile FX widget off the network after unmount
Client CI was red on this branch while dev was green, and no client file had changed. The 536 test files all passed; the job failed on three unhandled rejections, all `window is not defined` from MDashWidgets, attributed to whichever case happened to be running when they landed. The widget fetches api.frankfurter.dev directly and sets state from both handlers with no unmount guard. MSW runs with onUnhandledRequest: 'warn', so that request was never intercepted: it left the runner, came back after the environment was gone, and React tried to schedule an update against a torn down tree. Load dependent by construction, which is why dev happened to pass. Two halves, because either alone leaves the other broken. The widget now aborts in flight work on unmount and on refetch, and treats an abort as "no one is waiting" rather than as a failure to report. That matters outside the runner too: this is a third party call on a mobile screen a user can leave immediately. The handler set gains the third party endpoints the app calls directly rather than through /api, so no suite reaches the real host. An empty rate list is the shape the widget already handles. Also clears twenty more test files, each at its root cause. Two more were passing for the wrong reason: oidc.service.test.ts built AuthService with an AtlasService in the membership slot, undefined for userCleanup and mailer, and the mailer stub in the tokens slot, the same shift admin.service.test.ts had. And CAL-010 claims to serialise a flight byte for byte, but createReservation ignores reservation_time, so the row is written with a null time and CalendarService drops it: the recorded snapshot is truthful about the data the case builds and silent about the flight it advertises. |
||
|
Maurice
|
6cf5760383 |
Inject the notification senders instead of importing them lazily
Nine fire-and-forget sends reached notifications.bridge through
import('...').then(({ send }) => ...). The laziness was working around a
module cycle that no longer exists, and it cost twice: it hid the edge from
the module graph, and it handed each send a NotificationsService built
outside the container rather than the container's own.
The sweep needed a method rather than a grep. tests/ sits outside
tsconfig's include, so a service that gains a constructor parameter does not
break its hand-wired call sites at compile time — the argument arrives as
undefined and throws inside the send. Adding tests to a throwaway tsconfig
and diffing the errors before and after named all 46 sites exactly. The same
config reports 288 pre-existing errors in the suites, almost all deliberate
partial constructions in the harnesses, which is why tests/ is not in the
build's include; that cleanup is its own piece of work and is written down
in the nest README.
notifications.instance.ts holds the out-of-container instance, apart from
notifications.bridge.ts on purpose: several suites replace the bridge
wholesale with vi.mock(..., () => ({ send })), and a sibling bridge taking
its instance from there would break under those mocks.
The suites that assert on a send now hand their spy to the injected service.
The reservations e2e overrides the provider, because its app resolves the
real one out of its own container and never saw the module mock.
|
||
|
Maurice
|
6dda05f672 |
refactor(server): move the stateless helpers into nest/common (#1823)
Eight files in services/ hold no state and touch no database: avatarUrl, conflictResult, demo, passwordPolicy, timezoneService, cookie, and the two crypto helpers. They are imported from 65 places, 62 of them already inside src/nest, so having them live in the legacy layer was pure inertia. They stay plain functions rather than becoming providers. Three importers sit outside the container and would have no way to reach an injectable one: db/migrations.ts runs before the app is built, and middleware/auth.ts and middleware/mfaPolicy.ts run beside it. Wrapping these in DI would buy ceremony and nothing else. mfaCrypto and apiKeyCrypto move together into common/crypto — they read the same ENCRYPTION_KEY, and the directory says so. services/ drops from 44 files to 36, 9,870 lines to 8,493. One trap worth recording: src/app-config and src/nest/app-config both exist. cookie.ts imported '../app-config' from services/, which resolved to the former; after the move the same string silently pointed at the latter, which has no readEnv. It is now explicit about which one it wants. Nothing else changes — no signatures, no behaviour, no tests rewritten beyond their import paths. |
||
|
jubnl
|
287dceb532 |
chore: wire TripsService into the plugin RPC host and retire the legacy tripService
Swap the factory's six tripService imports for the injected TripsService; delete services/tripService.ts and the collab/vacay bridges with it; prune the days/budget bridges to their surviving exports; update the migration docs and regenerate migration-graph.md. |
||
|
jubnl
|
a2861d40f3 |
feat(server): injectable RealtimeService over the ws singleton
Nest services inject RealtimeService (call-time delegation to the websocket module, DatabaseService pattern) instead of importing broadcast/broadcastToUser globals. Zero behavior change; test edits are constructor-arg additions plus RealtimeModule in e2e harness imports only. |
||
|
jubnl
|
1d56de3897 | chore: move permissions backend to proper DI | ||
|
jubnl
|
07b8f1234c | fix(server): fix the quirks preserved by the collab DI migration | ||
|
jubnl
|
c08c595b7e | chore: move collab backend to proper DI |