mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-10-22 06:56:35 +00:00
8f6ed02cffa34becfb692feeac62df69d744f146
60 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 jubnl
|
76c8c349b1 |
Fold nest mcp (#2017)
* refactor(server): fold @trek/nest-mcp into server/src/nest-mcp * docs(server): point nest-mcp references at server/src/nest-mcp * remove tracked md file |
||
|
jubnl
|
305b113d5f |
Feat/storage driver (#1949)
* feat(storage): storage key validation, driver contract suite, LocalDriver * feat(storage): MirrorDriver composite backend * feat(server): add TREK_UPLOADS_DIR env var for the uploads root * feat(storage): settings-backed registry, StorageService facade, module wiring * test(storage): pin src/nest/storage coverage thresholds * feat(storage): multer upload options factory * refactor(files): spool trip-file uploads through storage * refactor(collab): spool note attachments through storage; normalize files/ prefix * refactor(journey): spool journey media uploads through storage * refactor(trips): spool trip cover upload through storage * refactor(places): spool place-image and collection uploads through storage * refactor(auth): spool avatar upload through storage * fix(server): consolidate allowed_file_types onto AllowedFileTypesService * fix(storage): default defParamCharset to utf8 in the upload factory * test(db): re-point the trailing-migration guard at the trip_files prefix strip * fix(storage): age-gate the boot spool sweep * test(files): key FILE-P02 residue checks on upload size * test(storage): raise src/nest/storage coverage pin * refactor(platform): serve /uploads routes through storage * refactor(files): serve trip-file downloads through storage * refactor(memories): serve photo bytes through storage * refactor(maps): stream place-photo bytes through storage * refactor(share): stream shared place-photo bytes through storage * refactor(journey): serve public journey assets through storage * fix(share): answer place-photo cache miss with 204 like the maps proxy * test(server): close serving-swap gaps surfaced by the full tier * test(storage): raise src/nest/storage branches pin * refactor(place-photos): route place-photo cache through storage * refactor(places): reclaim place images and collection covers through storage * refactor(trips): trip covers through storage * refactor(auth): avatar reclaim through storage * refactor(collab): note-file deletion through storage * refactor(files): trash deletion through storage * refactor(memories): trek photo cache internals through storage * refactor(memories): thumbnail service category-addressed * refactor(memories): immich upload reads through storage; drop uploads-root * refactor(journey): photo deletion through storage * fix(memories): reclaim orphaned trek photo cache objects in the ttl sweep * feat(journey): journey-thumbs orphan sweep and delete-site derived-artifact reclaim * test(server): close slice-4 gaps surfaced by the full tier * test(coverage): raise slice-4 domain pins * refactor(backup): list, existence and delete through storage * refactor(backup): download through storage sendToResponse * refactor(backup): create backup through per-category storage enumeration * refactor(backup): restore and upload-restore through storage * refactor(backup): auto-backup retention through storage * refactor(server): boot dir-creation to driver init * test(server): close slice-5 gaps surfaced by the coverage tier * test(coverage): raise slice-5 backup pin * test(server): resolve async backupFileExists mocks for the tests typecheck * test(server): pin ranged-read stat and zero-byte roundtrip in the storage driver contract * feat(server): TREK_S3_* env wiring — schema, s3 namespace, all-or-nothing boot rule * feat(server): S3 driver core — aws-lite client seam, stat/delete, driver-owned deadlines * refactor(server): type the aws-lite S3 client via s3-types reference * feat(server): S3 driver getStream and list — ranged full-object stat, iterator pagination * feat(server): S3 driver put — bounded-peek routing, Upload inactivity deadline * fix(server): S3 driver Upload monitoring — Transform interposition, stream cleanup * feat(server): env-declared s3-main storage backend; settings-declared s3 rejected * test(server): env-gated S3 driver contract suite against MinIO * fix(server): S3 driver zero-byte put — work around aws4 duplicate Content-Length signing defect * fix(server): scope the zero-byte put workaround to buffered bodies; bind test MinIO to localhost * ci(server): MinIO-backed S3 driver contract job * ci(server): align s3-contract shim comment and MinIO bind with references * docs: S3 storage backend configuration and off-box backup mirroring * docs: restore README formatting, keep S3 additions only * ci(server): make live-MinIO contract runs manual-only (AIStor licensing) MinIO's community line is discontinued entirely; the maintained successor, AIStor, requires a license key that can't safely live in a shared CI runner. The s3-contract CI job is removed; live-provider contract runs against MinIO/AIStor are now a manual, license-gated procedure documented in docker-compose.minio-test.yml's header. * chore(server): final-review polish — stale CI refs, dockerignore key, boundary pins, zero-byte tempfile contract case * expose admin ui * feat(shared): storage backend type registry, wire schemas, category relocation * feat(server): accept settings-declared s3 backends — shared-schema options, decrypt-at-build, registry preview() * feat(server): remove TREK_S3_*/TREK_UPLOADS_DIR — tombstone boot rule, built-in override as the relocation path * feat(server): deriveSecurity.encryptionKeySet and storage secret plumbing (gate, encrypt, unmask, mask, redact) * chore(server): drop storage-secrets import that only Task 8's probe helper will use * feat(server): seed-once storage-config.json boot import — fail closed on any invalid seed * feat(server): registry snapshot() — effective storage config with provenance * feat(server): api/admin/storage GET/PUT — masked secrets, preview-gated saves, wired reload, audits * feat(server): api/admin/storage/test — per-target ephemeral probes, mirror targets probed individually * test(server): e2e coverage for api/admin/storage — guards, managed refusal, envelope, probes * test(server): give e2e harness DBs the app_settings table the storage registry reads at boot * test(coverage): raise pins after the storage admin-config slice * fix(server): final-review polish — encryption-gate wording, all-field sentinel guard, README/e2e notes * test(server): storage slice-1 review follow-ups — mirror s3-replica probe decrypt, all-encrypted seed without key, test-route 401 * feat(i18n): storage admin domain + tab label in every locale * feat(client): adminApi storage endpoints — getStorage, updateStorage, testStorageBackend * feat(client): storage admin draft-document helpers + relative-time util * feat(client): schema-driven storage BackendForm — fields by kind, mask echo, encryption banner * feat(client): admin storage panel — backends list, category map, health strip, one-PUT save * feat(client): storage tab on the admin page * feat(client): storage admin panel and section on the phone shell * fix(client): block renaming a storage backend onto an existing name * feat(server): allow mirrors on any storage category — lift the backups-only rule * feat(i18n): replicas-on-primary mirror strings in every locale * feat(client): mirror fold/synthesize model helpers for replicas-on-primary * feat(client): BackendForm mirror-targets composer; raw mirror type hidden * feat(client): desktop storage panel folds mirrors into their primaries * feat(client): phone storage panel folds mirrors into their primaries * fix(client): strip removed backends from remaining mirror replicas — no phantom refs * feat(storage): retire photos as a configurable category — served-only via the ServedCategory seam * feat(i18n): descriptive storage category names and descriptions in every locale * feat(client): descriptive category names, descriptions and id badges in the storage panel * docs(storage): fix stale nine-category comments after the eight-category shrink * docs: storage configuration via the admin UI and seed file — README + compose * docs(wiki): Admin Storage page and storage cross-links * docs(server): storage domain blueprint entry; TREK_PLACE_PHOTO_DIR tombstone clarifier * refactor(server): drop the storage env-var tombstones — the variables never shipped * docs: storage config is simply new in v4 — drop the removed-variables story * feat(shared): backfill-status and usage wire schemas on the storage admin state * feat(server): MirrorDriver.backfill — prefix-scoped, size-compared, cancellable replica catch-up * feat(server): one-at-a-time storage backfill jobs + the storage events bridge * feat(server): storage usage scan — nightly cron, on-demand service, persisted with computedAt * feat(server): replica_failure admin notifications — events bridge, hourly debounce, channel registration * feat(i18n): replica-failure notification text in every locale * feat(server): storage backfill and stats-refresh admin endpoints * feat(client): adminApi backfill and stats-refresh endpoints * feat(i18n): storage sync and usage strings in every locale * feat(client): desktop storage panel — sync now, usage display, backfill polling * feat(client): phone storage panel — sync now, usage display, backfill polling * fix(client): per-category usage line on the phone storage panel * docs: sync now, usage stats and replica-failure notifications * test(server): admin scope now includes replica_failure, not just version_available Commit |
||
|
jubnl
|
04812702ec | docs: close the bridge chapter | ||
|
jubnl
|
f842073351 | refactor(server): delete airtrail.bridge and notifications.instance via a core/pull split | ||
|
jubnl
|
e1162c1c07 |
docs: bring the bridge docs in line
Roadmap census 14 -> 8 (5 seam-pinned, 3 verified-permanent), resources.ts DoD line closed, README in-container list and counts corrected. Includes the SystemNoticesService unit test the src/nest coverage ratchet demanded after the addonEnabled threading. |
||
|
jubnl
|
1a945baebe | refactor(server): shrink assignments.bridge to its two live exports | ||
|
Maurice
|
8e4261ed81 |
Delete four bridges with no consumer, and gate what runs outside the container
Both were left open by the audit and are closed here. days, packing, photos and reservations each built a service outside the container at module load for nobody: only their own delegation tests imported them, and three domain docblocks were still telling readers to go through them. photos.bridge's own comment pointed at src/services/, a directory deleted two PRs ago. The cases that exercised them kept their assertions and call the service the bridge delegated to, so nothing that was proved stopped being proved. Twelve bridges remain, each with a live out-of-container caller. Coverage measured src/** but gated only src/nest/**, so the boot path, the scheduler, the MCP handler and the migrations sat under no floor at all. That is the same 9,602 lines where this audit found its worst defect, and a swallowed EADDRINUSE is exactly the kind of thing an untested boot path hides. Seven globs now pin what is measured today. They are floors to raise, not targets that have been met: src/*.ts is at 39% statements and src/db at 38% branches, and writing those numbers down is the point. src/demo/** is deliberately left out. It measures 0%, and a floor of zero asserts nothing; it needs tests before it needs a threshold. |
||
|
Maurice
|
3a7c8e1d26 |
Make a bind failure fatal again, and undo four contract tightenings
An adversarial audit over the three merged migration PRs. Sixteen findings
survived triple refutation; these are the ones with a defect behind them.
The serious one is mine, from the gateway move. ws registers
`error: this.emit.bind(this, 'error')` on the http server it attaches to, and
the adapter now runs inside buildApp, before listen(). So from that moment the
http server HAS an error listener and Node stops throwing on one. The adapter's
handler was `() => {}`, which meant EADDRINUSE was forwarded there and dropped:
the process survived, the listen callback never ran, so the banner never
printed and the scheduler never started. A container that looks healthy and
serves nothing. Reproduced, then fixed at both ends: index.ts owns the fatal
decision on the http server, and the adapter logs instead of swallowing.
Four DTOs were tightening contracts rather than describing them, which is the
thing the pipe-before-handler ordering makes easy to do by accident:
- OauthClientCreateDto replaced 'Name is required' with a Zod field report, and
pre-empted the handler's 403 on an instance with the MCP addon off.
- BookingImportConfirmDto demanded per-item fields the service never reads, so
a caller assembling items itself got a 400 about `source`.
- TagCreateDto/TagUpdateDto typed name and color as strings; the legacy route
bound whatever arrived, so a numeric name used to create a tag.
- PluginUserSettingsUpdateDto typed config as a record, where the handler
answers 200 for anything it cannot use.
Two of my own tests were false-green, which the audit demonstrated by mutation
rather than by reading. WSGW-008 asserted on frames that never carry the user
object at all, so it passed with the password_version strip deleted; it reads
the retained identity now. GEO-009 made the same assertion before and after the
write; it distinguishes a cached negative from a miss now, through a hasCached
seam that says what getCached cannot. Both were re-run against the mutation
they exist to catch, and both fail on it.
booking-import geocodes every venue and endpoint in one request loop, up to
thirty sequential calls. Folding maps onto the shared Nominatim cursor put that
loop on the interactive lane, so an import starved every concurrent keystroke
for half a minute. It takes the background lane now, which is what the two-lane
split was built for.
Also: the README's in-container bridge list had outlived one entry and was
missing another, and the journey e2e still mocked a module this branch deleted.
|
||
|
jubnl
|
b82651bf93 | Merge remote-tracking branch 'origin/dev' into dev | ||
|
jubnl
|
3a48dd7c1c | docs: bring the scheduler docs in line | ||
|
Maurice
|
d981719941 |
Serve /ws from a Nest gateway with its own adapter
The transport was a hand-rolled setupWebSocket(server) that index.ts kicked
off with a dynamic import after listen(). It is a @WebSocketGateway now, with
DatabaseService and EphemeralTokenService injected instead of imported, and
the heartbeat owned by a lifecycle hook rather than an interval nobody stops.
The wire protocol does not move. Nest's WsAdapter dispatches on
{ event, data } and every deployed client sends { type, tripId }, so
TrekWsAdapter translates rather than asking clients to change. It also builds
the ws server itself instead of delegating: the base's port-0 branch uses
noServer and performs the upgrade from its own registry, which never consults
verifyClient, so delegating would have dropped the origin check and answered
101 to anyone. The per-socket error listener stays attached first, because an
unhandled one kills the process (#1576), and the flood guard counts every
inbound frame before parsing, because malformed frames are exactly the ones
worth limiting.
Four things had to stay where they were, and each is a silent failure if it
does not:
The socket registry stays module state. Seventeen bridges and
notifications.instance build `new RealtimeService()` outside the container;
rooms on a provider instance would leave their broadcasts going to an empty
map, with no error and no log.
The socket id stays a monotonic integer. The client echoes it as X-Socket-Id
and broadcast excludes the originator with Number(excludeSid). A uuid is NaN,
NaN === NaN is false, and every client would receive its own writes back,
which shows up as drag-and-drop that jumps rather than as an error.
buildApp creates the http server and binds the adapter before app.init(), and
callers take it from getHttpServer(). Nest binds gateways during init, so a
later adapter binds nothing; an adapter given the app binds to Nest's own
internal server, which this process never listens on. Both boot green with no
client able to connect.
RealtimeGatewayModule is imported by AppModule alone. RealtimeModule is
@Global, so a gateway there follows into every hand-assembled TestingModule,
and Nest's SocketModule then loads an adapter none of them set: it reaches for
@nestjs/platform-socket.io and calls process.exit(1). Eighteen e2e harnesses
died that way before the split.
src/websocket.ts survives as a re-export of the three fan-out primitives. 115
test files vi.mock that path to assert on broadcasts.
The two transport suites pass unchanged, and 30 new cases pin the pieces
directly: the pv and MFA gates, the numeric socket id, the plugin announcement
firing before the room check, both sides of the origin allowlist, and the
flood guard counting frames no handler wants.
|
||
|
Maurice
|
6cf5760383 |
Inject the notification senders instead of importing them lazily
Nine fire-and-forget sends reached notifications.bridge through
import('...').then(({ send }) => ...). The laziness was working around a
module cycle that no longer exists, and it cost twice: it hid the edge from
the module graph, and it handed each send a NotificationsService built
outside the container rather than the container's own.
The sweep needed a method rather than a grep. tests/ sits outside
tsconfig's include, so a service that gains a constructor parameter does not
break its hand-wired call sites at compile time — the argument arrives as
undefined and throws inside the send. Adding tests to a throwaway tsconfig
and diffing the errors before and after named all 46 sites exactly. The same
config reports 288 pre-existing errors in the suites, almost all deliberate
partial constructions in the harnesses, which is why tests/ is not in the
build's include; that cleanup is its own piece of work and is written down
in the nest README.
notifications.instance.ts holds the out-of-container instance, apart from
notifications.bridge.ts on purpose: several suites replace the bridge
wholesale with vi.mock(..., () => ({ send })), and a sibling bridge taking
its instance from there would break under those mocks.
The suites that assert on a send now hand their spy to the injected service.
The reservations e2e overrides the provider, because its app resolves the
real one out of its own container and never saw the module mock.
|
||
|
Maurice
|
fc9742357b |
One photo provider interface instead of three switches
Dispatch to a photo backend was `switch (photo.provider)` in photo-resolver.service.ts twice and, in journey-public.controller.ts, an if/else that handed anything not named immich to Synology — where it 404'd from inside that provider's id parser. Adding a third backend meant finding all three. PhotoProviderRegistry.get(id) replaces them, over adapters registered under the PHOTO_PROVIDERS token in memories.module.ts. Adapters rather than `implements` on the services themselves: ImmichService and SynologyService are also the album and browse surfaces, and folding the interface into them would drag their argument orders and return shapes along. PhotoAssetRef is the part worth having. The two services took the same three values in different orders — Immich as (userId, assetId, ownerId), Synology as (userId, targetUserId, photoId) — so every dispatch site had to remember which was which, and a transposition would have compiled and served another user's photo. One place builds the ref, one place decrypts the Synology share passphrase. `local` stays an explicit branch at both dispatch sites: the bytes are on this disk, there is no row for it in photo_providers, and its asset info comes from the stored row rather than from a backend. The rename memories/ -> media/ is deliberately not here. It is import churn across 275 files, and coverage-thresholds.mjs buckets on one path segment, so merging photos/ in would collapse two ratchet entries into one and keep the looser of the two. |
||
|
Maurice
|
2d0eca7938 |
Let MCP addon gates read an injected service
Nine *.mcp.ts files reached through addons.bridge for one boolean each, and platform.routes plus the legacy registrars still do. That was never a preference: a decorator's options object is built while the class is being defined, so the `when:` closure had no `this` to reach an injected service through, and the only thing in scope was a module-level singleton that builds its own AddonsService outside the container. Nine domains hid an edge to the addons domain from the module graph to answer "is this addon on". @trek/nest-mcp hands the gate its declaring instance now, resolved at attach rather than captured at module load, and addons/addon-gate.ts turns that into addonGate(ADDON_IDS.X) over the injected service. The option is declared with method syntax so a predicate written against its own controller class stays assignable. Two more edges went the same way, both checked for cycles first: reservations.mcp injects AssignmentsService (AssignmentsModule never reaches back), atlas.mcp and journey.mcp inject AuthService. The tests that forced a toggle by mocking the bridge module now write the addon row the admin panel writes, so they exercise the real read. Four in-container uses stay, each documented in the nest README: places.mcp (a real Days -> Places -> Assignments cycle), budget/packing/costs against trips.bridge, user-cleanup against budget.bridge, and the two multer interceptor options, whose fix is MulterModule.registerAsync. |
||
|
Maurice
|
328b5fcb2b |
One Nominatim client for atlas and maps
Atlas throttled to the 1.1s OSM asks for and cached its answers in a 50k map; maps did neither, so five interactive routes fired straight at the service while the two atlas backfill loops politely waited their turn. From Nominatim's side that is one instance ignoring the rate limit. Both go through geo/nominatim.client.ts now. Three things the naive merge would have got wrong: - The shared client sends UA, the instance-identifying User-Agent, not the generic string atlas used. A shared generic UA is throttled harder. - One cursor across five interactive routes and two background loops would have put a keystroke behind however many places the backfill enqueued, so there are two lanes over the one cursor and background yields. - The cache key carries the query shape. Atlas asks at zoom 3 and 8, maps at 18; a coordinate-only key answers a street lookup with a country. The throttle cursor and the cache stay module state, the way the permissions and FX caches do, because the bridges build their collaborators outside the container and a second copy of the cursor is the bug being removed here. GeocodingService owns only the cleanup timer's lifecycle, which atlas used to start as an import-time side effect. Also folds the three haversine implementations into common/geo.ts, keeping the clamped form: asin of a value a hair over 1 is NaN, and a NaN distance fails every comparison it feeds instead of throwing. tests/setup.ts zeroes the interval. Nineteen maps cases drive these paths back to back with real timers and were sleeping nineteen seconds for it. |
||
|
Maurice
|
4965281644 |
Delete src/services and wall off the way back
The directory is empty after the airtrail fold, so it goes. The ESLint rule is the part that matters: the directory disappearing is not what keeps it gone, because the way it grew was one file at a time, each one reasonable on its own. Any import of a services/ path is an error now, with the message pointing at the recipe in the module guide. Proved rather than assumed: a probe file importing the old airtrail path fails the rule, and the whole package still lints with zero errors. Not changed: the 91 provenance comments in shared/, client/ and server/ that name a services/*.ts file. They are statements about where a schema or a service came from, and they stay true as history. Rewriting them all would put 91 files of noise over the change worth reviewing, and the paths they cite are already gone from the tree, which is the honest signal that they are history. |
||
|
Maurice
|
22ee0bafdc |
Bring the module guide in line with the guards
The guard section still said BE-Phase 2 was in progress and listed TripAccessGuard, default-deny and the MFA policy as open. All three landed on this branch, and two of them changed how a contributor is supposed to write a controller, so the file that tells them how needs to say it: what GlobalAuthGuard does with a declared @UseGuards chain, why places splits along "does this handler take a DTO-typed body" rather than along the domain, and why TripOwnerGuard must not go near PermissionsService. It also still pointed at middleware/mfaPolicy.ts in two places, which this branch deleted. |
||
|
Maurice
|
94221a6d91 |
Roll TripAccessGuard onto the remaining trip-scoped controllers
assignments, budget, collab, accommodations, reservations and booking-import drop their copied requireTrip/requireEdit helpers for the guard. Status codes and bodies are unchanged. places is deliberately NOT converted, and that is the finding of this PR rather than an omission. Its e2e suite documents that the body pipe answers 400 ahead of the trip 404 — a parity shift somebody made on purpose — and a Nest guard always runs before the pipe, so converting it silently reverses that ordering. The first attempt also dropped the permission check on DELETE, which the integration suite caught as a 200 where a 403 belonged. places keeps its in-handler check, and README now says why so the next person does not rediscover it the same way. collab applies the guard per handler for the same reason files does: its note-file upload route would otherwise be refused before multer had read the body, and a response sent mid-upload destroys the socket. Two coverage notes. The per-domain thresholds are regenerated because deleting a well-covered helper lowers a file's ratio without anything becoming less tested — the drops are 0.4 to 1.2 points across six domains and match the helpers removed. And the atlas integration suite gets the same admin1-bundle warm-up the unit suite got: ATLAS-013 was paying 3.1s for the one-off decompression inside its own 5s budget, which is why it only ever failed under load. It now runs in 116ms. |
||
|
Maurice
|
fef338091c |
Fold backupService into the backup domain
src/services/backupService.ts moves to src/nest/backup/backup.impl.ts, imported only by its own domain now that the controller reaches it through BackupService. src/services/ is down to airtrail/. It stays a module of free functions rather than becoming methods on the service. The restore path closes and reinitializes the core DB handle, which is the single most dangerous sequence in the server, and rewriting its shape in the same step as the move would make a regression there impossible to bisect. The db-lifecycle question is the last step of this fold, not part of it. The move also pulled 512 lines out of ungated src/services/ into a gated domain, which is how the per-domain ratchet caught that the per-entry extraction loop had no tests at all: the zip-slip refusal, the running decompressed-byte cap (the declared size in the central directory is attacker-controlled, so only the running total catches a lying archive), a corrupt stream not being dressed up as a size refusal, and the reopen failure that has to report restart-required rather than success. Six cases now cover it, and backup goes from 84% to 96% statements. |
||
|
Maurice
|
f72aba0c91 |
Finish the plugin RPC decorator rollout
The last 20 wire methods move onto the decorators: db.* and the three unconditional methods (DbRpc), meta.* (MetaRpc) and the host-mediated surface of users/broadcasts/notify/ai/oauth/scheduler (HostSurfaceRpc). That closes the ledger at 113/113, so the legacy has() blocks in rpc-host.ts are gone and the router is down to what was never domain-specific: build the granted subset of the dispatch map, dispatch, audit, map errors. PluginHostDepsFactory was the 26-argument wiring sheet behind those blocks. With the handlers in the domains it has two dependencies left, so it is now PluginRpcHostFactory: the plugin's own sqlite handle, the inter-plugin router bound with this plugin as the caller, and the audit sink. The 18 host-to-plugin hook calls get the same treatment. They used to spell out the fn name and the timeout at each controller, which meant nothing checked that a hook:* grant on the consent screen had a consumer behind it, that the fn matched what the SDK documents, or that two call sites agreed on the budget. PluginHooks declares all 18 with @PluginHook and performs them, and the controllers call it by name. With both halves complete, requireTotalCoverage goes on: a KNOWN_METHOD without a handler, or a hook without a consumer, now fails app boot instead of surfacing as a PERMISSION_DENIED nobody can explain. |
||
|
Maurice
|
f0f69df67e |
Fold the journey domain into Nest and finish the MCP migration
journeyService (1324 LOC, 36 exports) and journeyShareService (203) become JourneyDomainService and JourneyShareService. Every SQL statement, error string and broadcast is the one that shipped. The two could never be split: the share service gates on isOwner, which lives in the domain service, so it injects it. JourneyDomainModule is a leaf carrying just the two services. Places, assignments and the plugin host each need a handful of journey functions; importing JourneyModule for that would drag MemoriesModule and both photo providers into their graphs. Same shape as TrekPhotosModule. The 23 tools in src/mcp/tools/journey.ts become JourneyMcp. src/mcp/tools/ is down to _shared.ts and registerTools() no longer fans out to anything. journey.bridge.ts keeps four exports, for the journey resources in src/mcp/resources.ts that register at import time. McpAccessMode is host-augmentable now. journey:share is neither read nor write, so the three share tools would have had to be opaque predicates - and predicates bypass the access policy entirely, invisible to the boot-time scope gate. A scope typo in one would ship as a silently ungated tool. McpAccessModeRegistry mirrors McpAccessGroupRegistry; unaugmented, mode stays 'read' | 'write'. Fixes a latent bug in tests/helpers/mcp-test-controllers.ts: it constructed PlacesService with five of eight arguments. tsconfig only includes src, so nothing typechecked it, and the missing dependencies were harmless until the journey skeleton hooks landed on the place write paths. journey.mcp.ts arrived from src/mcp/ at 14.5% branch coverage - the legacy registrar had no test file of its own. 40 tool tests take it to 93.4%, and 25 cases for the skeleton lifecycle, the photo gallery and the preference row take the domain service from 73.9% to 82.8%. |
||
|
Maurice
|
5b6c79299f |
docs(server): record what the coverage gate does and does not measure
PR #1844 went red with every suite green: the diff moved MCP tools from src/mcp/, which the gate ignores, into src/nest/**, which it measures. That did not change any behaviour - it revealed that airports.mcp.ts had never had a test in either world, and dragged the aggregate to 79.86%. Worth stating in the gotcha list, because journey.ts and _shared.ts still have to make the same move. |
||
|
Maurice
|
ac96644508 |
refactor(server): fold the notifications domain out of services/
Four files taken apart rather than moved, because they only worked as a knot. The mailer becomes its own leaf module. AuthService sends the password-reset mail while NotificationsModule imports AuthModule for the demo gate in NotificationsMcp - a mailer inside the notifications domain would have closed a hard cycle that only a forwardRef could open again. Webhook and ntfy stay with notifications; auth has no business importing them. The channel registry stays a module singleton and does NOT become a provider. PluginRuntimeService pushes its channel getter in at onModuleInit, and notifications.bridge.ts builds its own NotificationsService for the scheduler crons and the memories senders. A provider would hand the bridge a second, empty registry - every plugin channel would then go quiet with no error anywhere. CHOVR-015 drives that seam end to end: a channel published through the runtime, delivered through the bridge. Nothing pinned it before; the existing cases all set the source themselves. The built-ins no longer self-register by side-effect import. They used to exist only because notificationPreferencesService happened to `import './notifications/builtins'`; removing that line would have silenced email, webhook and ntfy without a failure. They are built from the injected transports and registered from NotificationsService's constructor, so both instances have them. No cached nodemailer transport: MailerService.createTransport re-reads the SMTP config and skipTlsVerify per send, exactly as the three inline call sites did, so an admin changing them does not have to restart. Preferences become a provider over DatabaseService. Eight places built NotificationsService by hand, so a tests/helpers/notifications.ts factory keeps the next constructor parameter a one-file diff instead of an eight-file one. |
||
|
Maurice
|
03493a2edd |
refactor(server): fold webauthn, account erasure, oauth and the wiki out of services/ (#1840)
* refactor(server): move the webauthn config and the account erasure into nest/auth webauthnConfig resolved the RP ID through its own getSetting() on the db singleton; it is a WebauthnConfigService over DatabaseService now, injected into AuthService (passkey_configured) and PasskeyService (four ceremonies). readEnv() is still read per call, so an operator flipping the RP ID at runtime takes effect on the next ceremony instead of after a restart. userCleanupService follows as UserCleanupService. It keeps taking removeUserFromBudgetItems from budget.bridge rather than injecting BudgetService: BudgetModule imports AuthModule for the demo guard in BudgetMcp, so AuthModule cannot import BudgetModule back, and breaking that edge the other way drags BudgetController, BudgetMcp and its trips.bridge chain into every graph that imports AuthModule. The legacy module had no tests of its own - the erasure only ever ran through AdminService.deleteUser and TripsService.deleteGuest - so USER-CLEANUP-001..009 pin both halves, including the orphan data dir that no permissions row can describe and the all-or-nothing deletion transaction. * refactor(server): fold the oauth 2.1 server out of services/ 694 lines move into OauthService over DatabaseService, AddonsService and AuditService. The thin wrapper that only forwarded is gone, and so is the second path to the MCP addon flag - validateAuthorizeRequest read it through addons.bridge while the wrapper read it through the injected service, which would have let the two drift. The pending authorization codes stay a module singleton in oauth.pending-codes.ts instead of becoming instance state. The SDK still serves /oauth/authorize, /oauth/register and /oauth/consent from its own router, mounted on Express before app.init(), so it reaches the domain through the new oauth.bridge.ts - two maps would break the authorization-code flow silently, since the SDK only ever answers "Authorization grant is invalid.". The import-time sweep keeps its .unref() and is a named function now so a test can drive it. scopes and sessionManager are imported deep, never through the src/mcp barrel: the barrel re-exports revokeUserSessionsForClient and pulls the domain bridges back in, which is the nest -> mcp -> nest cycle. Same comment as admin.service and auth.service carry. The 70 cases of the legacy suite moved over unchanged, bound to a hand-built service. Fifteen more cover what free functions never let anyone reach: the code store at capacity, an expired code, an expired access token, refresh against a public and an unknown client, the RFC 8707 resource checks, and that a code written through the container is read back through the bridge. * refactor(server): move the wiki reader into the help module wikiService had exactly one consumer, HelpController, and touches no database - it stays a plain module rather than becoming a provider, the way auth/ephemeral-tokens.ts does, so the text and asset caches keep being one per process without any ceremony around it. The WIKI_DIR anchor needed one more level: src/services sits three below the repo root, src/nest/help sits four. The existing path-resolution case caught it, which is what it is there for - a wrong anchor makes help degrade to the GitHub fallback without anybody noticing. Also brings the nest README back in line: services/ is down to five top-level files, and the passkey and budget entries no longer point at paths that were deleted since they were written. |
||
|
Maurice
|
7dea88120b |
docs(server): bring the nest README up to the foundation work (#1830)
The README is the binding spec, and seven PRs have landed patterns it does not mention. Someone reading it today would rebuild things that now exist, and would walk into two traps that cost real time this week. Added to the foundation section: common/ and why those helpers are free functions, auth/jwt-verify.ts and why JWT_SECRET must not move to app-config, query-helpers/, trip-membership/ and the cycle that put it in its own module, AddonGuard + @RequireAddon with its ordering requirement, and demo-write. New subsection on reaching the container from outside it. The choice is settled now: hand the dependency in, declared structurally, the way index.ts fills the scheduler's SchedulerDeps. Bridges stay correct for the MCP registrars, but the README should say what they cost — a bridge rebuilds its services with `new`, so one added constructor parameter meant editing five bridge files in a single PR. Two gotchas that are cheap to read and expensive to rediscover: Never gate a multipart upload with a guard. Guards run before the parser, so throwing leaves the body unread and Node resets the connection — the client gets ECONNRESET rather than the 403. @Global only applies to modules that are in the graph. An e2e TestingModule built around one domain module does not get AppConfigModule for free, and a provider injecting RuntimeEnvService takes every suite importing that module down with it. Also corrects an assumption the migration plan makes and this README implied: the trip-access and canEdit methods on the domain services are not dead weight waiting for TripAccessGuard. Across the five domains piloted for it, 40 of the 46 callers are *.mcp.ts tools, which never pass through an HTTP guard. Only 6 sit in controllers. Counted, not estimated. |
||
|
Maurice
|
e606ebd9b9 |
chore(server): pin the WebSocket envelope, dedupe the rate limiter (#1803)
* chore(nest): register LlmParseModule in AppModule
The module was only in the container because BookingImportModule and
PluginsModule happen to import it, so /api/admin/llm/local depended on those two
keeping that import. Register it where every other module is registered.
* refactor(nest): give the rate limiter one home and one instance
RateLimitService keeps its counters in a Map on the instance, but it was listed
in the providers of auth, oauth, transit and trip-invite, so each of them got its
own. Nothing is broken today — the bucket names in use (login/mfa/forgot/reset,
oauth_*, transit_*, trip_invite) don't overlap — but two modules picking the same
name would have quietly allowed twice the configured attempts.
Move it to nest/common, where it belongs now that four domains use it, and hand
it out through a small module they import. Not @Global: the e2e suites build a
container around one domain module, so a global AppModule never pulled in isn't
there at all.
transit.mcp.ts keeps its own instance on purpose — it keys by userId under
mcp_transit_* while the REST path keys by IP.
* test(ws): pin the wire envelope before touching the transport
Every frame on the socket is flat: {type, tripId, ...payload} out, {type, tripId}
in. Nest's WsAdapter dispatches on message.event and answers {event, data}, so
moving this onto a gateway without a custom serialiser would leave join silently
unhandled — no welcome.socketId to send back as X-Socket-Id, and every client
would then receive its own writes.
Nothing catches that today: the 100+ tests that touch realtime mock src/websocket
and never look at the bytes. WS-032 sends the exact shape the stock adapter would
and asserts the server ignores it.
* docs(nest): drop the parts of the README that no longer exist
strangler.ts, DEFAULT_NEST_PREFIXES/NEST_PREFIXES and tests/parity/ are all gone —
every request goes through Nest now — but the README still walks a new contributor
through adding a prefix and writing a parity test against the Express route.
Also spell out that the 80% gate is an average over src/nest/**, not a floor per
file, so a large untested module can sit behind the well-covered small ones.
|
||
|
jubnl
|
414d967223 |
docs: regenerate migration-graph after the admin fold
Records the fold and corrects two long-carried errors: there is no admin MCP surface (the '11 MCP consumers' figure predated the Phase-0 addons extraction) and systemNotices/conditions.ts was never a consumer. The memories/admin edge is documented in the right direction for the first time. journeyService and oauthService now head the frontier. |
||
|
jubnl
|
d203d8cccd |
chore: move notifications backend to proper DI
The send() dispatcher (services/notificationService.ts) and the in-app store (services/inAppNotifications.ts) fold together into the DI-native NotificationsService over DatabaseService + RealtimeService; the prefs matrix, transports, channel registry and inAppNotificationActions stay plain infra modules. A 1-export notifications.bridge covers scheduler, legacy adminService/memories and the six lazy fire-and-forget sends; AdminController and the plugin RPC host (24th factory dep) inject. The 5-tool registrar + notifications-in-app resource move to notifications.mcp.ts. Tests move with case IDs preserved (NSVC-*, INOTIF-*, + NSVC-020 bridge pin); ~18 suites repoint path mocks/warm-ups; the module e2e goes DI-native. Also carries the verified-quirk fixes that would normally trail as a separate fix(server) commit (file-overlap with the fold): toUtcIso on all three created_at paths (INOTIF-013/014), respond claims the response CAS before running the action handler with release-on-failure (INOTIF-011/012), synology's fire-and-forget send gains .catch, the dispatch-failure log unwraps Error messages (NSVC-021), and the MCP resource description drops its false 'unread first' claim. |
||
|
jubnl
|
e65f851446 | chore: move passkey backend to proper DI | ||
|
jubnl
|
ec0709741a | chore: move oidc backend to proper DI | ||
|
jubnl
|
45564b0a8a |
chore: move auth backend to proper DI
Fold the 1497-line services/authService.ts into the DI-native nest/auth
domain (Wave-5 chain opener, unblocked by the atlas fold):
- Pure crypto half (backup-code hash/match/generate, stripUserForClient,
key masking, the import-time DUMMY_PASSWORD_HASH timing equaliser and
avatarDir mkdir — documented parity exceptions) moves verbatim to the
plain module nest/auth/auth.helpers.ts.
- DB half folds into AuthService over DatabaseService + injected
PermissionsService (ex permissions.bridge) + AtlasService (ex
atlas.bridge, deleted on schedule). mfaSetupPending and the per-email
reset throttle stay module-scoped so the bridge and container instances
share them; avatarDir and require('../../../package.json') re-anchor one
directory deeper — with the two injection swaps, the only non-verbatim
lines. Controller-facing method surface unchanged.
- New 8-export auth.bridge.ts for out-of-container consumers: mcp/index.ts
token verification, the journey/notifications/transports registrars'
isDemoUser, and the still-legacy adminService/oidcService/passkeyService.
- 15 domain *.mcp.ts demo guards now inject AuthService (their modules
import AuthModule); atlas.mcp.ts alone stays on the bridge because
AuthService injects AtlasService and the reverse module edge would close
an AuthModule<->AtlasModule cycle (places.mcp precedent). OidcService and
PasskeyEnabledGuard inject AuthService too.
- Tests move with case IDs preserved: authService.test.ts ->
auth.helpers.test.ts, authServiceDb.test.ts -> auth.service.test.ts
(+ AUTH-BR-001..007 bridge delegation, + AUTH-DB-050..088 pinning the
previously untested getAppConfig/login/MFA/reset/admin-settings branches
now under the src/nest coverage gate). auth.e2e converts DI-native (real
bcrypt, real audit rows); oidc.e2e swaps its dead path mock for an
instance spy; websocket/integration suites repoint to bridge/helpers.
Quirks preserved byte-for-byte (0-coord drop in getTravelStats,
un-transactioned registerUser/verifyMfaLogin multi-writes, unguarded
revokeUserSessions in deleteMcpToken, current! assertions in
updateApiKeys) — fixes land separately.
|
||
|
jubnl
|
ae3ba18b9e |
chore: move atlas backend to proper DI
Fold the 1612-line services/atlasService.ts into the DI-native AtlasService: the stats aggregation, visited countries/regions with the #1490 tombstone and cascade logic, and the bucket-list CRUD become methods over the injected DatabaseService. The ~750-line pure-geo half — the bundled admin0/admin1 boundary stores with their #1576 streaming builders, the point-in-polygon indexes, Nominatim geocoding with its shared throttle, and every module-scoped cache including the import-time unref'd cleanup interval — moves verbatim to the plain module atlas-geo.ts so the caches stay process-global across the container instance, the bridge instance and test helpers (assetPath re-anchored one directory deeper, the only non-verbatim line). The 10-tool mcp/tools/atlas.ts registrar plus all four atlas resources in mcp/resources.ts move onto the decorator registry as atlas.mcp.ts — the when: atlas-addon gate is parity here (legacy tools and resources both gated; the REST controller deliberately does not). resources.test.ts retires with its last two cases, which move to tools-atlas-expanded.test.ts. The plugin RPC host swaps its 9 atlas imports for the injected AtlasService (23rd constructor dep); a minimal 2-export atlas.bridge.ts serves the one legacy consumer, authService.getTravelStats, and dies when authService migrates. Adopts atlas.dto.ts over the existing shared schemas and clears all three AtlasController entries from body-contract-allow-list.ts: the hand-rolled 'name and country_code are required' 400 becomes the ZodValidationPipe envelope (todo/places trade) while the whitespace-only bucket name keeps its legacy 'Name is required' trim guard. Fixes the verified defects the relocation had faithfully carried, each with a regression test (ATLAS-SVC-031..036 plus two MCP casing cases): the four multi-statement mark/unmark writes now run in db.transaction() (the region cascade nests as a savepoint), the trip-less countryPlaces early return honours manually_marked, the '|| null' bindings that dropped lat/lng 0 and empty-string notes on bucket update become '?? null', the mutating bucket SQL is user-scoped, and the MCP region/country-places tools uppercase their codes to match REST. Quirks deliberately preserved: the detached GET-path place_regions backfill writes, getStats' two divergent return shapes, the import-time cache-cleanup interval, getCountryFromAddress's unvalidated bare 2-letter codes, and the whitespace-only-name silent no-op on bucket update. Tests move with their case IDs: atlasService.test.ts becomes unit/nest/atlas.service.test.ts (pure-geo imports from atlas-geo, DB calls through the constructed service, plus bridge-delegation cases), the atlas e2e converts to the DI-native pattern (real SQL against the full temp-db schema), and the plugin-host path mock becomes a constructor stub. |
||
|
jubnl
|
c78134831f | fix(server): tidy collections quirks after the DI fold | ||
|
jubnl
|
cab49ea511 | chore: move collections backend to proper DI | ||
|
jubnl
|
e80ae79b77 | chore: move transitItineraryService into nest/transit as pure helpers | ||
|
jubnl
|
4f0f80ad69 |
chore: move place backend to proper DI
Fold the 1029-line services/placeService.ts into the DI-native PlacesService: the CRUD + ratings SQL, the GPX/KML/KMZ importers and the Google/Naver list importers become methods over the injected DatabaseService, byte-identical statements, COALESCE semantics and the If-Match conflict protocol (#1135). The pure half — frozen XML parsers, the KMZ unpacker, the dedup predicates, the Google hex-id parsers, reclaimPhotoCache — moves to places.helpers.ts (maps.helpers precedent). The 10-tool mcp/tools/places.ts registrar plus the trek://trips/{tripId}/places resource move onto the decorator registry as places.mcp.ts. search_place travels with them because its gate is places:read, not the read-only geo group, and now injects MapsService. TripsService, DaysMcp, BookingImportService and the plugin RPC host (PluginHostDepsFactory's 21st constructor dep) all inject PlacesService, so the domain needs no bridge at all. The two assignments.bridge imports that places.mcp.ts keeps are a deliberate cycle break: AssignmentsModule imports DaysModule and DaysModule now imports PlacesModule for days.mcp.ts's place creation, so injecting AssignmentsService there would close DaysModule -> PlacesModule -> AssignmentsModule -> DaysModule. Same seam reservations.mcp.ts uses, same trade trips.bridge.ts documents. Also folds services/placeEnrichment.ts in — its DB/websocket/Maps half becomes PlacesService methods over the injected DatabaseService/RealtimeService/ MapsService, its pure match selector joins places.helpers.ts — which retires nest/maps/maps.bridge.ts with its last consumer. Adopts places.dto.ts for the seven grandfathered body contracts and clears every PlacesController entry from body-contract-allow-list.ts. Two consequences worth knowing: the bespoke 400s 'Place name is required', 'ids must be an array of numbers' and 'URL is required' are now the ZodValidationPipe envelope, and because a pipe runs before the handler a malformed body 400s ahead of the trip-access 404 it used to follow — the same trade the todo and trips migrations took. placeBulkUpdateRequestSchema.ids drops .min(1) so the endpoint's empty-list short-circuit stays reachable. Fixes four defects the relocation had faithfully carried, each with a failing-first regression test: - create dropped lat/lng of exactly 0, so a place on the equator or the prime meridian lost its coordinates (`x || null` cannot tell absent from zero). - duration_minutes: 0 was unsettable — replaced by the 60-minute default on create, and read as "absent" by COALESCE on update. - the journey delete hook fired on unscoped ids: DELETE /:id, POST /bulk-delete and the delete_place MCP tool could detach another trip's journey entries for an id they then refused. All three scope first via the new PlacesService.scopedIds, mirroring the guard PluginHostDepsFactory already had. bulk_delete_places also moves its hook ahead of the DELETE — journey_entries.source_place_id is ON DELETE SET NULL, so running it afterwards left the entries as orphans. - the place search treated % and _ as LIKE wildcards; a bare % returned the whole trip. The three clauses now carry ESCAPE with the term escaped. Plus hardening on the Google list fetch: the response is capped at 8 MB (declared content-length and post-read length, the transit.service precedent) and its JSON.parse is guarded, so a malformed provider payload produces the existing 'Invalid list data received from Google Maps' 400 instead of throwing. Quirks deliberately preserved: the non-COALESCE route_color (#776), currency/transport_mode staying unclearable, importGpx returning null for a file that parses but yields no usable geometry, update skipping the If-Match check when the stored updated_at is null, and every string-valued `x || null`. Tests move with their case IDs: placeService.test.ts and the wrapper suite merge into unit/nest/places.service.test.ts, kmzUnpack into places.helpers.test.ts, placeEnrichment into places.enrichment.test.ts. The places e2e converts to the DI-native pattern (real SQL against temp-db DDL), tools-places.test.ts is kept and extended so it exercises the decorator path through the test registry, and the plugin-host and booking-import path mocks become constructor stubs. |
||
|
jubnl
|
f4bf7cf3dc |
chore: move transit backend to proper DI
Includes the quirk-repair pass (upstream timeout + size cap, LRU cache, NaN-duration guard) that maps landed as a sibling fix commit. |
||
|
jubnl
|
2e40c19249 | chore: move maps backend to proper DI | ||
|
jubnl
|
287dceb532 |
chore: wire TripsService into the plugin RPC host and retire the legacy tripService
Swap the factory's six tripService imports for the injected TripsService; delete services/tripService.ts and the collab/vacay bridges with it; prune the days/budget bridges to their surviving exports; update the migration docs and regenerate migration-graph.md. |
||
|
jubnl
|
65fb00d63b | feat(server): type MCP access groups against SCOPES + boot gate | ||
|
jubnl
|
7f34dadf70 | chore: move budget backend to proper DI | ||
|
jubnl
|
a3e6eb52a4 | chore: fold exchangeRateService into the nest budget domain | ||
|
jubnl
|
2944cd32a1 | docs: tick permissions and auditLog in migrate.md and refresh the migration graph | ||
|
jubnl
|
2d1eec76c3 | chore: move day backend to proper DI | ||
|
jubnl
|
203d67c45f |
chore: move reservations backend to proper DI
Folds the 626-line services/reservationService.ts into the DI-native ReservationsService — byte-identical SQL, statuses, bodies, error strings and broadcasts; the legacy file is deleted. - reservations.mcp.ts: the 5 legacy tools (create/update/delete_reservation, reorder_reservations, link_hotel_accommodation) + the trip-reservations resource move to the decorator registry (identical names, schemas, annotations, error strings, broadcasts); the canWrite/canRead registration gates become declarative access markers. Legacy registrar and resource entries removed. - reservations.bridge.ts: 9 legacy-named exports + 3 type re-exports for the outside-container consumers (tripService, airtrail import/sync, the still-legacy transit + transports registrars, transitItineraryService's type-only import). - notifyBookingChange folds to the legacy (tripId, actorId) signature; the controller passes user.id and the plugin-host factory drops its last plain-function reservation import for a private method over the injected service (constructor unchanged). - TripsService and BookingImportService inject ReservationsService instead of importing legacy functions. - Tests: reservations.service.test.ts rewritten as a real-SQL suite (RESV-SVC-001..030, RES-TRAV-001..005 moved verbatim, RESV-BRIDGE-001..009 pinning the bridge); both e2e suites converted to the DI-native pattern; path mocks repointed to the bridge or constructor stubs; tools-reservations kept and extended with the moved resource cases. Quirks preserved on purpose (parity first — see the follow-up fix commit): the raw-accommodation_id metadata-sync gate, the un-transactioned multi-statement writes, the truthy updatePositions dayId check, the empty-string COALESCE keeps, and the TEXT accommodation_id normalization. |
||
|
jubnl
|
588457710c | chore: move vacay backend to proper DI | ||
|
jubnl
|
c08c595b7e | chore: move collab backend to proper DI | ||
|
jubnl
|
f69731b72b | docs(server): chronicle the fileService DI migration | ||
|
jubnl
|
cece6509b9 |
chore: move settings backend to proper DI
Folds the legacy services/settingsService SQL into the Nest SettingsService (byte-identical statements; BEGIN/COMMIT blocks become db.transaction()). AdminService and ShareService inject it; llm-config.resolver becomes the injectable LlmConfigResolver (SettingsService + DatabaseService), injected by LlmParseService and PluginHostDepsFactory. No MCP surface, no bridge. Whole-file commit, so this also carries the post-migration quirk fixes that live in the same files: null serializes as '' (a stored "null" leaked out of getDecryptedUserSetting as the literal string), bulk upserts skip the masked sentinel (mirroring the single-upsert no-op) and return the written count. Unit suite moved to tests/unit/nest with SET-SVC-001..026 preserved, 027..030 pin the fixes; settings/admin e2e converted to DI-native. |
||
|
jubnl
|
72c3edda1a | chore: move share backend to proper DI |