mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-10-18 21:16:47 +00:00
532010aa790d7eb5f0f28896d2d47ea63ccc107b
57 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Maurice
|
48d573c60d |
refactor(trips): leaving a trip tidies its journal link in the roster's own repository
The departure port needed a global binding that every suite building TripMembersModule without AppModule lacked. The roster's repository now deletes the leaver's row on the trip's journal link itself (leaveJournalLink), inside the transaction that removes the member: no edge into journey, no DI, no test stub, and the eight suites the stub touched are back as they were. The schema inventory tests count the two new tables and their NOT NULL columns, the legacy step map lists the migration as post-legacy, and the TJL scenes give their users numbered names so cases sharing one database do not collide. |
||
|
Maurice
|
ce8d1e5a0a |
refactor(trips): leaving a trip reaches the journal link through a port, not an import
TripMembersService called TripJournalLinkService directly, which made trip-members and journey depend on each other (lint:boundaries). The roster now declares TRIP_MEMBER_DEPARTURE and calls memberLeft in the transaction that removes the member; the journey domain binds it globally (TripMemberDepartureModule), the way JourneyAccessModule binds the gateway's JOURNEY_ACCESS. Suites that build TripMembersModule alone import MemberDepartureStubModule. |
||
|
Maurice
|
80b6739362 |
feat(journey): a trip can link a journal and open it to the members it picks
A trip can now show one journal from the Journey addon as its own: any journal its owner picks, and one journal on as many trips as they like. The trip members chosen for the link get into the journal as viewers or editors, see it in their own journal list and follow it live, for as long as the link holds them and they stay on the trip. That access is never copied into journey_contributors. It is read where journal access is read: the contributor repository answers with the stronger of a direct role and a link role, the journal list and the live update recipients take the link members in, and the photo access check goes the same way. Every one of those reads starts from linkReach, which only counts members still on the linking trip, so unlinking, deselecting someone and their leaving the trip all end exactly what the link gave. A member who leaves is also taken off the link, so coming back later does not quietly reopen the journal. Linking or changing the members takes the journal's owner and the new journal_link trip permission (trip owner by default); unlinking takes either. /api/trips/:tripId/journal answers the link as each member may see it, /candidates says who a journal would be opened to for the first time, and four MCP tools do the same over the same service. A trip:journal:changed event tells the trip to look again. |
||
|
Maurice
|
89ad4ec56a |
fix(tours): paddle data reads stream, queue safely and keep their coverage during a refresh
The PBF reader streams the extract and refuses a varint longer than ten bytes; the block offsets are read as the plain int64 the format defines. Region reads queue without loose promises: a region queued while another is read is still picked up, removing a region calls its read off, a shutdown stops the read under way and the next boot queues it again and clears the download folder. The admin list shows how far a read has come. Geofabrik is asked only on its own servers, redirects included, and an outage of its list answers 502 instead of an unknown region. Nested ids such as us/texas are accepted. A region being read again or after a failed read still counts with the data it holds, a region without a boundary spans its objects, and an object two overlapping regions hold is answered once. Stored geometry, tags and boundaries go through the JSON column declarations. An obstacle is related to the route both ways, so a long weir passed in its middle and a route inside an area count. Tests cover the reader with extracts written in the test, the tag rules, the extraction and lock grouping, the geo helpers, the Geofabrik client, the region queue, the read side, both controllers, the job, the repositories and the HTTP routes; the domain gets its coverage entry. |
||
|
Maurice
|
2c3b20bb57 |
feat(tours): paddle portages and locks, paddle time, wind at the start and obstacle data from OpenStreetMap
PAD-03 portages and locks on planned tours and recorded tracks, PAD-07 paddle time with portage and lock stops, PAD-08 wind forecast for the start, PAD-05A regional obstacle import from Geofabrik extracts with admin endpoints and a monthly refresh, PAD-05B first cut of the obstacle layer on Leaflet, GL and 3D. Work in progress: checklist (PAD-06), check bench (PAD-09), admin UI, translations and tests follow. |
||
|
Maurice
|
782e613506 |
Tours: day GPX import, wanderer trails and planning over an import
- GPX files drop onto the map or a day; the import lands on that day - wanderer connection per user in Integrations; the workspace offers the user's own trails only, behind an admin switch under the Tours addon - wanderer difficulty is stored on the SAC scale (T1 to T3), backfilled - a GPX or wanderer import can be planned as an editable tour, either in place (the recorded track is kept and can be restored) or as a new tour that remembers the import it came from - rails, rows and detail dialog follow the plan sidebar's cards and badges - admin addon shelves with more than one row fold, collapsed by default - reverse-geocoded waypoint names are cached on the device |
||
|
Maurice
|
a9882a6415 | feat(tours): via ferrata layer mounted on the tours map (2D and 3D) | ||
|
Maurice
|
724c88c201 |
style: format server and shared with prettier
A mechanical prettier --write over server/src, server/tests, server/scripts and shared/src with the workspaces' .prettierrc; nothing but layout and import order changes, and every file keeps the same set of imports. shared is formatted throughout and its format baseline is empty. The server baseline keeps the 35 files that Prettier would grow past their lint:size limit (they need a split first, which is not a formatting change), and the generated entity files stay as their generator writes them. |
||
|
Maurice
|
66042ae1c7 | feat(server): services refuse with a DomainError that REST and MCP answer in their existing shapes | ||
|
Maurice
|
1389fe6984 | refactor(place-import): file and shared-list readers move into their own domain over one gpx and one kml codec | ||
|
Maurice
|
846d8d4952 |
refactor(realtime): the gateway asks a journey-access port instead of importing the journey domain
The websocket gateway injected JourneyDomainService to decide who may join a journey's book room, so the realtime domain, which nearly every domain imports for its broadcasts, depended on journey and closed a cycle through all of them. It now injects a JOURNEY_ACCESS port declared in realtime; JourneyAccessModule (global, imported by AppModule beside the gateway module) binds it to the same JourneyDomainService method. |
||
|
Maurice
|
cc79f561e5 |
refactor(auth): guards, auth decorators and ephemeral tokens move into an auth-core kernel
JwtAuthGuard, OptionalJwtAuthGuard, CookieAuthGuard, AdminGuard, the global and MFA guards, @CurrentUser, @Public, JWT verification and the ephemeral token service lived in the auth domain, so every controller that checked a session depended on the whole account module and its imports. They import no domain themselves; under src/nest/auth-core they are shared kernel like common/, every importer points there directly, and AtlasModule and NotificationsModule stop importing AuthModule, which they only did for the demo gate. lint:boundaries counts auth-core as kernel: 57 fewer cyclic domain edges, the largest cycle shrinks from 46 to 40 domains. |
||
|
Maurice
|
1a9855cf70 |
refactor(mcp): the demo-mode write block runs once, in the registry's tool gate
Every write tool used to open with its own demo check, 196 copies across 30 tool files, most of them injecting AuthService for nothing else. The registry now takes a toolGate (built from the container) that runs before each registered tool handler, and TREK's gate refuses a demo account on every tool not annotated read-only with the same canned result. The per-tool checks, the AuthService injections, AuthService.isDemoUser and the AuthModule imports that only fed them are gone; ESLint refuses a demo check in a tool file, and a registry-wide test calls every write tool as the demo account. |
||
|
Maurice
|
9aab9e7513 |
feat(auth): track sign-in sessions so a session can end before it expires
A session token was a signed JWT and nothing else. Logout only cleared the cookie, a 30-day "remember me" token stayed valid wherever a copy of it had ended up, and nobody could sign out one device short of changing the password or rotating the JWT secret for everyone. Every session token now carries an id (jti) naming a row in the new user_sessions table (id, user_id, created_at, last_seen_at, expires_at, revoked_at, user_agent cut to 256 characters; no address is kept). The rows live behind UserSessionsRepository and the new sessions domain, whose SessionsService is the one issuer: AuthService.generateToken and the OIDC sign-in both go through it, and sliding renewal re-signs a tracked token under the same id and moves its expiry. verifyJwtAndLoadUser takes the repository and refuses a token whose session is revoked, expired or gone, refreshing last_seen_at at most every five minutes. A token without a jti was issued before this change; it passes until its own expiry (or a password change), so the upgrade signs nobody out, and renewal turns it into a tracked session. Sessions end on logout (the one the request names, verified even when expired), on a password change (all, the device gets a new one), a password reset, an admin setting a password, reset-admin.js, an MFA disable (all but the current one), an admin clearing a user's MFA, and with the account. GET /api/auth/sessions lists the active sessions, DELETE /api/auth/sessions/:id ends one and POST /api/auth/sessions/revoke-others ends all but the current, with their contracts in @trek/shared and two new audit actions. There is no UI for them yet. MCP exposes no account or session tools, so there is no parallel tool to change; an MCP call made with a session JWT as bearer goes through the same check. The token in the login, register, demo-login, MFA and passkey bodies stays for API clients but is marked deprecated in the API docs and in the client type, which never read it. A nightly session-purge job on CronRegistrarService deletes the expired and revoked rows. The migration is 042500. The sessions domain gets its own coverage entry at the 80 floor until a full run measures it. |
||
|
Maurice
|
0a4c6fbace |
refactor(db): open the database through a DatabaseLifecycle provider
db/database.ts opened the connection as a side effect of being imported, so the moment anything reached it the database was open. index.ts had to defer loading the app until a first-boot restore was done, and nothing in the container decided when the connection opened or closed. The module now opens the connection on first use and opens nothing at import. A connection closed on purpose (a restore, the shutdown) stays closed until it is reopened, as before. The new DatabaseLifecycle provider owns the decisions: buildApp() opens through it, which also binds the ORM to later swaps (the hook attachOrm used to register) and runs the schema bootstrap in the same order as before, and the shutdown closes through it. Its file property is the one resolved database path. The module keeps every export, so the test suites that replace it with vi.mock keep working; buildDbMock gains the new openDb. |
||
|
Maurice
|
c956fe92ba |
Merge dev into feat/tours and port Tours onto MikroORM
dev now runs on MikroORM, so the positional migrations, seeds.ts and DatabaseService the Tours server side was built on are gone. Behaviour is unchanged; persistence moves onto dev's architecture: - one MikroORM migration for tour_types, tours and tour_waypoints, idempotent so instances that ran the old positional steps keep their data; the hike tour type is seeded inside it, the addon row in the AddonSeeder - entities and repositories for the three tables, tours domain on repositories and UnitOfWork - tour facet joins in places, assignments and days re-applied on the repository code Fixes found while porting: - GPX import checks addon, trip access and permission in the handler instead of guards, so a refused upload gets a 403/404 instead of a connection reset - copying a trip copies its tours and waypoints - a tour cannot be moved onto a day that already has it - updateTour checks existence inside its transaction Adds tours e2e tests and rewrites the tours unit tests onto the migrated test database. |
||
|
jubnl
|
de8caa81a2 |
Feat/orm : mikroorm (#2416)
* base entities
* update deps
* fix date as string
* rename entities
* migrations + seeders
* feat(db): DbTimestampType keeps DATETIME columns as the CURRENT_TIMESTAMP text
* feat(db): discover entities from an explicit list instead of a glob
* test(db): createTestOrm binds an ORM to a suite's own SQLite handle
* feat(db): dialect helper is the one place SQL functions are spelled
* test(db): dialect helper refuses malformed column references and day counts
* feat(db): snake_case entity shape and the first three repositories (Days, DayNotes, Trips)
* fix(db): repositories mirror the statement's column set and guard their row types
* feat(db): request-context and unit-of-work helpers for the ORM, with concurrency tests
* test(server): integration suites boot on the migrated schema snapshot
* test(server): snapshot harness keeps the legacy id semantics
* fix(test): route MikroORM's dynamic imports through vitest from the setup file, not the ORM config
* test(db): coverage ratchet entries for the ORM foundation
* test(server): websocket suites boot on the migrated schema snapshot
* test(server): coverage's uncovered-file pass goes through the SWC transform
* fix(db): dialect helpers dispatch on the platform; lint gate for the repository layer
* test(server): ORM suites open the schema snapshot; RESET_TABLES derived from the schema
* chore(server): promise safety net and sweep tooling for the async migration
* refactor(realtime, systemNotices, addons, airports, audit, categories, permissions, query-helpers, tags, transit, trip-membership): async DB-touching methods; transactions through UnitOfWork
* fix(server): observe every promise the async sweep left unobserved (timer, handshake, plugin-runtime test)
* test(doc-sync): unit suites await the methods the async sweep made async
* refactor(atlas, school-holidays, settings): async DB-touching methods; transactions through UnitOfWork
* chore(server): lint rule refusing a Promise used as a plain value, and the misses it found
* test(server): suites await the methods Task 1's forced edits made async
* refactor(budget, common, day-notes, days): async DB-touching methods; transactions through UnitOfWork
* test(oauth): controller suite awaits the async service
* chore(server): db-call-graph --unawaited lists calls to async methods whose result is used without await
* refactor(llm-parse, mcp-shared, oauth, place-shadow, route-usage, storage, todo, trip-invite): async DB-touching methods; transactions through UnitOfWork
* refactor(plugins): async DB-touching methods; transactions through UnitOfWork
* chore(server): db-call-graph --sync stops its transitive walk at an async callee
* chore(plugins): drop gate-only asyncs; observe the scheduler's rejection; async event-sink stub
* refactor(notifications, place-photos, backup): async DB-touching methods; transactions through UnitOfWork
* refactor(auth, files): async DB-touching methods; transactions through UnitOfWork
* test(server): suites await the methods earlier sweep groups made async
* refactor(tokens): async DB-touching methods; the auth token handlers await them
* test(files): upload filters fail closed on an excluded or unresolvable type
* refactor(collab, collections, maps): async DB-touching methods; transactions through UnitOfWork
* refactor(memories, photos, packing, share): async DB-touching methods; transactions through UnitOfWork
* chore(server): lint rule also refuses a Promise handed to expect() without resolves/rejects
* refactor(vacay, admin, doc-sync): async DB-touching methods; transactions through UnitOfWork
* fix(vacay): type the loop accumulators the async sweep introduced
* refactor(journey, oidc, place-enrichment, tokens, trip-members): async DB-touching methods; transactions through UnitOfWork
* fix(server): the journey hooks are awaited by their callers, not detached
* refactor(assignments, public-api, accommodations, places): async DB-touching methods; transactions through UnitOfWork
* refactor(trips): rewrite AccommodationsService.createAccommodation .bind alias
Recipe R4 trap named in the Task 12 brief: a .bind alias of an
AssignmentsService/AccommodationsService method is typed any and hides a
missing await from tsc and the promise lint rules. createAccommodation
belongs to this task's group, so rewritten as an arrow forwarding to the
method instead of a bound reference.
* refactor(assignments): drop the unused Trip type alias
* refactor(reservations, booking-import, calendar, integrations, roadtrip): async DB-touching methods; transactions through UnitOfWork
* refactor(roadtrip, reservations): return await on transactional; document the single-flight promises
* refactor(trip-read-model, trips, feeds): async DB-touching methods; transactions through UnitOfWork
* refactor(server): remove DatabaseService.transaction — every transaction runs through UnitOfWork
* refactor(trips, feeds): drop added return annotations; no double await on the feed guard
* fix(server): phase 1 review follow-ups — awaited returns, logged upload-filter rejections, covered handshake/notice branches, call-graph gate in CI, dead test imports
* fix(server): upload filters answer multer once on a throwing callback; positive call-graph gate test
* test(db): entity metadata must match the migrated schema; fix the four drifts it found
* feat(db): primary-relation entities proven (BudgetCategoryOrder, VacayUserSettings)
* test(db): parity gate also holds declared pivot entities; per-field column types; missing-set ratchets both ways
* test(db): numeric ids for the primary-relation spike
* feat(db): re-runnable entity generator encoding the snake_case/DbTimestampType rules
* fix(db): generator guard rails — validation cannot mask hidden()/index loss; typed JSON; loud failures
* fix(db): generator keeps NOT NULL boolean defaults; fails if a JSON interface import survives
* refactor(db): every entity in the snake_case shape; document-sync entities added; one repository per entity
* fix(db): no NaN class-field initialisers; entity generator --check gate in CI; parity normaliser keeps literal case
* chore(db): entity --check reports stray files; class-field census is a ratchet; migrate-encryption typechecked
* feat(db): AppSettings and Users repositories; forFeature wiring pattern
* refactor(permissions): permission flags through AppSettingsRepository
* fix(server): boot-time settings read runs inside an explicit ORM request context
* refactor(audit): audit log writes through AuditLogRepository
* fix(db): PK reads refresh past the identity map; typed owned-lookup generics; e2e ORM module mirrors production config
* docs(db): owned-lookup call-site convention; refresh side effect; e2e ORM module spread notes
* fix(audit): audit rows are native inserts, not a unit-of-work flush; closed test ORMs
* fix(server): non-HTTP entrypoints run inside an ORM request context; permission loads fail closed on context errors
* refactor(categories, tags): repositories replace raw SQL
* refactor(addons): addon catalog and photo providers through repositories
* fix(db): generator captures inline UNIQUE constraints
SqliteSchemaHelper.getIndexes() drops every unnamed sqlite_autoindex_
entry before entity-generator ever sees it, so a table-level
UNIQUE(...) constraint with no separate named CREATE UNIQUE INDEX
(settings' UNIQUE(user_id, key) among ~40 others) never reached the
generated entity's uniques: metadata. Adds a general
RULE9_addImplicitUniqueConstraints (skips a column set identical to
the table's own primary key — AppSettings.key/Addons.id noise) fed by
a PRAGMA index_list/index_info scan, and regenerates every entity.
* chore(addons): well-known gate wraps only the addon read in a request context; drop unused test factories
* fix(platform): well-known gate tolerates a synchronous addon read from the context helper
* fix(db): PK lookups inside patch refresh past the identity map; corrected identity-map notes
* refactor(settings): user and instance settings through repositories
SettingsRepository (getForUser/upsertForUser/getOne/deleteForUser) replaces
the per-user settings table's raw SQL; SettingsService drops DatabaseService
entirely. instance-api-keys.ts's readInstanceApiKey/writeInstanceApiKey/
resolveApiKey take an explicit AppSettingsRepository/UsersRepository instead
of a raw connection.
An earlier version resolved those two repositories itself from the active
MikroORM request context, so none of its six callers needed to change. That
worked in production and e2e (always inside a real request) but broke
service construction in plain unit tests across every calling domain
(addons, auth x2, maps, transit, unsplash - 185 failures on a full run),
since none of them wrap a request context. Each of those six services now
injects its own AppSettingsRepository/UsersRepository and passes them
through explicitly, matching how every other converted repository in this
migration is consumed; their own SQL is untouched (outside Plan 3a).
* test(server): harness constructors follow the Task 5 repository signatures; one test ORM per handle in the plugin host
* fix(db): generator names relation properties in captured UNIQUE constraints
* fix(settings): drop dead wiring and the speculative deleteForUser; strict row assertions
* fix(server): plan 3a review follow-ups — fail closed without an ORM at every choke point; seam and D5 ratchet tests; dead method and DDL copies removed
* fix(server): plan 3a re-review follow-ups — plugin boot activation and the airports backfill inside a request context; the supervisor answers the child before failing closed; BOOT-SWEEP-001 drives the real boot hooks; ws adapter seam pinned
* feat(db): generated entities declare their repository type; test casts dropped
* fix(server): plan 3a re-review 2 follow-ups — absent-ORM boot branch pinned; airports boot hook never rejects; BOOT-SWEEP-001 asserts the seven routes; duplicate comment removed
* feat(server): pre-init photo route and WS handshake run in a request context; InviteTokensRepository
* refactor(auth): users table through UsersRepository; JWT and MFA lookups repository-backed
* refactor(tokens): MCP and API tokens through McpTokensRepository
* refactor(auth): passkeys and registration invites through repositories
* fix(db): row-out repository reads bypass the identity map — stale write-back over nativeUpdate at flush; insertUser via em.insert; settings round-trip pinned
* fix(server): non-numeric route ids answer the legacy not-found, never NaN (tokens, categories, tags); mint path back to two statements; identity-map regression pinned
* refactor(oauth): OAuth server tables through repositories
OauthClientsRepository/OauthTokensRepository/OauthConsentsRepository
replace OauthService's raw SQL (OA1-OA33). revokeChain's recursive CTE
and the oauth_clients join (its FK actually targets client_id, not the
row id the generated relation assumes) go through em.getKysely();
OA19's chain revoke is one nativeUpdate($in); OA12's consent grant is
em.upsert on the (client, user) unique key. revokeAllForUser built now
for Task 5. Every row-out read is disableIdentityMap: true; inserts go
through em.insert.
* fix(auth): non-numeric passkey and invite ids answer the legacy 404, never NaN; invite join type pinned
* fix(db): generator emits the referenced column for non-PK foreign keys; OAuth client joins through the relation; FK parity un-skipped
* refactor(db): TrekRepository base — every repository path validates the request context and reads bypass the identity map by default
* refactor(auth): AuthService and account cleanup through repositories
* refactor(oidc): OIDC identities and settings through repositories
* fix(auth): invite validation keeps expires_at null on the wire; password transaction scopes and reset-token burn pinned
* fix(server): plan 3b close-out — case folding on both sides for identifier lookups (non-ASCII login regression); coverage ratchet restored; repository scaffold and instanceof ratchet on TrekRepository; D-shape tests load-bearing
* refactor(server): trip-access primitives async at every caller
* refactor(trips): trip access through TripsRepository; demo seed in a request context; dialect helpers for the trips cluster
* refactor(days): days and day notes through repositories
* refactor(assignments): day assignments through repositories
* refactor(server): trip-membership, place-shadow, place-enrichment, place-photos and query helpers through repositories
* refactor(trip-members): membership, guests and the trip read model through repositories
* refactor(places): places, tags, ratings and place images through repositories
* fix(server): place-shadow source counts ordered in SQL; identity-map ratchets load-bearing; photo-cache race pinned
* refactor(places): import paths through repositories
* refactor(trips): trips through repositories
* fix(server): trip ids parsed once at the gate for places and assignments; NUL-safe value quoting on the SQLite platform; places parity tests completed
* refactor(trips): trip copy through repositories for the owned tables
* fix(server): plan 3c close-out — non-finite ids bound as NULL, trip id parsed once at every places/days/ws gate
Platform: NulSafeSqlitePlatform now renders a bound NaN as NULL and
+/-Infinity as 9e999/-9e999, matching better-sqlite3's own legacy binding
(rule 22 extension) — the program-wide fail-safe for any caller that still
passes an unguarded Number(untrusted) into a converted statement.
PlacesService.verifyTripAccess and RealtimeGateway.handleJoin stop doing
that: verifyTripAccess parses once with toRowId and threads the parsed id
through create/import/bulk/rating/export.gpx (deleting the seven
toRowId(tripId) ?? -1 sites), and handleJoin gets a Number.isFinite guard.
DaysService gets the same treatment across list/create/getDay/reorder/
insert/resyncAccommodationDays, closing the gap where a hex-spelled trip id
could renumber a real trip's days while the raw-bind inverted-accommodation
guard silently matched nothing.
DatabaseService.getTripTitle is deleted; PlacesService injects
TripsRepository directly and calls getTitle itself.
Template/cheap-item cleanup: DayAssignmentsRepository.listForTimeSort uses
$castTo instead of an unknown bridge; the duplicated TagRow interface in
Places.repository.ts is now imported from Tags.repository.ts;
AssignmentWithPlaceRow extends the guarded DayAssignmentRow instead of
restating its columns; PlaceShadowPicksRepository.totals drops its
unreachable ??-fallback branch; Places.repository.ts's updatePlace types
the NOT NULL name column as string; trip-members.service.ts's
transferOwnership threads one parsed trip id through all three writes and
corrects two stale "raw-bind seam" docstrings that were actually Number()
seams; a dead canAccessTrip mock is removed from reservations.e2e.test.ts.
Tests: the 23 new no-explicit-any warnings in TRIP-SVC-069 are typed
(lint back to 2416); HTTP and WS regression coverage for the non-finite
and non-canonical id shapes across places/days routes, mutation-proved;
TRIP-JOIN-005 asserts the exact concurrent-join settlement instead of
>= 1; a new tests/unit/db/transaction-yield.test.ts pins rule 24's
measured transaction-yield/dirty-read/nested-BEGIN behaviour.
* test(server): plan 3c close-out — copy-source full-row parity, Kysely rollback proofs, identity-map relabels, trips e2e on the snapshot DB
* fix(server): plan 3c close-out — days create/insert answer the guard 404 for non-canonical trip ids instead of 500; exact settlement and cross-request yield ratchets
* feat(server): plan 3d groundwork — dialect helpers, visibility predicate, feed and cron context ratchets
* refactor(roadtrip): roadtrip tables through repositories
* refactor(reservations): reservations, endpoints and travelers through repositories
* refactor(feeds): ICS feed tokens and anonymous feeds through repositories
* refactor(accommodations): stays through repositories inside the existing transactions
* test(server): plan 3d — MCP test helper constructors follow the accommodations and days services
* refactor(trips): trip copy through repositories for the roadtrip, accommodation and reservation tables
* refactor(calendar): calendar reads and reservation visibility through one predicate
* feat(server): plan 3e groundwork — packing visibility predicate, memories access-gate audit
* fix(server): plan 3d close-out — legacy text shape for stored accommodation ids restores the date-change restamp; reservation ids parsed once at the gate; parity tests for the reservation repositories
* refactor(photos): trek photos, memories access and cache sweeps through repositories
* refactor(files): trip files and file links through repositories, with transactions added
* test(server): plan 3e — hand-built FilesService, FilesRpc and AllowedFileTypesService calls follow the repository constructors
* refactor(memories): unified, immich and synology photo sync through repositories
* refactor(collab): notes, polls, messages and links through repositories
* refactor(budget): budget items, members, payers and settlements through repositories
* refactor(todo): todo items through repositories
* refactor(packing): packing items, bags and templates through repositories
* feat(server): plan 3f groundwork — systemNotices DI shape, TRAVELER_OWNS predicate, ntfy/storage-health specs, composite-PK pins
* refactor(school-holidays): country and region catalog through repositories
* refactor(notifications): in-app notifications and channel preferences through repositories
* refactor(notifications): reminder crons, mailer, ntfy and webhook transports through repositories
* fix(server): plan 3e close-out — repository registrations restore the boot; packing assignee and bag updates parse and no-op as legacy; coverage and scoping ratchets for the budget, packing and collab repositories
* refactor(vacay): vacation plans, entries, shares and holiday calendars through repositories
* refactor(system-notices): notice dismissals through a Nest provider and repository
* refactor(atlas): visited countries, regions and bucket list through repositories
* feat(server): plan 3g groundwork — GALLERY_CHRONOLOGICAL_ORDER Kysely ordering, unixEpochToIsoKysely helper, composite-PK DDL pins, JB5/R7 confirmations
* refactor(journey): journey and trip access control, CRUD and the trip-sync engine through repositories
* feat(server): plan 3h groundwork — date-offset dialect helpers, entity/repository fitness confirmations, pre-init and encryption pattern confirmations, survivor re-grep
* fix(server): plan 3f close-out — ntfy operator-scope test on the live send path; vacay date-offset and shift-order parity; todo reminder window; coverage and parity ratchets for the atlas and vacay repositories
* refactor(journey): journey stats, entries, photos, contributors and suggestions through repositories
* refactor(journey): public share links, TREK Studio books and the journal facades through repositories
* refactor(collections): access model, hydration, export/import and list CRUD through repositories
* refactor(server): journey-table survivors in trips, photos, memories, auth-cleanup and plugin-contributions through repositories
* refactor(integrations): dawarich connection, sync, tracks and review-flow through repositories
* refactor(collections): saved places, copy-to-trip, labels and invitations through repositories
* refactor(integrations,booking-import,airports,maps,route-usage): airtrail, booking import, airport backfill, maps cache and route-usage counters through repositories
* refactor(doc-sync): connections, reconciler and controller-SQL relocation through repositories
* test(integrations): add AirTrail saveSettings tx-asymmetry ratchet (Plan 3h Task 4 addendum)
* fix(server): plan 3g close-out — transactional book first-save, skeleton reconcile and share-link create; unknown-key entry patches are no-ops again; parity and authorization pins for the journey repositories
* feat(server): plan 3i groundwork — AuditLogRepository's reserved methods, the demo connection-swap spike, repository-fitness confirmations
* feat(server): plan 3j groundwork — scheduler onto CronRegistrarService, SDK-contract confirmation, repository fitness
* refactor(storage): backend/category config and usage stats through AppSettingsRepository; fix the seed-file test race and the shared-replica migration-flip gap
* refactor(share): public share links, the pre-init photo fallback, and collection-places survivors through repositories
* feat(server): plan 3j — the withCamelCase utility, unwired, pending a contract decision
* refactor(admin): user CRUD, stats, audit log, version check and addon config through repositories
* test(admin): pin createUser's pre-existing non-transactional uniqueness-check race (AD2-4)
* refactor(plugins): the trip-write role gate through a repository, in isolation
* refactor(demo,backup): seed and reset through DemoRepository, live-connection backup statements through MaintenanceRepository, isDemoUserId through an injected DemoService
* fix(server): plan 3h close-out — new-value-wins COALESCE restores stay/reservation propagation; doc-sync state filter truthiness; request-context and parity ratchets for share, doc-sync and the integrations
* refactor(plugins): runtime lifecycle, config CRUD and the uninstall cascade through repositories
* fix(server): plan 3i close-out — DemoModule imported where its service is consumed (e2e restored); admin addons list selects the provider relation; saveDemoBaseline awaited; backup tests on the maintenance repository
* refactor(plugins): outbound OAuth tokens and per-user plugin settings through repositories
* refactor(plugins): install-time gates, the capability-audit hash chain and the registry through repositories
* refactor(plugins): contribution-hook controllers, the small plugin controllers and the plugins-table survivors through repositories
* test(server): plan 4 — collapse mcp tool test harnesses onto the ORM snapshot
* test(server): plan 4 — collapse nest service test harnesses onto the ORM snapshot
* fix(server): plan 3j close-out — serialized audit-chain appends, atomic metadata and scheduler limits, rollback and race proofs for the plugin repositories
* refactor(server): plan 4 — convert the program's orphaned raw-SQL callers
* fix(server): plan 4 — un-skip PARITY-009 (FK deleteRule and index-name parity)
* refactor(server): plan 4 — inline canAccessTrip/isOwner off DatabaseService (trip-domain services)
* refactor(server): plan 4 — inline canAccessTrip/isOwner off DatabaseService (plugins + MCP surfaces)
* fix(server): plan 4 — RULE12 paren-count scanner ignores string literals
The entity generator's RULE12_fixGarbledCheckExpressions CHECK-expression
paren-balance scanner counted every `(`/`)` character regardless of
whether it fell inside a single-quoted SQL string literal. A CHECK
expression like `status IN ('confirmed)', 'tentative')` has a literal `)`
inside its first string value; the scanner would hit depth 0 at that
in-string character and truncate the expression, losing its own real
closing paren. The scanner now tracks single-quote string state (handling
the doubled `''` SQLite escape) and skips parens while inside one.
Latent-bug fix — no shipped entity's CHECK expression has hit this shape
yet (3d's ledger: "nothing affected today").
* refactor(server): plan 4 — unify the trip-access predicate's Kysely copies
TripsRepository.findForViewer/listForUser/activeTrip each hand-wrote the
identical trip-access boolean (t.user_id = ? OR m.user_id IS NOT NULL)
through the Kysely expression builder. Task 7's security review (M1)
evaluated a shared helper and set it aside for time, falling back to the
TRIPREPO-041 cross-method parity test as a regression guard instead.
Extracted the boolean into one generic helper, tripAccessExpr
(_shared/trip-access.ts), following the reservation-visibility.ts
publicReservationExpr precedent: fully generic over <DB, TB> since the
predicate has no correlated subquery, so it composes across
TripSelectKyselyDB and ActiveTripKyselyDB without needing a narrower
fixed-alias shape. The LEFT JOIN itself stays hand-written at each call
site — tripSelectQuery's own docstring already explains why a shared join
helper can't be typed across these differently-joined outer queries.
feeds.service.ts and memories-access.service.ts no longer duplicate this
predicate at all (Task 2's facade-inline sweep already converted both onto
TripsRepository.findAccessible/getFeedTokenIfReachable/
listReachableActiveTrips, which read through the canonical MikroORM
QueryBuilder form, accessibleTripsQuery) — the remaining duplication this
task closes is the three Kysely copies inside this file.
TRIPREPO-041 (64/64 in the file) stays green as the unification's own
regression guard.
* fix(server): plan 4 — trek_photo_cache_meta.cache_key NOT NULL
The column was a bare TEXT PRIMARY KEY, which SQLite does not implicitly
make NOT NULL (unlike an INTEGER PRIMARY KEY rowid alias). Every writer
(TrekPhotoCacheMetaRepository.upsertMeta, the table's only INSERT path)
has always bound a real string — 3e's R5 confirmed this. Appends a new
migration rebuilding the table with cache_key NOT NULL (SQLite has no
ALTER COLUMN, so this follows the same rebuild recipe
Migration20200101020000 uses in the opposite direction) and updates the
entity/generated metadata to match. Schema tightening, not a behaviour
change.
* refactor(server): plan 4 — retire the safely-convertible string-form where conditions (rule 23)
Places.repository.ts's existsInTrip/findInTrip/findRaw/reclaimInputs/
listByTripAndIds and Trips.repository.ts's findPublicForShare all took a
raw `.where('col = ?', [v])` string condition even though their id/trip_id
parameters were already plain `number` (or, for findPublicForShare,
narrowed to `number` here after confirming its one caller,
share.service.ts#getSharedTripData, only ever passes a number sourced
from a Kysely-typed share_tokens row, never an unconverted URL param) —
converted all six to MikroORM's typed `.where({...})` filter object.
The remaining raw string-form conditions in src/db/repositories/** are
NOT converted here: they are the `number | string` raw-bind seam
findAccessible/isOwner's own docstrings describe (trip_id/id threaded
through unconverted from req.params), which structurally requires the
deferred guard-level single toRowId parse to land first — narrowing any
of those methods' signatures now would either break the '0x10'/'007' id
parity those docstrings warn about, or duplicate the deferred item's own
work ahead of it landing.
* refactor(server): plan 4 — trek-photos.repository.ts finishes its rename
3e Task 6 renamed the CLASS (TrekPhotosRepository -> TrekPhotoRegistrationService,
since the name collided with the generated ORM TrekPhotos.repository.ts) but
left the FILE and its test file's names unchanged. Renames both to
trek-photo-registration.service.ts / .test.ts.
The old path is kept as a thin re-export stub (the src/websocket.ts shape)
rather than repointed at every one of its ~20 importers in this pass: several
of those files sit in two sibling tasks' exclusively-owned windows on this
shared branch (Task 3's budget/packing/todo/accommodations/places ripple,
reached through tests/helpers/plugin-host.ts and mcp-test-controllers.ts;
Task 5c's held-back tests/unit/services/{trekPhotoMedia,conflictUpdate}.test.ts)
and are off limits here. Flagged for a follow-up once those tasks land: delete
the stub and repoint every remaining importer at ./trek-photo-registration.service.
* fix(server): plan 4 — plugin discovery's delete+reinsert pairs are atomic
install/discovery.ts#upsert refreshed a plugin's actions and settings-field
descriptors as two independent delete-then-reinsert pairs (DI5-DI8) with no
transaction spanning either pair, let alone both: a crash between a delete
and its own reinsert left that plugin with an EMPTY actions or
settings-field set until the next discovery run, instead of its previous
still-valid rows. Wraps both pairs in one uow.transactional, so a discovery
refresh for a plugin's descriptors is now all-or-nothing.
Deliberate behaviour change, named for the user per the task brief.
DiscoveryRepos grows a UnitOfWork; PluginRegistryService (the other
production caller, registry.service.ts) picks it up the same @Optional()
way PluginRuntimeService already does, with the same guard-throw. Updates
the two hand-constructed test call sites that actually exercise discovery
(discovery.test.ts, registry.test.ts) to pass a real UnitOfWork.
* refactor(server): plan 4 — facade inline sweep part B (rosterUserIds/getPlaceWithTags)
Inline DatabaseService.rosterUserIds onto TripMembersRepository.rosterUserIds
directly in budget.service.ts, budget.mcp.ts and todo.service.ts, and
DatabaseService.getPlaceWithTags onto PlacesRepository.findWithTagsAndRatings
in accommodations.service.ts (placesRepo was already injected there).
BudgetService/BudgetMcp/TodoService collapse their constructors, dropping
the now-fully-unused DatabaseService param (budget.module.ts/todo.module.ts
register TripMembers on forFeature). PackingService keeps DatabaseService as
a vestigial param behind an optional TripMembersRepository with a fallback
in tripRosterIds: an in-flight Task 5c file
(tests/unit/services/conflictUpdate.test.ts) hand-constructs PackingService
positionally without the new repository, so its arity cannot change this
task; the fallback keeps that file's DatabaseService.rosterUserIds spy
working until Task 4 drops the param with the class itself.
* test(server): plan 4 — constructor ripple for the rosterUserIds/getPlaceWithTags inline
Hand-built BudgetService/BudgetMcp/TodoService call sites drop the removed
DatabaseService constructor arg (budgetRepoArgs() now appends a
TripMembersRepository so most spread call sites need no other change);
budget.service.test.ts's rosterHas() helper and svc() factory move from a
spied DatabaseService to a stubbed TripMembersRepository. The
AccommodationsService helper (tests/helpers/accommodations-service.ts) and
its two direct callers drop the same arg. PackingService call sites are
untouched (its new param is optional, so every existing hand-built instance
still resolves rosterUserIds through the DatabaseService fallback).
* fix(server): plan 4 — thread the discovery UnitOfWork through PluginRuntimeService too
Complements the just-landed discovery atomicity fix (DiscoveryRepos.uow):
PluginRuntimeService's own discoveryRepos getter still built the bundle
without a uow field. Threads its already-injected, @Optional() uow through
so its own onApplicationBootstrap/reload/dev-link discovery calls also get
the transactional DI5-DI8 refresh, not just PluginRegistryService's.
No throwing guard here, unlike PluginRegistryService's: several hand-built
partial-DI-graph test instances of this class (boot-registry-order.test.ts,
in a sibling task's held file window) construct it with uow omitted and
exercise onApplicationBootstrap's discovery call directly. Discovery must
never refuse outright just because a test built a partial graph, so it
degrades to the pre-existing untransacted sequence when uow is absent
(DiscoveryRepos.uow's own docstring covers this), the same "boot must
never block app init" defensiveness this file already applies elsewhere.
* fix(server): plan 4 — guard-level single parse: verify + correct the overclaiming verifyTripAccess docstring (L-2)
TripAccessGuard/TripOwnerGuard already parse :tripId exactly once
(Number(), guarded by Number.isFinite) and hand the resolved TripAccess
row to the handler via request[TRIP_REQUEST_KEY], TripOwnerGuard reusing
TripAccessGuard's own lookup when both are on the same route rather than
re-querying — verified by reading both guards directly, no change needed
there.
PlacesService.verifyTripAccess's own docstring separately claimed its
returned `tid` gets "returned... for every downstream call to reuse
(never re-parsing)" (3c's Task 9 review, L-2, carried to Plan 4). Reading
every call site (not assumed) shows this is false: `.tid` has exactly one
reader (PlacesController.requireTrip itself), which discards it — none of
requireTrip's 13 callers read `.tid` off the trip it returns; each passes
the original raw tripId string on to the write/read method it calls next,
which toRowId-parses it again itself. That second parse cannot be
eliminated by this gate alone: the same service methods are called
directly by places.mcp.ts too, an entry point with no requireTrip gate at
all (its own numeric tripId is re-stringified, String(tripId), to call
them), so each method owns its own id validation regardless of entry
point. Threading `tid` all the way through would mean giving every one of
~15 methods a second, number-only call shape for the REST path alone to
use — a real refactor, out of this docstring fix's own scope. Corrected
the docstring to say so plainly instead of repeating the false claim.
* refactor(server): plan 4 — relocate JS7/10/13/14 off the journey-share fallback stub
JourneyShareTokensRepository's own JS7/JS10/JS13/JS14 Kysely reads were a
3g Task 3 fallback (Task 2's JourneyPhotosRepository/JourneyEntriesRepository/
JourneyEntryPhotosRepository were still mid-flight). Now that those
repositories are stable, relocate: JS7/JS10 (photo/asset validation joins)
onto JourneyPhotosRepository (already injected into JourneyShareService as
photosRepo, no constructor change); JS13 (public entry list) onto
JourneyEntriesRepository.listPublicEntries; JS14 (public per-entry photo
read) onto JourneyEntryPhotosRepository.listForPublicJourney, deduped
against that repository's own JP_COLUMNS select list instead of a
hand-duplicated copy. JourneyShareService gains two new @InjectRepository
params (both already registered in journey-domain.module.ts's forFeature),
rippled through mcp-test-controllers.ts and journey-share.service.test.ts
(P04/P05 parity tests now pin the new homes).
* refactor(server): plan 4 — document unreachable ?? default arms after single-row aggregates
3f Task 7's fix wave flagged a cluster of `row?.x ?? default` arms after
unqualified, ungrouped aggregate reads (MAX/COUNT/COALESCE(SUM,0)) as
"same class as OauthClients:147" but left them unexamined per-site. Decided
per site: document, don't delete — TypeScript still types
executeTakeFirst()'s result as `| undefined` even though SQL guarantees
exactly one row for these unqualified aggregates, so the optional chain
can't be removed without a non-null assertion (worse, not safer). Six
sites: JourneyEntries.repository.ts (maxSortOrderForDate JG42,
countStatsPlaces JG67), JourneyEntryPhotos.repository.ts
(maxSortOrderForEntry JG91), JourneyPhotos.repository.ts (maxSortOrder
JG88/JG99), VacayEntries.repository.ts (sumFraction VC1, sumCompFraction
VC2). No behavior change — comments only.
* refactor(server): plan 4 — unify the two user-data-erasure enqueue paths
UserCleanupService.erasePluginUserData (the account-deletion transaction's
own UC2/UC3 enqueue) and PluginRuntimeService.enqueueUserErasure (the
emitUserDeleted sink's post-commit enqueue) independently walked the
identical SELECT id, permissions FROM plugins + JSON.parse + hook:user-data
filter + INSERT OR IGNORE — and both genuinely run for every account
deletion, since every emitUserDeleted call site (auth.service.ts,
admin.service.ts, trip-members.service.ts's guest removal) fires
immediately after deleteUserCompletely.
Extracted the shared filter into enqueueHookUserDataErasures
(nest/plugins/user-erasure-enqueue.ts) and pointed both call sites at it.
UC2/UC3 also come off DatabaseService onto PluginsRepository/
PluginUserErasureQueueRepository in the same change (they were the last
raw statements standing in the way of sharing one implementation) — this
narrows the Plan 3b Task 5 "stays raw, owned by Plan 3j" ruling to UC1
only (plugin_user_config/plugin_oauth_tokens/plugin_oauth_state), which
still stays raw, unaffected. UserCleanupService's orphan-data-directory
scan (a plugin uninstalled with data retained) has no equivalent on the
enqueueUserErasure side and stays there, now via the same repository.
Both enqueue calls still run per deletion (INSERT OR IGNORE keeps this
idempotent) — this unifies the IMPLEMENTATION, not the call count; not
touching the emitUserDeleted event-sink wiring itself was a deliberate,
smaller-blast-radius choice for this task.
* refactor(server): plan 4 — narrow TripMembersRepository.remove/TripsRepository.setOwner to number
Both accepted trip_id: number | string per 3c's carry note (their "stays
loose for other callers" reasoning), pending the guard-level single parse
landing first. Re-checked every caller directly: setOwner's one production
caller (trip-members.service.ts#transferOwnership) already passes trip.id,
a real number resolved earlier in the same transaction; remove's callers
are trips.rpc.ts (num(), already a validated number), trips.mcp.ts (Zod-
typed tool input, already a number) and TripMembersController.removeMember
(the one genuine raw-string caller, :id off the route with no class-level
TripAccessGuard on this controller by design).
TripMembersController.removeMember now parses :id once (toRowId, not
Number() — rule 15's NaN-into-SQL trap) and threads the parsed number to
both its own access check and removeMember, instead of passing the raw
string through twice. TripMembersService.removeMember and both repository
methods narrow to number accordingly; setOwner's WHERE also moves off its
raw-bind escape hatch onto nativeUpdate now that trip_id is typed, and
remove's trip_id half moves onto a typed .where({ trip: trip_id }) filter
(user_id keeps its own raw bind — a different, still-live NaN-safety seam,
unrelated to this narrowing).
TRIPREPO-018 and TMEMREPO-023, which pinned the retired string-bind seam
on setOwner/remove, no longer typecheck against the narrowed signatures
and are removed with an explanatory comment in their place.
* refactor(server): plan 4 — U6 gate-level id parsing for todo item ids
Rule 21's carry from 3e Task 8's review: TodoController.update/.remove
passed the raw :id route string all the way to TodoItemsRepository,
relying on SQLite's column-affinity CAST to match it instead of a typed
filter. Parse :id ONCE at the controller (toRowId, matching Task 8a's
TripAccessGuard/removeMember shape), 404 on a non-canonical id, and thread
the number down through TodoService.updateItem/deleteItem into
TodoItemsRepository.findInTrip/existsInTrip/update/deleteById, all
narrowed from number | string to number. todo.rpc.ts's num()-derived
todoId no longer needs its String() wrapper. tripId stays string | number
— a separate, already-accepted carry (
|
||
|
elmocito
|
9e475f447a | feat(tours): add non-motorized day tour planning | ||
|
Maurice
|
7acf7f5655 |
feat(admin): a daily limit on Google API calls
Admin > Settings > Google options gets a daily limit with today's count beside it. Every call through the Places and Routes clients is counted per UTC day; once the limit is reached the Google key reads as absent until the next day, so search, details and photos fall back to OpenStreetMap the way a keyless install does, and the admin sees that the day is used up. (#1582) |
||
|
RemEx
|
b1c05bca83 |
feat(llm-parse): read a photo, as a booking or as a receipt in Costs
Whether the AI Parsing model reads images is an admin setting now, and it lands with the two features that send a photo, so it has something to control. The setting is `vision: 'auto' | 'on' | 'off'` on the addon config. On Automatic a local Ollama is asked what the model can do (`/api/show`), and a cloud provider means no; Yes and No are applied as set. The old `multimodal` flag is not read: the admin screens wrote it false on every save, so a stored false meant nothing. The per-user toggle keeps its on/off meaning and is read by the extraction now. The lookup is server-side only: capped read, validated body, never throws, cached per server and model. GET /api/llm/capabilities answers it for the signed-in caller, and both photo pickers below ask it. The booking import takes JPG, PNG and WEBP when the model reads images. The photo goes as an image to every provider (image_url, an Anthropic image block, Ollama's native chat with `images`), capped at 1600px, and skips KDE Itinerary. A scanned PDF, which used to reach the model as empty text, has its first two pages drawn by pdf-parse and sent the same way. The rest is the import as it was: background job, review, the file attached to the booking, the linked cost. On a real qwen3.5:4b a train ticket came back as bare clock times until the router gave the model every type's field meanings, and its price as an object inside a string, which read as 4900; both are handled. Costs gets Scan receipt: a dialog laid out like the booking import's takes one photo, read on the import job queue, then reviewed in the existing expense editor pre-filled with merchant, total, currency, day and lines, the photo waiting in Receipts & Invoices to be uploaded on save. No table, no migration, no category guess, no document-type detection. |
||
|
Maurice
|
2d7cb124b3 |
feat(documents): two-way sync with Paperless, Papra, Nextcloud, OpenCloud and Synology
A trip can be bound to a folder, tag or space in a self-hosted document store, and documents then move in both directions: what someone uploads in TREK appears there, and what someone files there appears in TREK. Three decisions shape the rest. The connection belongs to the TRIP, not to a person. Every other integration here is per user, and photos are shared only through an opt-in flag; neither can deliver "everyone on the trip sees the same documents", because visibility would depend on whose credentials fetched a file, and a member without their own Paperless account would see nothing. So the trip owner binds the trip once, the server talks to the provider under that one identity, and TREK's own membership decides who sees what. The provider never learns that TREK has members. When the credential holder leaves the trip the binding goes to `orphaned` rather than quietly carrying on with an ex-member's token. TREK keeps the bytes. A synced document is an ordinary trip_files row, so download, trash, offline cache, PDF export and backup keep working untouched, and a provider outage does not blank out a trip that starts tomorrow. Deletion never crosses the boundary on its own. A document that disappears upstream is recorded as missing and waits for a human, and a run that sees more than 40 percent of its known documents vanish at once refuses the whole listing rather than acting on it: an unmounted share answers exactly like a mass delete. The core reconciles by enumerating both sides, because the weakest provider sets the model. A changed-since query cannot see a deletion, Papra does not move updatedAt when a tag changes, a Paperless bulk edit leaves `modified` alone, and Synology FileStation has no change feed at all. A webhook is therefore only ever "look now", never a source of truth. Echo is caught by comparing content, not timestamps: pushed_sha256 records what TREK itself wrote, so a provider reporting TREK's own upload is not mistaken for a stranger's edit. Provider differences are answered by capability data rather than by branching on an id: stable ids where they exist (Paperless document id, oc:fileid, driveId) and a normalised path where they do not, replace-in-place where the API offers it, and a content hash from the listing only where the provider really supplies one. Admins switch providers on under Admin, Addons, Documents, mirroring how photo providers hang off Journey, including the cascade and the 409 when the parent addon is off. The per-trip setup lives in the file manager, where the person has the context that makes the folder choice obvious. |
||
|
Maurice
|
218d8baf0a |
feat(server): read visits and recorded routes from Dawarich
A new `integrations/dawarich` domain behind its own addon. Each reader connects their own instance; the API key is encrypted at rest and never returned. Nothing is written back to Dawarich, and no raw position is kept in TREK: the recorded route is fetched per request for the window being looked at and thrown away again. What it produces are suggestions, never facts. A recorded stay becomes a proposed place or journal entry, a country or city becomes a proposed Atlas mark, and a stay near a wish becomes a proposed tick-off; each one waits for the reader to confirm it. Identity is (user, Dawarich visit), so a second sync neither duplicates nor overwrites what was accepted, and deleting the result puts the suggestion back on the list. The sync runs as a job on the central scheduler, one user at a time, guarded against overlapping runs. |
||
|
Maurice
|
41b18f970c | feat(vacay): add manual school holiday calendars | ||
|
Maurice
|
d30b916f9c |
feat(maps): count how much routing an instance actually does
Every route is calculated in the browser against the public FOSSGIS hosts, so the server never sees a routing request and nobody has a number: not how many a real trip costs, not how big they are, not how much already goes to a self-hosted engine. Deciding whether TREK could run its own router without that number is guessing at the bill, and the counters have to be collecting before the question comes up. Counters, not a log. A row is a day, a profile, a request kind and an engine kind with totals on it: how many requests, how many waypoints, roughly how far, how many were refused. No query, no coordinate, no route, no user, no trip. Nothing leaves the instance: its own database holds them and its own admin reads them at GET /api/route-usage/summary. The client tallies in memory and flushes in batches — on a timer, past a threshold and when the page is hidden — because a counter that cost a request of its own would measure the load by adding to it. Routing all funnels through one fetch in RouteCalculator, so that is the only place instrumented. On by default and switchable off with route_usage_enabled, unlike the shadow log next door: that one stores what people typed, this one stores four integers about a day. |
||
|
Maurice
|
e11ae6f056 |
feat(roadtrip): shape the drive — vias, alternatives and stops along the way
The road-trip addon's second half: the parts that let a planned drive be argued with rather than only read. Reshaping. A via records which stop it follows by position, and that position is resolved fresh every time the day is routed — so inserting, reordering or removing a stop moved the ground under every via after it, and the drive silently snapped back to the road it had been steered away from while the handle still sat on the map where it was left. The correction is computed against the day as it stands and written with the same change, in one transaction: a half-applied re-anchoring mixes two numbering schemes in one day. The one judgement that needs geometry — which half of a split leg a via belongs to when a stop lands in the middle of it — is measured on the road actually driven, so a fuel stop added to a bent leg keeps the bend on the side it belongs to. Alternatives. OSRM answers most long legs with exactly one route, which read as "there is no other way" when a perfectly good slower road existed. When it offers nothing, the same drive is asked for again without the motorway, and failing that without the tolls — one question at a time, stopping as soon as one lands. The offer says what it is rather than how much slower. Stops along the way. Everything the corridor search can find is a pause, not a destination: fuel, charging, rest areas, campsites, food and sights all sit inside the leg they were found on, carry their own colour and icon, and are left out of every stop count. That colour is defined once and read by the rail, the corridor list, the map pin and the picker — and it replaced the chains' logos, which turned a corridor full of petrol stations into a row of advertisements and left the brands with no logo on file looking like something else entirely. How long a stop takes was readable in the rail and writable nowhere; new places were given an invented hour that nobody had entered. It is now set from the rail, cleared as a zero because the update statement folds a null into "leave it alone", and new places start with nothing. Both columns are rebuilt on one type scale, anchored to the day plan's own sizes so switching tabs does not change the size of the text beside it. The rail reads as trip totals, day cards and a chain of stops with the drives between them; the corridor column as what to look for, how the search is getting on, and what it found. Six category pills over three lines became one dropdown, which is the space the results wanted. Also: the offered routes are framed on the map when the picker opens (`fitKey` carries no coordinates and each renderer reads it as "the selected day"), road trip mode does not cluster at all so the whole drive stays readable zoomed out, and the drive-time labels finally render in the app's own typeface instead of Leaflet's Helvetica. All of it in both renderers. |
||
|
Maurice
|
b3d69aa760 |
feat(places): record which search result was picked, on this instance only
The corpus behind "is our own index good enough to replace Nominatim". When somebody searches and then picks a result, the client posts the query, the rank and the pick. Nothing else: no user, no trip, no session, and coordinates only to three decimals. Deliberately decoupled from the search endpoint. Correlating a response with a later pick would mean handing out a ticket on POST /api/maps/search and threading it through all fifteen call sites; the client already holds everything the row needs at the moment of the pick. Evaluation happens offline afterwards: export the corpus, run it against a candidate index, count how often the pick came back in the top five. That is why collection can start before such an index exists, which is the whole point, because the corpus needs weeks of real use and the index needs a server, and those two waits should not be serial. Off unless an admin switches it on, and it reads === 'true' rather than !== 'false'. The three older places switches fail open because a migration backfilled a row for installs already using them; nothing uses this one, so an instance that upgrades and never visits the admin panel logs nothing. Reading is admin only, retention is 180 days, and the switch state is on the unauthenticated app-config endpoint on purpose: whether an instance records what its users search for is something a visitor may know before logging in. |
||
|
Maurice
|
fd6381c814 |
feat(api): add a versioned read-only public API for integrations
TREK's REST surface is internal: it answers a session cookie, carries no version, and changes with the client. That is fine for our own frontend and useless for anyone building against us — the Dawarich integration people have been asking for since April needs something it can rely on not moving. So this adds a small one rather than opening the existing 549 endpoints: GET /api/v1/trips the trips the caller can reach GET /api/v1/trips/:id one trip, ?include= picks the sections Auth reuses the trek_ tokens users already manage under Settings, via Bearer or X-API-Key. No new credential type, no new table — the guard calls the same hash lookup the MCP transport does. It deliberately refuses a session JWT and, for now, OAuth bearers: a credential whose scopes this surface does not yet interpret is worse than one it rejects outright. Access is decided per trip against canAccessTrip, the same owner-or-member predicate as everywhere else, and a trip the caller may not read answers 404 identically to one that does not exist — a 403 would confirm the id is real and turn the endpoint into a way to count someone else's trips. The payload is not the rows. Ids, foreign keys and order_index stay inside; accommodation day ids resolve to ISO dates, places arrive in planned order, and every dated thing carries its date so a consumer can join on days rather than on ids it has no way to look up. |
||
|
jubnl
|
76c8c349b1 |
Fold nest mcp (#2017)
* refactor(server): fold @trek/nest-mcp into server/src/nest-mcp * docs(server): point nest-mcp references at server/src/nest-mcp * remove tracked md file |
||
|
jubnl
|
305b113d5f |
Feat/storage driver (#1949)
* feat(storage): storage key validation, driver contract suite, LocalDriver * feat(storage): MirrorDriver composite backend * feat(server): add TREK_UPLOADS_DIR env var for the uploads root * feat(storage): settings-backed registry, StorageService facade, module wiring * test(storage): pin src/nest/storage coverage thresholds * feat(storage): multer upload options factory * refactor(files): spool trip-file uploads through storage * refactor(collab): spool note attachments through storage; normalize files/ prefix * refactor(journey): spool journey media uploads through storage * refactor(trips): spool trip cover upload through storage * refactor(places): spool place-image and collection uploads through storage * refactor(auth): spool avatar upload through storage * fix(server): consolidate allowed_file_types onto AllowedFileTypesService * fix(storage): default defParamCharset to utf8 in the upload factory * test(db): re-point the trailing-migration guard at the trip_files prefix strip * fix(storage): age-gate the boot spool sweep * test(files): key FILE-P02 residue checks on upload size * test(storage): raise src/nest/storage coverage pin * refactor(platform): serve /uploads routes through storage * refactor(files): serve trip-file downloads through storage * refactor(memories): serve photo bytes through storage * refactor(maps): stream place-photo bytes through storage * refactor(share): stream shared place-photo bytes through storage * refactor(journey): serve public journey assets through storage * fix(share): answer place-photo cache miss with 204 like the maps proxy * test(server): close serving-swap gaps surfaced by the full tier * test(storage): raise src/nest/storage branches pin * refactor(place-photos): route place-photo cache through storage * refactor(places): reclaim place images and collection covers through storage * refactor(trips): trip covers through storage * refactor(auth): avatar reclaim through storage * refactor(collab): note-file deletion through storage * refactor(files): trash deletion through storage * refactor(memories): trek photo cache internals through storage * refactor(memories): thumbnail service category-addressed * refactor(memories): immich upload reads through storage; drop uploads-root * refactor(journey): photo deletion through storage * fix(memories): reclaim orphaned trek photo cache objects in the ttl sweep * feat(journey): journey-thumbs orphan sweep and delete-site derived-artifact reclaim * test(server): close slice-4 gaps surfaced by the full tier * test(coverage): raise slice-4 domain pins * refactor(backup): list, existence and delete through storage * refactor(backup): download through storage sendToResponse * refactor(backup): create backup through per-category storage enumeration * refactor(backup): restore and upload-restore through storage * refactor(backup): auto-backup retention through storage * refactor(server): boot dir-creation to driver init * test(server): close slice-5 gaps surfaced by the coverage tier * test(coverage): raise slice-5 backup pin * test(server): resolve async backupFileExists mocks for the tests typecheck * test(server): pin ranged-read stat and zero-byte roundtrip in the storage driver contract * feat(server): TREK_S3_* env wiring — schema, s3 namespace, all-or-nothing boot rule * feat(server): S3 driver core — aws-lite client seam, stat/delete, driver-owned deadlines * refactor(server): type the aws-lite S3 client via s3-types reference * feat(server): S3 driver getStream and list — ranged full-object stat, iterator pagination * feat(server): S3 driver put — bounded-peek routing, Upload inactivity deadline * fix(server): S3 driver Upload monitoring — Transform interposition, stream cleanup * feat(server): env-declared s3-main storage backend; settings-declared s3 rejected * test(server): env-gated S3 driver contract suite against MinIO * fix(server): S3 driver zero-byte put — work around aws4 duplicate Content-Length signing defect * fix(server): scope the zero-byte put workaround to buffered bodies; bind test MinIO to localhost * ci(server): MinIO-backed S3 driver contract job * ci(server): align s3-contract shim comment and MinIO bind with references * docs: S3 storage backend configuration and off-box backup mirroring * docs: restore README formatting, keep S3 additions only * ci(server): make live-MinIO contract runs manual-only (AIStor licensing) MinIO's community line is discontinued entirely; the maintained successor, AIStor, requires a license key that can't safely live in a shared CI runner. The s3-contract CI job is removed; live-provider contract runs against MinIO/AIStor are now a manual, license-gated procedure documented in docker-compose.minio-test.yml's header. * chore(server): final-review polish — stale CI refs, dockerignore key, boundary pins, zero-byte tempfile contract case * expose admin ui * feat(shared): storage backend type registry, wire schemas, category relocation * feat(server): accept settings-declared s3 backends — shared-schema options, decrypt-at-build, registry preview() * feat(server): remove TREK_S3_*/TREK_UPLOADS_DIR — tombstone boot rule, built-in override as the relocation path * feat(server): deriveSecurity.encryptionKeySet and storage secret plumbing (gate, encrypt, unmask, mask, redact) * chore(server): drop storage-secrets import that only Task 8's probe helper will use * feat(server): seed-once storage-config.json boot import — fail closed on any invalid seed * feat(server): registry snapshot() — effective storage config with provenance * feat(server): api/admin/storage GET/PUT — masked secrets, preview-gated saves, wired reload, audits * feat(server): api/admin/storage/test — per-target ephemeral probes, mirror targets probed individually * test(server): e2e coverage for api/admin/storage — guards, managed refusal, envelope, probes * test(server): give e2e harness DBs the app_settings table the storage registry reads at boot * test(coverage): raise pins after the storage admin-config slice * fix(server): final-review polish — encryption-gate wording, all-field sentinel guard, README/e2e notes * test(server): storage slice-1 review follow-ups — mirror s3-replica probe decrypt, all-encrypted seed without key, test-route 401 * feat(i18n): storage admin domain + tab label in every locale * feat(client): adminApi storage endpoints — getStorage, updateStorage, testStorageBackend * feat(client): storage admin draft-document helpers + relative-time util * feat(client): schema-driven storage BackendForm — fields by kind, mask echo, encryption banner * feat(client): admin storage panel — backends list, category map, health strip, one-PUT save * feat(client): storage tab on the admin page * feat(client): storage admin panel and section on the phone shell * fix(client): block renaming a storage backend onto an existing name * feat(server): allow mirrors on any storage category — lift the backups-only rule * feat(i18n): replicas-on-primary mirror strings in every locale * feat(client): mirror fold/synthesize model helpers for replicas-on-primary * feat(client): BackendForm mirror-targets composer; raw mirror type hidden * feat(client): desktop storage panel folds mirrors into their primaries * feat(client): phone storage panel folds mirrors into their primaries * fix(client): strip removed backends from remaining mirror replicas — no phantom refs * feat(storage): retire photos as a configurable category — served-only via the ServedCategory seam * feat(i18n): descriptive storage category names and descriptions in every locale * feat(client): descriptive category names, descriptions and id badges in the storage panel * docs(storage): fix stale nine-category comments after the eight-category shrink * docs: storage configuration via the admin UI and seed file — README + compose * docs(wiki): Admin Storage page and storage cross-links * docs(server): storage domain blueprint entry; TREK_PLACE_PHOTO_DIR tombstone clarifier * refactor(server): drop the storage env-var tombstones — the variables never shipped * docs: storage config is simply new in v4 — drop the removed-variables story * feat(shared): backfill-status and usage wire schemas on the storage admin state * feat(server): MirrorDriver.backfill — prefix-scoped, size-compared, cancellable replica catch-up * feat(server): one-at-a-time storage backfill jobs + the storage events bridge * feat(server): storage usage scan — nightly cron, on-demand service, persisted with computedAt * feat(server): replica_failure admin notifications — events bridge, hourly debounce, channel registration * feat(i18n): replica-failure notification text in every locale * feat(server): storage backfill and stats-refresh admin endpoints * feat(client): adminApi backfill and stats-refresh endpoints * feat(i18n): storage sync and usage strings in every locale * feat(client): desktop storage panel — sync now, usage display, backfill polling * feat(client): phone storage panel — sync now, usage display, backfill polling * fix(client): per-category usage line on the phone storage panel * docs: sync now, usage stats and replica-failure notifications * test(server): admin scope now includes replica_failure, not just version_available Commit |
||
|
jubnl
|
72221a2e45 | feat(auth): sliding session renewal past half the token lifetime (#1927) | ||
|
Maurice
|
3a0604b3f5 |
feat(config): put the operator-owned surfaces behind TREK_MANAGED (#1946)
* feat(config): put the operator-owned surfaces behind TREK_MANAGED The switch landed last release with nothing behind it. This is what it turns off, plus the four fixes that came out of walking every one of these surfaces. Nothing here changes anything for an install that does not set the flag. That is not a hope, it is what managed-inert.test.ts asserts: the mode stays off for any unparseable value, no other variable can turn it on, every settings key stays writable, and splitManagedKeys hands back the same object rather than a copy so no write path can quietly diverge on self-hosted installs only. WHAT THE MODE WITHHOLDS Fourteen routes carry @ManagedForbidden: key validation, static MCP tokens, JWT rotation, the OIDC admin pair, restore and auto-backup, plugin sideload and reload, the Ollama model list and pull, and the SMTP test send. Each names the reason it is not the instance admin's, and the reason is mandatory. Eighteen settings keys are filtered instead of refused. SMTP, the OIDC toggles and the WebAuthn pair share one handler with keys the admin does own, and the llm_* family is reachable through per-user settings, the instance defaults and the addon config alike, so a route-level refusal would be both too coarse and incomplete. A locked key is skipped and reported as managed_keys; the rest of the save goes through. Refusing the request would take the registration toggles down with the SMTP block. GET /api/auth/me/settings stops returning the three API keys in the clear. It is the only endpoint that decrypts a stored credential back to the caller, which is fine when the admin pasted it in and wrong when the operator supplied it. Two multipart routes check in the handler rather than the guard: a guard throws before the parser drains the body and the client gets an ECONNRESET instead of the 403 (PROFILE-015). They still carry the marker so the boot gate sees them. WHAT KEEPS IT FROM DRIFTING validate-managed-routes.ts refuses to boot on a marking that is not on the reviewed list, a list entry whose route lost its marking, or a marking with no reason. The third case has no equivalent in the guard gate next door and is the one that matters most: a blank reason destroys the only record of why a control was taken away, and nothing recovers that later. It found two wrong entries in my own list while I was writing this. managed-keys.test.ts asserts that every key in ADMIN_SETTINGS_KEYS and DEFAULTABLE_USER_SETTING_KEYS belongs to exactly one of two sets. A key added to either list belongs to neither until somebody decides, and that is the failure this prevents: the lists grow, the mode does not. PLACES_API_BASE AND PLACES_API_KEY The nine Places endpoints spell out googleapis.com, so an install that wants those calls to leave through something of its own has no way to say so. One variable, substituted where every call already funnels through. A second one supplies the key, and when it is set the users table is not consulted at all — a per-user key would route around whatever the endpoint counts, and those columns are readable by any admin of the instance. FOUR FIXES THAT ARE NOT ABOUT THE MODE An admin setting somebody's password now bumps password_version and clears their MCP tokens. An admin sets a password for one reason, and until now verifyJwtAndLoadUser kept accepting every cookie the intruder already had: the one action taken to lock them out was the one action that did not. The four OIDC provider calls go through the SSRF guard. Three are worse than they look — the token, userinfo and JWKS URLs come out of the discovery response, so they are chosen by the other end, not by an admin. safeFetchLlm is renamed safeFetchAdminConfigured and keeps its old name as an alias, because it is the variant that allows loopback and LAN: a self-hoster with Authentik on localhost:9000 must keep working. The token exchange follows no redirect at all, since one would hand client_secret to a second host. DELETE /api/admin/users/:id/mfa clears another account's TOTP. The passkey half has existed since passkeys landed; the TOTP half never did, which left "somebody on the trip lost their phone" with no answer short of an operator in the database. Never for the caller themselves. TREK_PLUGIN_PERMISSIONS=off refuses to boot when the install is centrally administered. A warning is right for a machine its owner chose to trust; where the person who set the flag and the people whose data is in the database are not the same person, fail closed. TEXTS Four strings lose the phrase "self-hosted" because they describe somebody else's software — an Ollama endpoint, an AirTrail instance, a service a plugin talks to — and read better without it in both modes. The one string that describes TREK itself keeps it, and the mode picks a second key instead, so self-hosters lose nothing. A grep gate in shared/ keeps the canonical locale honest, because the parity check compares the file set and the top-level keys and never looks at a value: re-wording a string is invisible to CI today, and so is putting it back. * feat(maps): supply the map credential with the instance when centrally administered MAPBOX_ACCESS_TOKEN joins PLACES_API_KEY as an operator-supplied value. SettingsService injects it when the settings are read, and picks GL as the provider if the reader is still on Leaflet, since a token is always present. The key moves from the customer set to the locked set. It is the one locked key whose value is public by design — a pk.* token has to reach the browser to be useful — so this is not about secrecy: a per-user save landing on top of the instance token would only break the map for that user. Both settings fields for it go away with it. * fix(auth): count the operator key when reporting whether Google features work has_maps_key only looked at the users table, but getMapsKey prefers the PLACES_API_KEY env over anything stored there. On an install that supplies the key that way and has an empty column, every lookup worked while the client was told there was no key and hid the search, the autocomplete and the place photos. Found by walking the running instance rather than the code: the map came up, the search box did not. * feat(admin): hide the settings a centrally administered install cannot act on A control that answers 403 is worse than no control: the AI section rendered the refusal as "Request failed with status code 403", which tells an admin they broke something rather than that the decision is not theirs. So the surfaces go rather than being disabled in place: - AI parsing and AirTrail leave the addon list, and the AI addon route now refuses outright. The endpoint, the model and what a document costs come with the instance, so there is no instance decision left — not even whether it runs. - Immich and Synology Photos leave it too. Both connect to a server the reader runs at home, and there is no route from here to one. - The SMTP block, the admin webhook and ntfy targets, and the passkey RP ID and origins follow the same rule: they configure something the operator owns, and a wrong RP ID on a shared parent domain reaches past this instance entirely. - The About tab goes. It is about the project — what TREK is, where to file a bug, where to support it — and is aimed at the person running the software. The version and the link to the source move to the sidebar footer, on both the desktop page and the mobile twin, because AGPL 13 wants that offer wherever people reach the software over a network. - The update check stops asking GitHub and reports nothing available. Separately, the version footers lose the "self-hosted" suffix everywhere, not just here. It read as a label on the software rather than on the install, and the version alone says what it needs to. * feat(config): refuse to boot centrally administered without an at-rest key Measured on a running instance rather than assumed: with ENCRYPTION_KEY unset, the at-rest key falls back to a file in the data volume, and backupService bundles that file into every archive it builds. The archive is downloadable by any instance admin, so the encryption ships with its own key in the same zip. The skip in backupService already handles this — it only omits the file when the key came from the env — but nothing made sure the env actually carried one. Provisioning sets it; forgetting it was silent. validateEnvAtBoot now carries a second kind of check next to the schema: cross- field rules that only apply with the flag set, for combinations where every value is individually valid and the result is wrong. Empty without the flag, so an install that never asked to be centrally administered cannot meet it. * feat(client): add an attachment point for install-specific screens Two empty exports on the client, an empty module on the server, and the four places that read them. Nothing here adds a route, a nav entry or an endpoint — the arrays are empty and the module contributes nothing, which managed-ext.test pins so it stays that way. Read unconditionally, never behind a flag. On the server a conditional import would tie the module graph to an environment variable, and the graph is built once per app while the flag is read live; on the client an empty array costs nothing to spread, so every build walks the same path instead of one build walking a path nobody tests. Anything attached here follows the same rules as the rest of src/nest and src/components: a real module with a controller and a provider, bodies through the Zod pipe, routes under the global guards, tests. * test(shared): move the wording gate out of src so shared typechecks again CI installs shared with its own dependencies only (npm ci --workspace shared), so there is no @types/node and the `node:fs` import inside src/ failed to compile. The tests passed the whole time — vitest supplies the module — which is why this only showed up in the typecheck step. The walk moves to scripts/, next to i18n-parity.mjs, and the spec imports it the same way that one does. It also runs on its own now (`node scripts/i18n-selfhost-wording.mjs`), which is what you want when CI names a line you have to go find. * test(server): repair the call sites typecheck:tests caught tests/ sits outside the build tsconfig, so none of this showed up in a green test run — which is exactly the gap `npm run typecheck:tests` exists to close. Three kinds, all mine: - Seven `import type` paths for RuntimeEnvService carried one `../` too many. Type-only imports are erased, so vitest never resolved them and nothing failed. - VersionCheckJob and PluginsController grew a constructor parameter and eleven hand-wired `new` calls kept passing the old count. The argument arrived as undefined and no assertion happened to read it. - ConditionContext grew `managed`, and the shared context object in the conditions suite did not. The condition also had no test of its own, so it has one now: both directions, since the registry uses `is: false` to keep a notice away and a notice that only belongs on a centrally administered install would use `is: true`. |
||
|
Maurice
|
bfb3bcd63c |
feat(config): mark an install as centrally administered
Some installs are run by somebody other than the person who administers them. The operator owns the credentials, the configuration and the upgrade schedule, and the admin of the instance is a user of the result rather than the one who set it up. Nothing in the app knows that today, so it keeps offering settings whose values are not the admin's to change, and the honest answer to half of them would be a 403 the UI never anticipated. TREK_MANAGED says it once. It is a single boolean and the surfaces that care read it themselves, so adding one never comes back to the config layer. Runtime-toggled, in the same class as DEMO_MODE, rather than a registerAs token: roughly sixty tests mutate env mid-lifetime, and a value frozen per built app breaks all of them. `=== true` on the way in, so an unset, empty or misspelled value means off — a managed install sets it deliberately, and a typo must not quietly take settings away from an admin who never asked for it. The guard keys on a decorator, not on a path. MfaPolicyGuard documents where the two literal path lists it replaced ended up: they stopped being maintained, and every endpoint added afterwards silently fell on the wrong side. A marker on the handler cannot drift away from the handler. It carries a mandatory reason, which is for the next reader of the source rather than for the client, so the marks stay auditable. It registers third, after the auth and MFA guards, so a stranger still gets the 401 those two would have given them instead of a 403 that would confirm the route exists. And it checks the mode before it touches metadata, because it runs on every request of every install and almost all of them are self-hosted. One 403 body, shared by REST and MCP. DEMO_MODE grew two texts for the same condition and neither can be changed now without hunting for the other. The flag reaches the client through app-config so the UI can stop offering what the server would refuse anyway, but it is deliberately not persisted the way demo_mode is: two installs can share a browser profile, and a stale "this one is managed" would take settings away on an install that never set it. No route carries the marker yet. The per-domain marks follow. Unset, which is every self-hosted install, nothing about this changes anything. |
||
|
jubnl
|
954a3ca0a7 | refactor(server): serve /mcp from a Nest controller; delete auth, audit, addons and oauth bridges | ||
|
jubnl
|
1390bfbe06 | refactor(server): move health, discovery and consent-COOP behind the container | ||
|
jubnl
|
b82651bf93 | Merge remote-tracking branch 'origin/dev' into dev | ||
|
jubnl
|
de5d196344 | refactor(server): move the cache sweeps, idempotency purge and airtrail poll in-container | ||
|
Maurice
|
d981719941 |
Serve /ws from a Nest gateway with its own adapter
The transport was a hand-rolled setupWebSocket(server) that index.ts kicked
off with a dynamic import after listen(). It is a @WebSocketGateway now, with
DatabaseService and EphemeralTokenService injected instead of imported, and
the heartbeat owned by a lifecycle hook rather than an interval nobody stops.
The wire protocol does not move. Nest's WsAdapter dispatches on
{ event, data } and every deployed client sends { type, tripId }, so
TrekWsAdapter translates rather than asking clients to change. It also builds
the ws server itself instead of delegating: the base's port-0 branch uses
noServer and performs the upgrade from its own registry, which never consults
verifyClient, so delegating would have dropped the origin check and answered
101 to anyone. The per-socket error listener stays attached first, because an
unhandled one kills the process (#1576), and the flood guard counts every
inbound frame before parsing, because malformed frames are exactly the ones
worth limiting.
Four things had to stay where they were, and each is a silent failure if it
does not:
The socket registry stays module state. Seventeen bridges and
notifications.instance build `new RealtimeService()` outside the container;
rooms on a provider instance would leave their broadcasts going to an empty
map, with no error and no log.
The socket id stays a monotonic integer. The client echoes it as X-Socket-Id
and broadcast excludes the originator with Number(excludeSid). A uuid is NaN,
NaN === NaN is false, and every client would receive its own writes back,
which shows up as drag-and-drop that jumps rather than as an error.
buildApp creates the http server and binds the adapter before app.init(), and
callers take it from getHttpServer(). Nest binds gateways during init, so a
later adapter binds nothing; an adapter given the app binds to Nest's own
internal server, which this process never listens on. Both boot green with no
client able to connect.
RealtimeGatewayModule is imported by AppModule alone. RealtimeModule is
@Global, so a gateway there follows into every hand-assembled TestingModule,
and Nest's SocketModule then loads an adapter none of them set: it reaches for
@nestjs/platform-socket.io and calls process.exit(1). Eighteen e2e harnesses
died that way before the split.
src/websocket.ts survives as a re-export of the three fan-out primitives. 115
test files vi.mock that path to assert on broadcasts.
The two transport suites pass unchanged, and 30 new cases pin the pieces
directly: the pv and MFA gates, the numeric socket id, the plugin announcement
firing before the room check, both sides of the origin allowlist, and the
flood guard counting frames no handler wants.
|
||
|
jubnl
|
399bbda0fa | feat(server): add the schedule wiring | ||
|
Maurice
|
328b5fcb2b |
One Nominatim client for atlas and maps
Atlas throttled to the 1.1s OSM asks for and cached its answers in a 50k map; maps did neither, so five interactive routes fired straight at the service while the two atlas backfill loops politely waited their turn. From Nominatim's side that is one instance ignoring the rate limit. Both go through geo/nominatim.client.ts now. Three things the naive merge would have got wrong: - The shared client sends UA, the instance-identifying User-Agent, not the generic string atlas used. A shared generic UA is throttled harder. - One cursor across five interactive routes and two background loops would have put a keystroke behind however many places the backfill enqueued, so there are two lanes over the one cursor and background yields. - The cache key carries the query shape. Atlas asks at zoom 3 and 8, maps at 18; a coordinate-only key answers a street lookup with a country. The throttle cursor and the cache stay module state, the way the permissions and FX caches do, because the bridges build their collaborators outside the container and a second copy of the cursor is the bug being removed here. GeocodingService owns only the cleanup timer's lifecycle, which atlas used to start as an import-time side effect. Also folds the three haversine implementations into common/geo.ts, keeping the clamped form: asin of a value a hair over 1 is NaN, and a NaN distance fails every comparison it feeds instead of throwing. tests/setup.ts zeroes the interval. Nineteen maps cases drive these paths back to back with real timers and were sleeping nineteen seconds for it. |
||
|
Maurice
|
55da861de6 |
Put the reservation imports behind one door
Two controllers declared the identical @Controller string from two modules and each rebuilt the trip access check its own way: booking-import ran it as a guard chain, integrations ran it by hand inside the handler, and BookingImportService carried a third copy that nothing called at all. They never actually collided - the four file-upload sub-paths and the one AirTrail sub-path are disjoint - so this is not a bug fix. It is one access path instead of three implementations of the same one, and the dead copy is gone. The addon requirement sits on the AirTrail handler rather than the class. Class-level would have 404d the four upload routes whenever AirTrail happened to be off. AddonGuard still runs first in the chain, because a disabled addon has to answer 404 to an anonymous caller before anything answers 401, and ADDON-GUARD-006 pins both halves of that. One deliberate behaviour change: the AirTrail route now gets its trip 404 from the guard, ahead of the body 400, because a guard runs before the pipe. That is the ordering every other trip-scoped controller already has and nothing pinned the other way for this route. The class rename would have stopped the server booting: the body-contract gate keys on ControllerClass.methodName and throws on stale entries. Three entries renamed, and the list re-sorted - GATE-006 caught that the rename had broken the ordering it asserts. booking-import lost the controller from its coverage denominator, so the branch floor was about to fall. It goes up instead: the off-request parse had no case for its catch arm, so a throw would have left the job on 'running' and the widget spinning, and the binary probe behind /api/health/features had no test at all - that one boolean decides whether the client offers booking import. |
||
|
Maurice
|
be8081c3b4 |
Enforce the MFA policy as a guard, and stop 403ing public routes
The last live non-shell middleware verified the JWT and read the users row a second time on every /api request, then threw the loaded user away - two jwt.verify calls and two SELECTs per authenticated request, because it ran before Nest and could not see what the auth guard had already resolved. The guard reads req.user. It also corrects an asymmetry that had grown quietly. The middleware decided on two literal path lists, and every public /api endpoint added after those lists were written - /api/config, /api/help/*, the public journey and share routes, /api/health/features - answered a logged-in user without MFA with a 403 while answering the same request from a stranger perfectly well. Nothing intended that; the lists simply stopped being maintained. Keying on @Public() means the exemption cannot drift from the route again. This is a deliberate behaviour change and the only one here. @MfaExempt carries the reason for the other kind of exemption: the routes that let somebody complete setup. Without them the policy locks out every new account, which is why the middleware had its second list. One ordering trap on the way: global guards run before route guards, so the auth guard standing down on a controller with its own @UseGuards chain left req.user unset when the MFA guard ran behind it, and it waved everything through. AUTH-020 caught it. It resolves the user now without refusing - the declared chain still owns the 401. |
||
|
Maurice
|
ffb73fac20 |
Turn protection from opt-in into default-deny
Nothing in src/nest/** was an APP_GUARD. Every protected route said so itself, 62 times, which means a forgotten @UseGuards was a silent bypass rather than an error. GlobalAuthGuard inverts that: a route is authenticated unless it carries @Public() or @OptionalAuth(), or declares its own guard chain. Reading the declared chain is what makes this behaviour-neutral. Nest runs global guards before route guards, so without that check the global guard would 401 an anonymous request to an addon-gated controller before its AddonGuard could answer 404 - inverting an order four e2e suites pin, and leaking which addons are installed. A controller that declared a chain keeps deciding for itself. It is a new class rather than JwtAuthGuard gaining a Reflector: that guard is constructed positionally in its own test with a context stub that has no getHandler, and the two want opposite behaviour anyway. @Public() takes a required reason, and the reasons are the diff worth reading: the invite token IS the credential, help assets are loaded by <img> which cannot send one, logout has to work with an expired session or nobody can sign out. The real deliverable is the inventory. validateRouteGuards runs in buildApp next to the body-contract gate and refuses to start when a route answers anonymously without being on a reviewed list of 25 - or when a listed one no longer does. An allow list nobody prunes stops being a list. Default-deny found two controllers on its own: SharedController and OidcController both sit in a file next to a guarded controller, so a file-level search for unguarded ones missed them. 37 tests went red and named them. That is the mechanism working before the change shipped, not after. |
||
|
Maurice
|
3d522be0e5 |
Give accommodations one home instead of two halves
The routes lived in reservations/ while the SQL lived on DaysService, so the controller delegated every call across a domain boundary to reach its own data, and neither module owned the thing outright. The seven statements move into AccommodationsService unchanged and the domain gets its own directory, module, DTO, RPC and MCP surface. What stays in days/ is the reorder side: assertNoInvertedAccommodation and resyncAccommodationDays are about re-dating day rows, and the stay is the thing being carried, not the thing doing the carrying. Two edges fall out of this. AccommodationsService drops its DaysService dependency, and DaysMcp drops DatabaseService and PlacesService, because create_place_accommodation was their only user - which lets DaysModule stop importing PlacesModule and closes one leg of the latent Days -> Places -> Assignments -> Days cycle. DAY-SVC-016..026, 029 and 035 move to accommodations.service.test.ts with their case ids intact. The reservations e2e container assembles both modules now: it builds a single domain module on purpose, which is exactly what catches a move like this, and it caught it. |
||
|
Maurice
|
ad312cdd43 |
Give day notes a domain of their own
days/ carried two fachlichkeiten with a full file set each: days.controller, days.service, days.mcp, days.rpc, days.dto next to the five day-notes ones. The note files move out unchanged; because the new directory sits at the same depth, not one import path inside them changed. The two modules deliberately do not reference each other in either direction: dayExists is raw SQL on the days table in the note service, and DaysService reads day_notes the same way, so a module edge would buy nothing and cost a cycle. DayNotesModule has to be registered in PluginsRuntimeModule as well as AppModule - DayNotesRpc owns four of the 113 wire methods and the registry boots with requireTotalCoverage, so leaving it out would stop every test app that assembles the plugin runtime. The days e2e container assembles both modules now: the note routes still nest under the days prefix, and a container that builds only DaysModule would 404 on them while production served them fine. |
||
|
Maurice
|
cc9587676a |
Give the feature-flag endpoint a health module of its own
GET /api/health/features lived in booking-import, which meant the client's
capability probe was reachable only because that domain happened to be
registered. Move it to health/ and give health an actual module instead of
two loose entries in app.module.
The extractor probe follows into a leaf module rather than health importing
BookingImportModule: features() needs isAvailable(), not llm-parse,
reservations, budget, maps and places.
The /api/_nest/{health,me,echo} smoke endpoints go. They proved boot, DI,
the cookie guard and the Zod pipe back when nothing else did; every one of
those now has real domain coverage, and nothing outside their own test ever
called them. The container probe is GET /api/health on the raw express layer
and is untouched.
|
||
|
Maurice
|
e9f9cfb5d5 |
Add the place enrichment endpoint
Fans the new provider primitives out into one answer for a place that is being looked at but not saved yet: a strip of pictures and a description, each with its source. Free sources are the normal case - most instances have no Google key at all, so Commons and the OSM tags have to carry this on their own and Google is what an admin adds on top. Pictures are copied into the existing photo cache under "<placeId>~pN" keys rather than hotlinked: Google's photo URLs expire (migration 107 had to repair exactly that once), and a provider URL in the strip would send every viewer's address to Google or Wikimedia. Whatever nobody picks is unreferenced and the nightly sweep clears it out. The result is cached for a week in place_details_cache under a third expanded value, so no migration. Rate limited per user, and a provider being down answers empty rather than failing the dialog. |
||
|
Maurice
|
9814480a96 |
refactor(server): move the JWT verify into nest/auth, delete four dead middlewares (#1828)
middleware/auth.ts exported seven things. Two of them — extractToken and verifyJwtAndLoadUser — are the canonical session check every auth surface goes through. The other five (authenticate, requireCookieAuth, optionalAuth, adminOnly, demoUploadBlock) are Express middlewares with no production caller left; the guards in nest/auth replaced them and only comments still mention them. The two live functions move to nest/auth/jwt-verify.ts and the file goes, along with middleware/validate.ts (nothing but its own test imported it) and middleware/idempotency.ts (its only caller was `authenticate`, and IdempotencyInterceptor has replaced it). AuthRequest and OptionalAuthRequest go with them — src/typings/express.d.ts is what types req.user now. Eight importers repoint: the three guards, auth/files/plugins-proxy services, platform.routes and the global MFA policy. Two deliberate departures from the plan for this package. JWT_SECRET does not move to app-config. It is not an env value: it is generated into data/.jwt_secret, and the admin panel rotates it at runtime through updateJwtSecret. `export let` is precisely what makes a rotation take effect in-process — a registerAs token would freeze the boot value and silently break rotation. The verify also keeps reading the db singleton rather than DatabaseService. That change is not a one-line swap: it gives the three guards a constructor dependency, which means registering them as providers everywhere they are used. Measured: 38 directories use those guards and 21 of them do not import AuthModule, on top of 34 e2e tests that each build a container around a single domain module. That is its own PR, not a rider on one that deletes dead files. tests/unit/middleware/auth.test.ts covered real behaviour, so it is not simply deleted. Its extractToken cases and the five verify cases it drove through `authenticate` come back as tests/unit/nest/auth/jwt-verify.test.ts, now hitting verifyJwtAndLoadUser directly. Four cases are new: the password_version gate — the whole reason this function exists — had no direct coverage at all. Its three adminOnly cases are dropped because AdminGuard, their successor, is covered by three e2e suites. |
||
|
Maurice
|
e606ebd9b9 |
chore(server): pin the WebSocket envelope, dedupe the rate limiter (#1803)
* chore(nest): register LlmParseModule in AppModule
The module was only in the container because BookingImportModule and
PluginsModule happen to import it, so /api/admin/llm/local depended on those two
keeping that import. Register it where every other module is registered.
* refactor(nest): give the rate limiter one home and one instance
RateLimitService keeps its counters in a Map on the instance, but it was listed
in the providers of auth, oauth, transit and trip-invite, so each of them got its
own. Nothing is broken today — the bucket names in use (login/mfa/forgot/reset,
oauth_*, transit_*, trip_invite) don't overlap — but two modules picking the same
name would have quietly allowed twice the configured attempts.
Move it to nest/common, where it belongs now that four domains use it, and hand
it out through a small module they import. Not @Global: the e2e suites build a
container around one domain module, so a global AppModule never pulled in isn't
there at all.
transit.mcp.ts keeps its own instance on purpose — it keys by userId under
mcp_transit_* while the REST path keys by IP.
* test(ws): pin the wire envelope before touching the transport
Every frame on the socket is flat: {type, tripId, ...payload} out, {type, tripId}
in. Nest's WsAdapter dispatches on message.event and answers {event, data}, so
moving this onto a gateway without a custom serialiser would leave join silently
unhandled — no welcome.socketId to send back as X-Socket-Id, and every client
would then receive its own writes.
Nothing catches that today: the 100+ tests that touch realtime mock src/websocket
and never look at the bytes. WS-032 sends the exact shape the stock adapter would
and asserts the server ignores it.
* docs(nest): drop the parts of the README that no longer exist
strangler.ts, DEFAULT_NEST_PREFIXES/NEST_PREFIXES and tests/parity/ are all gone —
every request goes through Nest now — but the README still walks a new contributor
through adding a prefix and writing a parity test against the Express route.
Also spell out that the 80% gate is an average over src/nest/**, not a floor per
file, so a large untested module can sit behind the well-covered small ones.
|
||
|
jubnl
|
a2861d40f3 |
feat(server): injectable RealtimeService over the ws singleton
Nest services inject RealtimeService (call-time delegation to the websocket module, DatabaseService pattern) instead of importing broadcast/broadcastToUser globals. Zero behavior change; test edits are constructor-arg additions plus RealtimeModule in e2e harness imports only. |
||
|
jubnl
|
65fb00d63b | feat(server): type MCP access groups against SCOPES + boot gate |