Commit Graph

1298 Commits

Author SHA1 Message Date
Maurice e316ccf3bb perf(db): the calendar export, the journey reads and the stop reorders read and write in batches
The ICS export reads the stops and notes of all dated days in two statements instead of two per day, the journey's full read and entry list take the source trip titles in one join instead of one read per entry, and reordering a day's stops or sorting it by time writes every position in one CASE update. Results are unchanged: each day keeps its order (the id settles a tie), a missing trip still names null, and an id listed twice keeps its last position.
2026-10-10 20:15:56 +02:00
Maurice 4a3c97e500 docs(server): name the query API per statement shape and hold QueryBuilder calls to a baseline
server/CLAUDE.md now says which API new repository code uses: Kysely for join, aggregate and set-based statements, the entity methods for single-table CRUD, no new QueryBuilder calls. The TrekRepository docstring no longer calls Kysely an escape hatch. lint:query-api counts this.qb()/createQueryBuilder() per repository file against a baseline that may only shrink and runs in the server CI job.

The same file documents the JSON column and timestamp rules from the two changes before.
2026-10-10 20:15:56 +02:00
Maurice 91deedae8b refactor(db): JSON columns are decoded through one declared schema per column
decodeJson/decodeJsonResult/encodeJson read and write a JSON-in-TEXT column against the schema and fallback declared for it in db/json-columns.ts, and log a row that does not decode. reservations.metadata, the OAuth scope and redirect lists, consents, API token scopes, MFA backup codes, journey entry tags and pros/cons and poll options go through it. Readers that used to throw on a broken row (a hotel update, the ICS export, the price mirror, the OAuth token and session reads, a consent, a journey entry, a poll) now take the tolerant fallback the other readers of the same column already used.
2026-10-10 20:15:55 +02:00
Maurice a926990e4a fix(db): timestamp columns are written and stored in one canonical spelling
A DbTimestamp brand, produced only by dbNow(), types the expiry writes of OAuth tokens and password reset links, and collection edits stamp updated_at the same way. parseDbTimestamp() reads the stored text as UTC and replaces the oauth-local parser; utcSuffix lives once beside it. A data migration rewrites the ISO values those writers left behind, and the OAuth session lists keep answering the expiries in ISO.

Two outputs change on purpose. collections.updated_at used to come back in two spellings: a collection never edited answered the column default (2026-01-02 03:04:05), an edited one the ISO text its writer stored (2026-01-02T03:04:05.678Z). It now answers the canonical text in every case, in the PUT and cover responses and, once the migration has run, in every read; the shared schema types it as a plain string and the client does not read it. The ISO expiries of the OAuth session lists are rebuilt from the stored second-precision text, so they end in .000Z instead of carrying the milliseconds of the write.
2026-10-10 20:15:55 +02:00
Maurice 1ea435d730 ci: server and shared lint stop rewriting files, and Prettier is checked per file
shared's lint ran eslint --fix with prettier/prettier as an error, so in
lint-prettier.yml the format:check after it only ever saw files it had just
rewritten, and the server's format scripts pointed at a test/ directory that
does not exist. Now lint is check-only in both (lint:fix applies fixes), the
server scripts cover src/ and tests/, and a lint:format ratchet on the shared
library holds every file outside scripts/format-baseline.json to Prettier in
both workspaces: in the server job of test.yml and in lint-prettier.yml,
which keeps providing the lint check. Formatting leaves ESLint in shared, and
the unused eslint-plugin-prettier goes. Once the tree is reformatted in one
commit, --update empties the baselines and the check is a full prettier
--check.
2026-10-10 20:15:54 +02:00
Maurice cfef8742f4 docs(server): the Nest README is a blueprint again, and guides may only name paths that exist
server/src/nest/README.md drops from 988 lines to under 200: the module
shape with weather as the reference, the cross-cutting pieces, the guards,
parity and the test layout, without the bridge playbook and the services
recipe that contradicted the code. The migration log moves verbatim to
server/docs/nest-migration-history.md, marked historical.

scripts/ci/doc-paths.mjs resolves every backtick path in the CLAUDE.md files,
the root README, the blueprint and the wiki against the tracked files. The
paths already missing are listed in a baseline that only shrinks, and the
changes job runs it so docs-only changes are checked too. The wiki no longer
says the Playwright flows stay out of CI.
2026-10-10 12:01:07 +02:00
Maurice 7ee8333aeb refactor(server): types.ts keeps only imported types and derives Trip from the schema
Thirteen interfaces nobody imported are gone. Trip is now the selectable row
of the generated TripsTable, so currency, is_archived and reminder_days read
as the nullable columns they are and feed_token and reminder_sent_for exist.
2026-10-10 11:52:07 +02:00
Maurice db8babffc7 chore(server): remove the MCP config re-export and the dormant row-shape helper
Only their own tests imported them. The MCP knobs are parsed in
app-config/parsers.ts, which tests them directly, and toCamelCase was never
wired into a plugin RPC handler.
2026-10-10 11:48:59 +02:00
Maurice 2b218a5528 refactor(server): drop the state helpers the injected stores replaced
setPermissionsCache and the free leaveRoom, leaveAllRooms and
leaveAllBooks lost their last caller when the services moved onto the
injected stores.
2026-10-08 21:15:17 +02:00
Maurice 68504b337f refactor(scheduling): take a cron lease in one conditional upsert
acquire() made sure a row existed and then raced on a conditional UPDATE,
two statements the registrar wrapped in a transaction of its own. lint:tx
cannot follow a repository reached through a local variable, so it
reported the method. One INSERT ... ON CONFLICT DO UPDATE ... WHERE now
takes a free lease, renews the holder's and leaves anyone else's alone,
atomically on SQLite and on Postgres, and needs no transaction.
2026-10-08 19:05:03 +02:00
Maurice fac139723e build(db): generate the Kysely types for the lease and session tables
Both tables arrived in migrations written before the generator existed,
so src/db/kysely/ did not describe them and check:db-types would fail.
Regenerated with gen:db-types.
2026-10-08 19:05:03 +02:00
Maurice be751ad521 fix(auth): start no new session for a password change over a Bearer token
A password change ends every session of the user and re-issues one for
the device it was made on, handed out as the session cookie. A caller
that authenticated with a Bearer token never reads that cookie, so the
new session sat in the session list as a live device nobody held until
it expired.

The controller now asks for the replacement session only when the
request came with the session cookie, the same check the sliding
renewal makes. The response body is unchanged.
2026-10-08 18:44:08 +02:00
Maurice 648f63c626 fix(sessions): keep an ended session's row until it expires
A session renewed from a token issued before sessions were tracked has
an id derived from that token, and the renewal inserts it only when no
row by that id exists. The nightly purge deleted revoked rows straight
away, so the next request with the still valid old token inserted the
session again as an active one and got a fresh tracked token with a new
lifetime. A logout, an ended session or a sign out of the others held
only until the purge ran.

The purge now deletes expired rows only, and a revoked row stays until
its own expiry. A derived row is recorded with an expiry no earlier
than the old token's, so by the time it goes the old token is refused
by its signature check anyway.
2026-10-08 18:44:08 +02:00
Maurice 0dde01baa2 fix(oidc): sign in after SSO on a success flag, not the deprecated token
The token in the sign-in bodies was declared deprecated and "read by
nothing of ours", but the web client still decided an SSO sign-in by
reading data.token from GET /api/auth/oidc/exchange, an endpoint the
deprecation notice left out. Removing the field later, as documented,
would have broken SSO sign-in.

The exchange now answers { success: true, token } and the login hook
signs in when the response is OK and success is true, so it never reads
token. The field stays for API clients and is now on the deprecation
list in the API docs, the controller comments, server/CLAUDE.md and the
wiki (Security-Hardening, OIDC-SSO). Adding a field to the body breaks
no existing client.
2026-10-08 18:44:00 +02:00
Maurice b872bb48a6 refactor(auth): check sessions through SessionsService, not its repository
The sessions domain owns user_sessions, yet AuthService injected
UserSessionsRepository next to SessionsService and AuthModule registered
the entity in its own forFeature, only so verifyJwtToken could hand the
repository to verifyJwtAndLoadUser.

verifyJwtAndLoadUser now takes a SessionLookup, the two repository
methods the session check uses (findActive, touchLastSeen).
SessionsService answers both, so AuthService passes itself through the
sessions domain's API and drops the repository and the forFeature entry.
The guards keep handing in the repository from their own entity
manager, which satisfies the same type. auth.service.ts lost lines, so
its size baseline goes down with it.
2026-10-08 18:43:50 +02:00
Maurice ac89895978 fix(sessions): renew a token from before tracking into one session
A token without a jti past half its life is renewed on every request
its cookie authenticates, and each renewal minted a session with a
random id. A first page load sends many requests at once, so one browser
ended up with several active sessions; only the last cookie survived,
and the rest showed up in GET /api/auth/sessions as separate devices
until they expired, up to 30 days for remember me.

The interceptor now hands the token to SessionsService.renew, which
derives the session id from it (a SHA-256 of the token, laid out as a
version 8 UUID so it passes the same id check) and inserts the row only
if it is not there yet. Every renewal of the same token lands on the
same session; if that session was ended in the meantime nothing is
renewed, just as for a tracked token. A caller that passes no token gets
the old behaviour.
2026-10-08 18:43:42 +02:00
Maurice 1ad6ae0ebc fix(auth): keep demo visitors out of each other's sessions
On a demo instance every visitor signs in as the one shared demo
account, so its sessions are other people's browsers. The session
routes had no demo check, unlike the password, MFA and account
changes, so any visitor could read the User-Agent of everyone else
signed in and sign them all out with revoke-others.

For the shared demo account in demo mode, GET /api/auth/sessions now
lists only the session the request came with, and DELETE
/api/auth/sessions/:id and POST /api/auth/sessions/revoke-others answer
403 "Sessions cannot be ended in demo mode." without ending anything.
Logout still ends the visitor's own session. Other accounts on a demo
instance keep the full routes. No MCP tool covers sessions, so there is
no parallel tool to change.

The demo sign-in now records the device's User-Agent on its session
like every other sign-in, which the session list needs to tell the
visitor's own entry apart.
2026-10-08 18:43:42 +02:00
Maurice 474beb0dfb fix(sessions): carry the signed-in sessions across a demo reset and a restore
Both swap the whole database file. The demo reset copies a baseline
saved at first seed, before anyone logged in, so it brings back an empty
user_sessions table; a backup restore brings the backup's own rows, or
none for a backup from before sessions were tracked. A session token
whose row is missing is refused, so the hourly reset signed every demo
visitor and the demo admin out, and the admin who ran a restore got a
401 on the next request. Before session tracking only the password
version applied, and those tokens survived.

Both paths now read the active rows with their owner's email before the
swap and put them back after the reopen, in a fresh fork, for every
account the new file still holds under the same id and email. A row the
new file already has is left as it is, so a session that ended there
stays ended, and a session whose id now names somebody else (a backup
from another install) is dropped instead of handed to them. The password
version check still refuses a token the restored account no longer
matches. Reading or writing them is best effort and never fails the
reset or the restore.
2026-10-08 18:43:42 +02:00
Maurice 9aab9e7513 feat(auth): track sign-in sessions so a session can end before it expires
A session token was a signed JWT and nothing else. Logout only cleared
the cookie, a 30-day "remember me" token stayed valid wherever a copy of
it had ended up, and nobody could sign out one device short of changing
the password or rotating the JWT secret for everyone.

Every session token now carries an id (jti) naming a row in the new
user_sessions table (id, user_id, created_at, last_seen_at, expires_at,
revoked_at, user_agent cut to 256 characters; no address is kept). The
rows live behind UserSessionsRepository and the new sessions domain,
whose SessionsService is the one issuer: AuthService.generateToken and
the OIDC sign-in both go through it, and sliding renewal re-signs a
tracked token under the same id and moves its expiry. verifyJwtAndLoadUser
takes the repository and refuses a token whose session is revoked,
expired or gone, refreshing last_seen_at at most every five minutes.
A token without a jti was issued before this change; it passes until its
own expiry (or a password change), so the upgrade signs nobody out, and
renewal turns it into a tracked session.

Sessions end on logout (the one the request names, verified even when
expired), on a password change (all, the device gets a new one), a
password reset, an admin setting a password, reset-admin.js, an MFA
disable (all but the current one), an admin clearing a user's MFA, and
with the account. GET /api/auth/sessions lists the active sessions,
DELETE /api/auth/sessions/:id ends one and POST
/api/auth/sessions/revoke-others ends all but the current, with their
contracts in @trek/shared and two new audit actions. There is no UI for
them yet. MCP exposes no account or session tools, so there is no
parallel tool to change; an MCP call made with a session JWT as bearer
goes through the same check.

The token in the login, register, demo-login, MFA and passkey bodies
stays for API clients but is marked deprecated in the API docs and in
the client type, which never read it.

A nightly session-purge job on CronRegistrarService deletes the expired
and revoked rows. The migration is 042500. The sessions domain gets its
own coverage entry at the 80 floor until a full run measures it.
2026-10-08 18:43:06 +02:00
Maurice 094e0b28f8 fix(booking-import): write an imported venue with its booking in one transaction
confirm() created the venue's place row in its own transaction and
broadcast place:created before the booking and its cost were written in
a second one. When that second write failed, the item was skipped but
the trip kept a place no booking pointed at, and clients had already
been told about it.

Per item, the venue and endpoint geocoding, the day lookups and the
cost's frozen rate still run first, outside any transaction. The place
row, the booking and its linked cost then go into one transaction
through the new ReservationsService.createWithCostInTx, the core that
createWithCost itself now wraps. place:created, reservation:created,
accommodation:created, the stay mirror and the cost events are sent
only after the commit. Each item keeps its own transaction, so a
failing item still leaves the others imported; confirm() is tagged
@txIndependent for that and leaves the lint:tx baseline.
2026-10-08 18:42:10 +02:00
Maurice 9cebadf67c fix(memories): save the Immich connection and auto_upload in one transaction
PUT /api/integrations/memories/immich/settings saved the connection through
one service call and then wrote auto_upload through a second one from
the controller, each committed on its own. A failure between them left
a new connection with the old upload choice.

saveImmichSettings takes the auto_upload flag and writes it with the
connection (or the disconnect) inside one transaction, after the SSRF
check that does the DNS lookup. An absent or non-boolean flag still
keeps the stored choice, a refused URL still writes nothing, and the
response is unchanged. The controller's write sequence leaves the
lint:tx baseline, and the MemoriesService facade loses the
immichSetAutoUpload pass-through it no longer needs. No MCP tool writes
these settings.
2026-10-08 18:42:10 +02:00
Maurice f40d92a69e fix(trips): write the budget currency rebase with the trip row
TripsService.update rebased the budget onto the new currency in its own
transaction and only then wrote the trip row in a second one. When the
day grid rebuild, the booking resync or the leave shift threw after the
rebase had committed, budget items and settlements stayed pinned and
re-frozen against a currency the trip never took (#1543).

rebaseTripCurrency is split into prepareCurrencyRebase, which fetches
the rates before any transaction opens, and applyCurrencyRebase, which
reads the outgoing currency and writes the rebase. update() fetches
first and hands the plan to updateTrip, which applies it inside the
transaction that writes the row, the leave shift and the day grid.
updateTrip called without a plan (the plugin RPC host) still never
rebases, and rebaseTripCurrency keeps working as before.

The updateTrip docstring no longer claims the row is written outside
the day regeneration transaction. TripsService.update leaves the lint:tx
baseline, and the size baselines shrink with the files.
2026-10-08 18:42:10 +02:00
Maurice 615a0b14f5 fix(reservations): roll a booking write back when its linked cost fails
createWithCost and updateWithCost run the booking and its cost in one
transaction, but the cost sync still caught and logged its own errors
inside it. A price that could not be written therefore committed the
booking edit and left the linked expense on its old figure; only the
category re-file path actually rolled back.

The sync no longer swallows a failing cost write. Both WithCost methods
are its only callers, so the error now reaches their transaction and
the booking and its cost commit or fail together. A booking import
item whose cost cannot be written is skipped like any other failing
item instead of landing without its cost.
2026-10-08 18:42:10 +02:00
Maurice c7feaf0a65 fix(reservations): link a cost created on a booking update on its insert
syncBudgetOnUpdate created the expense and then set its reservation_id in a
second statement, inside a try that logs and swallows the failure. Running
inside updateWithCost, a failed back-link committed an expense that was not
linked to its booking. The expense is now created through
linkBudgetItemToReservation, which writes the link on the insert, the same
way syncBudgetOnCreate does.
2026-10-08 18:42:09 +02:00
Maurice b84867b4c2 fix(mcp): update bookings through updateWithCost like REST and the plugin RPC
update_reservation and update_transport accept a type but still called the
bare ReservationsService.update. Only updateWithCost re-files a linked expense
that still carries the category derived from the old type, so changing a
booking from flight to train over MCP left its expense filed under flights
while the same change over REST or RPC moved it. Both tools now call
updateWithCost without a price entry, and send the cost events and the
accommodation change after the commit, as REST does.
2026-10-08 18:42:09 +02:00
Maurice 36a0d1eaba fix(auth): erase plugin user data in the transaction that deletes the user
erasePluginUserData deleted the host-side plugin rows and then queued the
plugin erasures as separate commits, and deleteGuest ran it before its own
transaction. When the guest delete failed, the guest kept its account but had
lost its plugin rows, with erasures queued for a user that still exists. The
erasure is now one transaction, and deleteGuest runs it inside the same
transaction as the budget re-split and the user delete, so it becomes a
savepoint of that write. Only DB writes and a synchronous readdir run inside.
2026-10-08 18:42:09 +02:00
Maurice b7e13e4af6 fix(booking-import): write an imported booking with its cost in one transaction
Confirming an import created the booking, then froze the FX rate and created
the linked cost in a second commit, so a failed cost write left the booking
without its expense. REST, MCP and the plugin RPC already go through
ReservationsService.createWithCost. The import now builds the cost entry and
freezes its rate first, outside the write since that can fetch rates, then
calls createWithCost and announces the cost after the commit. A rate that
cannot be frozen keeps the booking without its cost, as before. The place a
venue becomes still commits on its own, with endpoint geocoding between it
and the booking, so confirm stays in the lint:tx baseline at 4.
2026-10-08 18:41:57 +02:00
Maurice 60a42cfac1 fix(airtrail): import a flight with its AirTrail link in one transaction
The import created the booking and wrote its external_* link as two commits.
A failed link left the booking in the trip while the import reported the
flight as skipped. The flights are fetched before the loop, so nothing between
the two writes waits on the network: each flight or joined connection now goes
in with its link in one transaction, and the method is marked as one
transaction per imported booking instead of being held in the lint:tx baseline.
2026-10-08 18:41:57 +02:00
Maurice 8018f10273 feat(server): gate writes outside a transaction with lint:tx
The rule that a multi-statement write runs in uow.transactional was
written down but nothing held it. scripts/tx-writes.mjs reads every class
under src/ with the TypeScript parser and counts, per method, the writes
it issues outside a transactional callback: repository calls that build
an INSERT, UPDATE or DELETE, calls to methods that write (on the class
itself or on a constructor-injected one), and a transaction holding
writes as one. A write in a loop counts twice, of two branches only the
larger counts, local helpers count where they are called, and a helper
whose every call sits inside a transaction is not reported. --explain
lists the line of every call that counted.

Today's 102 reported methods sit in scripts/tx-writes-baseline.json and
may only shrink; most are syncs with network I/O between their writes,
plugin lifecycle steps and adapters calling two service writes. A new
method fails CI, and an entry above what its method holds fails until
--update lowers it, the same ratchet as lint:size and lint:boundaries.

Two JSDoc tags mark the deliberate cases with their reason in the tag:
@txStandalone for a write that stands on its own whatever the caller does
(the audit row, notification sends, the booking price mirror, the
assignment journey refresh, the quota retention purge), @txIndependent
for writes complete one by one on purpose (the OAuth token retention
sweep, the notification claim and its release). Runs in CI after
lint:boundaries; documented in server/CLAUDE.md, the root CLAUDE.md and
the wiki's dev setup page.
2026-10-08 18:41:57 +02:00
Maurice 9dd2e1b983 fix(server): take MAX+1 positions inside the transaction that inserts
Days, todo items, packing items and bags, packing template categories and
items, collections, collection labels and road-trip vias took their next
position as MAX(...) + 1 in one statement and inserted in the next, with
any other request free to run in between: two adds at once read the same
MAX and landed on the same position (the road-trip test pinned exactly
that duplicate). The read now sits in the transaction that inserts, and
the transaction holds the connection, so the second add waits and takes
the next position. A collection label's limit and name check move into
the same transaction, so two labels of one name added at once end in one
label and a 409.

Each site gets a test that adds several rows at once and reads back one
position each.
2026-10-08 18:41:32 +02:00
Maurice e015e5e5e8 fix(server): run the remaining multi-statement service writes in one transaction
A sweep of the services for write sequences that committed step by step,
each one leaving half a change behind when the next statement failed:

- accommodations: the linked bookings' times moved after the stay's own
  transaction had committed
- airports: the boot backfill wrote a flight's two endpoints one at a
  time, and a flight left with only the first was never picked up again;
  each flight is its own transaction now
- auth: the admin settings form and a password reset (retire the old
  link, issue the new one) each land whole
- oauth: a refresh revoked the old pair before issuing the new one, so a
  failure in between logged the client out; rotating a client secret and
  revoking its tokens go together too
- oidc: the identity link, the role from the claims and the avatar of a
  returning user are one write
- addons, notification preferences: a settings save lands whole
- places: create with its tags, update, and the GPX/KML imports with
  their track colours
- trips: the trip row, the shifted leave entries and the rebuilt day grid
- vacay: a first visit provisions the plan in one transaction with a
  second look against a concurrent first visit; updatePlan fetches the
  public holidays between two transactions instead of inside its writes
- memories: a photo and its trip link, and the deletes with the orphan
  sweep behind them

AddonsService, AirportsService and OauthService take the UnitOfWork for
this. Network I/O stays outside every transaction. Comments that only
narrated the migration plan make room under the lint:size entries, which
drop with them.
2026-10-08 18:41:32 +02:00
Maurice f76538b2b2 fix(plugins): uninstall a plugin and record its install in one transaction
Uninstall deleted the plugin row and then up to fourteen tables of rows
about it one by one. A failure halfway left grants, scheduled tasks or
per-user settings behind for a later plugin that reuses the id to
inherit. The rows now go in one transaction, and the data directory is
removed after the commit instead of before the rows that describe it.

A registry install records the discovered row, its provenance, the
pinned author key and the cleared update block together, so a failure
never leaves a registered plugin without its pinned key. Starting an
OAuth connect drops the stale state and stores the fresh one together,
and a disconnect drops tokens and state together; PluginOAuthService
takes the UnitOfWork for that. The scheduler's per-task claims stay
separate on purpose and say so for the transaction gate.
2026-10-08 18:41:06 +02:00
Maurice f6076328fe fix(journey): create a journey with its owner, trips and photos in one transaction
createJourney inserted the journey, then its owner row, then linked each
trip with its skeleton entries and set the cover, every step committed on
its own: a failure halfway left a journey nobody owned. It now runs as
one transaction and broadcasts the linked trips after the commit.
addTripToJourney links inside a transaction the same way.

The photo paths wrapped only the gallery insert and left the entry link
and the skeleton promotion outside; the photo row, the gallery row, the
link and the promotion are one write now. removeTripFromJourney,
updateEntry, updatePhoto, setPhotoProvider, the photo deletes and the
place hooks write in one transaction each.

createEntry and uploadGalleryPhotos read MAX(sort_order) inside the
transaction that inserts, so two entries or uploads at once take one
position each. reconcileTripSkeletons and the photo capture helpers are
tagged as deliberately split for the transaction gate that follows.
2026-10-08 18:41:06 +02:00
Maurice 5cbd9e9d36 fix(reservations): write a booking and its linked cost in one transaction
A booking's create and update ran their own transaction, and the linked
cost sync ran afterwards from the REST controller and the plugin RPC
handler, so a failing cost write left the booking edited and its expense
on the old category or price. MCP linked the cost in a second call of its
own the same way.

ReservationsService.createWithCost and updateWithCost now run the
booking write and the cost sync in one transaction and hand the cost
events back, so REST, MCP and the plugin RPC call the same method and
broadcast only after the commit, in the order they did before. The cost
sync keeps swallowing its own failures the way the REST route always
has, so a cost that cannot be written still leaves the booking; MCP now
behaves like REST there instead of failing the tool after the booking
was already saved.

The service sheds its inventory markers (RS1..RS54, plan history) to
stay under its lint:size entry.
2026-10-08 18:41:06 +02:00
Maurice 465480630b fix(plugins): report a plugin SQL call done once its last statement has run
The time budget for an exec script was read after every statement, the
last one included. A single statement without args, or a script whose
closing COMMIT crossed the budget, had already run and committed, yet the
call threw a time budget error. A plugin that retries on error then wrote
the same rows a second time.

The script now reads the clock before each statement after the first, so
it still stops before the next statement once the time is up and rolls
back a transaction it opened, but a script whose statements have all run
is reported as done. query() likewise reads the clock only once another
row has come back, so a result read to its end is returned.

The ctx.db doc in the SDK said a single statement was not timed, which
contradicted the query budget. It now names what is really untimed: an
exec with bound args and migrate. README and wiki describe the script
stopping before its next statement.
2026-10-08 18:41:02 +02:00
Maurice d3be6fb2b8 fix(plugins): hold exec scripts to the time budget between statements
The time budget covered query() and tx() only. exec() without bound args
handed the plugin's script to Database#exec, which runs any number of
statements in one call with nothing checked between them, so a long
script held the host loop as long as it liked.

An exec script now runs one statement at a time. sql-script.ts cuts it
after every semicolon outside a string, a quoted identifier or a
comment; a piece SQLite reports as incomplete input (a trigger body cut
at its inner semicolon) is joined with the next until it prepares. The
clock is read after every statement, and past the budget the call
throws. As with Database#exec, the statements before that point stay
applied, and an error is SQLite's own; a transaction the script opened
itself is rolled back so the connection is not left inside it.

exec() with bound args is one statement and stays as it was. A single
statement cannot be bounded at all on the host thread, a write such as
INSERT ... SELECT over a cross join included, and the SDK README and the
wiki now say so next to the cross-join read. migrate() keeps running
unbudgeted on purpose: a migration stopped part of the way is rolled back
and fails the same way on every later start, leaving the plugin unable to
update its schema.

The ctx.db rows of the SDK README and the plugin wiki pages describe the
budget for scripts, and the lines touched there and in plugin-sdk's db
comment lose their em dashes.
2026-10-08 18:41:02 +02:00
Maurice 9dec36995e feat(plugins): extend the output allowlist to child rows and whole-row read models
The output contract cut only the top level of an entity result. A derived
key's value went out untouched, so the children that are whole rows still
carried every column: a day's notes_items are full day_notes entities and
a reservation's endpoints are SELECT e.* on reservation_endpoints. About
fifty methods were also left as readModel and passed through whole, many
of them backed by SELECT * reads (day_accommodations, day_notes, todos,
tags, packing bags, journal entries, collab, collections, vacay plans).
A new column on any of those tables reached every plugin by default,
which is what the contract was meant to stop and what the SDK README and
the wiki already claimed it did.

PLUGIN_ENTITY_NESTED now names, per entity, the derived keys that hold
rows of another entity, and the shaper cuts those rows down to their own
field list, recursively. dayNote and reservationEndpoint are the child
tables that were whole rows; day.assignments and place.tags are held to
the assignment and tag lists too, although their reads already name
their columns. The remaining derived children (travelers, participants,
ratings, budget members, payers and receipts, packing recipients,
attachments, voters, bag members) are built field by field, which the
contract comment now states as the rule for a derived key without a
nested entity.

The whole-row read models get entity contracts: accommodation, dayNote,
todo, tag, category, packingBag, fileLink, collabNote, collabPoll,
collabMessage, journey, journalEntry, bucketItem and collection, and
vacayPlan with its holiday calendars. collections.listMine,
collections.get and vacay.mine return envelopes around rows; an entity
with a null table describes those, so their rows are reached through the
same nesting. Every list holds exactly the keys each read returned
before, so no plugin loses a field. What stays readModel is built field
by field from named columns, and each such entry says what it returns:
setBagMembers, atlas.visited, collections.savePlace and copyToTrip, the
two vacay toggles and journal.addEntryPhoto.

The schema test now covers every child and new table, and calls each
newly shaped method on the real services, comparing nested rows key for
key so the only field allowed to disappear is a withheld one. A further
case proves the nested rows it compares are not empty. gen:plugin-facts
publishes PLUGIN_ENTITY_NESTED to the SDK, and the SDK README, the plugin
wiki pages and server/CLAUDE.md describe the nested rows and list the new
entities.
2026-10-08 18:41:02 +02:00
Maurice 0c9e1cd639 fix(plugins): hold plugin SQL to a time budget between rows
A plugin's own database runs on the host's event loop, and the only bounds
were a regex denylist, the 256 MB quota and a 100,000 row cap. A select
whose every row is expensive, or a long scan under the cap, could hold the
loop for as long as it liked.

query() and tx() now get a wall-clock budget of 2 s, read between the rows
a statement yields and between the statements of a batch. Past it the call
throws, and a tx rolls back whole like any other failure inside it. The
row cap and the budget are overridable per PluginDataDb for tests; the
defaults are what production runs.

better-sqlite3 has no interrupt or progress handler, so a statement that
yields nothing until it is done, such as an aggregate over a cross join,
still runs to its first row. Only moving plugin SQL off the host thread can
bound that; the SDK README and the plugin wiki state the budget and this
limit.
2026-10-08 18:41:01 +02:00
Maurice 95f9f031c4 feat(plugins): cut entity results down to a published field list
The entity reads of the plugin RPC surface handed plugins whatever their
query produced, mostly SELECT t.*, with credentials stripped by a denylist
(withoutFeedToken). Every column a migration added reached every plugin
holding the read grant, and renaming one was an unversioned break of a
published API.

protocol/output-contract.ts now states, for every wire method, what its
result is: rows of one of the nine entities the SDK types (trip, place,
day, reservation, packingItem, tripFile, budgetItem, assignment, user), a
value the host builds itself, or a read model passed on as the REST route
returns it. The router applies it to each result on the way out, so an
entity row keeps exactly the fields plugins received when the list was
drawn, in the same order and with the same values, minus the trip's
feed_token (listMine, create and update used to carry it as null). The
table is keyed by every method, so a new one does not compile without an
entry.

output-contract.schema.test.ts holds the lists to the migrated schema: a
new column on a whole-row table fails until it is published or withheld,
and each entity method is called on the real services to prove the
contract drops nothing but the withheld fields.

gen:plugin-facts publishes the lists to the SDK as PLUGIN_ENTITY_FIELDS
and PLUGIN_METHOD_RESULT, and the SDK's entity interfaces are type-checked
against them. Place.day_id, which no read ever delivered, is deprecated
rather than removed. KNOWN_ADDONS is generated from ADDON_IDS as well,
which adds the dawarich, roadtrip and tours ids the hand-kept copy lacked.
The SDK goes to 1.9.0, and its README and the plugin wiki pages describe
what a ctx call returns.
2026-10-08 18:41:01 +02:00
Maurice 8c6157aa46 test(server): pin the request id through the body parsers
A handler behind express.json() is reached from the request stream's 'end'
event, long after the correlation middleware returned, so it looked as if
every write lost its id. It does not: raw-body and multer bind their
callbacks with AsyncResource at the point they are called, which is inside
the request's correlation. Nothing tested that, though, because every
REQID case was a GET. REQID-007 posts a JSON body and REQID-008 one that
arrives in several chunks, and both expect the handler to see the id the
response carries. The middleware comment now says why this holds.
2026-10-08 18:41:01 +02:00
Maurice aa8d9b4037 refactor(server): inject the transit base URL and the kitinerary paths as config tokens
The ownership rule says a boot-stable value that only Nest providers read
belongs to a registerAs token, yet TransitService froze TRANSIT_API_URL in a
module-top readEnv() const and KitineraryExtractorService read
KITINERARY_EXTRACTOR_PATH and PATH through readEnv() in its probe. Both now
come from tokens: transitConfig and kitineraryConfig, derived in
boot-derive.ts and dropped from deriveIntegrations, so readEnv() no longer
reads those variables at all. TransitModule and KitineraryExtractorModule
import AppConfigModule, which keeps them buildable on their own in tests.

Behaviour is unchanged: the same defaults, the same trailing-slash strip and
PATH split, now frozen per built app instead of at import. The app-config
README names each boot-stable variable that stays on readEnv() together with
the reader the container cannot inject into.
2026-10-08 18:41:01 +02:00
Maurice a6f635f3e0 docs(server): keep the managed-hosting switches out of the env reference
The env reference parity test forced TREK_MANAGED, MAPBOX_ACCESS_TOKEN,
CARTO_API_KEY and PLACES_API_BASE into server/.env.example with how-to text.
They are managed hosting only and stay out of the public docs, the same way
TREK_DB_FILE and APP_VERSION do, so they move to NOT_OPERATOR_SETTINGS with
that reason and their lines leave the self-hoster reference again. The
variables themselves and their behaviour are unchanged.
2026-10-08 18:41:01 +02:00
Maurice 558203213c fix(server): keep idle connections open longer than Traefik's upstream timeout
The 65 s default sat above nginx's 60 s but below the 90 s Traefik keeps an
idle upstream connection, so a default Traefik deployment could still reuse a
socket Node had just closed and answer that request with a 502. 95 s clears
both, and the headers timeout one second above it stays well under Node's
300 s request timeout and the 290000 cap the schema accepts.

Also moves the stray RawEnv import in boot-derive.ts up to the other import.
2026-10-08 18:41:01 +02:00
Maurice b3a085df3f refactor(server): make the state ports async and route every reader through the provided store
The process-state ports were meant to let a store shared between
processes replace the in-memory one, but they could not. The readers
outside the container (the broadcast functions, the backup restore's
permissions-cache flush, the pending-code sweep, hand-built consumers)
went straight to the module-level in-memory instances, so an overridden
provider was used by the gateway and the services while the broadcasts
and the restore flush kept using the old store. And every port was
synchronous, while a store in the database or on the network is async,
so plugging one in would have changed every signature anyway.

RateLimitStore, PendingCodeStore, OidcFlowStore and
PermissionsCacheStore now return promises, and RateLimitService,
OauthService, OidcService, PermissionsService and their callers await
them. The in-memory stores still decide synchronously, so who gets a
429, a code or a cached level is unchanged. RoomRegistry stays
synchronous: membership holds live sockets and broadcast() is a
synchronous call every domain makes, so several processes need a
fan-out in front of broadcast(), not a different place for the sets.

The process-wide ports get a ProcessStoreSlot. PermissionsModule,
OauthModule and RealtimeGatewayModule install the store the container
resolved and release it on shutdown, and the free functions, the sweep
and hand-built consumers read the slot, so overriding the provider
changes every reader. Tests provide fake stores and check that the
broadcasts, the restore flush and the sweep use them.

Two callers needed more than an await. The transit MCP tools counted in
a RateLimitService of their own that no provider could reach; they now
take the one RateLimitModule provides (their mcp_transit_* buckets never
overlap the REST ones). The link preview claimed its in-flight slot only
after the budget check, which now yields, so simultaneous askers would
each have been charged; it now claims the slot first, and an asker that
joined someone out of budget asks again on its own budget.

The unused putPendingCode/takePendingCode helpers are gone.
2026-10-08 18:41:01 +02:00
Maurice fa0e6a3eae refactor(server): take the storage roots from DataPathsService
The data-directory change said Nest classes inject DataPathsService, but
only AdminService did. StorageRegistryService still read the built-in
uploads and backups roots and the scratch dir from module constants
frozen at import, and StorageJobsService read the scratch dir the same
way.

The registry now takes DataPathsService through its constructor (a
hand-built one resolves the same layout), and the jobs service asks the
registry for its temp dir. The plain helper modules keep calling
resolveDataPaths() because they run without a container or are shared
with code that does: config.ts, the database path, index.ts, the logger,
the backup functions and the plugin trees, which the out-of-process
supervisor reads too. The app-config README now lists both sides, and
the nest README, server/CLAUDE.md and the provider's own comment say the
same.
2026-10-08 18:40:52 +02:00
Maurice d1b461a37f docs(server): name HTTP_KEEP_ALIVE_TIMEOUT_MS as an httpConfig variable
The keep-alive change moved HTTP_KEEP_ALIVE_TIMEOUT_MS onto the httpConfig
token, but the app-config README still listed only TRUST_PROXY and
HSTS_INCLUDE_SUBDOMAINS for it and left the variable out of the
boot-stable list. Also drops the dashes from two lines the recent
scheduling and entity changes rewrote.
2026-10-08 18:40:37 +02:00
Maurice a72497b3c4 fix(server): trace a cron tick lost to another process as skipped
A tick whose lease another process held returned from runLeased without
running, and traceEntry still wrote `cron <name> ok`, so both processes
logged the tick as run and the per-tick line could not tell which
replica did the work. runLeased now says whether the tick ran, and
traceEntry takes a `skipped` option that logs such a call at debug as
`cron <name> skipped <ms>: lease held by another process`.

Taking the lease is two statements (make sure the row exists, then the
conditional UPDATE that decides the race), so it now runs inside
UnitOfWork.transactional like every other multi-statement write.
2026-10-08 18:40:37 +02:00
Maurice 06dfdc6943 refactor(server): put process-local state behind injectable stores
Rate-limit counters, OAuth pending codes, OIDC login states and codes,
the permissions cache and the WebSocket rooms were Maps on module or
instance scope. That is right while TREK runs as one process, but nothing
could be swapped without editing every consumer.

Each now sits behind a port with an in-memory implementation that keeps
the old behaviour exactly: RateLimitStore, PendingCodeStore,
OidcFlowStore, PermissionsCacheStore and RoomRegistry. The owning module
provides the implementation with the same lifetime the state had
(process-wide for the pending codes, the permissions cache and the rooms,
per instance for the rate limits and the OIDC flows), the consumers take
it through their constructor, and a hand-built consumer falls back to the
same in-memory store, so tests and the free-function paths (the backup
restore's cache flush, the broadcasts) see what they saw before. A store
shared between processes can replace a provider later without touching
the services.
2026-10-08 18:40:37 +02:00
Maurice 5cb87a4df5 feat(server): add last-resort error handlers, a draining readiness probe and proxy keep-alive
Three small gaps in how the process behaves at its edges:

- An unhandled rejection or uncaught exception crashed the process with
  the stack on stderr only. index.ts now logs it to trek.log and runs the
  same orderly shutdown a SIGTERM gets, exiting with 1; a second fatal
  error during that shutdown exits at once.
- /api/health/ready kept answering 200 while a SIGTERM was draining the
  process, so an orchestrator sent traffic for another probe period. A
  shutdown now marks the new ReadinessService first (and Nest's own
  shutdown hook marks it too), and the probe answers 503 from then on.
- Node closes an idle keep-alive connection after 5 seconds while reverse
  proxies keep upstream connections for about a minute, which shows up as
  intermittent 502s. The server now keeps them for 65 seconds, configurable
  through HTTP_KEEP_ALIVE_TIMEOUT_MS (owned by the httpConfig token), with
  the headers timeout one second above.
2026-10-08 18:40:37 +02:00
Maurice c5bc6798a9 feat(server): take a database lease before a cron tick runs
waitForCompletion only kept a tick from overlapping itself inside one
process. Two processes on the same database (a rolling update's overlap,
a second replica, a dev container pointed at the same file) both ran
every job: two auto-backups, double syncs and retention sweeps, and
reminder runs racing each other.

A new migration adds scheduler_leases, and CronRegistrarService now runs
every registered tick through SchedulerLeasesRepository.acquire: one
conditional UPDATE moves the job's row to this process when it is free
(no holder, a lapsed holder, or this process already), so exactly one
process wins. A running tick renews its lease every 20 seconds, a crashed
holder frees the job after a minute, and a finished tick keeps the lease
for 30 seconds so a peer whose timer fires late does not run the same
tick again. The per-row reminder claims stay as they are; jobs need no
change of their own.
2026-10-08 18:40:15 +02:00