acquire() made sure a row existed and then raced on a conditional UPDATE,
two statements the registrar wrapped in a transaction of its own. lint:tx
cannot follow a repository reached through a local variable, so it
reported the method. One INSERT ... ON CONFLICT DO UPDATE ... WHERE now
takes a free lease, renews the holder's and leaves anyone else's alone,
atomically on SQLite and on Postgres, and needs no transaction.
The server suite seeds and reads its database through some 5,500 raw
better-sqlite3 statements on the sync handle. They restate the schema by
hand, skip the repositories the app goes through and run on SQLite only,
so nothing in the suite could ever exercise the dialect layer on another
engine.
tests/helpers/factories/ adds typed factories per domain (users, trips and
days, places, assignments, bookings, budget, packing, to-dos, files,
journeys, tours, collections, notifications, settings, plugins, tokens,
atlas, vacay, photos, collab) and generic readers (findRow, findRows,
countRows, updateRows, deleteRows) that go through MikroORM in a request
context of their own, against the app's ORM or a createTestOrm handle.
They refuse a write to a read-only foreign-key twin, which MikroORM would
otherwise drop without a word. The README there is the conversion guide.
The categories integration suite and the tags and calendar-feed e2e suites
are converted in full as worked examples; the tags suite also moves from
hand-rolled tables to the migrated snapshot.
lint:test-sql (scripts/test-sql-ratchet.mjs) holds the raw statements left
to a per-file baseline: growth and stale entries fail, files whose subject
is raw SQL are exempt with a reason, and a single statement can be allowed
with a test-sql-allow comment. CI runs it next to lint:boundaries.
A session renewed from a token issued before sessions were tracked has
an id derived from that token, and the renewal inserts it only when no
row by that id exists. The nightly purge deleted revoked rows straight
away, so the next request with the still valid old token inserted the
session again as an active one and got a fresh tracked token with a new
lifetime. A logout, an ended session or a sign out of the others held
only until the purge ran.
The purge now deletes expired rows only, and a revoked row stays until
its own expiry. A derived row is recorded with an expiry no earlier
than the old token's, so by the time it goes the old token is refused
by its signature check anyway.
The token in the sign-in bodies was declared deprecated and "read by
nothing of ours", but the web client still decided an SSO sign-in by
reading data.token from GET /api/auth/oidc/exchange, an endpoint the
deprecation notice left out. Removing the field later, as documented,
would have broken SSO sign-in.
The exchange now answers { success: true, token } and the login hook
signs in when the response is OK and success is true, so it never reads
token. The field stays for API clients and is now on the deprecation
list in the API docs, the controller comments, server/CLAUDE.md and the
wiki (Security-Hardening, OIDC-SSO). Adding a field to the body breaks
no existing client.
A session token was a signed JWT and nothing else. Logout only cleared
the cookie, a 30-day "remember me" token stayed valid wherever a copy of
it had ended up, and nobody could sign out one device short of changing
the password or rotating the JWT secret for everyone.
Every session token now carries an id (jti) naming a row in the new
user_sessions table (id, user_id, created_at, last_seen_at, expires_at,
revoked_at, user_agent cut to 256 characters; no address is kept). The
rows live behind UserSessionsRepository and the new sessions domain,
whose SessionsService is the one issuer: AuthService.generateToken and
the OIDC sign-in both go through it, and sliding renewal re-signs a
tracked token under the same id and moves its expiry. verifyJwtAndLoadUser
takes the repository and refuses a token whose session is revoked,
expired or gone, refreshing last_seen_at at most every five minutes.
A token without a jti was issued before this change; it passes until its
own expiry (or a password change), so the upgrade signs nobody out, and
renewal turns it into a tracked session.
Sessions end on logout (the one the request names, verified even when
expired), on a password change (all, the device gets a new one), a
password reset, an admin setting a password, reset-admin.js, an MFA
disable (all but the current one), an admin clearing a user's MFA, and
with the account. GET /api/auth/sessions lists the active sessions,
DELETE /api/auth/sessions/:id ends one and POST
/api/auth/sessions/revoke-others ends all but the current, with their
contracts in @trek/shared and two new audit actions. There is no UI for
them yet. MCP exposes no account or session tools, so there is no
parallel tool to change; an MCP call made with a session JWT as bearer
goes through the same check.
The token in the login, register, demo-login, MFA and passkey bodies
stays for API clients but is marked deprecated in the API docs and in
the client type, which never read it.
A nightly session-purge job on CronRegistrarService deletes the expired
and revoked rows. The migration is 042500. The sessions domain gets its
own coverage entry at the 80 floor until a full run measures it.
The rule that a multi-statement write runs in uow.transactional was
written down but nothing held it. scripts/tx-writes.mjs reads every class
under src/ with the TypeScript parser and counts, per method, the writes
it issues outside a transactional callback: repository calls that build
an INSERT, UPDATE or DELETE, calls to methods that write (on the class
itself or on a constructor-injected one), and a transaction holding
writes as one. A write in a loop counts twice, of two branches only the
larger counts, local helpers count where they are called, and a helper
whose every call sits inside a transaction is not reported. --explain
lists the line of every call that counted.
Today's 102 reported methods sit in scripts/tx-writes-baseline.json and
may only shrink; most are syncs with network I/O between their writes,
plugin lifecycle steps and adapters calling two service writes. A new
method fails CI, and an entry above what its method holds fails until
--update lowers it, the same ratchet as lint:size and lint:boundaries.
Two JSDoc tags mark the deliberate cases with their reason in the tag:
@txStandalone for a write that stands on its own whatever the caller does
(the audit row, notification sends, the booking price mirror, the
assignment journey refresh, the quota retention purge), @txIndependent
for writes complete one by one on purpose (the OAuth token retention
sweep, the notification claim and its release). Runs in CI after
lint:boundaries; documented in server/CLAUDE.md, the root CLAUDE.md and
the wiki's dev setup page.
The time budget covered query() and tx() only. exec() without bound args
handed the plugin's script to Database#exec, which runs any number of
statements in one call with nothing checked between them, so a long
script held the host loop as long as it liked.
An exec script now runs one statement at a time. sql-script.ts cuts it
after every semicolon outside a string, a quoted identifier or a
comment; a piece SQLite reports as incomplete input (a trigger body cut
at its inner semicolon) is joined with the next until it prepares. The
clock is read after every statement, and past the budget the call
throws. As with Database#exec, the statements before that point stay
applied, and an error is SQLite's own; a transaction the script opened
itself is rolled back so the connection is not left inside it.
exec() with bound args is one statement and stays as it was. A single
statement cannot be bounded at all on the host thread, a write such as
INSERT ... SELECT over a cross join included, and the SDK README and the
wiki now say so next to the cross-join read. migrate() keeps running
unbudgeted on purpose: a migration stopped part of the way is rolled back
and fails the same way on every later start, leaving the plugin unable to
update its schema.
The ctx.db rows of the SDK README and the plugin wiki pages describe the
budget for scripts, and the lines touched there and in plugin-sdk's db
comment lose their em dashes.
The output contract cut only the top level of an entity result. A derived
key's value went out untouched, so the children that are whole rows still
carried every column: a day's notes_items are full day_notes entities and
a reservation's endpoints are SELECT e.* on reservation_endpoints. About
fifty methods were also left as readModel and passed through whole, many
of them backed by SELECT * reads (day_accommodations, day_notes, todos,
tags, packing bags, journal entries, collab, collections, vacay plans).
A new column on any of those tables reached every plugin by default,
which is what the contract was meant to stop and what the SDK README and
the wiki already claimed it did.
PLUGIN_ENTITY_NESTED now names, per entity, the derived keys that hold
rows of another entity, and the shaper cuts those rows down to their own
field list, recursively. dayNote and reservationEndpoint are the child
tables that were whole rows; day.assignments and place.tags are held to
the assignment and tag lists too, although their reads already name
their columns. The remaining derived children (travelers, participants,
ratings, budget members, payers and receipts, packing recipients,
attachments, voters, bag members) are built field by field, which the
contract comment now states as the rule for a derived key without a
nested entity.
The whole-row read models get entity contracts: accommodation, dayNote,
todo, tag, category, packingBag, fileLink, collabNote, collabPoll,
collabMessage, journey, journalEntry, bucketItem and collection, and
vacayPlan with its holiday calendars. collections.listMine,
collections.get and vacay.mine return envelopes around rows; an entity
with a null table describes those, so their rows are reached through the
same nesting. Every list holds exactly the keys each read returned
before, so no plugin loses a field. What stays readModel is built field
by field from named columns, and each such entry says what it returns:
setBagMembers, atlas.visited, collections.savePlace and copyToTrip, the
two vacay toggles and journal.addEntryPhoto.
The schema test now covers every child and new table, and calls each
newly shaped method on the real services, comparing nested rows key for
key so the only field allowed to disappear is a withheld one. A further
case proves the nested rows it compares are not empty. gen:plugin-facts
publishes PLUGIN_ENTITY_NESTED to the SDK, and the SDK README, the plugin
wiki pages and server/CLAUDE.md describe the nested rows and list the new
entities.
A plugin's own database runs on the host's event loop, and the only bounds
were a regex denylist, the 256 MB quota and a 100,000 row cap. A select
whose every row is expensive, or a long scan under the cap, could hold the
loop for as long as it liked.
query() and tx() now get a wall-clock budget of 2 s, read between the rows
a statement yields and between the statements of a batch. Past it the call
throws, and a tx rolls back whole like any other failure inside it. The
row cap and the budget are overridable per PluginDataDb for tests; the
defaults are what production runs.
better-sqlite3 has no interrupt or progress handler, so a statement that
yields nothing until it is done, such as an aggregate over a cross join,
still runs to its first row. Only moving plugin SQL off the host thread can
bound that; the SDK README and the plugin wiki state the budget and this
limit.
The entity reads of the plugin RPC surface handed plugins whatever their
query produced, mostly SELECT t.*, with credentials stripped by a denylist
(withoutFeedToken). Every column a migration added reached every plugin
holding the read grant, and renaming one was an unversioned break of a
published API.
protocol/output-contract.ts now states, for every wire method, what its
result is: rows of one of the nine entities the SDK types (trip, place,
day, reservation, packingItem, tripFile, budgetItem, assignment, user), a
value the host builds itself, or a read model passed on as the REST route
returns it. The router applies it to each result on the way out, so an
entity row keeps exactly the fields plugins received when the list was
drawn, in the same order and with the same values, minus the trip's
feed_token (listMine, create and update used to carry it as null). The
table is keyed by every method, so a new one does not compile without an
entry.
output-contract.schema.test.ts holds the lists to the migrated schema: a
new column on a whole-row table fails until it is published or withheld,
and each entity method is called on the real services to prove the
contract drops nothing but the withheld fields.
gen:plugin-facts publishes the lists to the SDK as PLUGIN_ENTITY_FIELDS
and PLUGIN_METHOD_RESULT, and the SDK's entity interfaces are type-checked
against them. Place.day_id, which no read ever delivered, is deprecated
rather than removed. KNOWN_ADDONS is generated from ADDON_IDS as well,
which adds the dawarich, roadtrip and tours ids the hand-kept copy lacked.
The SDK goes to 1.9.0, and its README and the plugin wiki pages describe
what a ctx call returns.
The ownership rule says a boot-stable value that only Nest providers read
belongs to a registerAs token, yet TransitService froze TRANSIT_API_URL in a
module-top readEnv() const and KitineraryExtractorService read
KITINERARY_EXTRACTOR_PATH and PATH through readEnv() in its probe. Both now
come from tokens: transitConfig and kitineraryConfig, derived in
boot-derive.ts and dropped from deriveIntegrations, so readEnv() no longer
reads those variables at all. TransitModule and KitineraryExtractorModule
import AppConfigModule, which keeps them buildable on their own in tests.
Behaviour is unchanged: the same defaults, the same trailing-slash strip and
PATH split, now frozen per built app instead of at import. The app-config
README names each boot-stable variable that stays on readEnv() together with
the reader the container cannot inject into.
The 65 s default sat above nginx's 60 s but below the 90 s Traefik keeps an
idle upstream connection, so a default Traefik deployment could still reuse a
socket Node had just closed and answer that request with a 502. 95 s clears
both, and the headers timeout one second above it stays well under Node's
300 s request timeout and the 290000 cap the schema accepts.
Also moves the stray RawEnv import in boot-derive.ts up to the other import.
The process-state ports were meant to let a store shared between
processes replace the in-memory one, but they could not. The readers
outside the container (the broadcast functions, the backup restore's
permissions-cache flush, the pending-code sweep, hand-built consumers)
went straight to the module-level in-memory instances, so an overridden
provider was used by the gateway and the services while the broadcasts
and the restore flush kept using the old store. And every port was
synchronous, while a store in the database or on the network is async,
so plugging one in would have changed every signature anyway.
RateLimitStore, PendingCodeStore, OidcFlowStore and
PermissionsCacheStore now return promises, and RateLimitService,
OauthService, OidcService, PermissionsService and their callers await
them. The in-memory stores still decide synchronously, so who gets a
429, a code or a cached level is unchanged. RoomRegistry stays
synchronous: membership holds live sockets and broadcast() is a
synchronous call every domain makes, so several processes need a
fan-out in front of broadcast(), not a different place for the sets.
The process-wide ports get a ProcessStoreSlot. PermissionsModule,
OauthModule and RealtimeGatewayModule install the store the container
resolved and release it on shutdown, and the free functions, the sweep
and hand-built consumers read the slot, so overriding the provider
changes every reader. Tests provide fake stores and check that the
broadcasts, the restore flush and the sweep use them.
Two callers needed more than an await. The transit MCP tools counted in
a RateLimitService of their own that no provider could reach; they now
take the one RateLimitModule provides (their mcp_transit_* buckets never
overlap the REST ones). The link preview claimed its in-flight slot only
after the budget check, which now yields, so simultaneous askers would
each have been charged; it now claims the slot first, and an asker that
joined someone out of budget asks again on its own budget.
The unused putPendingCode/takePendingCode helpers are gone.
The data-directory change said Nest classes inject DataPathsService, but
only AdminService did. StorageRegistryService still read the built-in
uploads and backups roots and the scratch dir from module constants
frozen at import, and StorageJobsService read the scratch dir the same
way.
The registry now takes DataPathsService through its constructor (a
hand-built one resolves the same layout), and the jobs service asks the
registry for its temp dir. The plain helper modules keep calling
resolveDataPaths() because they run without a container or are shared
with code that does: config.ts, the database path, index.ts, the logger,
the backup functions and the plugin trees, which the out-of-process
supervisor reads too. The app-config README now lists both sides, and
the nest README, server/CLAUDE.md and the provider's own comment say the
same.
The keep-alive change moved HTTP_KEEP_ALIVE_TIMEOUT_MS onto the httpConfig
token, but the app-config README still listed only TRUST_PROXY and
HSTS_INCLUDE_SUBDOMAINS for it and left the variable out of the
boot-stable list. Also drops the dashes from two lines the recent
scheduling and entity changes rewrote.
A tick whose lease another process held returned from runLeased without
running, and traceEntry still wrote `cron <name> ok`, so both processes
logged the tick as run and the per-tick line could not tell which
replica did the work. runLeased now says whether the tick ran, and
traceEntry takes a `skipped` option that logs such a call at debug as
`cron <name> skipped <ms>: lease held by another process`.
Taking the lease is two statements (make sure the row exists, then the
conditional UPDATE that decides the race), so it now runs inside
UnitOfWork.transactional like every other multi-statement write.
Rate-limit counters, OAuth pending codes, OIDC login states and codes,
the permissions cache and the WebSocket rooms were Maps on module or
instance scope. That is right while TREK runs as one process, but nothing
could be swapped without editing every consumer.
Each now sits behind a port with an in-memory implementation that keeps
the old behaviour exactly: RateLimitStore, PendingCodeStore,
OidcFlowStore, PermissionsCacheStore and RoomRegistry. The owning module
provides the implementation with the same lifetime the state had
(process-wide for the pending codes, the permissions cache and the rooms,
per instance for the rate limits and the OIDC flows), the consumers take
it through their constructor, and a hand-built consumer falls back to the
same in-memory store, so tests and the free-function paths (the backup
restore's cache flush, the broadcasts) see what they saw before. A store
shared between processes can replace a provider later without touching
the services.
Three small gaps in how the process behaves at its edges:
- An unhandled rejection or uncaught exception crashed the process with
the stack on stderr only. index.ts now logs it to trek.log and runs the
same orderly shutdown a SIGTERM gets, exiting with 1; a second fatal
error during that shutdown exits at once.
- /api/health/ready kept answering 200 while a SIGTERM was draining the
process, so an orchestrator sent traffic for another probe period. A
shutdown now marks the new ReadinessService first (and Nest's own
shutdown hook marks it too), and the probe answers 503 from then on.
- Node closes an idle keep-alive connection after 5 seconds while reverse
proxies keep upstream connections for about a minute, which shows up as
intermittent 502s. The server now keeps them for 65 seconds, configurable
through HTTP_KEEP_ALIVE_TIMEOUT_MS (owned by the httpConfig token), with
the headers timeout one second above.
waitForCompletion only kept a tick from overlapping itself inside one
process. Two processes on the same database (a rolling update's overlap,
a second replica, a dev container pointed at the same file) both ran
every job: two auto-backups, double syncs and retention sweeps, and
reminder runs racing each other.
A new migration adds scheduler_leases, and CronRegistrarService now runs
every registered tick through SchedulerLeasesRepository.acquire: one
conditional UPDATE moves the job's row to this process when it is free
(no holder, a lapsed holder, or this process already), so exactly one
process wins. A running tick renews its lease every 20 seconds, a crashed
holder frees the job after a minute, and a finished tick keeps the lease
for 30 seconds so a peer whose timer fires late does not run the same
tick again. The per-row reminder claims stay as they are; jobs need no
change of their own.
A 5xx stack and its access-log line were two unrelated lines, the
exception filter printed the stack with console.error and no request
context, and the MCP, WebSocket, plugin RPC and cron entry points logged
nothing per call. A self-hosted bug report could not be traced from a
failure back to the call that caused it.
Every HTTP request now runs under an id held in AsyncLocalStorage: a
well-formed incoming X-Request-Id is kept, anything else is replaced by a
UUID, and the response carries it in X-Request-Id. The logger tags every
line written inside a unit of work with `[http <id>]`. The exception
filter leaves a server-side failure on res.locals and the access log
writes the request line and the stack as one entry under that id; without
an access log watching the response it still prints the error itself.
MCP tool, resource and prompt calls (through a new `around` seam on the
registry), WebSocket messages, plugin RPC dispatches and cron ticks each
run through traceEntry(), which gives the call its own id (remembering the
outer one as its parent) and logs one line for it: debug on success, like
the HTTP access log, and warn or error with the reason on failure. The
call's result and timing are unchanged, a failed cron tick is logged once
rather than twice, and a WebSocket handler rejection no longer reaches
rxjs's asynchronous rethrow.
Every log line ran existsSync, statSync and appendFileSync on the event
loop before it returned. On a slow data volume (an SMB or NFS bind mount
on a NAS) that stalled every request that logged, and rotation renamed
files without checking whether another process sharing data/logs had
rotated already.
The file half of the logger is now BufferedLogFile: a line is queued in
memory and the queue goes out with one appendFile per batch, through a
single serial chain, so no append runs while files are being renamed. The
files and the rotation are the same as before (trek.log rotates to .1 at
10 MB, five files in total), but the size is re-read before a rotation
and a file another process already rotated is left alone. A stalled disk
costs dropped lines with a count instead of unbounded memory, a failed
write is reported instead of thrown, and index.ts writes whatever is
still queued synchronously when the process exits.
Eight registerAs tokens existed while every consumer read the same values
through readEnv() instead, so the boot-stable layer the repo rules describe
was unused and nothing said which path owned a variable. FORCE_HTTPS, for
instance, was frozen by the middleware through the http token and read live
by the cookie code at the same time.
A variable is now owned by exactly one side. TRUST_PROXY and
HSTS_INCLUDE_SUBDOMAINS belong to httpConfig, which buildApp() already
injects into the pre-init middleware, and TREK_PLACE_PHOTO_DIR to the new
storageConfig, which the storage registry now injects instead of reading it
through RuntimeEnvService. Their derive functions moved to boot-derive.ts,
which deriveAll() never calls. The six tokens nothing injected are gone:
their values are needed before the container exists (config.ts, the
database, src/mcp, the SSRF guard) and stay on readEnv().
config-ownership.test.ts records the keys each side reads through a Proxy
and fails on a variable read through both, on a token nothing injects and
on a variable the schema does not validate. env-reference.test.ts holds
server/.env.example to the schema, which had drifted by 34 variables (SMTP,
WebAuthn, plugins, MCP session tuning, TREK_MANAGED and others); they are
documented now. jwt-verify.ts no longer cites a middleware file that is
gone.
The data and uploads directories were derived independently in about ten
files, each counting `..` hops from its own `__dirname`, and the file
logger used `process.cwd()` instead. Moving a file or starting the server
from another directory quietly moved where secrets, backups or logs went,
and the backup code named `data/travel.db` directly, so with TREK_DB_FILE
set a backup archived and a restore replaced a file the connection never
opened.
`resolveDataPaths()` in app-config now anchors data/, uploads/, backups,
tmp, logs, the JWT secret and encryption key files and the database file
(TREK_DB_FILE when set, else data/travel.db) on the server package root.
Code that runs before the container calls it directly; Nest classes inject
the new `DataPathsService`. Every previous anchor uses it, and the resolved
paths are unchanged for the image and for `npm run dev`.
The Postgres probe only ratcheted the methods it managed to call. A method
it could not build arguments for, one that threw before its first
statement and one that timed out were counted and logged but held to
nothing, so a new repository method with a callback parameter could land
SQLite-only SQL unseen, and the part of the code the probe reaches could
shrink without anyone noticing.
pg-probe-baseline.json gains an `uncovered` list of those methods. A
method outside it that goes unmeasured fails the run, and so does an
entry whose method is measured now or gone, so the list only shrinks.
It is seeded from the same first measurement as `failing`; a baseline
without the key reads as unmeasured.
The failing count also counted every refusal, including data errors the
sample values cause (a "probe" string in a date column, SQLSTATE 22007),
so renaming a parameter could fail the gate on a change that never
touched the SQL. Only classes 42 and 0A count now; the rest is reported
in the log and the step summary without being held.
The docs said the probe measures every repository statement. They now
say it measures what each method reaches with sample arguments against
empty tables, and that the methods it cannot reach are held to the list.
The probe shipped with scripts/pg-probe-baseline.json at `failing: null`,
and an unmeasured baseline only reported. Every repository statement
Postgres refused, new SQLite-only SQL included, still passed the job;
only the dialect helper cases could fail it, so the ratchet was off until
some later commit seeded the file.
An unmeasured baseline now fails the run outside --update, and the log,
step summary and annotation say which artifact file to commit. The job
stays red until the first measurement lands, so the gate cannot reach dev
holding nothing. A run that called no repository method (an unreadable
tsconfig, a moved repositories directory) fails as well, also under
--update, so a broken plan can neither pass nor seed an empty baseline.
Nothing in CI ran a statement against anything but SQLite, so the claim
that the dialect layer keeps the repositories portable was never measured
and new SQLite-only SQL could land unseen. The new postgres-probe job
(server/scripts/pg-probe.ts, npm run probe:pg) does that against a
postgres:17 service container:
- it derives a Postgres schema from the entities with MikroORM's schema
generator, keeping TEXT timestamps, mapping the SQLite-only defaults and
the NOCASE collation, and leaving foreign keys out;
- it runs every dialect helper on an in-memory SQLite and on Postgres
against the same expectation, and every case has to pass;
- it calls every public repository method once with arguments the
TypeScript checker builds from the parameter types, through a Postgres
driver that runs each statement alone in a rolled-back transaction with
a statement timeout and records whether Postgres accepted it.
The statements Postgres refuses are held per method to
scripts/pg-probe-baseline.json, which may only shrink. The baseline starts
unmeasured (failing: null): the job reports, passes and uploads the first
measurement as the pg-probe-baseline artifact, together with a JSON report,
so the commit that seeds it and every later commit that lowers it needs no
local Postgres. The probe refuses a database holding tables it did not
create.
@mikro-orm/postgresql joins the server's devDependencies at the 7.2.1 the
other MikroORM packages resolve to; the image's production install leaves
it out. The job needs the preflight gates like every test job, runs only
for the code area, stops after ten minutes and is part of ci-ok.
The dialect layer promised that a second engine would be a branch in
src/db/dialect/ and no sweep through the repositories, but every helper
only knew SQLite and threw for anything else. Each one now has a Postgres
branch, and the helpers whose text both engines read alike (COALESCE,
LOWER, TRIM, substr, ||, the aggregates, the CASE shapes) share one
spelling behind isKnownPlatform.
The Postgres branches keep the storage model SQLite has: the clock helpers
render UTC as the same YYYY-MM-DD HH:MM:SS text (to_char over CURRENT_TIMESTAMP
AT TIME ZONE 'UTC'), GLOB becomes an anchored regular expression, the integer
cast truncates through numeric so '14.0' still reads as 14, JULIANDAY
distance comes from the epoch difference, SQLite's null-safe IS becomes IS
NOT DISTINCT FROM, and NOCASE names a nondeterministic ICU collation a
Postgres schema creates. Bound values on the Kysely side are cast to text
where pg would otherwise send an untyped parameter the planner cannot
resolve.
The SQLite branches are untouched, so nothing changes for a running
install. The Postgres text is pinned in sql-functions.postgres.test.ts.
sql-functions.ts sat at 1007 lines, held there by the size baseline, so it
had no room for the Postgres branch every helper is meant to grow. The
Kysely-expression twins are a separate family already (a Kysely Expression
and a MikroORM RawQueryFragment do not convert into each other), so they
move to src/db/dialect/kysely-functions.ts, and the platform check plus the
fail-closed unsupported() move to platform.ts, which both files share.
No helper changes behaviour or spelling. lint:dialect now skips the whole
src/db/dialect/ directory instead of the one file, and the size baseline
drops its sql-functions.ts entry.
The datetime rule only matched a literal holding a call such as
datetime('now'). The query builder spells the same function as
eb.fn('datetime', [...]), where the literal is just the name, so that form
slipped through while strftime, json_extract and glob are caught bare. A
literal that is exactly datetime, julianday or unixepoch now counts as well,
the same way the glob rule treats a bare glob. The only such call lives in the
dialect layer, which is not scanned, so the baseline does not change.
The repositories already lean on the query builder, the generated table
types and sql-functions.ts, but the rules behind that were only implied.
server/CLAUDE.md now states them: which builder call replaces each SQLite
spelling, which helper covers time, strings and casts, where PRAGMA may live,
how a new migration is treated, and that the generated types describe the
SQLite storage model.
It also says plainly that MySQL and MariaDB are out of scope. The
repositories rely on INSERT ... ON CONFLICT and RETURNING, which MySQL lacks,
so Postgres is the second engine the code is prepared for.
ESLint keeps raw SQL out of the repositories, but nothing looked at the rest
of src/: the entities, the domain services, the src/db lifecycle files or a
migration added from now on. scripts/dialect-lint.mjs (npm run lint:dialect)
reads every string and template literal under src/, skipping comments, and
counts INSERT OR IGNORE/REPLACE, datetime(), strftime, json_extract, GLOB,
last_insert_rowid, AUTOINCREMENT, PRAGMA outside the lifecycle files, ||
concatenation in SQL and .insertId reads. scripts/dialect-baseline.json
holds today's thirteen hits; the counts may only go down, like lint:size.
The dialect layer is not scanned, and neither are the migrations that have
shipped.
The four seeders that still spelled INSERT OR IGNORE (addons, photo and
document providers, the schema_version marker) are Kysely inserts that do
nothing on conflict now, so a re-run still leaves an operator's rows alone.
They leave the SQL-string exemption list in eslint.config.mjs and their pins
in the selector test.
Kysely fills InsertResult.insertId from SQLite's last_insert_rowid. On
Postgres it is undefined, so the sixteen repository inserts that returned
Number(result.insertId) would hand NaN ids to their services without an
error. They now add .returning('id') and read the id from the row, which
SQLite (3.35 and later, bundled by better-sqlite3) and Postgres both answer
with the same value.
DemoRepository ran its example-trip seed as nine SQL strings through
connection.execute, including INSERT OR IGNORE and a bare CURRENT_TIMESTAMP.
Those are Kysely inserts on the generated table types now: ON CONFLICT
(trip_id, user_id) DO NOTHING for the membership row, the dialect helper for
the timestamp, RETURNING id where the caller needs it. With no SQL text left
it drops off the ESLint exemption list and its pin.
ESLint now refuses any read of insertId in src/db/repositories and
src/db/seeders, so the pattern cannot come back.
The repositories typed their Kysely statements against 159 table interfaces
written by hand, one or more per repository. Nothing compared them with the
schema: a migration that renamed a column or changed its type left them
stale while tsc stayed green, and several had already drifted
(reservations.accommodation_id is TEXT, not INTEGER).
scripts/generate-db-types.ts migrates a throwaway database with the real
migrations, reads every table back and writes one interface per table into
src/db/kysely/ plus the DB map in db.ts. Every hand-written interface is now
Pick<DB, ...> of that type, and npm run check:db-types fails when the
committed files no longer match the migrations.
A column an insert may leave out (rowid, DEFAULT, nullable) is
InsertOptional<T>, declared as ColumnType<T, T | void, T>. The server
compiles without strictNullChecks, where Kysely's own Generated<T> still
demands the column; that is why the old code kept separate insert-only
interfaces. The ids callers pass as number | string are cast to the column
type at the bind, so every statement binds the same value as before.
The demo seed runs in the schema bootstrap, at boot and again after every
reopen. Since the baseline save moved out of the seed into DemoResetJob,
only the boot read the seed's mark. A restore onto a demo instance with no
baseline, whose archive held no admin trips, seeded the example trips
during the reopen and then saved nothing, so the hourly reset kept logging
"No baseline found"; the mark also stayed set for the rest of the process.
Before the move, that path saved the baseline from inside the seed.
DatabaseLifecycle now runs listeners after a reopen's schema bootstrap,
and DemoResetJob registers one in demo mode that does the same check as at
boot: save through the backup port when the seed just ran and no baseline
exists. Each check clears the mark. A failed save is logged, never turned
into a failed reopen. AdminModule imports DatabaseLifecycleModule from the
shared database kernel for it.
The DB paragraph said MaintenanceRepository is never built with `new` in
src/, but MaintenanceModule's factory does exactly that. Say what is
actually true: the module's factory is the one place that builds it, and
every consumer injects it.
Backup, restore and the demo reset each handled the SQLite file
themselves: backup.impl closed and reinitialized the connection, built
MaintenanceRepository with new, and assumed the database was
data/travel.db. That tied the whole chain to one local file and hid two
bugs.
They now go through DATABASE_BACKUP, a port declared in the shared
database kernel (snapshot, verify, keep a copy before a restore, replace).
SqliteDatabaseBackup implements it with what the code did before:
VACUUM INTO for a consistent copy, and a close, copy, sidecar drop,
rename and reopen through DatabaseLifecycle for a restore. BackupService
injects it and hands it to backup.impl. The demo reset and the baseline
save use the same port; the first baseline is saved by DemoResetJob once
the app is up, because the boot-time seed runs before anything can be
injected. Another engine can plug in a logical dump behind the port.
Fixes on the way:
- The backup and the restore ignored TREK_DB_FILE. The backup looked for
data/travel.db and, with the database elsewhere, silently left it out
of the archive; the restore wrote data/travel.db and never replaced
the live database. Both now use DatabaseLifecycle.file, the path the
connection runs on, and the pre-restore copy sits next to it.
- A failed VACUUM INTO fell back to archiving the live WAL-mode file
without a word, the torn copy the snapshot exists to prevent. The
backup now fails: nothing is archived or committed, the existing
backups are untouched, a partial snapshot is removed, and the log says
why. A missing database file and a failed checkpoint are logged too.
db/database.ts opened the connection as a side effect of being imported,
so the moment anything reached it the database was open. index.ts had to
defer loading the app until a first-boot restore was done, and nothing
in the container decided when the connection opened or closed.
The module now opens the connection on first use and opens nothing at
import. A connection closed on purpose (a restore, the shutdown) stays
closed until it is reopened, as before. The new DatabaseLifecycle
provider owns the decisions: buildApp() opens through it, which also
binds the ORM to later swaps (the hook attachOrm used to register) and
runs the schema bootstrap in the same order as before, and the shutdown
closes through it. Its file property is the one resolved database path.
The module keeps every export, so the test suites that replace it with
vi.mock keep working; buildDbMock gains the new openDb.
Both server ratchets only logged an entry the tree had moved past. A
file that shrank could grow back to its old size entry unseen, a
deleted file left its allowance to the next file at its path, and a
boundary violation that was fixed stayed allowed until someone ran
--update. The client ratchets now fail on such entries, so the server
ones do the same.
lint:size fails on an entry above its file's size or for a file that is
gone, and lint:boundaries on an entry that no longer occurs; both tell
to run --update, which still only lowers or drops. Both baselines match
the tree today.
The foreign key index check took every index from PRAGMA index_list as a
usable prefix, partial ones included. An index like
CREATE INDEX ix ON c(p_id) WHERE state = 'open' would satisfy FKIDX-001,
yet the cascade lookup WHERE p_id = ? still scans the table, which is the
regression the check exists to catch. A partial index now counts only when
its predicate is IS NOT NULL on the key's own columns, the one form SQLite
can prove from p_id = ?. Nothing in today's schema depends on a partial
index alone, so the result on the migrated snapshot is unchanged.
contracts:open in shared only counts exports named *RequestSchema, but
the name is a convention: 37 body DTOs wrap a schema called *BodySchema,
*InputSchema or plain *Schema, and tags.dto.ts declares its schemas
locally. pluginInstanceConfigUpdateSchema (a record of unknown behind
PluginConfigDto), createAlbumLinkSchema with five open shapes and a dozen
more were never counted, and a new open body under one of those names
passed both checks.
tests/unit/contracts/dto-open-shapes.test.ts walks the other way round.
It finds every class that extends createZodDto under src/, imports it,
and counts the open shapes in the schema the Zod pipe validates with
shared's walk. Each class is held to scripts/dto-open-shapes-baseline.json,
61 DTOs with 101 open shapes today, and a new DTO may hold none. It fails
closed: a createZodDto call that is not an exported class, two DTO
classes with one name, an empty tree or an unreadable baseline stop it.
It runs with the unit tests; npm run contracts:dto-open runs it on its
own and --update only lowers the baseline.
The five DTO classes declared inside roadtrip controllers are exported so
the walk can reach their schemas. The selector test picks up prettier's
wrapping of the cases added before.
MaintenanceRepository, DemoRepository and six seeders are exempt from the
SQL string selectors as whole files, so new SQL text added to any of them
passed lint unseen, and nothing noticed when a file no longer needed its
exemption. The comment promised a list that can only shrink, but nothing
held it to that.
The selector test now works out which files under src/db/repositories and
src/db/seeders the config really exempts and compares that set with a
pinned table, then runs the SQL selectors over each exempt file's source
and compares the hits with the count pinned for it (23 today). One more
statement fails, one fewer asks for the pin to come down, and a file at
zero has to leave the ignores. Per-line disables would have done the same
job but add suppressions the codebase is meant to lose.
The outbound-timeout selectors flagged safeFetch(url, undefined,
{ timeoutMs }) and safeFetch(url, { headers }, { timeoutMs }), although
timeoutMs is the deadline over the whole exchange that server/CLAUDE.md
tells callers to pass. At the same time an init spelled null or void 0
passed unseen, so a guarded call could still go without any timeout.
A call whose third argument is an object literal with a timeoutMs key is
now exempt, and undefined, null and void 0 (also behind an as cast) all
count as no init. The cast forms pin the inner node type, because esquery
compares String(value) and a missing name would otherwise read as
'undefined' and flag every cast init. The selector test covers both
sides.
The repository rule that keeps dialect SQL in sql-functions.ts only
matched raw() and the sql tag, so a SQL string handed straight to the
connection, em.getConnection().execute('INSERT OR IGNORE ...'), passed
unseen, and the seeders sat outside every repository rule. A new block
refuses a string or template whose text starts like a statement as the
first argument of execute, run, all or get in src/db/repositories and
src/db/seeders; a result mode such as qb.execute('run') stays legal.
The files that already hold such SQL are named one by one so the list
can only shrink: MaintenanceRepository (PRAGMA, VACUUM INTO),
DemoRepository and six of the seven seeders.
The selector groups are named once in eslint.config.mjs, and the
selector test now also checks the new block per path, its exceptions,
and that raw() stays forbidden in repositories.
journey-gallery-order.ts exported raw SQLite ORDER BY text from
src/nest that no production code read any more since the gallery reads
moved onto JourneyPhotosRepository. Two parity tests still compare the
Kysely rebuild against it, so it moves to tests/helpers as their oracle.
The ESLint rule from #2610 that requires a signal on outbound fetches
only matched the platform fetch. The guarded wrappers hand it a spread
init, which the rule trusts, so safeFetch(url, undefined, options) passed
lint with no timeout, and the dispatcher left undici's five-minute
default in place. Nothing capped the body either. Two callers did exactly
that on hosts TREK does not control: the Synology thumbnail read (a
user-configured NAS) and the place enrichment photo download (a URL out
of third-party place data), both buffering the whole answer.
The wrappers now carry defaults. A call without a signal waits 30 s for
the response headers; the body stays under undici's idle timeout, so a
proxied video that pauses with its player is not cut off. timeoutMs adds
a deadline over the whole exchange and joins the caller's signal, once
for the whole redirect chain. Every body is capped at 64 MB, refused up
front when the declared length is larger and failed mid-read otherwise;
callers pass a tighter maxBytes, or null where the body is streamed on
(pipeAsset, the doc-sync transfers that count their own bytes, the
Ollama pull).
The two call sites get a signal and an 8 MB cap. The lint selectors now
cover safeFetch, safeFetchFollow and safeFetchAdminConfigured as well,
including an explicit undefined init, and are named once so the
repository block restates the same list. A new test asks ESLint for the
options it applies to a path and runs them over snippets, so a selector
that stops matching fails a test instead of passing silently.
google-quota, managed, mcp-shared, mcp-transport, public-api,
receipt-scan, route-usage, scheduling, school-holidays, tokens and tours
had no entry in the per-domain ratchet, so they sat in the pooled
src/nest/** catch-all, where a domain can lose twenty points behind the
headroom of the files beside it. That includes the cron registrar, the
MCP HTTP transport, the public API and the new tours domain.
Each is pinned at floor(measured) minus 1 as scripts/coverage-thresholds.mjs
prints it, measured over the unit, integration and e2e suites that load
the domain. A full run can only measure higher. mcp-shared (59 branches)
and route-usage (66 branches, 77 functions) measure below the 80 floor
and are pinned where they are; managed holds an empty module.
A new unit test fails when a folder under src/nest has no entry, or an
entry names a folder that is gone, so the next domain cannot land in the
catch-all again.
Migration20200101042200 wrote 86 indexes by hand so that deleting a trip
or a user stops scanning every child table while it holds the single
connection. Nothing kept it that way: the next table with an unindexed
trip_id would bring the scans back, and the entity parity tests compare
index names, not whether a key is covered.
The test reads PRAGMA foreign_key_list off the migrated test database
and requires the columns of each foreign key to lead some index or the
primary key. Today every one does, so the allow-list for deliberate
exceptions starts empty, and a second case keeps that list honest. A
third builds a small schema to prove the check fails on an unindexed
key, a composite one, and an index that holds the column second.
Module boundaries were held by review alone. storage.module.ts documents
a cycle that only surfaced at runtime as "imports[1] is undefined", three
modules exist just to break cycles, and nothing stopped the next feature
from reaching into another domain's helpers.
scripts/import-boundaries.mjs parses every import under src/ with the
TypeScript parser and checks five rules against a baseline of today's
violations: load-time file cycles (none today once type-only and lazy
imports and the two-way MikroORM entity relations are left out), new
dependencies between domains that already depend on each other (262
pairs, one tangle around plugins and auth), shared-kernel files importing
a domain (8), imports of another domain's internals (69) and src/db
importing src/nest (5). A domain's public surface is what the code
already treats that way: modules, services, guards, decorators, types,
constants, the audit logger, the plugin RPC kit and the addon gates.
New violations fail, --update only drops entries, and an unreadable
baseline, a missing src/ or an import it cannot resolve stops the run.
The client has had lint:size since #2612, the server had nothing, and
the server is where the biggest files live: maps.service.ts holds 3231
lines and grew by 200 in a month, and nine more services are past 1100.
scripts/size-lint.mjs carries the client's semantics over to src/ and
scripts/: a file may hold 1000 lines, the 23 that already hold more are
pinned at today's length in scripts/size-baseline.json, and --update
only ever lowers an entry.
A line past prettier's 120 columns counts once per 120 columns it spans,
so joining lines does not make room under the limit. The check fails
closed: a missing or malformed baseline or a missing src/ stops it
instead of passing with nothing checked.
Login, password reset and the sign-up checks all match emails
case-insensitively, but the only constraint was the column's own
UNIQUE, which SQLite compares case-sensitively. Two accounts whose
addresses differ only in case (legacy rows, two sign-ups at once) left a
login to resolve to whichever row came first, and LOWER(email) lookups
had no index to use.
A migration adds a unique index on lower(email). An install that
already holds such a pair must still start, so the migration then
leaves the index out, logs how many addresses are shared and by which
user ids (not the addresses), and records email_case_index_pending.
Since a migration runs once, db/email-case-index.ts runs on every boot
while that flag is set: it repeats the warning until the accounts are
told apart, then creates the index and clears the flag. The app-level
checks keep guarding meanwhile.
A write that passes the up-front check and still loses the race is now
refused by the database. The users repository turns that unique
violation into UserIdentityTakenError, and sign-up, admin create and
edit, and the profile update answer it with the same 409 their checks
give, instead of a 500 or an unhandled rejection (ADMIN-SVC-094 pinned
the old behaviour and now pins the 409).
The entity generator could not see an expression index (PRAGMA
index_info names its column NULL), so PARITY-009c found a db index with
no entity declaration. Rule 14 carries such an index into the entity as
a named entry with its own statement; Users.entity.ts is regenerated.
Maurice