Commit Graph

1738 Commits

Author SHA1 Message Date
Maurice f13d60e7e9 test(db): run the AirTrail, atlas expanded, collab and categories MCP suites on the real database module
They seed through the booted container's ORM instead of a second ORM over
the SQLite snapshot mock, and reset through resetDatabase, so they run in
the Postgres lane as well. The PermissionsService the AirTrail cases save
through and the CategoriesMcp and DemoService the categories write
harness attaches are the container's own, called in a request context,
instead of instances built by hand over the SQLite handle.

include_coords compares the coordinates as a set: getTravelStats reads
them through a SELECT DISTINCT without an order, and Postgres returns
them in hash order.
2026-11-17 18:30:05 +01:00
Maurice 1b16fd1683 test(db): run the demo tool gate, addon gating, atlas and assignments MCP suites on the real database module
They seed through the booted container's ORM instead of a second ORM over
the SQLite snapshot mock, and reset through resetDatabase, so they run in
the Postgres lane as well. The tour the assignment cases put on a day is
made with makeTour instead of a raw insert.
2026-11-17 18:20:16 +01:00
Maurice 965a7f2ea8 test(db): merge the factory imports of the newly ported integration suites
One import per module, and the resetTestDb import that no longer has a
caller dropped.
2026-11-17 18:15:31 +01:00
Maurice 88c2b8b991 test(db): run the trips and vacay suites on the real database module
They reset through resetDatabase and seed through the ORM factories on the
booted app's ORM instead of the SQLite snapshot mock, so they run in the
Postgres lane as well.
2026-11-17 18:14:09 +01:00
Maurice 3764efde7a test(db): run the share, system notices and trip access primitives suites on the real database module
They reset through resetDatabase and seed through the ORM factories on the
booted app's ORM instead of the SQLite snapshot mock, so they run in the
Postgres lane as well.
2026-11-17 18:10:13 +01:00
Maurice 11013711d6 test(db): run the security, settings, tags and todo suites on the real database module
They reset through resetDatabase and seed through the ORM factories on the
booted app's ORM instead of the SQLite snapshot mock, so they run in the
Postgres lane as well.
2026-11-17 18:06:52 +01:00
Maurice e083e2d480 test(db): run the scheduler gate, storage health context and uploads suites on the real database module
They drop the SQLite snapshot mock and, where they seed, reset through
resetDatabase and seed through the ORM factories, so they run in the
Postgres lane as well.
2026-11-17 18:03:33 +01:00
Maurice 59adaf2f0c test(db): keep the legacy upgrade suites on SQLite by name
They boot on a SQLite file the retired positional runner built, which no
Postgres database ever came from, so they belong in SQLITE_ONLY rather
than in the snapshot baseline. The lane scan now tells such a suite apart
from the snapshot mock: it mocks src/db/database onto a database of its
own, never names createSnapshotTestDb, and is named in SQLITE_ONLY with
its reason. A snapshot suite added to SQLITE_ONLY stays in the baseline.
2026-11-17 17:55:42 +01:00
Maurice a09d16d6f7 test(db): run the Immich, Synology and unified memories suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.

IMMICH-057 and SYNO-035 need a shared photo row whose trip is gone, which
the foreign key's cascade rules out on a database that enforces it; they
write it with SQLite's foreign keys off as before and are skipped on
Postgres, which has no such switch.
2026-11-17 17:54:16 +01:00
Maurice e1677e7d17 test(db): run the MCP and OAuth suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well. The OAuth service they drove by hand over a second ORM on the
SQLite handle is now the container's own, called in a request context.

The MCP audit cases wait for the mcp.tool_call row: the transport writes
it without awaiting it, so on Postgres it can land after the response.
2026-11-17 17:44:13 +01:00
Maurice 7b0f75a3a4 test(db): run the places and incoming leg mode suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.

The two cases about a NUL byte in a place name move to
places-nul-sqlite.test.ts, listed in SQLITE_ONLY: they pin SQLite's value
quoting, and Postgres text cannot hold U+0000. The control characters
0x01 to 0x1F stay in places.test.ts and run on both engines.
2026-11-17 17:44:07 +01:00
Maurice 8a31a71e1a test(db): run the packing, reservations and journey suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.
2026-11-17 17:19:20 +01:00
Maurice 741bb651e5 test(db): run the OIDC, passkey, profile, notifications and ORM request context seam suites on the real database module
They reset through resetDatabase (the seam suite needs no reset) and seed
through the ORM factories instead of the SQLite snapshot mock, so they run
in the Postgres lane as well.
2026-11-17 17:00:56 +01:00
Maurice 97bafd3ab3 test(db): run the journey place order, maps, misc and local OIDC provider suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.
2026-11-17 16:47:38 +01:00
Maurice 27c043be09 test(db): run the admin and auth suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well, the user erasure cases included.
2026-11-17 16:35:24 +01:00
Maurice 6159cb9011 test(db): run the files and collab suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.
2026-11-17 16:25:48 +01:00
Maurice 6d2bf31bb6 test(db): run the atlas and backup suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.
2026-11-17 16:18:55 +01:00
Maurice 62228af0d5 test(db): run the assignments, day notes and async closure guard suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.
2026-11-17 16:13:02 +01:00
Maurice 3572058917 test(db): run the feeds, atlas geocode, collections and Immich suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.
2026-11-17 16:06:58 +01:00
Maurice cfab220015 test(db): run the boot, boot sweep, demo seed, AirTrail and doc sync context suites on the real database module
They boot the app without the SQLite snapshot mock, so they run in the
Postgres lane as well. The demo seed suite spies on the real runDemoSeed
export instead of replacing the database module to wrap it.
2026-11-17 15:59:20 +01:00
Maurice b11d8d492c test(db): run the session revocation, api docs, app config and help suites on the real database module
They boot the app without the SQLite snapshot mock, so they run in the
Postgres lane as well. sessionRowsOf reads a user's session rows through
a booted app's ORM, next to sessionRows for the suites that still hand in
a better-sqlite3 handle.
2026-11-17 15:50:49 +01:00
Maurice 53ab10afdc test(realtime): run the websocket connection suite on the real database module
Seeds through the ORM factories, resets through resetDatabase and mints
ws-tokens through the container's TokenService instead of one built by hand
over the SQLite snapshot mock, so it runs in the Postgres lane as well.
2026-11-17 15:40:08 +01:00
Maurice 9631a59c2b test(todo): run the todo MCP tool suite on the real database module
Seeds through the booted container's ORM instead of a second ORM over the
SQLite snapshot mock, and resets through resetDatabase, so it runs in the
Postgres lane as well.
2026-11-17 15:36:13 +01:00
Maurice 40385267fc fix(db): keep the stay filter inside src/db and cover the flag type's missing read
src/db may not import src/nest, so the '<id>.0' shape is spelled where the
filter is. The flag type's undefined read had no test, which left the types
folder below its branch threshold.
2026-11-17 15:33:15 +01:00
Maurice 13b896f384 test(days): run the days suite on the real database module
Seeds through the ORM factories and resets through resetDatabase instead of
the SQLite snapshot mock, so it runs in the Postgres lane as well.
2026-11-17 15:32:00 +01:00
Maurice 7bbeec9d9a test(budget): run the budget suite on the real database module
Seeds through the ORM factories and resets through resetDatabase instead of
the SQLite snapshot mock, so it runs in the Postgres lane as well.
2026-11-17 15:27:49 +01:00
Maurice 33f777bcf8 test(categories): run the categories suite on the real database module
It resets through resetDatabase instead of the SQLite snapshot mock, so it
runs in the Postgres lane as well.
2026-11-17 15:24:54 +01:00
Maurice 6010694c08 test(db): reset a suite's database between tests on either engine
resetTestDb needs the SQLite snapshot mock's better-sqlite3 handle, so a
suite resetting with it can only run on SQLite. resetDatabase takes the
booted app's ORM and leaves the same state on SQLite and Postgres: every
table but the kept ones emptied (read from the live schema), sequences left
counting, seedDefaults' rows back through the ORM, and on the first call the
users sequence restarted as the snapshot mock normalises it.
2026-11-17 15:22:48 +01:00
Maurice 03ee902c3d fix(db): type the move's value conversion and client for the server's compiler
Formats the two files the format gate listed.
2026-11-17 15:15:48 +01:00
Maurice 6a9c1c287f test(auth): a plugin table missing from the schema fails the user erasure
The erasure no longer skips a missing plugin table in a savepoint of its own (61ea6b648); USER-CLEANUP-005 still expected the skip.
2026-11-17 14:57:24 +01:00
Maurice d253267e49 test(db): the move command's own output and target ORM 2026-11-17 14:49:42 +01:00
Maurice b931b92fc0 test(realtime): read each parity case after its notification sends have settled
On Postgres a fire-and-forget notification still ran when the case read its recording, so it landed in the next case. The harness now waits for NotificationsService.settled() after each write and each exemption; a late send of any other kind still fails the next case.
2026-11-17 14:33:07 +01:00
Maurice da7d4ff237 feat(notifications): know when the sends a write left running have finished
Most writes start their notification without waiting for it. NotificationsService now tracks each send until it settles, and settled() resolves once every send started before it has finished. The channel check moves to channel-gate.ts, which keeps the service under the size limit.
2026-11-17 14:33:07 +01:00
Maurice f508e0b4c2 test(demo): read the baseline refusal from the logger 2026-11-17 14:33:06 +01:00
Maurice 7e4561b469 test(db): name the restore's plugin schemas as plugin ids allow 2026-11-17 14:24:27 +01:00
Maurice 40e904e237 test(integrations): give the AirTrail suites' UnitOfWork its engine 2026-11-17 14:24:09 +01:00
Maurice c848429820 test(db): no await inside a loop in the Postgres tests 2026-11-17 14:23:13 +01:00
Maurice c46e34dc3a test(db): stand-in pg tools write --file=- to their own stdout
The CI runner gives a child process a socket for stdout, which /dev/stdout cannot open, so every case whose pg_restore streams its script failed there. The COPY terminators in PGTOOLS-020 are spelled as a dump spells them.
2026-11-17 14:23:12 +01:00
Maurice 203c7dee7e feat(lint): refuse a transaction outside UnitOfWork under src/
trek/transactions-through-unit-of-work reads the receiver's type and reports
transactional() on anything but a UnitOfWork (an untyped receiver
included), and begin() or transaction() on an EntityManager, a connection
or Kysely. src/db/engine/ and the two boot-time schema files are left out.
It runs with the rest of the server's ESLint, so lint:warnings fails on a
new one, and its message names UnitOfWork, writeTransaction and the rule in
server/CLAUDE.md.
2026-11-17 14:14:49 +01:00
Maurice 0eeee1e884 fix(db): read the checks a write relies on in the transaction that writes
On Postgres a check read before a transaction, or outside one, no longer
holds at the write that follows: the pool runs other requests' writes in
between. Moved into the unit of work that writes:

- the last-admin guard of an admin's role edit and of an account deletion
  (deleteUserCompletely takes a precondition run in its transaction)
- the login lockout guards of the auth settings save and the SSO settings
  save
- the first-account check of an SSO sign-up, which starts over as an
  ordinary sign-up when another account won
- the OAuth client caps, the token cap and the mail intake limits
- the year check before a vacay year is opened, and the label name check of
  a rename

A password sign-up that loses the last use of an invite is now refused, as
the SSO sign-up already was, and a reset link is burned by one conditional
update, so two resets with one link cannot both change the password.
2026-11-17 14:14:40 +01:00
Maurice 61ea6b648c fix(db): take the write lock in every transaction that writes
UnitOfWork took the engine's write lock, but four repositories opened
transactions of their own: the demo addon set, the plugin user erasure's
savepoints, and the plugin metadata and scheduler cap checks. On Postgres
those ran beside the others on the pool.

writeTransaction (src/db/engine/write-transaction.ts) is now the one
transaction that writes: UnitOfWork wraps it, and the demo seed and reset,
which run outside the container, call it with the engine the environment
names. The capped plugin writes run in their caller's unit of work instead
of a transaction of their own, and the plugin erasure runs in the erasure's
transaction without savepoints, since its three tables come with the
migrations on both engines. lint:tx counts writeTransaction as a
transaction.
2026-11-17 14:14:29 +01:00
Maurice f311fed5c9 test(db): the key rotation on Postgres, and its refusals before a key is asked 2026-11-17 13:54:25 +01:00
Maurice 7065f87dc0 fix(db): rotate the encryption key on Postgres through the server's engine
The rotation ran on the SQLite file only and refused when TREK_DB_URL was
set. Its statements now go through a small store per engine: better-sqlite3
for the file, unchanged, and on Postgres the server's own MikroORM
connection, in one transaction that takes the engine's write lock first. The
Postgres path checks that it reaches a database holding TREK's tables and
that the data directory is writable before it asks for a key, writes a
pg_dump copy before it changes anything, refuses DB_PATH or TREK_DB_FILE
beside the URL, and reads the jsonb addon config the driver hands back
parsed. The SQLite transaction is now BEGIN IMMEDIATE, since better-sqlite3
takes no async function.
2026-11-17 13:54:25 +01:00
Maurice 6d8721ee82 test(db): move a factory-seeded SQLite install into Postgres and read it back through the app 2026-11-17 13:41:21 +01:00
Maurice 36f64c51c8 feat(db): move a SQLite install into an empty Postgres database
npm run db:move-to-postgres (dist/move-to-postgres.js in the image) copies
a SQLite file at this build's migration level into an empty Postgres
database. It refuses a file at another level, rows whose references point
at nothing and a target holding any table or plugin schema, creates the
schema with the engine's own bootstrap, then in one transaction copies every
table in foreign key order (setting a self-reference once its rows are in),
converts each value to its column's type and refuses one Postgres cannot
hold exactly, copies each plugin's plugin.db into its schema, moves the
sequences past the copied ids and compares every table's row count and an
order-free checksum over all rows before it commits. A failure drops the
schema again. The SQLite files and the uploads are never written.
2026-11-17 13:41:20 +01:00
Maurice d177227519 fix(db): refuse a Postgres boot that would leave a SQLite install behind
Setting TREK_DB_URL on an existing SQLite install booted a new, empty
instance beside its data. The Postgres bootstrap now refuses an empty
database, or a migrated one without a single account, while the data
directory's travel.db holds accounts, and names the move command. A file
that cannot be read counts as holding them. TREK_DB_IGNORE_SQLITE_FILE=true
starts anyway, with a warning; the file is opened read-only and only when
the database holds no TREK data.
2026-11-17 13:41:12 +01:00
Maurice 9f7efd49ea test(db): the Postgres plugin data store against a stand-in client
Covers what every call sets up, the schema check before plugin SQL runs, the
row cap, the time budget between chunks and at statement_timeout, the quota,
a script committed statement by statement, the tx and migrate shapes and one
call at a time per plugin, without a database. The Postgres lane runs the
same store on a real one.
2026-11-17 13:02:48 +01:00
Maurice b82cd03a28 fix(db): carry the plugin data schemas in Postgres backups and restores
pg_dump now names the trek_plugin_ schemas TREK's role owns beside its own
schema. A restore reads the dump's plugin schemas from its table of
contents, drops the plugin schemas the role owns, creates the dump's and
loads them with TREK's schema, all in the one psql transaction. Schemas of
another role and every other schema of the database are never read or
dropped.

The dump's migration history is read from TREK's mikro_orm_migrations only,
so a plugin table of that name cannot pass for it.
2026-11-17 13:00:55 +01:00
Maurice 3460511fa4 feat(plugins): store plugin data on the engine TREK runs on
PLUGIN_DATA binds the plugin data backend to the engine (the same
{ sqlite, postgres }[engine.id] binding the backup port uses): the SQLite
file per plugin, unchanged, or the Postgres schema per plugin. The host
opens a plugin's store through it and an uninstall that deletes data erases
through it, which drops the schema on Postgres. Nothing outside db/engine
asks which engine runs.

A plugin's SQL is passed through as written, so a plugin needs to know the
dialect: db.engine (db:own) answers 'sqlite' or 'postgres', and the SDK
exposes it as ctx.db.engine(), in the mock host (dbEngine option) and in
dev. The plugin facts are regenerated and the SDK goes to 1.10.0.

The Postgres lane's PGPLUG suite installs, uses and erases plugin data on
Postgres and shows that a plugin reaches neither TREK's users table nor
another plugin's schema.
2026-11-17 13:00:47 +01:00
Maurice b345c92493 feat(db): keep a plugin's own data in a Postgres schema of its own
A plugin with db:own got a SQLite WAL file in the data volume whatever the
engine, so a Postgres install still needed a local disk (discussion #386).
This adds the plugin data port (db/engine/plugin-data.ts) and its Postgres
side: one schema per plugin, trek_plugin_<id>, reached through TREK's pool.

Every call runs in a transaction that creates the schema if missing, puts
only it on the search_path and sets statement_timeout to the call's budget.
Reads stream through a cursor and stop at the row cap, a write that grows
the schema past the quota rolls back, and the calls of one plugin run one at
a time on at most half of the pool.

TREK's role may read every table, and a role per plugin would need
CREATEROLE, so plugin-sql-guard.ts checks each statement first: it lexes the
SQL as Postgres does and refuses another schema as a qualifier, the pg_
catalogs and functions, functions that run SQL from a string or reach an
object by a name in one, temporary tables, and every statement but DML and
DDL on tables, indexes, views, sequences and types.
2026-11-17 13:00:39 +01:00