Commit Graph

281 Commits

Author SHA1 Message Date
Maurice f4a6411511 fix(db): hold SQLite-only functions, LIKE and the sqlite member of a post-cut migration in lint:dialect
lint:dialect missed several classes of SQL that SQLite reads and Postgres does
not, or reads differently:

- group_concat, ifnull, instr, iif, printf, typeof, total(), changes() and
  json_object/json_array/json_valid/json_type/json_quote, as calls in SQL text
  and as the bare name handed to eb.fn.
- LIKE in SQL text, Kysely's 'like' operator and MikroORM's $like, which ignore
  case on SQLite only. A LIKE that is exact on purpose now says so in a
  like-exact comment; the six existing ones (setting keys, storage paths, the
  lowercased reference search, sqlite_master) carry it.
- The sqlite member of engineDdl was not read at all, so a trigger or a data
  backfill put there never reached Postgres. It may now hold only the schema
  change and a rebuild's copy and sqlite_sequence bookkeeping, and SQLite
  statements with an empty postgres member need a sqlite-only reason.
- execBestEffort, bestEffort and execUnlessTableMissing in a post-cut
  migration: on Postgres a swallowed failure aborts the migration transaction.
- The post-cut rules follow POSTGRES_FIRST_CUT as well as the manifest, so
  regenerating the baseline over a new migration no longer releases it.

src/db/engine/sqlite/ is SQLite's own code and is no longer scanned.
2026-11-17 21:36:21 +01:00
Maurice dc75c200b1 fix(db): count every spelling of an engine check in lint:engine-dispatch
The gate matched only isSqlite(/isPostgres(, instanceof on a platform class
and a comparison on a property named engine or id. A destructured engine name,
the literal on the left, a switch over the engine, instanceof on a driver class
and a comparison with a driver class all passed with an empty baseline, and
standalone-orm.ts already held one such check. The regex now matches every one
of them, the guard in standalone-orm.ts moves into the engine port as
assertSqliteOrmConfig, and the advice names where a check belongs.
2026-11-17 21:27:41 +01:00
Maurice 143e2ae447 fix(db): find no nearest day for a date the calendar does not have
The nearest-day fallback of a booking's time cast the bound date to a
timestamp on Postgres, which refuses 2024-02-30 with a 500, while SQLite's
JULIANDAY rolled it over into March and placed the booking there.
dayDistance now refuses anything but a calendar date on both engines
(isCalendarDate), so a caller that forgets the check fails on SQLite too,
and findNearestDayId answers no day for such a date. The size baseline of
Reservations.repository.ts drops by one line.
2026-11-17 21:23:31 +01:00
Maurice fcddde71a7 test(db): run the notifications and advanced packing MCP suites on the real database module
They seed through the booted container's ORM instead of the SQLite snapshot
mock and reset through resetDatabase, so they run in the Postgres lane as
well. The tag notification case waits for the fire-and-forget row with
vi.waitFor instead of one turn of the event loop: on SQLite the insert
lands within that turn, on Postgres it needs a few round trips more.
2026-11-17 20:13:29 +01:00
Maurice 280e9ab0d9 test(realtime): run the websocket wire envelope suite on the real database module
It takes the ORM from the app buildApp boots, seeds through the factories
and resets through resetDatabase instead of the SQLite snapshot mock, so
it runs in the Postgres lane as well.
2026-11-17 20:07:53 +01:00
Maurice 4284a90b03 test(db): run the trips and vacay MCP suites and the route inventory on the real database module
The MCP suites seed through the booted container's ORM instead of the
SQLite snapshot mock and reset through resetDatabase. The route inventory
never touched the database and only mocked it; without the mock it
compiles AppModule over the real module, so it runs in the Postgres lane
as well.
2026-11-17 20:06:54 +01:00
Maurice 475597a6d7 test(db): run the transports, trip members and trip warnings MCP suites on the real database module
They seed through the booted container's ORM instead of the SQLite snapshot
mock and reset through resetDatabase, so they run in the Postgres lane as
well. The trip warnings suite seeded through the factories but only closed
the snapshot handle; it now boots the container to reach the ORM the reset
needs.
2026-11-17 19:56:40 +01:00
Maurice 4f8ae98e7d test(db): run the share, tags, maps and weather, and transit MCP suites on the real database module
They seed through the booted container's ORM instead of the SQLite snapshot
mock and reset through resetDatabase, so they run in the Postgres lane as
well. The transit suite saves its permission levels through the
container's PermissionsService in a request context instead of one built
by hand over the SQLite handle.
2026-11-17 19:46:27 +01:00
Maurice 65d55add5a test(db): run the reservations, road trip and settings MCP suites on the real database module
They seed through the booted container's ORM instead of the SQLite snapshot
mock and reset through resetDatabase, so they run in the Postgres lane as
well. The road trip suite saves its permission levels through the
container's PermissionsService in a request context instead of one built
by hand over the SQLite handle.
2026-11-17 19:36:14 +01:00
Maurice 17fff1a152 test(db): run the packing, places and plugin MCP suites on the real database module
They seed through the booted container's ORM instead of the SQLite snapshot
mock and reset through resetDatabase, so they run in the Postgres lane as
well. The plugin search suite seeded nothing through the snapshot handle
and only closed it; it now boots the container to reach the ORM the reset
needs.
2026-11-17 19:26:01 +01:00
Maurice f795508fcb test(db): run the feeds and invite, journey, leg transport mode, memories and notes MCP suites on the real database module
They seed through the booted container's ORM instead of the SQLite snapshot
mock and reset through resetDatabase, so they run in the Postgres lane as
well.
2026-11-17 19:09:36 +01:00
Maurice 8254ffd23b test(db): run the days, accommodations, files and help MCP suites on the real database module
They seed through the booted container's ORM instead of the SQLite snapshot
mock and reset through resetDatabase, so they run in the Postgres lane as
well. The collab sub-feature case in the help suite toggles through the
container's AddonsService in a request context instead of one built by
hand over the SQLite handle.
2026-11-17 18:49:07 +01:00
Maurice b6618eac73 test(db): run the budget and collections MCP suites on the real database module
They seed through the booted container's ORM instead of a second ORM over
the SQLite snapshot mock, and reset through resetDatabase, so they run in
the Postgres lane as well. The settlement parity cases call the
container's BudgetController in a request context instead of one built by
hand over the SQLite handle, and the collections suite drops its raw
per-table DELETE for the same reset.
2026-11-17 18:30:12 +01:00
Maurice f13d60e7e9 test(db): run the AirTrail, atlas expanded, collab and categories MCP suites on the real database module
They seed through the booted container's ORM instead of a second ORM over
the SQLite snapshot mock, and reset through resetDatabase, so they run in
the Postgres lane as well. The PermissionsService the AirTrail cases save
through and the CategoriesMcp and DemoService the categories write
harness attaches are the container's own, called in a request context,
instead of instances built by hand over the SQLite handle.

include_coords compares the coordinates as a set: getTravelStats reads
them through a SELECT DISTINCT without an order, and Postgres returns
them in hash order.
2026-11-17 18:30:05 +01:00
Maurice 1b16fd1683 test(db): run the demo tool gate, addon gating, atlas and assignments MCP suites on the real database module
They seed through the booted container's ORM instead of a second ORM over
the SQLite snapshot mock, and reset through resetDatabase, so they run in
the Postgres lane as well. The tour the assignment cases put on a day is
made with makeTour instead of a raw insert.
2026-11-17 18:20:16 +01:00
Maurice 88c2b8b991 test(db): run the trips and vacay suites on the real database module
They reset through resetDatabase and seed through the ORM factories on the
booted app's ORM instead of the SQLite snapshot mock, so they run in the
Postgres lane as well.
2026-11-17 18:14:09 +01:00
Maurice 3764efde7a test(db): run the share, system notices and trip access primitives suites on the real database module
They reset through resetDatabase and seed through the ORM factories on the
booted app's ORM instead of the SQLite snapshot mock, so they run in the
Postgres lane as well.
2026-11-17 18:10:13 +01:00
Maurice 11013711d6 test(db): run the security, settings, tags and todo suites on the real database module
They reset through resetDatabase and seed through the ORM factories on the
booted app's ORM instead of the SQLite snapshot mock, so they run in the
Postgres lane as well.
2026-11-17 18:06:52 +01:00
Maurice e083e2d480 test(db): run the scheduler gate, storage health context and uploads suites on the real database module
They drop the SQLite snapshot mock and, where they seed, reset through
resetDatabase and seed through the ORM factories, so they run in the
Postgres lane as well.
2026-11-17 18:03:33 +01:00
Maurice a09d16d6f7 test(db): run the Immich, Synology and unified memories suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.

IMMICH-057 and SYNO-035 need a shared photo row whose trip is gone, which
the foreign key's cascade rules out on a database that enforces it; they
write it with SQLite's foreign keys off as before and are skipped on
Postgres, which has no such switch.
2026-11-17 17:54:16 +01:00
Maurice e1677e7d17 test(db): run the MCP and OAuth suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well. The OAuth service they drove by hand over a second ORM on the
SQLite handle is now the container's own, called in a request context.

The MCP audit cases wait for the mcp.tool_call row: the transport writes
it without awaiting it, so on Postgres it can land after the response.
2026-11-17 17:44:13 +01:00
Maurice 7b0f75a3a4 test(db): run the places and incoming leg mode suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.

The two cases about a NUL byte in a place name move to
places-nul-sqlite.test.ts, listed in SQLITE_ONLY: they pin SQLite's value
quoting, and Postgres text cannot hold U+0000. The control characters
0x01 to 0x1F stay in places.test.ts and run on both engines.
2026-11-17 17:44:07 +01:00
Maurice 8a31a71e1a test(db): run the packing, reservations and journey suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.
2026-11-17 17:19:20 +01:00
Maurice 741bb651e5 test(db): run the OIDC, passkey, profile, notifications and ORM request context seam suites on the real database module
They reset through resetDatabase (the seam suite needs no reset) and seed
through the ORM factories instead of the SQLite snapshot mock, so they run
in the Postgres lane as well.
2026-11-17 17:00:56 +01:00
Maurice 97bafd3ab3 test(db): run the journey place order, maps, misc and local OIDC provider suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.
2026-11-17 16:47:38 +01:00
Maurice 27c043be09 test(db): run the admin and auth suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well, the user erasure cases included.
2026-11-17 16:35:24 +01:00
Maurice 6159cb9011 test(db): run the files and collab suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.
2026-11-17 16:25:48 +01:00
Maurice 6d2bf31bb6 test(db): run the atlas and backup suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.
2026-11-17 16:18:55 +01:00
Maurice 62228af0d5 test(db): run the assignments, day notes and async closure guard suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.
2026-11-17 16:13:02 +01:00
Maurice 3572058917 test(db): run the feeds, atlas geocode, collections and Immich suites on the real database module
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.
2026-11-17 16:06:58 +01:00
Maurice cfab220015 test(db): run the boot, boot sweep, demo seed, AirTrail and doc sync context suites on the real database module
They boot the app without the SQLite snapshot mock, so they run in the
Postgres lane as well. The demo seed suite spies on the real runDemoSeed
export instead of replacing the database module to wrap it.
2026-11-17 15:59:20 +01:00
Maurice b11d8d492c test(db): run the session revocation, api docs, app config and help suites on the real database module
They boot the app without the SQLite snapshot mock, so they run in the
Postgres lane as well. sessionRowsOf reads a user's session rows through
a booted app's ORM, next to sessionRows for the suites that still hand in
a better-sqlite3 handle.
2026-11-17 15:50:49 +01:00
Maurice 53ab10afdc test(realtime): run the websocket connection suite on the real database module
Seeds through the ORM factories, resets through resetDatabase and mints
ws-tokens through the container's TokenService instead of one built by hand
over the SQLite snapshot mock, so it runs in the Postgres lane as well.
2026-11-17 15:40:08 +01:00
Maurice 9631a59c2b test(todo): run the todo MCP tool suite on the real database module
Seeds through the booted container's ORM instead of a second ORM over the
SQLite snapshot mock, and resets through resetDatabase, so it runs in the
Postgres lane as well.
2026-11-17 15:36:13 +01:00
Maurice 13b896f384 test(days): run the days suite on the real database module
Seeds through the ORM factories and resets through resetDatabase instead of
the SQLite snapshot mock, so it runs in the Postgres lane as well.
2026-11-17 15:32:00 +01:00
Maurice 7bbeec9d9a test(budget): run the budget suite on the real database module
Seeds through the ORM factories and resets through resetDatabase instead of
the SQLite snapshot mock, so it runs in the Postgres lane as well.
2026-11-17 15:27:49 +01:00
Maurice 33f777bcf8 test(categories): run the categories suite on the real database module
It resets through resetDatabase instead of the SQLite snapshot mock, so it
runs in the Postgres lane as well.
2026-11-17 15:24:54 +01:00
Maurice ac5f8ad67e chore(lint): lower the server's no-console baseline to 231
AuthService no longer warns through console when an invite's last use was taken: the registration now refuses it inside its transaction.
2026-11-17 14:43:14 +01:00
Maurice 0eeee1e884 fix(db): read the checks a write relies on in the transaction that writes
On Postgres a check read before a transaction, or outside one, no longer
holds at the write that follows: the pool runs other requests' writes in
between. Moved into the unit of work that writes:

- the last-admin guard of an admin's role edit and of an account deletion
  (deleteUserCompletely takes a precondition run in its transaction)
- the login lockout guards of the auth settings save and the SSO settings
  save
- the first-account check of an SSO sign-up, which starts over as an
  ordinary sign-up when another account won
- the OAuth client caps, the token cap and the mail intake limits
- the year check before a vacay year is opened, and the label name check of
  a rename

A password sign-up that loses the last use of an invite is now refused, as
the SSO sign-up already was, and a reset link is burned by one conditional
update, so two resets with one link cannot both change the password.
2026-11-17 14:14:40 +01:00
Maurice 61ea6b648c fix(db): take the write lock in every transaction that writes
UnitOfWork took the engine's write lock, but four repositories opened
transactions of their own: the demo addon set, the plugin user erasure's
savepoints, and the plugin metadata and scheduler cap checks. On Postgres
those ran beside the others on the pool.

writeTransaction (src/db/engine/write-transaction.ts) is now the one
transaction that writes: UnitOfWork wraps it, and the demo seed and reset,
which run outside the container, call it with the engine the environment
names. The capped plugin writes run in their caller's unit of work instead
of a transaction of their own, and the plugin erasure runs in the erasure's
transaction without savepoints, since its three tables come with the
migrations on both engines. lint:tx counts writeTransaction as a
transaction.
2026-11-17 14:14:29 +01:00
Maurice 7065f87dc0 fix(db): rotate the encryption key on Postgres through the server's engine
The rotation ran on the SQLite file only and refused when TREK_DB_URL was
set. Its statements now go through a small store per engine: better-sqlite3
for the file, unchanged, and on Postgres the server's own MikroORM
connection, in one transaction that takes the engine's write lock first. The
Postgres path checks that it reaches a database holding TREK's tables and
that the data directory is writable before it asks for a key, writes a
pg_dump copy before it changes anything, refuses DB_PATH or TREK_DB_FILE
beside the URL, and reads the jsonb addon config the driver hands back
parsed. The SQLite transaction is now BEGIN IMMEDIATE, since better-sqlite3
takes no async function.
2026-11-17 13:54:25 +01:00
Maurice d177227519 fix(db): refuse a Postgres boot that would leave a SQLite install behind
Setting TREK_DB_URL on an existing SQLite install booted a new, empty
instance beside its data. The Postgres bootstrap now refuses an empty
database, or a migrated one without a single account, while the data
directory's travel.db holds accounts, and names the move command. A file
that cannot be read counts as holding them. TREK_DB_IGNORE_SQLITE_FILE=true
starts anyway, with a warning; the file is opened read-only and only when
the database holds no TREK data.
2026-11-17 13:41:12 +01:00
Maurice 7a9a7c1831 refactor(db): keep the touched files within their size baselines
The users identity guard moves to repositories/_shared/user-identity.ts
(Users.repository.ts re-exports UserIdentityTakenError), the flag text pass
of the entity generator to scripts/generate-entities-flags.ts, and the
Wallet pass query takes the repository's EntityManager. The size baseline
of Users.repository.ts is lowered to what it holds now (1445 to 1421); a
few exports only their own file used are no longer exported.
2026-11-17 12:24:08 +01:00
Maurice 7a99190550 fix(db): reset the admin on Postgres and keep the key rotation off it
reset-admin.js opened data/travel.db with better-sqlite3 whatever the
engine, so on a Postgres install it created a stray SQLite file or reset
the admin in a leftover one. With TREK_DB_URL set it now resets the account
in Postgres through the built server's engine (its pool, SSL and write
lock, in one transaction), and refuses a TREK_DB_FILE beside the URL; the
SQLite branch is unchanged. migrate-encryption.ts cannot re-encrypt the
secrets in Postgres yet, so it now stops before it opens or writes any file
when TREK_DB_URL is set. The wiki says what each does on Postgres.
2026-11-17 12:24:08 +01:00
Maurice ec733fc927 fix(db): store addons.enabled as the integer flag on Postgres too
addons.enabled was the one flag MikroORM's boolean type declared as a
Postgres boolean, while SQLite and every other flag hold 0/1, so an addon
migration's usual SET enabled = 1 would fail there and a Kysely read would
answer true instead of 1. The column now uses IntegerFlagType: the entity
still reads a boolean, both engines store 0/1, and the regenerated Postgres
baseline declares it bigint default 0. The entity generator renders the
type for the columns in BOOLEAN_COLUMNS.
2026-11-17 12:24:07 +01:00
Maurice 9c14f941ba build(docker): ship the PostgreSQL 18 client tools from the PGDG repository
A client handles every server up to its own major and refuses a newer one,
so the image needs the newest supported major, 18, which Debian 13 does not
ship. The PGDG repository is added with its signing key kept in the
repository and its component pinned to 18.
2026-11-17 12:23:51 +01:00
Maurice 22973036b5 fix(db): match a stay-linked booking on Postgres in the public stay check
publicStayExists cast reservations.accommodation_id straight to integer,
which Postgres refuses for the stored '14.0', so the calendar feed and the
share links failed for every trip once any booking was linked to a stay.
The correlation now goes through accommodationIdMatches, which compares
through the new dialect helper integerTextEqualsKysely, and the Wallet pass
context uses the same helper instead of its own numeric cast.
2026-11-17 12:23:36 +01:00
Maurice e4b990b75e ci(db): run the Postgres probe and lane on Postgres 14 and 18
TREK supports Postgres 14 to 18, and CI tested 17 only. One server-postgres
job per major now runs the probe and then the lane against the same service,
so the change keeps the job count of the two jobs it replaces. The lane's
global setup refuses a service whose major is not TREK_TEST_PG_MAJOR, and
the image's Postgres boot moves to 18, the major its client tools target.

scripts/ci/workflow-jobs.test.mjs fails when a job other than the scan is
missing from ci-ok's needs, and changed-areas pins that migrations,
repositories, the engine and the lane's files always run the job.
2026-11-17 12:04:45 +01:00
Maurice 53d875aecb fix(db): keep check:pg-baseline speaking once the baseline is frozen
A frozen baseline still has its manifest verified, and the check now lists the
migrations that run on Postgres after the cut and what holds them. Every
problem it reports says what to do about it.
2026-11-17 11:48:18 +01:00
Maurice a748d6c554 feat(db): hold migrations after the Postgres cut to engineDdl in lint:dialect
A migration newer than the baseline's through (read from
assets/postgres/baseline.json) fails on a schema statement outside engineDdl,
a plain string column for addColumnIfMissing or a SQLite-only helper, and on a
SQLite spelling in an engineDdl postgres member. The advice names the two
steps, and the baseline cannot hold an allowance for these rules.
2026-11-17 11:48:17 +01:00