The token guard kept leaking: on a superuser connection an unqualified
`alter type int4 rename to int4x` renamed pg_catalog's type for the whole
server, sequences and catalog-only objects grew the database past the quota,
regtype/regconfig/regdictionary looked TREK's objects up by name, and a
column alias named insert or merge in front of INTO made a table. Instead
of a longer word list, the store now asks the database:
- No plugin SQL while TREK's role is privileged in the sense of the restore
check (PRIVILEGED_ROLE_SQL, imported from server-checks.ts): every db:own
call fails with PRIVILEGED_ROLE_PLUGIN_REFUSAL, read once per pool
connection, and the boot log says why.
- The objects an ALTER, DROP, REFRESH, TRUNCATE or CREATE INDEX ... ON
changes, and the tables a CREATE or ALTER TABLE ties to its own, are
resolved with to_regclass/to_regtype in the transaction right before the
statement, and it is refused unless each one found lies in the plugin's
schema. An unqualified int4, text or box is pg_catalog's and refused even
with the role check off.
- The quota counts every relation of the schema with its TOAST table, and a
schema holds at most PLUGIN_SCHEMA_MAX_OBJECTS catalog objects (relations,
types, constraints, enum labels).
- The guard refuses every reg* type and its array in every spelling, the
has_*_privilege functions, currtid2 and the BRIN/GIN maintenance
functions; INTO only after the INSERT or MERGE that is the statement's
verb or a WITH body's; TEMP only where it makes a temporary object
(CREATE TEMP/TEMPORARY, SELECT ... INTO TEMP), so a column named temp
works again; a base type; and a NUL anywhere in the SQL text.
- A bound value or a migration id holding NUL is refused with the reason
instead of being stripped, so the plugin's keys never change meaning.
Every bypass from the re-review is a test on the real database, on the
role the plugin data is meant to run as and on a superuser connection with
the role check off.
The privileged-role check missed three ways a role can reach the database
host from SQL. Before PostgreSQL 16 a member of a CREATEROLE role sets its
role to it and grants itself pg_execute_server_program, since role
attributes are not inherited but the grant checks the role in effect. A role
granted EXECUTE on lo_export, lo_import, pg_read_file, pg_ls_dir and their
siblings reads or writes server files, and lo_export keeps the file after the
transaction rolls back. From PostgreSQL 15 on a role can be granted ALTER
SYSTEM on a setting or SET on a superuser-only one.
PRIVILEGED_ROLE_SQL now counts all three. The parameter grants are read
through query_to_xml inside a CASE taken only on 15 or later, because a
statement naming pg_parameter_acl does not parse on 14. A SET grant on a
setting every role may change anyway, and pg_monitor, stay ordinary. The
lane test creates each kind of role and shows the escalation the check is
for on the server it runs against.
createFromBaseline created the migration history's table through MikroORM's
storage before, and outside, the transaction that runs the baseline. Without a
master transaction MikroORM commits that table on its own, so a first boot that
failed in the baseline (a server without ICU, a timeout, a dropped connection)
left an empty mikro_orm_migrations behind. The next start with
RESTORE_FROM_BACKUP took that for a database in use, skipped the restore with a
warning and brought up an empty instance, after which the restore never ran.
The table is now created on the baseline's own transaction, so a failed first
boot leaves the database empty. The boot restore also treats a lone migration
history that records nothing as no database, which a restore clears anyway; any
other answer keeps it, so the check fails closed. Database-Postgres says both.
Plugin statements on Postgres went over the extended query protocol through a
marker parameter and a hook that replaced query() on every client of TREK's
pool. That kept a plugin to one statement per round trip, but it changed
TREK's own hot path for a concern only plugin data has.
The plugin data store now checks a pg client out of the ORM's pool for each
transaction, through MikroORM's getNativeClient, and runs the transaction on
it directly. Every plugin write goes as a query with queryMode 'extended' and
every read through a pg-cursor on that client, so Postgres parses exactly one
statement either way. The settings the transaction makes are local, and
before the client goes back to the pool the store drops temporary tables,
cursors, listeners, session advisory locks and sequence state, or closes a
client it cannot bring back to that state. The row cap, quota, time budget,
untimed migrate, per-plugin queue, pool share and script commits are as
before, and plugin args are still sent without NUL.
The marker, the hook and its install in the Postgres driver are gone.
Postgres text cannot hold U+0000, so any write or filter carrying one (a
request body, a query string, an import, a feed, a forwarded mail, a
plugin's own SQL) answered a 500 on Postgres while SQLite stored it. Both
engines now drop the character where every value leaves for the database
(src/db/nul-free.ts): the MikroORM platforms when they inline a string or
serialize a JSON value, the Kysely dialects for every parameter, a
plugin's own SQLite file for its arguments, and the move from SQLite,
which used to refuse such text. The same request now answers the same
status and leaves the same row on both engines, so the places NUL suite
runs on Postgres as well and leaves SQLITE_ONLY.
The privileged-role check looked only at the role's own rolsuper and at
pg_execute_server_program and pg_write_server_files. A role that is no
superuser itself but a member of one passed it and could SET ROLE to the
superuser from a crafted archive; on PostgreSQL 14 and 15 a role with
CREATEROLE could grant itself pg_execute_server_program; and
pg_read_server_files let COPY read the server's files into TREK's tables.
The check now counts membership in any superuser role, the three
predefined server roles, and CREATEROLE before 16. The boot warning and
the restore refusal describe the same list from one constant. A Postgres
lane suite creates each kind of role and evaluates the check as it, and
shows on the server it runs against whether CREATEROLE can grant itself
pg_execute_server_program.
The guard ended a -- comment at LF only, while Postgres ends it at CR as
well, so "select 1 --\rfrom public.users" hid a qualified name from the
guard that Postgres then read, and with no args a second statement after
the CR ran as TREK's role. It also only looked for INTO at the top level
of a SELECT, so "(select ... into temp users)" made a temporary table that
shadowed TREK's own users table on the pooled connection.
The tokenizer now skips exactly what Postgres skips as whitespace, ends a
line comment at CR or LF, and refuses any character the two read
differently (other control characters, non-ASCII whitespace such as
U+00A0, U+FEFF and U+2028, and the non-decimal and underscored numbers
Postgres before 16 reads another way) instead of guessing. INTO is refused
anywhere but after INSERT or MERGE, at any depth, and the words TEMP and
TEMPORARY anywhere outside a string or quoted name.
The guard tests carry both payloads and their variants; the Postgres lane
runs them against the real database, proves a second statement in one
text is refused by Postgres over the extended protocol, and that a
temporary table never shadows TREK's or a plugin's tables.
They seed through the booted container's ORM instead of the SQLite snapshot
mock and reset through resetDatabase, so they run in the Postgres lane as
well. The tag notification case waits for the fire-and-forget row with
vi.waitFor instead of one turn of the event loop: on SQLite the insert
lands within that turn, on Postgres it needs a few round trips more.
It takes the ORM from the app buildApp boots, seeds through the factories
and resets through resetDatabase instead of the SQLite snapshot mock, so
it runs in the Postgres lane as well.
The MCP suites seed through the booted container's ORM instead of the
SQLite snapshot mock and reset through resetDatabase. The route inventory
never touched the database and only mocked it; without the mock it
compiles AppModule over the real module, so it runs in the Postgres lane
as well.
They seed through the booted container's ORM instead of the SQLite snapshot
mock and reset through resetDatabase, so they run in the Postgres lane as
well. The trip warnings suite seeded through the factories but only closed
the snapshot handle; it now boots the container to reach the ORM the reset
needs.
They seed through the booted container's ORM instead of the SQLite snapshot
mock and reset through resetDatabase, so they run in the Postgres lane as
well. The transit suite saves its permission levels through the
container's PermissionsService in a request context instead of one built
by hand over the SQLite handle.
They seed through the booted container's ORM instead of the SQLite snapshot
mock and reset through resetDatabase, so they run in the Postgres lane as
well. The road trip suite saves its permission levels through the
container's PermissionsService in a request context instead of one built
by hand over the SQLite handle.
They seed through the booted container's ORM instead of the SQLite snapshot
mock and reset through resetDatabase, so they run in the Postgres lane as
well. The plugin search suite seeded nothing through the snapshot handle
and only closed it; it now boots the container to reach the ORM the reset
needs.
They seed through the booted container's ORM instead of the SQLite snapshot
mock and reset through resetDatabase, so they run in the Postgres lane as
well.
They seed through the booted container's ORM instead of the SQLite snapshot
mock and reset through resetDatabase, so they run in the Postgres lane as
well. The collab sub-feature case in the help suite toggles through the
container's AddonsService in a request context instead of one built by
hand over the SQLite handle.
They seed through the booted container's ORM instead of a second ORM over
the SQLite snapshot mock, and reset through resetDatabase, so they run in
the Postgres lane as well. The settlement parity cases call the
container's BudgetController in a request context instead of one built by
hand over the SQLite handle, and the collections suite drops its raw
per-table DELETE for the same reset.
They seed through the booted container's ORM instead of a second ORM over
the SQLite snapshot mock, and reset through resetDatabase, so they run in
the Postgres lane as well. The PermissionsService the AirTrail cases save
through and the CategoriesMcp and DemoService the categories write
harness attaches are the container's own, called in a request context,
instead of instances built by hand over the SQLite handle.
include_coords compares the coordinates as a set: getTravelStats reads
them through a SELECT DISTINCT without an order, and Postgres returns
them in hash order.
They seed through the booted container's ORM instead of a second ORM over
the SQLite snapshot mock, and reset through resetDatabase, so they run in
the Postgres lane as well. The tour the assignment cases put on a day is
made with makeTour instead of a raw insert.
They reset through resetDatabase and seed through the ORM factories on the
booted app's ORM instead of the SQLite snapshot mock, so they run in the
Postgres lane as well.
They reset through resetDatabase and seed through the ORM factories on the
booted app's ORM instead of the SQLite snapshot mock, so they run in the
Postgres lane as well.
They reset through resetDatabase and seed through the ORM factories on the
booted app's ORM instead of the SQLite snapshot mock, so they run in the
Postgres lane as well.
They drop the SQLite snapshot mock and, where they seed, reset through
resetDatabase and seed through the ORM factories, so they run in the
Postgres lane as well.
They boot on a SQLite file the retired positional runner built, which no
Postgres database ever came from, so they belong in SQLITE_ONLY rather
than in the snapshot baseline. The lane scan now tells such a suite apart
from the snapshot mock: it mocks src/db/database onto a database of its
own, never names createSnapshotTestDb, and is named in SQLITE_ONLY with
its reason. A snapshot suite added to SQLITE_ONLY stays in the baseline.
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.
IMMICH-057 and SYNO-035 need a shared photo row whose trip is gone, which
the foreign key's cascade rules out on a database that enforces it; they
write it with SQLite's foreign keys off as before and are skipped on
Postgres, which has no such switch.
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well. The OAuth service they drove by hand over a second ORM on the
SQLite handle is now the container's own, called in a request context.
The MCP audit cases wait for the mcp.tool_call row: the transport writes
it without awaiting it, so on Postgres it can land after the response.
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well.
The two cases about a NUL byte in a place name move to
places-nul-sqlite.test.ts, listed in SQLITE_ONLY: they pin SQLite's value
quoting, and Postgres text cannot hold U+0000. The control characters
0x01 to 0x1F stay in places.test.ts and run on both engines.
They reset through resetDatabase (the seam suite needs no reset) and seed
through the ORM factories instead of the SQLite snapshot mock, so they run
in the Postgres lane as well.
They reset through resetDatabase and seed through the ORM factories
instead of the SQLite snapshot mock, so they run in the Postgres lane as
well, the user erasure cases included.
They boot the app without the SQLite snapshot mock, so they run in the
Postgres lane as well. The demo seed suite spies on the real runDemoSeed
export instead of replacing the database module to wrap it.
They boot the app without the SQLite snapshot mock, so they run in the
Postgres lane as well. sessionRowsOf reads a user's session rows through
a booted app's ORM, next to sessionRows for the suites that still hand in
a better-sqlite3 handle.
Seeds through the ORM factories, resets through resetDatabase and mints
ws-tokens through the container's TokenService instead of one built by hand
over the SQLite snapshot mock, so it runs in the Postgres lane as well.
Seeds through the booted container's ORM instead of a second ORM over the
SQLite snapshot mock, and resets through resetDatabase, so it runs in the
Postgres lane as well.
PLUGIN_DATA binds the plugin data backend to the engine (the same
{ sqlite, postgres }[engine.id] binding the backup port uses): the SQLite
file per plugin, unchanged, or the Postgres schema per plugin. The host
opens a plugin's store through it and an uninstall that deletes data erases
through it, which drops the schema on Postgres. Nothing outside db/engine
asks which engine runs.
A plugin's SQL is passed through as written, so a plugin needs to know the
dialect: db.engine (db:own) answers 'sqlite' or 'postgres', and the SDK
exposes it as ctx.db.engine(), in the mock host (dbEngine option) and in
dev. The plugin facts are regenerated and the SDK goes to 1.10.0.
The Postgres lane's PGPLUG suite installs, uses and erases plugin data on
Postgres and shows that a plugin reaches neither TREK's users table nor
another plugin's schema.
SQLite takes NULL as the smallest value and Postgres as the largest, so any
list ordered by a nullable column (bookings without a time, trips without
dates) came back in another order on Postgres. Instead of every ORDER BY
spelling its NULL position, the Postgres engine renders SQLite's wherever a
statement names none: a Kysely query compiler for this.kysely() statements
and a MikroORM platform for find() and the query builder. An explicit
position is kept.
A new Postgres-lane suite checks the answers the probe cannot see: the NULL
order, a stay linked as '14.0' in the public reads, the traveller owner
flag, the identity errors for a lost race, and the plugin caps under a race
on the pool.
TREK supports Postgres 14 to 18, and CI tested 17 only. One server-postgres
job per major now runs the probe and then the lane against the same service,
so the change keeps the job count of the two jobs it replaces. The lane's
global setup refuses a service whose major is not TREK_TEST_PG_MAJOR, and
the image's Postgres boot moves to 18, the major its client tools target.
scripts/ci/workflow-jobs.test.mjs fails when a job other than the scan is
missing from ci-ok's needs, and changed-areas pins that migrations,
repositories, the engine and the lane's files always run the job.
The lane ran a hand-kept list of 26 suites that had already missed
trip-tags-mcp and the broadcast-parity suites, and listed two that never
touch a database. tests/postgres/lane.ts now takes every test file that
reaches src/bootstrap.ts or src/nest/app.module.ts through its imports and
does not vi.mock src/db/database, minus a named SQLITE_ONLY map with the
reason for each.
tests/unit/postgres-lane.test.ts (PGLANE, in the SQLite lane) fails on a
suite that names a boot entry point the discovery does not see, on a stale
exclusion, and on a new suite that boots the app over the SQLite snapshot
mock; the 105 that still do are held in snapshot-suites-baseline.json,
which only shrinks.