Commit Graph

16 Commits

Author SHA1 Message Date
Maurice f2cfbc4041 fix(db): close the move's remaining silent changes
An attack on the move to Postgres with real SQLite files found five values
that changed without a refusal or a listing, or a valid file that failed:

- Negative zero in a plugin column became 0: pg sends the number -0 as "0",
  and the checksum printed both as "0". It is sent as text and kept apart in
  the checksum on both sides.
- Text whose bytes are not valid UTF-8 was read with each bad sequence
  replaced by U+FFFD, and the checksum was built from the replaced text.
  Text is now read as its bytes and decoded strictly in the file's encoding
  (UTF-8, or UTF-16 for a file created so); such a value is refused in
  TREK's and in plugin data, with --drop-nul too, and listed with its bytes.
- --drop-nul serialized a JSON value again, so a large number lost digits
  and 1.0 became 1. Only the \u0000 escapes are removed now.
- A plugin default Postgres refuses for the column (DEFAULT 0 on a BLOB
  column) failed the move, and DEFAULT 1.5 on an integer column would have
  been rounded. Such a default is left off and listed as not carried over.
- A plugin index or primary key SQLite compared with COLLATE NOCASE or RTRIM
  became one that compares exactly. It is still created, and the summary
  lists the collation it lost.

Each case has a regression test that fails without its fix, on SQLite and
on the Postgres lane.
2026-11-17 23:30:35 +00:00
Maurice 636fc7d0a6 fix(db): refuse the column notation without parentheses and aclitem in plugin SQL
Postgres reads x.f as the call f(x) when x has no column f. The guard counted
that spelling only after ")." or "].", so over a function in FROM that
returns a single value, or a parameter, the allowlist was bypassed:
"select g.obj_description from generate_series(...) g" returned the comments
on every object of the database, and g.current_schemas, g.txid_status and
$1.ts_token_type passed as well. A refused catalog function after a dot is
now refused too, except the names in COLUMN_NAMES (money, name, oid, version,
xid, xml), which take no argument or only convert it to their type, so t.name
keeps working. PGPLUG-021 holds that list to pg_proc on the lane's server.

The type aclitem names roles: '=r'::aclitem printed the cluster's first
superuser and 'x=r/x'::aclitem told whether a role exists. It is refused in
every spelling like the reg* types.

The allowlist itself moves to assets/postgres/plugin-functions.txt, beside
the catalog list it is checked against. As TypeScript it spelled 'like' and
'json_each', names of Postgres functions that the SQLite dialect gate reads
as SQLite SQL.
2026-11-17 23:30:29 +00:00
Maurice c75fb22289 docs(db): describe the move's refusals and --drop-nul
The wiki's move section, the server guide and the nul-free header said
the move drops NUL as every write does. They now say what it does: a
plugin's data moves unchanged or the move is refused with the values
listed, TREK's own NUL is refused unless --drop-nul is given, which lists
every value it changed, and the checksum comes from the SQLite values.
2026-11-17 23:28:42 +01:00
Maurice 2edf5f807a fix(db): let a plugin call only an allowlist of Postgres's catalog functions
The plugin SQL guard refused functions by name, so every function that reads
the catalogs or the server and was not listed passed: obj_description,
col_description and shobj_description returned the comments on TREK's
tables, columns, database and roles by OID, format_type named a type by its
OID, and version(), current_database(), inet_server_addr() and the session
keywords described the server.

A call to a pg_catalog function is now refused unless the function is on
PLUGIN_FUNCTIONS (string, number, date and time, JSON, array, aggregate,
window, UUID, range, geometry, network and text search functions), in
either spelling of a call, f(...) and the column notation (x).f. The
catalog's function names of Postgres 14 to 18 are kept in
assets/postgres/catalog-functions.txt, and PGPLUG-021 fails on the lane
when the server has a function that list does not name, so a new major
cannot widen what a plugin may call. The session keywords current_user,
session_user, current_role, user, system_user, current_catalog and
current_schema are refused unquoted.
2026-11-17 23:22:59 +01:00
Maurice 9fd77b04fa docs(db): describe the layers that hold plugin SQL on Postgres and the NUL rule for plugin data
The plugin-sdk README, the Postgres wiki page and server/CLAUDE.md now name
the layers as the code enforces them: the refusal on a privileged role and
the boot log line, the guard (the reg* types, INTO after the statement's
own INSERT or MERGE only, TEMP only where it makes a temporary object), the
targets Postgres resolves before an ALTER, DROP, REFRESH, TRUNCATE or
CREATE INDEX, one statement per round trip, pg_temp last, and the quota over
every relation with the object cap. They also say that a plugin argument
holding NUL is refused on Postgres and bound exactly on SQLite.
2026-11-17 23:04:12 +01:00
Maurice 9808e113e4 fix(db): refuse restores for CREATEROLE members, server file grants and parameter grants
The privileged-role check missed three ways a role can reach the database
host from SQL. Before PostgreSQL 16 a member of a CREATEROLE role sets its
role to it and grants itself pg_execute_server_program, since role
attributes are not inherited but the grant checks the role in effect. A role
granted EXECUTE on lo_export, lo_import, pg_read_file, pg_ls_dir and their
siblings reads or writes server files, and lo_export keeps the file after the
transaction rolls back. From PostgreSQL 15 on a role can be granted ALTER
SYSTEM on a setting or SET on a superuser-only one.

PRIVILEGED_ROLE_SQL now counts all three. The parameter grants are read
through query_to_xml inside a CASE taken only on 15 or later, because a
statement naming pg_parameter_acl does not parse on 14. A SET grant on a
setting every role may change anyway, and pg_monitor, stay ordinary. The
lane test creates each kind of role and shows the escalation the check is
for on the server it runs against.
2026-11-17 23:04:11 +01:00
Maurice e6f4f7a5f3 fix(db): leave nothing behind when a first Postgres boot fails, so RESTORE_FROM_BACKUP still runs
createFromBaseline created the migration history's table through MikroORM's
storage before, and outside, the transaction that runs the baseline. Without a
master transaction MikroORM commits that table on its own, so a first boot that
failed in the baseline (a server without ICU, a timeout, a dropped connection)
left an empty mikro_orm_migrations behind. The next start with
RESTORE_FROM_BACKUP took that for a database in use, skipped the restore with a
warning and brought up an empty instance, after which the restore never ran.

The table is now created on the baseline's own transaction, so a failed first
boot leaves the database empty. The boot restore also treats a lone migration
history that records nothing as no database, which a restore clears anyway; any
other answer keeps it, so the check fails closed. Database-Postgres says both.
2026-11-17 23:04:11 +01:00
Maurice 77d90be4f8 docs(db): describe plugin SQL on its own pool client instead of the marker hook
The guard's header, server/CLAUDE.md and the Postgres wiki page named
extended-query.ts and a hook on every pool connection. They now say that the
store sends each plugin statement over the extended protocol on a client it
checks out per call, and drops that client's temporary tables and session
state before handing it back.
2026-11-17 23:04:10 +01:00
Maurice f211a1c614 docs(db): describe the NUL, whole-number and calendar-date rules
server/CLAUDE.md names where NUL is dropped, INTCOL and the .int() rule
for integer columns, and the calendar-date check for a bound date, and no
longer lists the places NUL suite as SQLite-only. The Postgres wiki says
the move drops a NUL instead of refusing it.
2026-11-17 23:04:09 +01:00
Maurice a9cdb2f91c docs(db): describe the plugin SQL layers and the refused restore roles
The Postgres wiki page and the plugin author guide list what the plugin
SQL guard now refuses and the two layers behind it, and the wiki and
server/CLAUDE.md name every kind of role a restore refuses, including
CREATEROLE on PostgreSQL 14 and 15, which the managed postgres role on
such a server holds.
2026-11-17 23:04:07 +01:00
Maurice 11f4373076 docs(db): rotating the encryption key on Postgres 2026-11-17 23:03:55 +01:00
Maurice 3f51153dcc docs(db): moving an existing install to Postgres, and the refusal of an empty database beside one 2026-11-17 23:03:55 +01:00
Maurice 17c9c0758f docs(db): plugin data on Postgres, its SQL rules and one TREK per database
The plugin-sdk README says what dialect a plugin writes when TREK runs on
Postgres, what the guard refuses and how the limits hold there. The wiki's
Postgres page describes the plugin schemas, their backup and restore and
the one-instance-per-database limit, and its key rotation path now names
the file where it is.
2026-11-17 23:03:54 +01:00
Maurice 693acd5580 fix(db): reset the admin on Postgres and keep the key rotation off it
reset-admin.js opened data/travel.db with better-sqlite3 whatever the
engine, so on a Postgres install it created a stray SQLite file or reset
the admin in a leftover one. With TREK_DB_URL set it now resets the account
in Postgres through the built server's engine (its pool, SSL and write
lock, in one transaction), and refuses a TREK_DB_FILE beside the URL; the
SQLite branch is unchanged. migrate-encryption.ts cannot re-encrypt the
secrets in Postgres yet, so it now stops before it opens or writes any file
when TREK_DB_URL is set. The wiki says what each does on Postgres.
2026-11-17 23:03:52 +01:00
Maurice 633e9bcdc1 docs(db): Postgres 14 to 18, the database role, restore limits and the manual rollback
- Database-Postgres: supported versions, the version 18 client and the tool
  search order, sslrootcert with a client before 16, how to create a role
  that is not a superuser and move an older compose install to one, a
  current schema other than public, and the space a restore needs.
- Updating: the manual rollback clears the whole schema and loads the copy
  in one transaction, as TREK's restore does; dropping only the tables left
  the collation, the trigger functions and the sequences behind. A unit
  test keeps the printed statement equal to CLEAR_SCHEMA_SQL, and the boot's
  rollback hint points at that section.
- Troubleshooting, Install-Docker-Compose, Environment-Variables, README
  and server/CLAUDE.md follow.
2026-11-17 23:03:52 +01:00
Isaias Tavares 2e3762afe1 feat(db): run TREK on PostgreSQL
TREK runs on PostgreSQL when TREK_DB_URL is set, and on SQLite as before
otherwise: an engine port under src/db/engine, the dialect helpers under
src/db/dialect, a Postgres baseline for the migrations up to the cut,
pg_dump/pg_restore backups, a probe of every repository method against a
real server and a Postgres test lane in CI.
2026-11-17 23:03:33 +01:00