From 2edf5f807aa48f4803f9b3f6b95e2858aa09b0fc Mon Sep 17 00:00:00 2001 From: Maurice Date: Tue, 17 Nov 2026 23:05:37 +0100 Subject: [PATCH] fix(db): let a plugin call only an allowlist of Postgres's catalog functions The plugin SQL guard refused functions by name, so every function that reads the catalogs or the server and was not listed passed: obj_description, col_description and shobj_description returned the comments on TREK's tables, columns, database and roles by OID, format_type named a type by its OID, and version(), current_database(), inet_server_addr() and the session keywords described the server. A call to a pg_catalog function is now refused unless the function is on PLUGIN_FUNCTIONS (string, number, date and time, JSON, array, aggregate, window, UUID, range, geometry, network and text search functions), in either spelling of a call, f(...) and the column notation (x).f. The catalog's function names of Postgres 14 to 18 are kept in assets/postgres/catalog-functions.txt, and PGPLUG-021 fails on the lane when the server has a function that list does not name, so a new major cannot widen what a plugin may call. The session keywords current_user, session_user, current_role, user, system_user, current_catalog and current_schema are refused unquoted. --- plugin-sdk/README.md | 21 + server/CLAUDE.md | 2 +- server/assets/postgres/catalog-functions.txt | 2810 +++++++++++++++++ .../engine/postgres/plugin-sql-functions.ts | 496 +++ .../db/engine/postgres/plugin-sql-guard.ts | 41 + server/tests/postgres/plugin-data.test.ts | 52 +- .../postgres/plugin-sql-functions.test.ts | 102 + wiki/Database-Postgres.md | 2 +- 8 files changed, 3523 insertions(+), 3 deletions(-) create mode 100644 server/assets/postgres/catalog-functions.txt create mode 100644 server/src/db/engine/postgres/plugin-sql-functions.ts create mode 100644 server/tests/unit/db/engine/postgres/plugin-sql-functions.test.ts diff --git a/plugin-sdk/README.md b/plugin-sdk/README.md index 2cf7efdda..633a32f6c 100644 --- a/plugin-sdk/README.md +++ b/plugin-sdk/README.md @@ -210,6 +210,27 @@ What changes on Postgres: `regconfig`, `regdictionary`, `regnamespace`, `regrole`, `regcollation` and their arrays (`'x'::regtype`, `CAST('x' AS regtype)`, `regtype 'x'`). An alias that shares a schema's name is refused too, so pick another. +- **Postgres's own functions: an allowlist.** You may call the functions that work only on + their arguments: strings and bytes (`lower`, `substring`, `regexp_replace`, `format`, + `md5`, `sha256`, ...), numbers (`round`, `abs`, `random`, ...), dates and times (`now`, + `date_trunc`, `extract`, `to_char`, `make_date`, ...), the conversions (`numeric(10, 2)`, + `varchar(20)`, `int8`, ...), JSON and JSONB (`jsonb_build_object`, `jsonb_set`, + `jsonb_path_query`, ...), arrays (`unnest`, `array_agg`, `generate_series`, ...), + aggregates and window functions (`count`, `sum`, `string_agg`, `row_number`, `lag`, ...), + UUIDs (`gen_random_uuid`), ranges, geometry, network addresses and full text search + (`to_tsvector`, `ts_rank`, ...). Any other function of Postgres's catalog is refused + however you spell the call (`f(...)`, `"f"(...)`, the column notation `(x).f`): the ones + that read the catalogs or the server rather than their arguments (`obj_description`, + `col_description`, `format_type`, `version`, `current_database`, `inet_server_addr`, + `txid_current`, ...), and so is every new function a later Postgres adds until TREK has + decided about it. The keywords `current_user`, `session_user`, `current_role`, `user`, + `system_user`, `current_catalog` and `current_schema` are refused too. `coalesce`, + `nullif`, `greatest`, `least`, `CASE` and `CAST` are syntax and always work. The complete + list is `PLUGIN_FUNCTIONS` in TREK's + [`plugin-sql-functions.ts`](https://github.com/liketrek/TREK/blob/main/server/src/db/engine/postgres/plugin-sql-functions.ts). + A table or type of yours may still share a refused function's name where it cannot be a + call (`INSERT INTO version (...)`, `CREATE TABLE version (...)`, `REFERENCES version (id)`); + as the table of `CREATE INDEX ... ON` it cannot, so name it differently. - **You change your own objects only.** Right before an `ALTER`, `DROP`, `REFRESH`, `TRUNCATE` or `CREATE INDEX ... ON` runs, Postgres resolves the object it names (and the tables a `CREATE`/`ALTER TABLE` ties to with `REFERENCES`, `INHERITS`, `PARTITION OF`, diff --git a/server/CLAUDE.md b/server/CLAUDE.md index b178fdb2c..b41a2a88a 100644 --- a/server/CLAUDE.md +++ b/server/CLAUDE.md @@ -79,7 +79,7 @@ Nest owns everything. Every domain is a DI module under `src/nest//` (`c - **Realtime**: `/ws` is a Nest gateway in `src/nest/realtime/`. Inject `RealtimeService`; tests hand in a `FakeRealtimeService` (`tests/helpers/fake-realtime.ts`) or spy on the container's instance instead of mocking the transport module. Trip mutations forward `X-Socket-Id` so the originating client doesn't echo its own change. - **MCP**: `src/nest-mcp/` is the decorator/registry layer (`@McpController`/`@Tool`/`@Resource`/`@Prompt`; see its `README.md`); tools live in `.mcp.ts` files beside each module; the HTTP transport is `src/nest/mcp-transport/`; `src/mcp/` holds process-wide state only (scopes, session manager, boot policy). Every tool invocation writes an audit row through the registry's `onInvoke` seam. The demo-mode write block is the registry's `toolGate` (`trekDemoToolGate`): every tool not annotated read-only refuses a demo account there (except the write tools in `DEMO_UNGATED_WRITE_TOOLS`, which never had a demo check), so a tool never checks demo mode itself (ESLint refuses it in `*.mcp.ts`). MCP tools are adapters over the same services and permission checks as REST — a change on one side lands on the other in the same change. - **Scheduling**: `src/nest/scheduling/` (`CronRegistrarService`) is the one way to schedule a cron. Jobs are `*.job.ts` providers in their owning domain. The registrar refuses to schedule under `NODE_ENV=test` (an integration test pins it). **Never use a `@Cron` decorator**: it bypasses that gate. Every tick first takes the job's row in `scheduler_leases` (`SchedulerLeasesRepository.acquire`, one conditional upsert that decides the race), renews it while it runs and keeps it for a short settle window afterwards, so two processes on one database never run the same tick; a job needs nothing of its own for this. Claims on individual rows (the reminder jobs' `claimReminder`) stay where they are: the lease decides who runs a tick, the claim what a tick may send. -- **Plugins** (`src/nest/plugins/`): the child-process runtime (`supervisor/`, `runtime/plugin-host-entry.ts`, `protocol/envelope.ts`) is **intentionally not Nest**; it is the security boundary. `host/rpc-host.ts` registers an RPC method at spawn only if the plugin holds the unlocking permission (registration = authorization). Install-time gates live in `install/`; the manifest's `trek` version range fails closed, and `TREK_PLUGINS_IGNORE_TREK_RANGE` downgrades that to a warning. The protocol tables in `protocol/envelope.ts` (+ `src/plugin-event-sink.ts`) are **generated** into `plugin-sdk/` and `shared/`: run `gen:plugin-facts` after touching them and commit the output. What a plugin gets back is `protocol/output-contract.ts`: every wire method has an entry, and an entity result (`trips.getById`, `todos.list`, `vacay.mine`, ...) is cut down to its published fields in `rpc-host.ts` on the way out, together with every child row it carries whose field is listed in `PLUGIN_ENTITY_NESTED` (a day's `notes_items`, a reservation's `endpoints`). A new column on any whole-row table behind those entities, child tables included, fails `tests/unit/plugins/output-contract.schema.test.ts` until it is listed as published or withheld (the SDK's `PLUGIN_ENTITY_FIELDS` and `PLUGIN_ENTITY_NESTED` are generated from it); a method left as `readModel` must return a value built field by field from named columns. A plugin's own SQL (`host/plugin-data.service.ts`) is held to a row cap, and `query`, `tx` and an `exec` script to a time budget checked between rows and statements (`migrate` is deliberately not timed). Where that data lives is the engine's: `PLUGIN_DATA` (`src/nest/plugins/host/plugin-data.backend.ts`) binds the SQLite file per plugin, or on Postgres a schema per plugin (`trek_plugin_`, `src/db/engine/postgres/plugin-data-store.ts`) whose SQL is kept inside that schema in layers that `src/db/engine/postgres/plugin-sql-guard.ts`'s header names, the ones after the guard enforced by the database: no plugin SQL at all while TREK's role is privileged in the sense of `PRIVILEGED_ROLE_SQL` (`src/db/engine/postgres/plugin-data-checks.ts`, read once per pool connection, and the boot log says why); the guard, which refuses what it would read differently from the Postgres lexer, other schemas, the catalogs, temporary objects, `SELECT ... INTO`, the `reg*` types and the functions that look an object up by name; the objects an ALTER, DROP, REFRESH, TRUNCATE or CREATE INDEX changes, resolved by Postgres (`to_regclass`/`to_regtype`) right before the statement and refused outside the plugin's schema, so an unqualified `int4` is `pg_catalog`'s and refused even on a superuser connection; one statement per round trip over the extended query protocol, which the store sends on a pool client it checks out for each call and runs its transaction on directly; `pg_temp` last on the plugin's and on every pool connection's `search_path`, `session-search-path.ts`, with the store dropping its client's temporary tables and other session state before the client goes back to the pool; and a quota over every relation of the schema with its TOAST table plus a cap on its catalog objects (`PLUGIN_SCHEMA_MAX_OBJECTS`). A plugin argument holding NUL is refused there, since Postgres text cannot hold it, while the plugin's SQLite file binds it exactly; the plugin's SQL is never translated, and `db.engine` tells the plugin which dialect to write. Backups and restores carry the plugin schemas the role owns (`pg-tools.ts`). `runtime/egress-policy.ts` is a hand-kept mirror guarded by a parity test in `plugin-sdk/`; change both sides together. +- **Plugins** (`src/nest/plugins/`): the child-process runtime (`supervisor/`, `runtime/plugin-host-entry.ts`, `protocol/envelope.ts`) is **intentionally not Nest**; it is the security boundary. `host/rpc-host.ts` registers an RPC method at spawn only if the plugin holds the unlocking permission (registration = authorization). Install-time gates live in `install/`; the manifest's `trek` version range fails closed, and `TREK_PLUGINS_IGNORE_TREK_RANGE` downgrades that to a warning. The protocol tables in `protocol/envelope.ts` (+ `src/plugin-event-sink.ts`) are **generated** into `plugin-sdk/` and `shared/`: run `gen:plugin-facts` after touching them and commit the output. What a plugin gets back is `protocol/output-contract.ts`: every wire method has an entry, and an entity result (`trips.getById`, `todos.list`, `vacay.mine`, ...) is cut down to its published fields in `rpc-host.ts` on the way out, together with every child row it carries whose field is listed in `PLUGIN_ENTITY_NESTED` (a day's `notes_items`, a reservation's `endpoints`). A new column on any whole-row table behind those entities, child tables included, fails `tests/unit/plugins/output-contract.schema.test.ts` until it is listed as published or withheld (the SDK's `PLUGIN_ENTITY_FIELDS` and `PLUGIN_ENTITY_NESTED` are generated from it); a method left as `readModel` must return a value built field by field from named columns. A plugin's own SQL (`host/plugin-data.service.ts`) is held to a row cap, and `query`, `tx` and an `exec` script to a time budget checked between rows and statements (`migrate` is deliberately not timed). Where that data lives is the engine's: `PLUGIN_DATA` (`src/nest/plugins/host/plugin-data.backend.ts`) binds the SQLite file per plugin, or on Postgres a schema per plugin (`trek_plugin_`, `src/db/engine/postgres/plugin-data-store.ts`) whose SQL is kept inside that schema in layers that `src/db/engine/postgres/plugin-sql-guard.ts`'s header names, the ones after the guard enforced by the database: no plugin SQL at all while TREK's role is privileged in the sense of `PRIVILEGED_ROLE_SQL` (`src/db/engine/postgres/plugin-data-checks.ts`, read once per pool connection, and the boot log says why); the guard, which refuses what it would read differently from the Postgres lexer, other schemas, the catalogs, temporary objects, `SELECT ... INTO`, the `reg*` types, the functions that look an object up by name, and every function of `pg_catalog` outside the allowlist in `src/db/engine/postgres/plugin-sql-functions.ts` (its catalog list, `assets/postgres/catalog-functions.txt`, is held to `pg_proc` by PGPLUG-021); the objects an ALTER, DROP, REFRESH, TRUNCATE or CREATE INDEX changes, resolved by Postgres (`to_regclass`/`to_regtype`) right before the statement and refused outside the plugin's schema, so an unqualified `int4` is `pg_catalog`'s and refused even on a superuser connection; one statement per round trip over the extended query protocol, which the store sends on a pool client it checks out for each call and runs its transaction on directly; `pg_temp` last on the plugin's and on every pool connection's `search_path`, `session-search-path.ts`, with the store dropping its client's temporary tables and other session state before the client goes back to the pool; and a quota over every relation of the schema with its TOAST table plus a cap on its catalog objects (`PLUGIN_SCHEMA_MAX_OBJECTS`). A plugin argument holding NUL is refused there, since Postgres text cannot hold it, while the plugin's SQLite file binds it exactly; the plugin's SQL is never translated, and `db.engine` tells the plugin which dialect to write. Backups and restores carry the plugin schemas the role owns (`pg-tools.ts`). `runtime/egress-policy.ts` is a hand-kept mirror guarded by a parity test in `plugin-sdk/`; change both sides together. - **Storage**: `src/nest/storage/` is the upload driver abstraction (local / S3 / mirror behind one interface, with an admin surface and a usage job). Domain services never touch the filesystem or the S3 SDK directly. Where the data lives (`data/`, `uploads/`, backups, logs, key files, the DB file) is resolved in one place, `src/app-config/data-paths.ts`; a provider that reads one injects `DataPathsService`, plain helper modules shared with code outside the container call `resolveDataPaths()` (the list is in `src/app-config/README.md`), and nothing anchors a data path on its own `__dirname` or on `process.cwd()`. - **DB** (`src/db/`): `better-sqlite3`, WAL, FK on. `database.ts` only holds the connection and applies pragmas. It does not build a schema, and importing it opens nothing (the first reader does). The connection's lifecycle belongs to the `DatabaseLifecycle` provider (`nest/database/database-lifecycle.service.ts`): `buildApp()` opens it through `open()` (which binds the ORM to later swaps and runs the schema bootstrap below), the shutdown closes it through `close()`, and a restore or the demo reset swaps the file only through the database backup port, `DATABASE_BACKUP` (`nest/database/database-backup.interface.ts`; the SQLite implementation is `nest/backup/sqlite-database-backup.ts`: `VACUUM INTO` snapshot, close, atomic file swap, `reopen()`). The port works on `DatabaseLifecycle.file`, the one resolved path (`TREK_DB_FILE` or `data/travel.db`), and a snapshot that fails fails the backup rather than archiving the live file. `MaintenanceRepository` is provided only by `MaintenanceModule`'s factory and injected everywhere else; no consumer constructs it. A provider that has to follow every reopen (the demo reset job saves the first demo baseline after a restore's re-bootstrap seeded the example trips) registers with `DatabaseLifecycle.onReopened()`, which runs after the reopen's schema bootstrap; a listener contains its own failures, since one that throws fails the reopen. MikroORM's migrations (`db/migrations/`) are the only schema source, **including for tests**: the hand-kept `schema.ts`/`migrations.ts`/`seeds.ts` builders and the parity test that guarded them are gone. Seeding is the MikroORM seeders in `db/seeders/`, applied by `db/orm.ts` from `buildApp()` before anything reads. An install the retired positional runner migrated (a `schema_version` row, no `mikro_orm_migrations` rows) is baselined by `db/legacy-baseline.ts`: the migrations whose `Legacy migration step N` docstring is ≤ its version are recorded as executed (never replayed) in the same transaction as the migrator run, while the baseline migration itself still runs first, as the legacy runner's `createTables()` did on every boot. It refuses the boot when `schema_version` is not exactly one row holding an integer ≥ 1, or is above the last mapped step (258); a value from 1 to 258 is read as dev's steps of those numbers, so a pre-ORM feature-branch database whose steps differ is not detected (the Tours branch is the one case it cuts back). Recorded `mikro_orm_migrations` names this build does not ship refuse it too (`refuseNewerDatabase`), whether a newer release or a development build wrote them. 4.3.x and earlier have no such check and start on a database 5.x migrated. Before a boot applies pending migrations to a file database that holds anything, `db/pre-migrate-snapshot.ts` writes a `VACUUM INTO` copy next to it (`pre-migrate--