fix(db): run plugin SQL one statement at a time and search pg_temp last

The plugin SQL guard was the only thing between a plugin and TREK's tables
on Postgres, and two gaps in it were enough to leave the plugin schema.
Two layers now hold even when the guard is fooled.

A plugin write with no bound args went over the simple query protocol,
which runs every statement in its text. Plugin statements now go through
extendedQuery, which pg sends over the extended protocol whatever the
args, so a second statement is a syntax error instead of a statement that
runs. A marker parameter carries the request to the pool's clients, and a
client without the hook binds one value too many, so it fails closed.

pg_temp is searched before the search_path unless the path names it, and
a temporary table lives as long as its pooled connection, so one left
behind would stand in for users, sessions and every other table for the
next caller. Every connection of TREK's pool now puts pg_temp last on the
search_path the role configured, and a plugin transaction sets its schema
with pg_temp last.
This commit is contained in:
Maurice
2026-11-17 20:45:39 +01:00
parent ff57883e3d
commit 87612c0c79
9 changed files with 259 additions and 11 deletions
@@ -0,0 +1,55 @@
import { CompiledQuery } from 'kysely';
/**
* Statements that must run alone: sent over Postgres's extended query
* protocol, whose Parse message takes exactly one statement, so a second one
* in the text is a syntax error instead of a statement that runs.
*
* `pg` picks the protocol by the values it is handed: with none it sends the
* text over the simple protocol, which runs every statement in it. Kysely
* hands it a query's parameters as they are, so {@link extendedQuery} adds a
* marker as the last parameter, and the connection hook
* {@link sendMarkedQueriesExtended} (installed on every client of TREK's pool
* by `postgres-driver.ts`) takes the marker off again and asks `pg` for the
* extended protocol. A query that reached a client without the hook still goes
* over the extended protocol, with one parameter too many, and Postgres
* refuses it: the marker fails closed.
*
* Plugin SQL (`plugin-data-store.ts`) runs through here. A read streamed
* through a cursor needs none of this: a cursor always uses the extended
* protocol.
*/
/** The last parameter of a query {@link extendedQuery} built. */
const EXTENDED_ONLY = Object.freeze({ trekExtendedProtocol: true });
/** `sql` with `args` as a query sent over the extended protocol, so it runs as one statement or not at all. */
export function extendedQuery(sql: string, args: readonly unknown[]): CompiledQuery {
return CompiledQuery.raw(sql, [...args, EXTENDED_ONLY]);
}
/** The values `pg` was handed, without the marker, when {@link extendedQuery} built them; otherwise null. */
export function unmarkedValues(values: unknown): unknown[] | null {
if (!Array.isArray(values) || values.length === 0 || values[values.length - 1] !== EXTENDED_ONLY) return null;
return values.slice(0, -1);
}
/** The part of a `pg` client this hook replaces. */
interface QueryClient {
query: (...args: unknown[]) => unknown;
}
/**
* Makes `client` send a query {@link extendedQuery} built over the extended
* protocol (`queryMode: 'extended'`, which `pg` honours even without values),
* and every other query as before.
*/
export function sendMarkedQueriesExtended(client: QueryClient): void {
const query = client.query.bind(client);
client.query = (...args: unknown[]) => {
const [text, values, ...rest] = args;
const unmarked = unmarkedValues(values);
if (typeof text !== 'string' || unmarked === null) return query(...args);
return query({ text, values: unmarked, queryMode: 'extended' }, ...rest);
};
}
@@ -1,4 +1,5 @@
import { PLUGIN_DATA_CAPS, type PluginDataStore, type PluginTxOp, type PluginTxResult } from '../plugin-data';
import { extendedQuery } from './extended-query';
import { isPluginId, pluginSchemaName, quotePluginSchema } from './plugin-schema';
import { checkPlaceholders, checkQualifiers, parsePluginSql, type PluginStatement } from './plugin-sql-guard';
@@ -9,10 +10,13 @@ import { CompiledQuery, type Kysely, type Transaction } from 'kysely';
* database (`plugin-schema.ts`), reached through TREK's pool.
*
* Every call runs in one transaction that first makes sure the schema and its
* migration table exist, then puts only that schema on the `search_path` and
* sets `statement_timeout` to the call's time budget. The plugin's SQL is
* checked by `plugin-sql-guard.ts` before it runs, so it cannot name another
* schema. Reads stream through a cursor and stop at the row cap. A call that
* migration table exist, then puts only that schema on the `search_path`, with
* `pg_temp` explicitly last so a temporary table is never found before the
* plugin's own, and sets `statement_timeout` to the call's time budget. The
* plugin's SQL is checked by `plugin-sql-guard.ts` before it runs, so it cannot
* name another schema, and each statement goes to Postgres on its own over the
* extended query protocol (`extended-query.ts`), which runs one statement and
* no more. Reads stream through a cursor and stop at the row cap. A call that
* may write measures the schema's size before and after, and rolls back one
* that leaves it past the quota and larger than it found it, so a plugin at
* its quota can still delete.
@@ -267,7 +271,7 @@ export class PostgresPluginDataStore implements PluginDataStore {
const setup = [
`create schema if not exists ${this.quoted}`,
`create table if not exists ${this.quoted}._plugin_migrations (id text primary key, applied_at bigint)`,
`set local search_path to ${this.quoted}`,
`set local search_path to ${this.quoted}, pg_temp`,
'set local standard_conforming_strings to on',
];
if (timeoutMs !== null) setup.push(`set local statement_timeout to ${Math.max(1, Math.floor(timeoutMs))}`);
@@ -313,7 +317,11 @@ export class PostgresPluginDataStore implements PluginDataStore {
return Number(result.rows[0]?.bytes ?? 0);
}
/** Runs a statement for its effect: the rows it changed, or 0 for a read, whose rows are streamed past and dropped. */
/**
* Runs a statement for its effect: the rows it changed, or 0 for a read,
* whose rows are streamed past and dropped. Both ways use the extended
* protocol: a cursor always does, and a write goes through `extendedQuery`.
*/
private async run(trx: Trx, statement: PluginStatement, args: readonly unknown[]): Promise<number> {
if (!statement.mayWrite) {
const chunks = trx.getExecutor().stream(CompiledQuery.raw(statement.text, [...args]), CHUNK_ROWS);
@@ -322,7 +330,7 @@ export class PostgresPluginDataStore implements PluginDataStore {
}
return 0;
}
const result = await trx.executeQuery(CompiledQuery.raw(statement.text, [...args]));
const result = await trx.executeQuery(extendedQuery(statement.text, args));
return Number(result.numAffectedRows ?? 0);
}
@@ -1,7 +1,13 @@
import { sendMarkedQueriesExtended } from './extended-query';
import { SqliteNullOrderPlatform, withSqliteNullOrder } from './null-order';
import { type Configuration } from '@mikro-orm/core';
import { PostgreSqlConnection, PostgreSqlDriver } from '@mikro-orm/postgresql';
/** The `pg` pool event a new client is handed to before anyone uses it. */
interface ClientEvents {
on(event: 'connect', listener: (client: { query: (...args: unknown[]) => unknown }) => void): void;
}
/** `int8` (bigint), a `pg_type.oid`: every integer column and every `count(*)` on Postgres. */
const INT8_OID = 20;
/** `numeric`: what `sum()` of a bigint and `avg()` answer. */
@@ -51,9 +57,19 @@ class NumberTypesConnection extends PostgreSqlConnection {
return options;
}
/** The pool's Kysely dialect, compiling every ORDER BY with SQLite's NULL position (`null-order.ts`). */
/**
* The pool's Kysely dialect, compiling every ORDER BY with SQLite's NULL
* position (`null-order.ts`), on a pool whose every client sends the queries
* `extendedQuery` builds over the extended protocol (`extended-query.ts`).
*/
override createKyselyDialect(overrides: Parameters<PostgreSqlConnection['createKyselyDialect']>[0]) {
return withSqliteNullOrder(super.createKyselyDialect(overrides)) as ReturnType<
const given = (overrides ?? {}) as { onPoolCreated?: (pool: unknown) => unknown };
const onPoolCreated = (pool: unknown) => {
(pool as ClientEvents).on('connect', sendMarkedQueriesExtended);
return given.onPoolCreated?.(pool);
};
const withHook = { ...given, onPoolCreated } as Parameters<PostgreSqlConnection['createKyselyDialect']>[0];
return withSqliteNullOrder(super.createKyselyDialect(withHook)) as ReturnType<
PostgreSqlConnection['createKyselyDialect']
>;
}
@@ -5,9 +5,12 @@ import type { DatabaseEngine } from '../engine';
import { redactUrl } from '../redact-url';
import { TrekPostgreSqlDriver } from './postgres-driver';
import { bootstrapPostgresSchema, defaultPostgresBootstrapDeps, type PostgresBootstrapDeps } from './schema-bootstrap';
import { putTempSchemaLast } from './session-search-path';
import { sslOptionFor } from './ssl';
import type { EntityManager } from '@mikro-orm/core';
import type { DatabaseConnection } from 'kysely';
/** What `TREK_DB_URL` and `TREK_DB_POOL_MAX` resolve to, when they name a Postgres database. */
type PostgresSettings = Extract<DbEngineSettings, { engine: 'postgres' }>;
@@ -78,6 +81,7 @@ export class PostgresEngine implements DatabaseEngine {
return `Postgres ${redactUrl(this.settings.url)}`;
}
/** Every new pool connection first puts `pg_temp` last on its `search_path` (`session-search-path.ts`). */
ormOptions() {
const ssl = sslOptionFor(this.settings.url);
return {
@@ -85,6 +89,7 @@ export class PostgresEngine implements DatabaseEngine {
clientUrl: this.settings.url,
pool: { min: 0, max: this.settings.poolMax },
driverOptions: ssl === undefined ? {} : { ssl },
onCreateConnection: (connection: unknown) => putTempSchemaLast(connection as DatabaseConnection),
};
}
}
@@ -0,0 +1,43 @@
import { CompiledQuery, type DatabaseConnection } from 'kysely';
/**
* Puts `pg_temp` last on the `search_path` of every connection of TREK's pool.
*
* Postgres searches the session's temporary schema before everything on the
* `search_path` unless the path names `pg_temp` itself, and a temporary table
* lives as long as its connection. On a pool, whatever one caller left there
* would stand in for TREK's tables (`users`, `sessions`, ...) for every later
* caller of that connection. Named last, the temporary schema is searched
* after every schema TREK uses, so its tables are never found first. The rest
* of the path stays as the role or the database configured it: TREK may run
* in a schema other than `public`, and finds it through that path.
*
* Plugin transactions set their own path the same way
* (`plugin-data-store.ts`); `plugin-sql-guard.ts` explains the layers.
*/
/** `path` (as `SHOW search_path` prints it) with every `pg_temp` entry taken out and one put last. */
export function searchPathWithTempLast(path: string): string {
const entries: string[] = [];
let current = '';
let quoted = false;
for (const ch of path) {
if (ch === '"') quoted = !quoted;
if (ch === ',' && !quoted) {
entries.push(current.trim());
current = '';
} else {
current += ch;
}
}
entries.push(current.trim());
const kept = entries.filter((entry) => entry !== '' && entry.toLowerCase() !== 'pg_temp' && entry !== '"pg_temp"');
return [...kept, 'pg_temp'].join(', ');
}
/** The connection hook (`onCreateConnection`) that sets {@link searchPathWithTempLast} for the session. */
export async function putTempSchemaLast(connection: DatabaseConnection): Promise<void> {
const shown = await connection.executeQuery<{ search_path: string }>(CompiledQuery.raw('show search_path'));
const path = searchPathWithTempLast(shown.rows[0]?.search_path ?? '');
await connection.executeQuery(CompiledQuery.raw("select set_config('search_path', $1, false)", [path]));
}