fix(plugins): hold plugin data on Postgres to its own schema and to one contract on both engines

The plugin SQL guard now refuses what still reached outside a plugin's
schema or depended on the server:
- reads of the system columns (xmin, xmax, cmin, cmax, ctid, tableoid),
  which show transaction ids and storage layout and do not exist on
  SQLite; a value of the plugin's own may still be named xmin with AS
- the xid and xid8 types wherever the word stands for the type, since
  age() of an xid is the cluster's transaction counter
- the text search functions that coerce a text argument to regconfig, a
  catalog lookup by name, which leave the allowlist
- collations and access methods outside the set every server has, which
  could tell whether a locale or an extension exists

Every plugin transaction pins, with SET LOCAL, the settings that decide
how values print and parse (TimeZone UTC, DateStyle, the C locales, float,
bytea and xml output, the text search configuration), so a plugin's output
neither depends on nor reveals the server's configuration.

Both stores now keep one contract. Its texts and caps live once, in
plugin-data-contract.ts, and both stores throw only those: NUL in SQL,
an argument or a migration id is refused on SQLite as on Postgres,
transaction control is refused in every call on both engines (tx() is
how a batch becomes atomic), a closed store refuses every call, and the
Postgres store answers the cases better-sqlite3 names with its words.
The SQLite store moves beside the Postgres one as SqlitePluginDataStore
in db/engine/sqlite, takes its file from the backend so src/db never
imports src/nest, and no longer needs the driver exemption. A contract
test runs every violation on both stores and fails when they answer
differently. The Postgres lane holds the allowlist, the pinned settings
and the SQL plugins commonly write against a real database.
This commit is contained in:
Maurice
2026-11-18 13:20:07 +01:00
parent cc0fe6ce2b
commit c970245e0d
26 changed files with 1932 additions and 513 deletions
@@ -138,24 +138,3 @@ export async function schemaUsage(client: PluginSqlClient, schema: string): Prom
const [row] = result.rows as Array<{ bytes?: number | string; objects?: number | string }>;
return { bytes: Number(row?.bytes ?? 0), objects: Number(row?.objects ?? 0) };
}
/**
* Refuses a plugin's bound value that holds U+0000 (NUL): Postgres text
* cannot store the character, and TREK will not change what a plugin asked
* to store or compare (`'a\0b'` and `'ab'` are different keys on SQLite).
* A string, or a string anywhere in an array, is checked; a Buffer goes to
* `bytea`, which holds any byte.
*/
export function refuseNulValues(values: readonly unknown[]): void {
if (values.some(holdsNul)) {
throw new Error(
'a plugin argument holds the character U+0000 (NUL), which Postgres text cannot store; ' +
'TREK refuses it instead of changing the value. Remove it, or bind binary data as a Buffer (bytea)',
);
}
}
function holdsNul(value: unknown): boolean {
if (typeof value === 'string') return value.includes('\u0000');
return Array.isArray(value) && value.some(holdsNul);
}
@@ -1,14 +1,21 @@
import { PLUGIN_DATA_CAPS, type PluginDataStore, type PluginTxOp, type PluginTxResult } from '../plugin-data';
import type { PluginDataStore, PluginTxOp, PluginTxResult } from '../plugin-data';
import {
PLUGIN_DATA_CAPS,
PLUGIN_DATA_ERRORS,
refuseNulSql,
refuseNulValues,
type PluginDataBudgetedCall,
} from '../plugin-data-contract';
import {
PLUGIN_SCHEMA_MAX_OBJECTS,
PrivilegedRoleGate,
refuseForeignTargets,
refuseNulValues,
schemaUsage,
type PluginRoleGate,
type SchemaUsage,
} from './plugin-data-checks';
import { isPluginId, pluginSchemaName, quotePluginSchema } from './plugin-schema';
import { pluginSessionSql } from './plugin-session';
import { checkPlaceholders, checkQualifiers, parsePluginSql, type PluginStatement } from './plugin-sql-guard';
import { CompiledQuery, type Kysely } from 'kysely';
@@ -25,8 +32,11 @@ import Cursor from 'pg-cursor';
* once per connection). The transaction first makes sure the schema and its
* migration table exist, then puts only that schema on the `search_path`,
* with `pg_temp` explicitly last so a temporary table is never found before
* the plugin's own, and sets `statement_timeout` to the call's time budget;
* every setting is `local` and ends with the transaction. The plugin's SQL is
* the plugin's own, sets `statement_timeout` to the call's time budget, and
* pins the settings that decide how values print and parse
* (`plugin-session.ts`: the time zone, the locale, ...), so what a plugin
* reads does not depend on the server's configuration; every setting is
* `local` and ends with the transaction. The plugin's SQL is
* checked by `plugin-sql-guard.ts` before it runs, so it cannot name another
* schema; right before each statement, the existing objects it changes are
* resolved by Postgres and must lie in the plugin's schema
@@ -34,8 +44,9 @@ import Cursor from 'pg-cursor';
* over the extended query protocol, which runs one statement and no more: a
* write as a query with `queryMode: 'extended'`, which `pg` honours even
* without values, and a read through a `pg-cursor`, which always uses that
* protocol. A bound value or a migration id holding U+0000 is refused before
* anything runs, since Postgres text cannot hold it and changing it would
* protocol. SQL, a bound value or a migration id holding U+0000 is refused
* before anything runs, on this engine as on SQLite (`refuseNulValues` in
* `../plugin-data-contract.ts`): Postgres text cannot hold it, and changing it would
* change the plugin's data. Reads stop at the row cap. A call that may write
* measures the schema before and after (`schemaUsage`: the bytes of every
* relation with its TOAST table, and the catalog objects), and rolls back one
@@ -50,12 +61,15 @@ import Cursor from 'pg-cursor';
* (`pluginDataSlots`), so a busy plugin never takes the connections TREK's
* own requests need.
*
* The SQLite store's semantics carry over: `query` and `tx` are held to the
* row cap, `query`, a `tx` batch and an `exec` script to the time budget, a
* script's statements each commit on their own, and `migrate` runs once per
* id with no time budget. What Postgres adds is that the budget also stops a
* single statement (`statement_timeout`), which SQLite cannot, and that an
* `exec` with bound args is held to it as well.
* The SQLite store's semantics (`../sqlite/plugin-data-store.ts`) carry
* over: `query` and `tx` are held to the row cap, `query`, a `tx` batch and
* an `exec` script to the time budget, a script's statements each commit on
* their own, and `migrate` runs once per id with no time budget. What
* Postgres adds is that the budget also stops a single statement
* (`statement_timeout`), which SQLite cannot, and that an `exec` with bound
* args is held to it as well. A call that breaks the contract is refused
* with the words the SQLite store uses (`PLUGIN_DATA_ERRORS`),
* better-sqlite3's own among them.
*/
/** The limits one store enforces. Overridable for tests; production uses `PLUGIN_DATA_CAPS`. */
@@ -81,8 +95,6 @@ const DEFAULT_LIMITS: PostgresPluginDataLimits = {
const QUERY_CANCELED = '57014';
/** Rows a cursor fetches at once. */
const CHUNK_ROWS = 500;
/** What better-sqlite3 says about a closed handle, which the host recognizes. */
const CLOSED = 'The database connection is not open';
/**
* What a client runs before it goes back to the pool, outside any
@@ -206,7 +218,7 @@ export class PostgresPluginDataStore implements PluginDataStore {
const { maxRows } = this.limits;
return this.call('query', (budget) =>
this.transaction(budget.ms, [statement], (client) => {
const rows = { count: 0, max: maxRows, message: `query returned more than ${maxRows} rows` };
const rows = { count: 0, max: maxRows, message: PLUGIN_DATA_ERRORS.queryTooManyRows(maxRows) };
return this.stream(client, statement, args, rows, budget.check);
}),
);
@@ -222,6 +234,7 @@ export class PostgresPluginDataStore implements PluginDataStore {
);
}
const statements = this.parse(sql);
this.refuseIfClosed();
return this.call('exec', async (budget) => {
// A script runs as SQLite runs one: each statement commits on its own, and
// the clock is read before every statement but the first.
@@ -234,16 +247,19 @@ export class PostgresPluginDataStore implements PluginDataStore {
}
async tx(ops: PluginTxOp[]): Promise<PluginTxResult> {
if (!Array.isArray(ops)) throw new Error('tx requires an array of { sql, args }');
if (ops.length === 0) return { results: [] };
if (!Array.isArray(ops)) throw new Error(PLUGIN_DATA_ERRORS.txNotArray);
if (ops.length === 0) {
this.refuseIfClosed();
return { results: [] };
}
if (ops.length > PLUGIN_DATA_CAPS.maxTxOps) {
throw new Error(`tx allows at most ${PLUGIN_DATA_CAPS.maxTxOps} statements`);
throw new Error(PLUGIN_DATA_ERRORS.txTooManyOps(PLUGIN_DATA_CAPS.maxTxOps));
}
const statements = ops.map((op) => this.single(op?.sql, op?.args ?? []));
const { maxRows } = this.limits;
return this.call('tx', (budget) =>
this.transaction(budget.ms, statements, async (client) => {
const rows = { count: 0, max: maxRows, message: `tx returned more than ${maxRows} rows in total` };
const rows = { count: 0, max: maxRows, message: PLUGIN_DATA_ERRORS.txTooManyRows(maxRows) };
const results = await inOrder(statements, async (statement, index) => {
const args = ops[index].args ?? [];
// One budget for the whole batch: each statement after the first gets what is left of it.
@@ -263,6 +279,7 @@ export class PostgresPluginDataStore implements PluginDataStore {
async migrate(id: string, sql: string): Promise<{ applied: boolean }> {
const statements = this.parse(sql);
refuseNulValues([id]);
this.refuseIfClosed();
return this.enqueue(() =>
this.transaction(null, statements, async (client) => {
const seen = await client.query({
@@ -280,23 +297,37 @@ export class PostgresPluginDataStore implements PluginDataStore {
);
}
/** `sql` as exactly one statement, checked against `args`. */
/**
* `sql` as exactly one statement, checked against `args`. The checks run in
* the order the SQLite store meets them (the text, the args, then what
* better-sqlite3's prepare says: a closed handle, no statement, more than
* one), so a call that breaks several rules is told the same on both.
*/
private single(sql: unknown, args: readonly unknown[]): PluginStatement {
const statements = this.parse(sql);
if (statements.length !== 1) throw new Error('sql must hold exactly one statement here; use exec() for a script');
checkPlaceholders(statements[0], args);
refuseNulValues(args);
this.refuseIfClosed();
if (statements.length === 0) throw new Error(PLUGIN_DATA_ERRORS.noStatement);
if (statements.length > 1) throw new Error(PLUGIN_DATA_ERRORS.moreThanOneStatement);
checkPlaceholders(statements[0], args);
return statements[0];
}
/** The statements of `sql`, refused when the guard does not allow one or none is there. */
/**
* The statements of `sql`, refused when it is no string, too long, holds NUL
* or holds a statement the guard does not allow. A script may hold none,
* as on SQLite, where an empty script runs nothing.
*/
private parse(sql: unknown): PluginStatement[] {
if (!this.open) throw new Error(CLOSED);
if (typeof sql !== 'string') throw new Error('sql must be a string');
if (sql.length > PLUGIN_DATA_CAPS.maxSqlLength) throw new Error('sql too long');
const statements = parsePluginSql(sql);
if (statements.length === 0) throw new Error('The supplied SQL string contains no statements');
return statements;
if (typeof sql !== 'string') throw new Error(PLUGIN_DATA_ERRORS.sqlNotString);
if (sql.length > PLUGIN_DATA_CAPS.maxSqlLength) throw new Error(PLUGIN_DATA_ERRORS.sqlTooLong);
refuseNulSql(sql);
return parsePluginSql(sql);
}
/** Refuses a call once {@link close} ran, with better-sqlite3's words for a closed handle. */
private refuseIfClosed(): void {
if (!this.open) throw new Error(PLUGIN_DATA_ERRORS.closed);
}
/**
@@ -304,7 +335,7 @@ export class PostgresPluginDataStore implements PluginDataStore {
* turn comes, and reports a statement Postgres cancelled at
* `statement_timeout` as the budget running out.
*/
private call<T>(what: string, work: (budget: Budget) => Promise<T>): Promise<T> {
private call<T>(what: PluginDataBudgetedCall, work: (budget: Budget) => Promise<T>): Promise<T> {
const { now, timeBudgetMs } = this.limits;
return this.enqueue(async () => {
const startedAt = now();
@@ -312,14 +343,14 @@ export class PostgresPluginDataStore implements PluginDataStore {
ms: timeBudgetMs,
remaining: () => Math.max(1, timeBudgetMs - (now() - startedAt)),
check: () => {
if (now() - startedAt > timeBudgetMs) throw new Error(`${what} exceeded its ${timeBudgetMs} ms time budget`);
if (now() - startedAt > timeBudgetMs) throw new Error(PLUGIN_DATA_ERRORS.overBudget(what, timeBudgetMs));
},
};
try {
return await work(budget);
} catch (err) {
if (sqlState(err) === QUERY_CANCELED) {
throw new Error(`${what} exceeded its ${timeBudgetMs} ms time budget`, { cause: err });
throw new Error(PLUGIN_DATA_ERRORS.overBudget(what, timeBudgetMs), { cause: err });
}
throw err;
}
@@ -382,7 +413,7 @@ export class PostgresPluginDataStore implements PluginDataStore {
`create schema if not exists ${this.quoted}`,
`create table if not exists ${this.quoted}._plugin_migrations (id text primary key, applied_at bigint)`,
`set local search_path to ${this.quoted}, pg_temp`,
'set local standard_conforming_strings to on',
pluginSessionSql(),
];
if (timeoutMs !== null) setup.push(`set local statement_timeout to ${Math.max(1, Math.floor(timeoutMs))}`);
await client.query(setup.join(';\n'));
@@ -0,0 +1,61 @@
/**
* The settings every plugin transaction on Postgres runs with, so that what
* a plugin reads never depends on how the server is configured.
*
* Postgres formats and parses values by settings the server, the database or
* TREK's role may set: `TimeZone` decides how a `timestamptz` prints and
* where `date_trunc('day', now())` cuts, `lc_monetary` how `money` prints,
* `lc_numeric` and `lc_time` what `to_char` writes, `DateStyle` how
* `'01/02/2020'` reads, `default_text_search_config` what `text @@ text`
* matches. Left to the server, a plugin would answer differently on two
* installs, and its output would tell it how the server is set up (its time
* zone, its locale). SQLite has no such settings: its dates are UTC and its
* numbers plain.
*
* So the store sets each of these with `SET LOCAL` at the start of every
* plugin transaction (`plugin-data-store.ts`), to Postgres's own built-in
* default (what a server with an untouched configuration gives), with the
* time zone UTC as on SQLite. `SET LOCAL` ends with the transaction, so
* nothing reaches TREK's own use of the pool connection. A plugin cannot
* change them: the guard refuses `SET`, `set_config` and `current_setting`.
*
* The `search_path` and `statement_timeout` are set by the store per call,
* since they depend on the plugin and its time budget. The Postgres lane
* (PGPLUG-023) lists every setting of the server a client may change in the
* categories that cover output and statement behaviour, and fails on one
* that is neither pinned here nor named there as one that cannot change what
* a plugin reads, so a setting a new Postgres adds is decided on, not missed.
*/
export const PLUGIN_SESSION_SETTINGS: Readonly<Record<string, string>> = {
// How dates, times and intervals print and parse.
TimeZone: 'UTC',
DateStyle: 'ISO, MDY',
IntervalStyle: 'postgres',
timezone_abbreviations: 'Default',
// How money, numbers and month and day names print (`money`, `to_char`, `to_number`).
lc_monetary: 'C',
lc_numeric: 'C',
lc_time: 'C',
// How floating point numbers, bytea and xml print and parse.
extra_float_digits: '1',
bytea_output: 'hex',
xmlbinary: 'base64',
xmloption: 'content',
// The configuration `text @@ text` and the other text search defaults use.
default_text_search_config: 'pg_catalog.simple',
// How SQL reads: string literals, NULL in array input, `x = NULL`, `quote_ident`.
standard_conforming_strings: 'on',
backslash_quote: 'safe_encoding',
array_nulls: 'on',
transform_null_equals: 'off',
quote_all_identifiers: 'off',
// How many rows a GIN index search may return; 0 is all of them.
gin_fuzzy_search_limit: '0',
};
/** The `SET LOCAL` statements for {@link PLUGIN_SESSION_SETTINGS}, in one string. */
export function pluginSessionSql(): string {
return Object.entries(PLUGIN_SESSION_SETTINGS)
.map(([name, value]) => `set local ${name} to '${value}'`)
.join(';\n');
}
@@ -19,9 +19,18 @@ import path from 'node:path';
* refused unless its name is in {@link PLUGIN_FUNCTIONS_FILE}, the string, math,
* date and time, JSON, array, range, aggregate, window, UUID and text search
* functions a data plugin plausibly needs, each of which works only on its
* arguments. A name counts as called when `(` follows it (`version()`,
* `"version" ()`) or when it follows a dot, Postgres's column notation for
* a call (`(16386::oid).obj_description`, `u.obj_description` over `FROM
* arguments. None of them takes or returns a type that looks an object up by
* a name (`regconfig`, `regclass`, ..., `aclitem`): such a parameter turns a
* text argument into a catalog lookup, so `to_tsvector('schema.cfg', ...)`
* would tell whether a schema exists. None returns a transaction id type
* either, so the only ways to an `xid` value, which `age(xid)` turns into
* the server's transaction counter, are the system columns and the type's
* name, which the guard refuses. PGPLUG-022 checks every overload of every
* listed name on the lane's server.
*
* A name counts as called when `(` follows it (`version()`, `"version" ()`)
* or when it follows a dot, Postgres's column notation for a call
* (`(16386::oid).obj_description`, `u.obj_description` over `FROM
* unnest(...) u`, `$1.obj_description`), unless it is one of
* {@link COLUMN_NAMES}. A plugin's own table or type may still carry such a
* name where it cannot be a call: after INTO, TABLE, VIEW, REFERENCES or IF
@@ -63,12 +72,13 @@ export const SESSION_KEYWORDS: ReadonlySet<string> = new Set([
* returns a single value (`FROM unnest(...) u`, `FROM generate_series(...)
* g`) or a parameter (`$1.f`), that value is the argument. None of these
* reads anything called that way: `version` takes no argument, so the
* notation never calls it, and `money`, `name`, `oid`, `xid` and `xml` only
* notation never calls it, and `money`, `name`, `oid` and `xml` only
* convert their argument to the type of that name. Every other refused name
* is refused after a dot too. PGPLUG-021 holds each name here to that on the
* lane's server.
* is refused after a dot too. `xid` is not here: `u.xid` casts `u` to the
* transaction id type, whose `age()` reads the server's transaction counter.
* PGPLUG-021 holds each name here to that on the lane's server.
*/
export const COLUMN_NAMES: ReadonlySet<string> = new Set(['money', 'name', 'oid', 'version', 'xid', 'xml']);
export const COLUMN_NAMES: ReadonlySet<string> = new Set(['money', 'name', 'oid', 'version', 'xml']);
/** The function names of `pg_catalog` on every supported major, one per line, sorted. */
export const CATALOG_FUNCTIONS_FILE = path.join(
+209 -15
View File
@@ -1,3 +1,4 @@
import { PLUGIN_DATA_ERRORS, PLUGIN_TRANSACTION_CONTROL, refuseNulSql } from '../plugin-data-contract';
import { PLUGIN_SCHEMA_PREFIX } from './plugin-schema';
import { refusedCatalogCall, refusedCatalogField, SESSION_KEYWORDS } from './plugin-sql-functions';
@@ -83,6 +84,24 @@ import { refusedCatalogCall, refusedCatalogField, SESSION_KEYWORDS } from './plu
* `regcollation`, and their array types `_regclass`, ...), in every
* spelling: `'x'::regtype`, `CAST('x' AS regtype)`, `regtype 'x'`, quoted;
* and `aclitem`, whose input and output name roles;
* - a read of a system column every Postgres table has (`xmin`, `xmax`,
* `cmin`, `cmax`, `ctid`, `tableoid`), quoted or not: they show the
* server's transaction ids, where a row sits on disk and the table's OID,
* and SQLite has none of them. Nothing of TREK's reads them either; a
* plugin that wants optimistic locking keeps a version column of its own.
* The name may still be given to a value of the plugin's own, after AS or
* in the column list of a WITH query or an alias (`min(lng) AS xmin`);
* - the transaction id types `xid` and `xid8` (and `_xid`, `_xid8`) where
* the word stands for the type: `age('3'::xid)` is how many transactions
* the cluster ran since. That is after `::`, before a string, as a call
* or after a dot (`xid(x)` and `x.xid` are casts when `x` has no such
* column), after AS in a CAST, and after any word but the keywords that
* precede a column (`c xid` in a column definition, `TYPE xid`). A column
* of the plugin's own may be called `xid` (`SELECT xid FROM payments`);
* - a collation other than `"C"`, `"POSIX"`, `"default"` and `ucs_basic`,
* and an access method other than the built-in ones (`USING bloom`): which
* others exist depends on the server's locales, ICU and extensions, so
* naming one would tell a plugin about the server;
* - a temporary object: CREATE TEMP or TEMPORARY (also after OR REPLACE,
* LOCAL or GLOBAL), and SELECT ... INTO TEMP; a temporary table outlives
* the call on a pooled connection. Elsewhere `temp` is an ordinary name;
@@ -104,11 +123,23 @@ import { refusedCatalogCall, refusedCatalogField, SESSION_KEYWORDS } from './plu
/** Why a plugin's SQL is refused. The message is the plugin's to read, so it says what to change. */
export class PluginSqlRefused extends Error {
constructor(reason: string) {
super(`statement type not allowed for plugin databases: ${reason}`);
super(`${PLUGIN_DATA_ERRORS.statementNotAllowed}: ${reason}`);
this.name = 'PluginSqlRefused';
}
}
/**
* Transaction control, refused with the contract's whole sentence
* (`PLUGIN_DATA_ERRORS.txControl`), which the SQLite store and the SDK throw
* too, so a plugin reads the same words on both engines.
*/
class TransactionControlRefused extends PluginSqlRefused {
constructor() {
super('');
this.message = PLUGIN_DATA_ERRORS.txControl;
}
}
/** One token of plugin SQL. A word is a name or a keyword: `value` is folded to lower case unless it was quoted. */
type Token = (
| { kind: 'word'; value: string; quoted: boolean }
@@ -177,9 +208,6 @@ const STATEMENTS = new Set([
'drop',
]);
/** Transaction control: every call already runs in a transaction of the host's. */
const TRANSACTION_CONTROL = new Set(['begin', 'start', 'commit', 'end', 'rollback', 'abort', 'savepoint', 'release']);
/** What a CREATE, ALTER or DROP may act on. */
const OBJECTS = new Set(['table', 'index', 'view', 'materialized', 'sequence', 'type']);
@@ -217,6 +245,48 @@ const NAME_LOOKUP_TYPES = [
*/
const ROLE_LOOKUP_TYPES = ['aclitem', '_aclitem'];
/** The system columns of every Postgres table, refused where they are read ({@link checkSystemColumn}). */
const SYSTEM_COLUMNS = new Set(['xmin', 'xmax', 'cmin', 'cmax', 'ctid', 'tableoid']);
/**
* The transaction id types and their array types, refused where the word
* stands for the type ({@link checkXidType}): an `xid` value is all
* `age(xid)` needs to show the server's transaction counter.
*/
const XID_TYPES = new Set(['xid', '_xid', 'xid8', '_xid8']);
/**
* The keywords after which a word names a column, a table or an alias and
* never a type: `SELECT xid`, `WHERE xid = $1`, `ORDER BY xid`, `ADD xid
* text`, `INSERT INTO xid`. After AS and RETURNING it depends on what
* encloses them ({@link namesType}); after any other word the word is a type
* (`c xid` in a column definition, `TYPE xid`) or a bare alias, which a
* plugin then writes with AS.
*/
const NAME_AFTER = new Set(
(
'select distinct where and or not by set on then else when case between like ilike having from join into ' +
'update table column add alter drop rename to with index view exists only lateral over window constraint ' +
'references using'
).split(' '),
);
/** The calls in which AS is followed by a type: `CAST(x AS xid)`. */
const CAST_CALLS = new Set(['cast', 'treat', 'xmlserialize']);
/** The words a WITH query's body starts with, after `AS (`. */
const QUERY_START = new Set(['select', 'values', 'table', 'insert', 'update', 'delete', 'merge', 'with']);
/**
* The collations every Postgres server has. Which others exist depends on
* the server's locales and on ICU, so naming one tells a plugin whether the
* server has it. `"C"` sorts by code point, as SQLite's BINARY does.
*/
const COLLATIONS = new Set(['C', 'POSIX', 'default', 'ucs_basic']);
/** The built-in access methods. Another one comes from an extension, whose presence it would tell. */
const ACCESS_METHODS = new Set(['btree', 'hash', 'gist', 'gin', 'spgist', 'brin', 'heap']);
/** Functions and types refused under any spelling, quoted or not. */
const REFUSED_NAMES = new Set([
'set_config',
@@ -317,12 +387,10 @@ function truncateName(name: string): string {
/** Splits `sql` into tokens, or refuses what this guard does not read the way Postgres would. */
export function tokenize(sql: string): Token[] {
if (sql.includes('\u0000')) {
// The protocol ends a statement's text at a NUL, so what Postgres would read is not what was checked.
throw new PluginSqlRefused(
'the SQL holds the character U+0000 (NUL), which Postgres cannot read in a statement or store in text',
);
}
// The protocol ends a statement's text at a NUL, so what Postgres would read
// is not what was checked. The store refuses it before the guard runs, with
// the words every engine uses; the guard refuses it too, so it holds on its own.
refuseNulSql(sql);
const tokens: Token[] = [];
let i = 0;
const n = sql.length;
@@ -466,11 +534,8 @@ function describe(text: string, tokens: Token[]): PluginStatement {
const first = tokens[0];
const opensWithParen = first.kind === 'punct' && first.value === '(';
const verb = opensWithParen ? 'select' : keyword(first);
if (verb !== null && TRANSACTION_CONTROL.has(verb)) {
throw new PluginSqlRefused(
'transaction-control statements are not allowed: each call runs in its own transaction, and tx() makes a batch atomic',
);
}
// Transaction control: every call already runs in a transaction of the host's.
if (verb !== null && PLUGIN_TRANSACTION_CONTROL.has(verb)) throw new TransactionControlRefused();
if (verb === null || !STATEMENTS.has(verb)) {
throw new PluginSqlRefused(`a statement may not start with ${describeToken(first)}`);
}
@@ -490,11 +555,14 @@ function describe(text: string, tokens: Token[]): PluginStatement {
if (token.kind === 'punct' && token.value === ')') depth--;
if (token.kind !== 'word') continue;
checkName(token.value);
checkSystemColumn(tokens, index);
checkXidType(tokens, index);
checkCall(tokens, index);
const after = tokens[index + 1];
if (after?.kind === 'punct' && after.value === '.') qualifies = true;
if (token.quoted) continue;
checkKeyword(token.value);
checkServerNames(verb, tokens, index);
if (token.value === 'into') checkInto(tokens, index, mainAt);
if (verb === 'alter' && (token.value === 'schema' || token.value === 'owner')) {
throw new PluginSqlRefused(`an ALTER may not change the ${token.value}`);
@@ -720,6 +788,132 @@ function depthAt(tokens: readonly Token[], index: number): number {
return depth;
}
/** The token before the `(` that encloses the token at `index`, the name of a call: `cast` in `CAST(x AS t)`. */
function opener(tokens: readonly Token[], index: number): Token | undefined {
let depth = 0;
for (let i = index - 1; i >= 0; i--) {
if (isPunct(tokens[i], ')')) depth++;
if (isPunct(tokens[i], '(')) {
if (depth === 0) return tokens[i - 1];
depth--;
}
}
return undefined;
}
/**
* Refuses a read of a system column ({@link SYSTEM_COLUMNS}): the word
* anywhere but where it gives a name ({@link givesName}). `t.xmin`, `xmin`
* in WHERE or RETURNING and `"xmin"` read the column of the table in FROM.
*/
function checkSystemColumn(tokens: readonly Token[], index: number): void {
const token = tokens[index];
if (token.kind !== 'word' || !SYSTEM_COLUMNS.has(token.value) || givesName(tokens, index)) return;
throw new PluginSqlRefused(
`"${token.value}" reads a Postgres system column, which a plugin may not read: xmin, xmax, cmin, cmax, ` +
"ctid and tableoid show the server's transaction ids and where rows are stored, and SQLite has none of " +
'them. Keep a column of your own (a version number, an updated_at) for optimistic locking. A value of ' +
`your own may carry the name where you give it (min(lng) AS ${token.value}) and be read from the result, ` +
'but the SQL may not refer to it by that name',
);
}
/**
* Whether the word at `index` gives a name rather than reads one: right
* after AS (`min(lng) AS xmin`, `FROM t AS xmin`), or in the column list of
* an alias (`AS b(xmin, xmax)`) or of a WITH query (`WITH b(xmin) AS
* (SELECT ...)`). A word read wrongly here as a reference is only refused; a
* reference read wrongly as a name would let the column through, so a list
* counts only when it holds names alone and stands where a list of names
* does. `f(xmin) AS (a int)` is a call with a column definition list.
*/
function givesName(tokens: readonly Token[], index: number): boolean {
if (keyword(tokens[index - 1]) === 'as') return true;
let first = index;
while (isPunct(tokens[first - 1], ',') && tokens[first - 2]?.kind === 'word') first -= 2;
let last = index;
while (isPunct(tokens[last + 1], ',') && tokens[last + 2]?.kind === 'word') last += 2;
if (!isPunct(tokens[first - 1], '(') || !isPunct(tokens[last + 1], ')') || tokens[first - 2]?.kind !== 'word') {
return false;
}
if (keyword(tokens[first - 3]) === 'as') return true;
if (keyword(tokens[first - 3]) !== 'with' && !isPunct(tokens[first - 3], ',')) return false;
const as = last + 2;
if (keyword(tokens[as]) !== 'as') return false;
const body = skip(tokens, skip(tokens, as + 1, 'not'), 'materialized');
return (
isPunct(tokens[body], '(') && (isPunct(tokens[body + 1], '(') || QUERY_START.has(keyword(tokens[body + 1]) ?? ''))
);
}
/** Refuses a transaction id type ({@link XID_TYPES}) where the word stands for the type ({@link namesType}). */
function checkXidType(tokens: readonly Token[], index: number): void {
const token = tokens[index];
if (token.kind !== 'word' || !XID_TYPES.has(token.value) || !namesType(tokens, index)) return;
throw new PluginSqlRefused(
`"${token.value}" stands for a transaction id type here, which a plugin may not use: age() of one shows ` +
`how many transactions the server ran. A column of your own may be called ${token.value}, but only ` +
`without its table: Postgres reads x.${token.value} as the cast ${token.value}(x) when x has no such ` +
`column, so where you need x.${token.value} (excluded.${token.value} in an upsert) rename the column`,
);
}
/**
* Whether the word at `index` stands for a type: after `::`, before a string
* (`xid '3'`), as a call or after a dot (both casts: `xid(x)`, `x.xid`),
* after `subtype =` (a range type), after AS in a CAST, after RETURNING in
* a JSON function, and after any word but {@link NAME_AFTER} (`c xid`,
* `TYPE xid`). Anywhere else, after a comma, a parenthesis or an operator,
* it is a column.
*/
function namesType(tokens: readonly Token[], index: number): boolean {
const before = tokens[index - 1];
const after = tokens[index + 1];
if (isPunct(before, '::') || isPunct(before, '.') || after?.kind === 'string') return true;
if (isPunct(after, '(')) return !namesObject(tokens, index);
if (isPunct(before, '=')) return keyword(tokens[index - 2]) === 'subtype';
if (before?.kind !== 'word') return false;
const word = keyword(before);
if (word === 'as') return CAST_CALLS.has(keyword(opener(tokens, index)) ?? '');
if (word === 'returning') return (keyword(opener(tokens, index)) ?? '').startsWith('json');
return word === null || !NAME_AFTER.has(word);
}
/**
* Refuses a collation or an access method a plugin may not name: the word
* after COLLATE, after `collation =` in a CREATE TYPE, after USING where it
* names an index's or a table's access method, and after ACCESS METHOD. Only
* {@link COLLATIONS} and {@link ACCESS_METHODS}, unqualified, pass.
*/
function checkServerNames(verb: string, tokens: readonly Token[], index: number): void {
const word = keyword(tokens[index]);
const before = keyword(tokens[index - 1]);
let collation = -1;
if (word === 'collate') collation = index + 1;
else if (word === 'collation' && verb === 'create' && isPunct(tokens[index + 1], '=')) collation = index + 2;
if (collation >= 0 && !named(tokens, collation, COLLATIONS)) {
throw new PluginSqlRefused(
'a collation other than "C", "POSIX", "default" or ucs_basic may not be named: which others exist ' +
'depends on the server\'s locales and ICU. COLLATE "C" sorts by code point, as SQLite does',
);
}
const method =
(word === 'using' && (before === 'exclude' || (verb === 'create' && tokens[index + 1]?.kind === 'word'))) ||
(word === 'method' && before === 'access');
if (method && !named(tokens, index + 1, ACCESS_METHODS)) {
throw new PluginSqlRefused(
'only the built-in access methods may be named (btree, hash, gist, gin, spgist and brin for an index, ' +
'heap for a table): whether another exists depends on the extensions installed on the server',
);
}
}
/** Whether the token at `at` is one of `names`, unqualified. */
function named(tokens: readonly Token[], at: number, names: ReadonlySet<string>): boolean {
const token = tokens[at];
return token?.kind === 'word' && names.has(token.value) && !isPunct(tokens[at + 1], '.');
}
/** Refuses a name that reaches past the plugin's schema whatever qualifies it. */
function checkName(name: string): void {
if (REFUSED_NAMES.has(name) || REFUSED_PREFIXES.some((p) => name.startsWith(p)) || XML_EXPORT.test(name)) {