mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-11-20 02:56:10 +00:00
docs(db): describe the layers that hold plugin SQL on Postgres and the NUL rule for plugin data
The plugin-sdk README, the Postgres wiki page and server/CLAUDE.md now name the layers as the code enforces them: the refusal on a privileged role and the boot log line, the guard (the reg* types, INTO after the statement's own INSERT or MERGE only, TEMP only where it makes a temporary object), the targets Postgres resolves before an ALTER, DROP, REFRESH, TRUNCATE or CREATE INDEX, one statement per round trip, pg_temp last, and the quota over every relation with the object cap. They also say that a plugin argument holding NUL is refused on Postgres and bound exactly on SQLite.
This commit is contained in:
+43
-12
@@ -193,21 +193,41 @@ What changes on Postgres:
|
||||
of `AUTOINCREMENT`, `now()` instead of `datetime('now')`, `ON CONFLICT` works on both.
|
||||
A `bigint` comes back as a number, and one past 2^53 is refused. A boolean comes back as
|
||||
`true`/`false`, not `1`/`0`.
|
||||
- **Not on a privileged role.** While TREK connects to Postgres as a superuser, a member of
|
||||
one, or another role that could reach the database server (the same roles TREK refuses
|
||||
restores for), every `ctx.db` call fails with an error that starts `Plugin databases
|
||||
(db:own) are refused on Postgres`, and TREK's boot log says so. Your SQL would run as
|
||||
that role. The admin fixes it by connecting TREK as a role that owns its database and is
|
||||
not a superuser; your plugin needs no change.
|
||||
- **Your SQL stays in your schema.** Only your schema is on the `search_path`, and a
|
||||
statement is refused before it runs when it names another schema (`public.users`,
|
||||
another plugin's), anything starting with `pg_` or `information_schema`, or a function
|
||||
that runs SQL from a string, reads settings or reaches an object by a name in a string
|
||||
(`query_to_xml`, `ts_stat`, `set_config`, `current_setting`, `nextval`, `setval`, the
|
||||
`reg*` casts, `lo_*`, `dblink`). An alias that shares a schema's name is refused too, so
|
||||
pick another.
|
||||
(`query_to_xml`, `ts_stat`, `set_config`, `current_setting`, `nextval`, `setval`,
|
||||
`has_table_privilege` and the other `has_*_privilege` functions, `currtid2`, `to_reg*`,
|
||||
`lo_*`, `dblink`). The types that look an object up by name are refused in every
|
||||
spelling: `regclass`, `regtype`, `regproc`, `regprocedure`, `regoper`, `regoperator`,
|
||||
`regconfig`, `regdictionary`, `regnamespace`, `regrole`, `regcollation` and their arrays
|
||||
(`'x'::regtype`, `CAST('x' AS regtype)`, `regtype 'x'`). An alias that shares a schema's
|
||||
name is refused too, so pick another.
|
||||
- **You change your own objects only.** Right before an `ALTER`, `DROP`, `REFRESH`,
|
||||
`TRUNCATE` or `CREATE INDEX ... ON` runs, Postgres resolves the object it names (and the
|
||||
tables a `CREATE`/`ALTER TABLE` ties to with `REFERENCES`, `INHERITS`, `PARTITION OF`,
|
||||
`ATTACH`/`DETACH PARTITION` or `INHERIT`), and the statement is refused unless each one
|
||||
that exists is in your schema. Postgres searches its catalog before your schema, so
|
||||
`ALTER TYPE int4 ...` or `DROP TYPE text` names the built-in type and is refused; name
|
||||
your own types so they do not clash with built-in ones.
|
||||
- **Statements**: `SELECT`, `INSERT`, `UPDATE`, `DELETE`, `MERGE`, `WITH`, `VALUES`,
|
||||
`TABLE`, `TRUNCATE`, `REFRESH`, and `CREATE`/`ALTER`/`DROP` of a table, index, view,
|
||||
materialized view, sequence or type. No functions, triggers, procedures, `DO` blocks,
|
||||
temporary tables, `SELECT ... INTO`, `COPY`, `SET`, transaction control (`tx()` makes a
|
||||
batch atomic), `WITH RECURSIVE`, and no `ALTER` of a schema, owner or tablespace.
|
||||
Dollar-quoted strings, `E'...'` and `U&'...'` are refused; write plain single-quoted
|
||||
strings. The words `TEMP` and `TEMPORARY` are refused anywhere outside a string (quote a
|
||||
column named `"temp"`), and so is `INTO` anywhere but `INSERT INTO` and `MERGE INTO`.
|
||||
materialized view, sequence or type (a composite, enum or range type, `CREATE TYPE x AS
|
||||
...`). No functions, triggers, procedures, `DO` blocks, base types, temporary tables,
|
||||
`SELECT ... INTO`, `COPY`, `SET`, transaction control (`tx()` makes a batch atomic),
|
||||
`WITH RECURSIVE`, and no `ALTER` of a schema, owner or tablespace. Dollar-quoted strings,
|
||||
`E'...'` and `U&'...'` are refused; write plain single-quoted strings. `CREATE TEMP` and
|
||||
`CREATE TEMPORARY` (also after `OR REPLACE`, `LOCAL` or `GLOBAL`) are refused, and so is
|
||||
`INTO` anywhere but right after the `INSERT` or `MERGE` that is the statement's verb (or
|
||||
the verb of a `WITH` query's body): `SELECT 1 insert INTO t` is refused, because `insert`
|
||||
there is a column alias. A column named `temp` needs no quoting.
|
||||
Numbers are decimal: `0x1F`, `0o17`, `0b1` and `1_000` are refused, because Postgres
|
||||
before 16 reads them differently. Outside strings, quoted names and comments, whitespace is
|
||||
space, tab, newline, carriage return or form feed; any other control character or non-ASCII
|
||||
@@ -216,11 +236,22 @@ What changes on Postgres:
|
||||
- **One statement at a time.** Each statement goes to Postgres on its own, over the protocol
|
||||
that runs exactly one: `query` and each `tx` entry take one statement, and an `exec`
|
||||
script is split at its semicolons and sent statement by statement.
|
||||
- **No NUL in text.** Postgres text cannot hold the character U+0000. A bound argument
|
||||
holding one (a string, or a string in an array) and a `migrate` id holding one are refused
|
||||
with an error that names it, and so is SQL text with one anywhere, a string literal
|
||||
included; TREK never drops or changes the character, since `'a\0b'` and `'ab'` are
|
||||
different keys. Bind binary data as a `Buffer` into a `bytea` column, and use another
|
||||
separator for composite keys. On SQLite your arguments are bound exactly as you send
|
||||
them, NUL included, as they always were.
|
||||
- **Limits**: the same row cap and quota as on SQLite. The 2 s budget is a
|
||||
`statement_timeout` here, so it also stops a single long statement, and an `exec` with
|
||||
bound args is held to it too; `migrate` stays untimed. A write that takes your schema past
|
||||
256 MB and grows it is rolled back; a delete still runs. Rows deleted or rolled back keep
|
||||
their space until Postgres vacuums them.
|
||||
bound args is held to it too; `migrate` stays untimed. The quota counts every relation of
|
||||
your schema (tables, indexes, sequences, materialized views and each table's TOAST
|
||||
storage), and a schema may hold at most 5,000 catalog objects (relations, types, each
|
||||
table brings its row type and that type's array, constraints and enum labels). A write
|
||||
that takes your schema past 256 MB or past 5,000 objects and grows it there is rolled
|
||||
back; a delete or a drop still runs. Rows deleted or rolled back keep their space until
|
||||
Postgres vacuums them.
|
||||
|
||||
## Settings
|
||||
|
||||
|
||||
+2
-2
File diff suppressed because one or more lines are too long
@@ -78,13 +78,13 @@ CREATE DATABASE trek OWNER trek;
|
||||
ALTER SCHEMA public OWNER TO trek;
|
||||
```
|
||||
|
||||
Then use `postgres://trek:<password>@<host>:5432/trek` as `TREK_DB_URL`. A restore refuses a role that is a superuser or a member of one (`SET ROLE` would make it a superuser), that holds `pg_execute_server_program`, `pg_read_server_files` or `pg_write_server_files`, that may run a server file function (`lo_import`, `lo_export`, `pg_read_file`, `pg_read_binary_file`, `pg_ls_dir`, `pg_stat_file` or adminpack's `pg_file_*`), that was granted ALTER SYSTEM on any setting or SET on a superuser-only one (PostgreSQL 15 and later), or, on PostgreSQL 14 and 15, that has CREATEROLE or is a member of a role that has it (there it may grant itself those roles). A crafted archive could use any of these to run commands on the database server or to read or write its files. A member of `pg_monitor` is not refused. TREK also logs a warning about such a role when it starts.
|
||||
Then use `postgres://trek:<password>@<host>:5432/trek` as `TREK_DB_URL`. A restore refuses a role that is a superuser or a member of one (`SET ROLE` would make it a superuser), that holds `pg_execute_server_program`, `pg_read_server_files` or `pg_write_server_files`, that may run a server file function (`lo_import`, `lo_export`, `pg_read_file`, `pg_read_binary_file`, `pg_ls_dir`, `pg_stat_file` or adminpack's `pg_file_*`), that was granted ALTER SYSTEM on any setting or SET on a superuser-only one (PostgreSQL 15 and later), or, on PostgreSQL 14 and 15, that has CREATEROLE or is a member of a role that has it (there it may grant itself those roles). A crafted archive could use any of these to run commands on the database server or to read or write its files. A member of `pg_monitor` is not refused. TREK also logs a warning about such a role when it starts. Plugins' own databases (`db:own`) are refused for the same roles, since a plugin's SQL runs as TREK's role (see [Plugin data](#plugin-data)).
|
||||
|
||||
TREK creates its tables in the role's current schema, normally `public`. A role whose `search_path` points at a schema of its own works too: TREK keeps its tables and its migration history there, and backups and restores cover that schema.
|
||||
|
||||
### Moving a compose install off the superuser
|
||||
|
||||
An install that started from an older `docker-compose.postgres.yml` connects as `trek`, which the `postgres` image created as the superuser. Its backups work, but restores are refused until TREK runs as a role that is not a superuser. With the containers running:
|
||||
An install that started from an older `docker-compose.postgres.yml` connects as `trek`, which the `postgres` image created as the superuser. Its backups work, but restores and plugin databases are refused until TREK runs as a role that is not a superuser. With the containers running:
|
||||
|
||||
1. Create a second superuser for administration, so the database keeps one: `docker exec -it trek-db psql -U trek -d trek -c "CREATE ROLE postgres LOGIN SUPERUSER PASSWORD '<a long random value>'"`.
|
||||
2. Take the superuser right from `trek`: `docker exec -it trek-db psql -U postgres -d trek -c "ALTER ROLE trek NOSUPERUSER NOCREATEDB NOCREATEROLE"`.
|
||||
@@ -187,7 +187,16 @@ Restoring a backup on Postgres follows these rules:
|
||||
A plugin with the `db:own` permission keeps its own database. On SQLite that is a `plugin.db` file in its data directory. On Postgres it is a schema of its own in TREK's database, named `trek_plugin_<plugin id>` with each dash an underscore, so a Postgres install writes no SQLite file to the data volume, and the data volume can sit on network storage (NFS). Each plugin's directory under `data/plugins-data` is still created, for files a plugin keeps beside its database.
|
||||
|
||||
- **TREK's role creates the schemas**, so it needs the `CREATE` privilege on the database, which the role that owns the database has.
|
||||
- **A plugin's SQL stays in its schema.** Only that schema is on the plugin's `search_path`, and TREK refuses a statement before it runs when it names another schema, the system catalogs or a function that could reach past it. It also refuses temporary objects (the words `TEMP` and `TEMPORARY`, and `SELECT ... INTO`), hexadecimal, octal and binary numbers, and control characters or non-ASCII whitespace outside strings, which Postgres reads differently from the check. Each statement is sent on its own over the extended query protocol, which runs one statement and no more. Every connection of TREK's pool searches the temporary schema last, and a plugin call drops the temporary tables and other session state of the connection it used before handing it back, so a temporary table can never stand in for one of TREK's. The rules are in the [plugin-sdk README](https://github.com/liketrek/TREK/blob/main/plugin-sdk/README.md#your-own-database-on-postgres).
|
||||
- **No plugin SQL on a privileged role.** While `TREK_DB_URL` connects as a role TREK refuses restores for (a superuser or a member of one, or a role that may reach the server's files or programs; see [A role that is not a superuser](#a-role-that-is-not-a-superuser)), every plugin database call fails with "Plugin databases (db:own) are refused on Postgres ...", and the boot log says so. A plugin's SQL runs as TREK's role, and such a role could change the system catalogs for the whole server. Connect TREK as a role that owns its database and is not a superuser, and restart; TREK reads the role once per pool connection.
|
||||
- **A plugin's SQL stays in its schema**, in layers the database enforces as well as TREK's own check:
|
||||
- TREK refuses a statement before it runs when it names another schema, the system catalogs, a function that could reach past the schema, or a type that looks an object up by name (`regclass`, `regtype`, `regconfig` and the other `reg*` types, `has_table_privilege` and its siblings). It also refuses temporary objects (`CREATE TEMP`/`TEMPORARY` and `SELECT ... INTO TEMP`; a column named `temp` is fine), `INTO` anywhere but after the `INSERT` or `MERGE` that is the statement's verb, hexadecimal, octal and binary numbers, and control characters or non-ASCII whitespace outside strings, which Postgres reads differently from the check.
|
||||
- Right before an `ALTER`, `DROP`, `REFRESH`, `TRUNCATE` or `CREATE INDEX ... ON`, Postgres itself resolves the objects the statement changes, and TREK refuses it unless each one lies in the plugin's schema. An unqualified `int4` or `text` is the built-in type and is refused, whatever the role may do.
|
||||
- Each statement is sent on its own over the extended query protocol, which runs one statement and no more.
|
||||
- Every connection of TREK's pool searches the temporary schema last, and a plugin call drops the temporary tables and other session state of the connection it used before handing it back, so a temporary table can never stand in for one of TREK's.
|
||||
- A call that writes is held to the quota (256 MB, counted over every table, index, sequence and materialized view of the schema with their TOAST storage) and to 5,000 catalog objects (relations, types, constraints, enum labels), so objects that take no disk space are bounded too.
|
||||
|
||||
The rules are in the [plugin-sdk README](https://github.com/liketrek/TREK/blob/main/plugin-sdk/README.md#your-own-database-on-postgres).
|
||||
- **No NUL character in plugin text.** Postgres text cannot hold U+0000. A plugin argument holding one is refused with an error that names the reason, instead of being changed. On SQLite a plugin's own file keeps storing and finding it as before. TREK's own data holds no NUL on either engine.
|
||||
- **A plugin writes Postgres SQL** on a Postgres install, not SQLite SQL. TREK does not translate it. A plugin written for SQLite only fails its `db:own` calls with the database's error; ask its author for Postgres support (`ctx.db.engine()` tells a plugin which engine it runs on).
|
||||
- **Plugin calls share TREK's pool**, at most half of `TREK_DB_POOL_MAX` at once, one at a time per plugin. Each call takes a connection of its own from the pool for its transaction and gives it back when it ends.
|
||||
- **Uninstalling a plugin with "delete data"** drops its schema.
|
||||
|
||||
Reference in New Issue
Block a user