docs(db): plugin data on Postgres, its SQL rules and one TREK per database

The plugin-sdk README says what dialect a plugin writes when TREK runs on
Postgres, what the guard refuses and how the limits hold there. The wiki's
Postgres page describes the plugin schemas, their backup and restore and
the one-instance-per-database limit, and its key rotation path now names
the file where it is.
This commit is contained in:
Maurice
2026-11-17 13:00:55 +01:00
parent 7340087353
commit 17c9c0758f
5 changed files with 63 additions and 6 deletions
+45 -1
View File
@@ -169,6 +169,50 @@ Every ctx method falls into one of three kinds, listed per method in the exporte
answer, an access token) or data you own (`ctx.db`, `ctx.meta`, another plugin's
answer), returned as it is.
### Your own database on Postgres
`ctx.db` is a SQLite file of your own on a default TREK. When the instance runs on
PostgreSQL (`TREK_DB_URL`), it is a schema of your own in TREK's database instead,
named `trek_plugin_<your id>` with each dash an underscore, and no SQLite file is
written. TREK hands your SQL to the engine as you wrote it and translates nothing, so
ask which one you are on and write that dialect:
```js
const pg = (await ctx.db.engine()) === 'postgres'
await ctx.db.migrate('001', pg
? 'CREATE TABLE notes (id bigserial PRIMARY KEY, body text NOT NULL)'
: 'CREATE TABLE notes (id INTEGER PRIMARY KEY AUTOINCREMENT, body TEXT NOT NULL)')
await ctx.db.exec(pg ? 'INSERT INTO notes (body) VALUES ($1)' : 'INSERT INTO notes (body) VALUES (?)', body)
```
What changes on Postgres:
- **Placeholders** are `$1`, `$2`, ..., not `?`. A `?` with bound args is refused with a
message that says so.
- **Types and functions** are Postgres's: `bigserial`/`GENERATED ... AS IDENTITY` instead
of `AUTOINCREMENT`, `now()` instead of `datetime('now')`, `ON CONFLICT` works on both.
A `bigint` comes back as a number, and one past 2^53 is refused. A boolean comes back as
`true`/`false`, not `1`/`0`.
- **Your SQL stays in your schema.** Only your schema is on the `search_path`, and a
statement is refused before it runs when it names another schema (`public.users`,
another plugin's), anything starting with `pg_` or `information_schema`, or a function
that runs SQL from a string, reads settings or reaches an object by a name in a string
(`query_to_xml`, `ts_stat`, `set_config`, `current_setting`, `nextval`, `setval`, the
`reg*` casts, `lo_*`, `dblink`). An alias that shares a schema's name is refused too, so
pick another.
- **Statements**: `SELECT`, `INSERT`, `UPDATE`, `DELETE`, `MERGE`, `WITH`, `VALUES`,
`TABLE`, `TRUNCATE`, `REFRESH`, and `CREATE`/`ALTER`/`DROP` of a table, index, view,
materialized view, sequence or type. No functions, triggers, procedures, `DO` blocks,
temporary tables, `SELECT ... INTO`, `COPY`, `SET`, transaction control (`tx()` makes a
batch atomic), `WITH RECURSIVE`, and no `ALTER` of a schema, owner or tablespace.
Dollar-quoted strings, `E'...'` and `U&'...'` are refused; write plain single-quoted
strings.
- **Limits**: the same row cap and quota as on SQLite. The 2 s budget is a
`statement_timeout` here, so it also stops a single long statement, and an `exec` with
bound args is held to it too; `migrate` stays untimed. A write that takes your schema past
256 MB and grows it is rolled back; a delete still runs. Rows deleted or rolled back keep
their space until Postgres vacuums them.
## Settings
Declare settings in `trek-plugin.json`; TREK renders the form, you write no settings UI.
@@ -252,7 +296,7 @@ for a legitimate plugin and only bite a runaway or abusive one — build against
| `ctx.ai` | 200 calls/day per plugin (UTC midnight rollover); past it throws `"daily AI budget exhausted (resets at UTC midnight)"` |
| `ctx.notify` | 100 calls/day per plugin (UTC midnight rollover); past it throws `"daily notification budget exhausted (resets at UTC midnight)"` |
| RPC (every `ctx.*` call) | burst 60, sustained 20/s, 16 in-flight per plugin; a throttled call is refused with `HOST_ERROR: rate limit exceeded — slow down ctx.* calls` |
| `ctx.db` (your own sqlite) | 256 MB per plugin; a `query` or `tx` returns at most 100,000 rows (a whole `tx` batch shares one count). A `query`, a whole `tx` batch and an `exec` script without bound args each have 2 s of wall-clock time, checked between the rows a statement reads and between the statements of a batch or script; past either limit the call throws (`query exceeded its 2000 ms time budget`), a `tx` rolls back, and an `exec` script stops before its next statement once the time is up (the statements that ran stay applied, as they always did, and a transaction the script opened itself is rolled back; a script whose last statement has run counts as done, however long it took). Split a longer script over several calls, or batch it with `tx`. `migrate` has no time budget, since a migration stopped part of the way would fail again on every start. `ATTACH`, `DETACH`, `VACUUM`, `PRAGMA`, `WITH RECURSIVE` and `load_extension` are refused. The budget cannot stop a single statement, because SQLite runs it on TREK's main thread in one step: a write runs to its end (an `INSERT ... SELECT` over a cross join of large tables), and a read runs until its first row (an aggregate over one). Keep your statements indexed and bounded |
| `ctx.db` (your own database) | 256 MB per plugin; a `query` or `tx` returns at most 100,000 rows (a whole `tx` batch shares one count). A `query`, a whole `tx` batch and an `exec` script without bound args each have 2 s of wall-clock time, checked between the rows a statement reads and between the statements of a batch or script; past either limit the call throws (`query exceeded its 2000 ms time budget`), a `tx` rolls back, and an `exec` script stops before its next statement once the time is up (the statements that ran stay applied, as they always did, and a transaction the script opened itself is rolled back; a script whose last statement has run counts as done, however long it took). Split a longer script over several calls, or batch it with `tx`. `migrate` has no time budget, since a migration stopped part of the way would fail again on every start. `ATTACH`, `DETACH`, `VACUUM`, `PRAGMA`, `WITH RECURSIVE` and `load_extension` are refused. The budget cannot stop a single statement, because SQLite runs it on TREK's main thread in one step: a write runs to its end (an `INSERT ... SELECT` over a cross join of large tables), and a read runs until its first row (an aggregate over one). Keep your statements indexed and bounded. On a TREK running on Postgres the same caps hold, and the budget also stops a single statement (see [Your own database on Postgres](#your-own-database-on-postgres)) |
| `ctx.meta` | 64 KB per value, 256 chars per key, 100 keys per (plugin, entity) |
| Plugin process | 300 MB RSS ceiling; auto-disabled after 5 crashes in 5 minutes |
| Event redelivery buffer | 200 events held per plugin, dropped unreplayed after 15 minutes |
+1 -1
View File
File diff suppressed because one or more lines are too long
+15 -2
View File
@@ -123,11 +123,12 @@ The copy is a safety net for an update. It does not replace a backup, and it doe
## Backups and restore
A Postgres install produces the same backup archive as a SQLite install, with one difference: the database is stored as `travel.pgdump`, a `pg_dump` custom-format file, instead of `travel.db`. The uploads, plugin data and the encryption key are in the archive as before. See [Backups](Backups).
A Postgres install produces the same backup archive as a SQLite install, with one difference: the database is stored as `travel.pgdump`, a `pg_dump` custom-format file, instead of `travel.db`. The uploads, the files plugins keep in their data directories and the encryption key are in the archive as before, and the dump holds each plugin's data schema beside TREK's own (see [Plugin data](#plugin-data)). See [Backups](Backups).
Restoring a backup on Postgres follows these rules:
- **The current database is replaced.** TREK first keeps a copy of it as `data/pre-restore-<time>.pgdump`. `pg_restore` then turns the archive's dump into a SQL script, and `psql` clears the current schema, which drops its tables, views, sequences, functions, types and collations, and runs the script. Both steps run in one transaction, so a restore that fails leaves the database as it was.
- **Plugin data is replaced with the archive's.** The same transaction drops every `trek_plugin_*` schema the role owns and loads the ones the dump holds, so a plugin installed after the backup loses its data, as it does on SQLite. Schemas the role does not own, and every other schema of the database, are never touched.
- **TREK checks the archive before it replaces anything.** The archive must hold a Postgres dump, not a SQLite file. The dump must carry TREK's migration history, and it must not come from a newer TREK. TREK refuses the restore with a message saying which check failed.
- **The client tools must be recent enough.** The restore refuses `pg_restore` or `psql` older than 18.0, 17.6, 16.10, 15.14 or 14.19, and tools older than the server.
- **The role must not be a superuser.** The restore refuses a role that is a superuser or holds `pg_execute_server_program` or `pg_write_server_files`. See [A role that is not a superuser](#a-role-that-is-not-a-superuser).
@@ -135,10 +136,21 @@ Restoring a backup on Postgres follows these rules:
`RESTORE_FROM_BACKUP` works on Postgres too. On the first start, TREK loads the archive's dump into the database when that database holds no tables. When the database already has tables, the variable is ignored and TREK logs a warning. See [Backups: Restoring on a new install, before setup](Backups#restoring-on-a-new-install-before-setup).
## Plugin data
A plugin with the `db:own` permission keeps its own database. On SQLite that is a `plugin.db` file in its data directory. On Postgres it is a schema of its own in TREK's database, named `trek_plugin_<plugin id>` with each dash an underscore, so a Postgres install writes no SQLite file to the data volume, and the data volume can sit on network storage (NFS). Each plugin's directory under `data/plugins-data` is still created, for files a plugin keeps beside its database.
- **TREK's role creates the schemas**, so it needs the `CREATE` privilege on the database, which the role that owns the database has.
- **A plugin's SQL stays in its schema.** Only that schema is on the plugin's `search_path`, and TREK refuses a statement before it runs when it names another schema, the system catalogs or a function that could reach past it. The rules are in the [plugin-sdk README](https://github.com/liketrek/TREK/blob/main/plugin-sdk/README.md#your-own-database-on-postgres).
- **A plugin writes Postgres SQL** on a Postgres install, not SQLite SQL. TREK does not translate it. A plugin written for SQLite only fails its `db:own` calls with the database's error; ask its author for Postgres support (`ctx.db.engine()` tells a plugin which engine it runs on).
- **Plugin calls share TREK's pool**, at most half of `TREK_DB_POOL_MAX` at once, one at a time per plugin.
- **Uninstalling a plugin with "delete data"** drops its schema.
- **One TREK per database.** The schema names depend only on the plugin id, so two TREK instances on one database would share their plugins' data.
## Maintenance tools
- **Resetting the admin account** works as on SQLite: `docker exec -it -u node trek node server/reset-admin.js` resets or creates the account in the Postgres database, with the server's connection settings and its write lock. It never opens or creates a `travel.db`, and it refuses to run when `TREK_DB_FILE` is set beside `TREK_DB_URL`. See [Troubleshooting](Troubleshooting).
- **Rotating the encryption key** is not available on Postgres yet. `scripts/migrate-encryption.ts` stops before it changes anything when `TREK_DB_URL` is set. Keep the current `ENCRYPTION_KEY`. See [Encryption Key Rotation](Encryption-Key-Rotation).
- **Rotating the encryption key** is not available on Postgres yet. `server/scripts/migrate-encryption.ts` stops before it changes anything when `TREK_DB_URL` is set. Keep the current `ENCRYPTION_KEY`. See [Encryption Key Rotation](Encryption-Key-Rotation).
## Limitations
@@ -146,6 +158,7 @@ Restoring a backup on Postgres follows these rules:
- **No more than one TREK replica.** TREK keeps state in memory: rate limits, pending one-time codes, the permission cache and the WebSocket connections. Each process has its own copy, so two replicas would disagree with each other. Keep `replicas: 1` in the Helm chart and one `trek` container. Postgres does not change that.
- **Two processes booting on one database at once are not coordinated.** Start one process for the first boot and for each upgrade, then continue with one.
- **PostgreSQL 14 to 18 only.** A newer server is refused until a TREK release supports it, because the image's client tools would not be able to back it up.
- **One TREK instance per database.** Plugin data schemas are named after the plugin alone (see [Plugin data](#plugin-data)), and a restore drops the plugin schemas the role owns. Give each instance its own database.
- **Writes are serialized.** TREK takes a database-wide advisory lock for each write transaction, as the SQLite engine does with its single writer. Heavy concurrent writes wait on that lock.
## Supabase
+1 -1
View File
@@ -280,7 +280,7 @@ exports the list per method as `PLUGIN_METHOD_RESULT`:
| Area | Methods | Requires |
|---|---|---|
| `ctx.db` | `query(sql, …args)` / `exec(sql, …args)` / `migrate(id, sql)` / `tx(ops)` against your **own** SQLite file. `tx([{sql, args?}, …])` runs up to 100 statements in one transaction (all commit or all roll back; reads see the batch's own earlier writes) → `{ results: [{changes?}\|{rows?}, …] }`. Your file is capped at **256 MB** (a write past it fails `SQLITE_FULL`, contained to your plugin), a single result set at **100,000 rows** and a `query`, a `tx` or an `exec` script at **2 s** (see [Runtime limits](#runtime-limits)): page your reads instead of materialising a cartesian product, and split long scripts | `db:own` |
| `ctx.db` | `query(sql, …args)` / `exec(sql, …args)` / `migrate(id, sql)` / `tx(ops)` against your **own** SQLite file, or your own schema when TREK runs on Postgres (`engine()` says which; write that engine's SQL, with `$1` placeholders on Postgres, see the plugin-sdk README's "Your own database on Postgres"). `tx([{sql, args?}, …])` runs up to 100 statements in one transaction (all commit or all roll back; reads see the batch's own earlier writes) → `{ results: [{changes?}\|{rows?}, …] }`. Your file is capped at **256 MB** (a write past it fails `SQLITE_FULL`, contained to your plugin), a single result set at **100,000 rows** and a `query`, a `tx` or an `exec` script at **2 s** (see [Runtime limits](#runtime-limits)): page your reads instead of materialising a cartesian product, and split long scripts | `db:own` |
| `ctx.trips` | `getById` / `getPlaces` / `getReservations` / `getDays` / `getAccommodations` / `listMine()` — enumerate every trip the acting user can access (membership-checked). `getDays` includes each day's `assignments` + `notes_items`; `getReservations` includes `endpoints` + `day_positions` | `db:read:trips` |
| `ctx.trips.update(tripId, fields)` | update trip fields (title/dates/currency/reminder_days/…) | `db:write:trips` |
| `ctx.trips.create(input)` | create a **new trip owned by the acting user** (importers) — `title` required, plus `description?`/`start_date?`/`end_date?`/`currency?`/`reminder_days?`/`day_count?`; without `currency` the trip takes the acting user's display currency, then the instance default, then EUR | `db:create:trips` (+ `trip_create`) |
+1 -1
View File
@@ -10,7 +10,7 @@ ungranted capability is physically unreachable**, not just disallowed. See
| Permission | Grants | Notes |
|---|---|---|
| `db:own` | Read/write the plugin's **own** SQLite file via `ctx.db`: `db.query`, `db.exec`, `db.tx` **and `db.migrate`** | A separate `plugin.db` per plugin under the plugin's data dir, never TREK's own `travel.db`. `db.migrate` runs a keyed, idempotent migration (schema/table creation, e.g. `CREATE TABLE`) once per id. `ATTACH`/`DETACH`/`VACUUM`/`PRAGMA`/`WITH RECURSIVE` are refused, a `query` or `tx` stops at 100,000 rows, and a `query`, `tx` or `exec` script stops at 2 s. |
| `db:own` | Read/write the plugin's **own** database via `ctx.db`: `db.query`, `db.exec`, `db.tx` **and `db.migrate`**, plus `db.engine` to ask which engine it runs on | A separate `plugin.db` per plugin under the plugin's data dir, never TREK's own `travel.db`. On a TREK running on Postgres it is a schema of the plugin's own instead (`trek_plugin_<id>`), and a statement naming any other schema is refused (see [Database: Postgres](Database-Postgres#plugin-data)). `db.migrate` runs a keyed, idempotent migration (schema/table creation, e.g. `CREATE TABLE`) once per id. `ATTACH`/`DETACH`/`VACUUM`/`PRAGMA`/`WITH RECURSIVE` are refused, a `query` or `tx` stops at 100,000 rows, and a `query`, `tx` or `exec` script stops at 2 s. |
| `db:read:trips` | Read-only trip data via `ctx.trips` (`getById`, `getPlaces`, `getReservations`, `getDays`, `getAccommodations`, `listMine`, `members`) | Every call is **membership-checked** against the acting user: a plugin can't read a trip that user can't see. `getDays` returns each day with its `assignments` + `notes_items`; `getReservations` rows carry `endpoints` + `day_positions` like the REST list. `members` returns the roster (id + display fields only). Each row, and each row nested in it such as a day's notes or a reservation's endpoints, carries the fields TREK publishes for it (`PLUGIN_ENTITY_FIELDS` and `PLUGIN_ENTITY_NESTED` in the SDK), never the trip's calendar `feed_token`, and a column added in a later release is not delivered until it is published. |
| `db:read:users` | Read-only public profile via `ctx.users.getById` | Returns id, username, display name, avatar only — **never** password hashes, tokens, or secrets. |
| `db:read:packing` | Read-only packing items of a trip via `ctx.packing.list(tripId)` | Membership-checked, and scoped to the acting user's visibility — a plugin never sees another member's private packing items. |