Commit Graph

5386 Commits

Author SHA1 Message Date
Maurice 55607ff2f9 feat(db): move a SQLite install into an empty Postgres database
npm run db:move-to-postgres (dist/move-to-postgres.js in the image) copies
a SQLite file at this build's migration level into an empty Postgres
database. It refuses a file at another level, rows whose references point
at nothing and a target holding any table or plugin schema, creates the
schema with the engine's own bootstrap, then in one transaction copies every
table in foreign key order (setting a self-reference once its rows are in),
converts each value to its column's type and refuses one Postgres cannot
hold exactly, copies each plugin's plugin.db into its schema, moves the
sequences past the copied ids and compares every table's row count and an
order-free checksum over all rows before it commits. A failure drops the
schema again. The SQLite files and the uploads are never written.
2026-11-17 23:03:54 +01:00
Maurice 1ea08dcf0a fix(db): refuse a Postgres boot that would leave a SQLite install behind
Setting TREK_DB_URL on an existing SQLite install booted a new, empty
instance beside its data. The Postgres bootstrap now refuses an empty
database, or a migrated one without a single account, while the data
directory's travel.db holds accounts, and names the move command. A file
that cannot be read counts as holding them. TREK_DB_IGNORE_SQLITE_FILE=true
starts anyway, with a warning; the file is opened read-only and only when
the database holds no TREK data.
2026-11-17 23:03:54 +01:00
Maurice c6f6dd2d57 fix(db): find a stay's bookings on Postgres
reservations.accommodation_id is TEXT and holds '14.0'. Compared with a bare
number, Postgres binds the number as the text '14' and finds nothing, so
deleting a stay, a day or a place left its bookings behind with no error. The
three filters read the column as an integer, as the joins already do.
2026-11-17 23:03:54 +01:00
Maurice 579df97bba fix(db): type the integer flag and the owner literal for the server's compiler
The server compiles without strictNullChecks, where a narrowed value returned
as is keeps its full type, and two literal arms of one UNION column must share
a type.
2026-11-17 23:03:54 +01:00
Maurice aa2f7ca8b4 test(db): the Postgres plugin data store against a stand-in client
Covers what every call sets up, the schema check before plugin SQL runs, the
row cap, the time budget between chunks and at statement_timeout, the quota,
a script committed statement by statement, the tx and migrate shapes and one
call at a time per plugin, without a database. The Postgres lane runs the
same store on a real one.
2026-11-17 23:03:54 +01:00
Maurice 17c9c0758f docs(db): plugin data on Postgres, its SQL rules and one TREK per database
The plugin-sdk README says what dialect a plugin writes when TREK runs on
Postgres, what the guard refuses and how the limits hold there. The wiki's
Postgres page describes the plugin schemas, their backup and restore and
the one-instance-per-database limit, and its key rotation path now names
the file where it is.
2026-11-17 23:03:54 +01:00
Maurice 7340087353 fix(db): carry the plugin data schemas in Postgres backups and restores
pg_dump now names the trek_plugin_ schemas TREK's role owns beside its own
schema. A restore reads the dump's plugin schemas from its table of
contents, drops the plugin schemas the role owns, creates the dump's and
loads them with TREK's schema, all in the one psql transaction. Schemas of
another role and every other schema of the database are never read or
dropped.

The dump's migration history is read from TREK's mikro_orm_migrations only,
so a plugin table of that name cannot pass for it.
2026-11-17 23:03:53 +01:00
Maurice 910c98b971 feat(plugins): store plugin data on the engine TREK runs on
PLUGIN_DATA binds the plugin data backend to the engine (the same
{ sqlite, postgres }[engine.id] binding the backup port uses): the SQLite
file per plugin, unchanged, or the Postgres schema per plugin. The host
opens a plugin's store through it and an uninstall that deletes data erases
through it, which drops the schema on Postgres. Nothing outside db/engine
asks which engine runs.

A plugin's SQL is passed through as written, so a plugin needs to know the
dialect: db.engine (db:own) answers 'sqlite' or 'postgres', and the SDK
exposes it as ctx.db.engine(), in the mock host (dbEngine option) and in
dev. The plugin facts are regenerated and the SDK goes to 1.10.0.

The Postgres lane's PGPLUG suite installs, uses and erases plugin data on
Postgres and shows that a plugin reaches neither TREK's users table nor
another plugin's schema.
2026-11-17 23:03:53 +01:00
Maurice b144781e9f feat(db): keep a plugin's own data in a Postgres schema of its own
A plugin with db:own got a SQLite WAL file in the data volume whatever the
engine, so a Postgres install still needed a local disk (discussion #386).
This adds the plugin data port (db/engine/plugin-data.ts) and its Postgres
side: one schema per plugin, trek_plugin_<id>, reached through TREK's pool.

Every call runs in a transaction that creates the schema if missing, puts
only it on the search_path and sets statement_timeout to the call's budget.
Reads stream through a cursor and stop at the row cap, a write that grows
the schema past the quota rolls back, and the calls of one plugin run one at
a time on at most half of the pool.

TREK's role may read every table, and a role per plugin would need
CREATEROLE, so plugin-sql-guard.ts checks each statement first: it lexes the
SQL as Postgres does and refuses another schema as a qualifier, the pg_
catalogs and functions, functions that run SQL from a string or reach an
object by a name in one, temporary tables, and every statement but DML and
DDL on tables, indexes, views, sequences and types.
2026-11-17 23:03:53 +01:00
Maurice a26db47cb6 refactor(db): keep the touched files within their size baselines
The users identity guard moves to repositories/_shared/user-identity.ts
(Users.repository.ts re-exports UserIdentityTakenError), the flag text pass
of the entity generator to scripts/generate-entities-flags.ts, and the
Wallet pass query takes the repository's EntityManager. The size baseline
of Users.repository.ts is lowered to what it holds now (1445 to 1421); a
few exports only their own file used are no longer exported.
2026-11-17 23:03:53 +01:00
Maurice 9e5379d092 fix(demo): keep a demo baseline per engine and check it before every reset
The baseline was travel-baseline.db on both engines and went straight into
the swap, so a SQLite baseline left in the data directory made every hourly
reset of a Postgres demo fail inside pg_restore, and never got replaced. It
is now named after the engine's archive entry (travel-baseline.db on
SQLite, travel-baseline.pgdump on Postgres), and the backup port's new
checkCopy refuses one the engine cannot restore before the swap; the reset
then saves a fresh baseline and logs the reason instead of failing hourly.
2026-11-17 23:03:53 +01:00
Maurice 693acd5580 fix(db): reset the admin on Postgres and keep the key rotation off it
reset-admin.js opened data/travel.db with better-sqlite3 whatever the
engine, so on a Postgres install it created a stray SQLite file or reset
the admin in a leftover one. With TREK_DB_URL set it now resets the account
in Postgres through the built server's engine (its pool, SSL and write
lock, in one transaction), and refuses a TREK_DB_FILE beside the URL; the
SQLite branch is unchanged. migrate-encryption.ts cannot re-encrypt the
secrets in Postgres yet, so it now stops before it opens or writes any file
when TREK_DB_URL is set. The wiki says what each does on Postgres.
2026-11-17 23:03:52 +01:00
Maurice c3f6177be1 fix(db): sort NULLs where SQLite sorts them on Postgres
SQLite takes NULL as the smallest value and Postgres as the largest, so any
list ordered by a nullable column (bookings without a time, trips without
dates) came back in another order on Postgres. Instead of every ORDER BY
spelling its NULL position, the Postgres engine renders SQLite's wherever a
statement names none: a Kysely query compiler for this.kysely() statements
and a MikroORM platform for find() and the query builder. An explicit
position is kept.

A new Postgres-lane suite checks the answers the probe cannot see: the NULL
order, a stay linked as '14.0' in the public reads, the traveller owner
flag, the identity errors for a lost race, and the plugin caps under a race
on the pool.
2026-11-17 23:03:52 +01:00
Maurice 60642cc18b fix(plugins): hold the write lock for the plugin cap checks on Postgres
meta.set and scheduler.set checked their caps in a transaction of the
repository's own, which keeps check and write together on SQLite's single
writer but not on Postgres, where two pool connections both read the same
count. Both RPCs now run the capped write inside UnitOfWork.transactional,
which takes the engine's write lock; the repository's transaction becomes a
savepoint of it.
2026-11-17 23:03:52 +01:00
Maurice 54b843ae29 fix(db): store addons.enabled as the integer flag on Postgres too
addons.enabled was the one flag MikroORM's boolean type declared as a
Postgres boolean, while SQLite and every other flag hold 0/1, so an addon
migration's usual SET enabled = 1 would fail there and a Kysely read would
answer true instead of 1. The column now uses IntegerFlagType: the entity
still reads a boolean, both engines store 0/1, and the regenerated Postgres
baseline declares it bigint default 0. The entity generator renders the
type for the columns in BOOLEAN_COLUMNS.
2026-11-17 23:03:52 +01:00
Maurice 633e9bcdc1 docs(db): Postgres 14 to 18, the database role, restore limits and the manual rollback
- Database-Postgres: supported versions, the version 18 client and the tool
  search order, sslrootcert with a client before 16, how to create a role
  that is not a superuser and move an older compose install to one, a
  current schema other than public, and the space a restore needs.
- Updating: the manual rollback clears the whole schema and loads the copy
  in one transaction, as TREK's restore does; dropping only the tables left
  the collation, the trigger functions and the sequences behind. A unit
  test keeps the printed statement equal to CLEAR_SCHEMA_SQL, and the boot's
  rollback hint points at that section.
- Troubleshooting, Install-Docker-Compose, Environment-Variables, README
  and server/CLAUDE.md follow.
2026-11-17 23:03:52 +01:00
Maurice eab501925f fix(deploy): run TREK as a role that is not a superuser in the Postgres compose file
The compose file let the postgres image create TREK's role as the
superuser, so a crafted backup restored by TREK could run commands in the
database container. The superuser is now postgres, with a password of its
own, and an init script creates the role trek that owns TREK's database and
nothing more. The Helm values say the same about the role in the URL.
2026-11-17 23:03:51 +01:00
Maurice 2630d7e8ec build(docker): ship the PostgreSQL 18 client tools from the PGDG repository
A client handles every server up to its own major and refuses a newer one,
so the image needs the newest supported major, 18, which Debian 13 does not
ship. The PGDG repository is added with its signing key kept in the
repository and its component pinned to 18.
2026-11-17 23:03:51 +01:00
Maurice 116ba52fbb fix(db): check the Postgres server and tools before a dump or a restore
- The client tools are looked for newest first, from 18 down: a client
  handles every server up to its own major and refuses a newer one. A dump
  or a restore now refuses a client older than the server up front, naming
  both versions, and the boot refuses a server outside PostgreSQL 14 to 18.
- sslrootcert=system is used only by a client from libpq 16 on; an older
  client is refused with a message that names sslrootcert and the bin dir.
- The restore's CVE-2025-8714 table covers 18.
- A restore refuses a role that is a superuser or may run programs or write
  files on the database host, since the archive's SQL runs as that role;
  the boot warns about such a role.
- pg_restore's script is streamed to its work file and stopped once it
  passes BACKUP_MAX_DECOMPRESSED_MB, so a small dump can no longer expand
  without limit on the data volume.
- A failed tool's error keeps only the summarized stderr; its cause names
  the exit code and signal instead of carrying the process error, whose
  message and stderr quoted row data into the log.
- The migration history is looked for in the current schema, where the
  tables and the backups are, so a role whose search_path is not public
  boots a second time.
2026-11-17 23:03:51 +01:00
Maurice e5fb6d9107 fix(db): read unique violations and missing tables the same way on both engines
guardIdentity matched only SQLite's message text, so on Postgres a lost
race for an email or a username escaped as a 500 instead of a 409. The new
dialect module driver-errors.ts reads a unique violation by Postgres's
SQLSTATE 23505 and constraint name, or by SQLite's message, and the users
repository maps the key names of both engines. isMissingTableError moves
there too and also knows Postgres's 42P01 for an unconverted error.
2026-11-17 23:03:51 +01:00
Maurice 98529bc84c fix(db): return the traveller owner flag as a number on Postgres
The owner and member branches of listTravellerUsernames selected bound
values, which Postgres resolves to text in a UNION column, so the public
API reported every traveller as owner: false. The literals are written into
the statement as integers now.
2026-11-17 23:03:51 +01:00
Maurice 4a3ad562a7 fix(db): match a stay-linked booking on Postgres in the public stay check
publicStayExists cast reservations.accommodation_id straight to integer,
which Postgres refuses for the stored '14.0', so the calendar feed and the
share links failed for every trip once any booking was linked to a stay.
The correlation now goes through accommodationIdMatches, which compares
through the new dialect helper integerTextEqualsKysely, and the Wallet pass
context uses the same helper instead of its own numeric cast.
2026-11-17 23:03:50 +01:00
Maurice 518287ff12 docs(db): describe the discovered Postgres lane and the 14/18 job 2026-11-17 23:03:50 +01:00
Maurice 0cdfb9eb42 ci(db): run the Postgres probe and lane on Postgres 14 and 18
TREK supports Postgres 14 to 18, and CI tested 17 only. One server-postgres
job per major now runs the probe and then the lane against the same service,
so the change keeps the job count of the two jobs it replaces. The lane's
global setup refuses a service whose major is not TREK_TEST_PG_MAJOR, and
the image's Postgres boot moves to 18, the major its client tools target.

scripts/ci/workflow-jobs.test.mjs fails when a job other than the scan is
missing from ci-ok's needs, and changed-areas pins that migrations,
repositories, the engine and the lane's files always run the job.
2026-11-17 23:03:50 +01:00
Maurice 4d0067a455 test(db): find the Postgres lane's suites instead of listing them
The lane ran a hand-kept list of 26 suites that had already missed
trip-tags-mcp and the broadcast-parity suites, and listed two that never
touch a database. tests/postgres/lane.ts now takes every test file that
reaches src/bootstrap.ts or src/nest/app.module.ts through its imports and
does not vi.mock src/db/database, minus a named SQLITE_ONLY map with the
reason for each.

tests/unit/postgres-lane.test.ts (PGLANE, in the SQLite lane) fails on a
suite that names a boot entry point the discovery does not see, on a stale
exclusion, and on a new suite that boots the app over the SQLite snapshot
mock; the 105 that still do are held in snapshot-suites-baseline.json,
which only shrinks.
2026-11-17 23:03:50 +01:00
Maurice 66d4431d26 docs(db): the routine for a migration after the Postgres cut 2026-11-17 23:03:50 +01:00
Maurice 293a48c26a fix(db): keep check:pg-baseline speaking once the baseline is frozen
A frozen baseline still has its manifest verified, and the check now lists the
migrations that run on Postgres after the cut and what holds them. Every
problem it reports says what to do about it.
2026-11-17 23:03:50 +01:00
Maurice 80fca10784 feat(db): hold migrations after the Postgres cut to engineDdl in lint:dialect
A migration newer than the baseline's through (read from
assets/postgres/baseline.json) fails on a schema statement outside engineDdl,
a plain string column for addColumnIfMissing or a SQLite-only helper, and on a
SQLite spelling in an engineDdl postgres member. The advice names the two
steps, and the baseline cannot hold an allowance for these rules.
2026-11-17 23:03:49 +01:00
Maurice 830cd6e3b5 feat(db): print the Postgres side of a new migration with gen:pg-migration
The script builds a scratch database the way a boot does (the baseline, then
the migrations after its cut) up to the target migration, compares it with the
entities by PARITY-PG's rules and prints the remaining statements as the
postgres member of engineDdl. The scratch database is always dropped.
2026-11-17 23:03:49 +01:00
Maurice 0f338d5484 refactor(db): move PARITY-PG's schema comparison into the Postgres engine
Which comparator statements change nothing (drop not null on a live primary
key, spelling-only defaults and checks proven by a rolled-back fingerprint, the
hand-ported triggers) is now one module the parity test and the migration
generator share, with unit tests for the sorting and the rollback.
2026-11-17 23:03:49 +01:00
Maurice a99cd40f4d feat(db): write a migration's schema change once per engine
A migration after the Postgres baseline's cut runs on both engines, but its DDL
is not one language. engineDdl(this, { sqlite, postgres }) queues the statements
of the engine the migration runs on, like addSql, and addColumnIfMissing takes a
column definition per engine, both picked in the dialect layer so a migration
never branches on the engine itself.
2026-11-17 23:03:49 +01:00
Maurice f3ea50b17b fix(db): cut the Postgres baseline at the newest migration again
The merge with dev brought eight migrations written for SQLite (DATETIME,
INTEGER PRIMARY KEY without identity); run verbatim on Postgres they stop the
first boot. No Postgres install has been released, so the baseline moves to
the newest migration and those tables come from the entities.
2026-11-17 23:03:49 +01:00
Isaias Tavares 2e3762afe1 feat(db): run TREK on PostgreSQL
TREK runs on PostgreSQL when TREK_DB_URL is set, and on SQLite as before
otherwise: an engine port under src/db/engine, the dialect helpers under
src/db/dialect, a Postgres baseline for the migrations up to the cut,
pg_dump/pg_restore backups, a probe of every repository method against a
real server and a Postgres test lane in CI.
2026-11-17 23:03:33 +01:00
Maurice 17b47b38b9 test(places): wait for the new tag's tick, not only the tag
The tag reaches the trip's list before the form ticks it, so the button
could be found in the moment between and read as not pressed.
2026-11-17 10:12:33 +01:00
Maurice 494c848ec9 chore(i18n): drop the place form's old text placeholders
The texts card explains each text in a line of its own and the phone sheet
follows it, so the description, notes and day note placeholders are no
longer read anywhere.
2026-11-17 09:41:30 +01:00
Maurice c203b0a597 test(places): look for the hidden tab's formatting buttons too 2026-11-17 09:27:49 +01:00
Maurice b36fefdf5d test(places): keep the place form's test file at its size
The tab changes had grown it past its line budget; the labels of the texts
are covered by PlaceFormTexts' own tests.
2026-11-17 09:20:46 +01:00
Maurice 8611359f7c chore(lint): lower the tests' non-null count to what they hold 2026-11-17 09:18:50 +01:00
Maurice c8e48460b6 fix(places): settle the gates for the place form rework
The text tabs name their field type for the switch, so it type-checks. The
colour field's sliders run left to right in every language and say so with
dir=ltr and a logical inset. The new phone fields take their sizes from the
type tiers or fs() instead of rem literals, the category field builds its
icon as an element, and the ESLint and theme baselines come down to what the
tree now holds. The tests follow the tabs (only the open tab's field and
formatting bar are shown), the category list and the files row, whose link
button keeps its place before attach.
2026-11-17 09:08:41 +01:00
Maurice 5ecf3001af fix(places): keep the hooks' helper types to their files
knip counts an exported type nothing imports; the tabs' row type and the
hours' field type are only used where they are declared.
2026-11-17 08:57:18 +01:00
Maurice 2c5beee849 fix(deps): hold jsdom at 29 and isomorphic-dompurify at 3, and settle the gates
jsdom 31 failed 56 client test files: it simplifies calc() and reads an
unset border as none in computed styles, and its File no longer carries
what the upload tests hand it. It moves back to 29 with isomorphic-dompurify
3, which needs it; the lockfile is resolved from the one before the bump, so
both sit at the root as they did. jsdom stays a root devDependency, so npm
keeps the copy vitest resolves at the root.

knip 6.45 reports tsconfig-paths, which test.yml loads from the server, as
unlisted rather than unresolved; it is the server's dependency. The removed
RTL text plugin took one swallowed catch with it, so that ESLint count is
lowered to what the tree holds.
2026-11-17 08:49:37 +01:00
Maurice a3f6635633 feat(places): colour tags, tabs for the texts and opening hours on the phone
Tags on the place form are pills in their own colour, with a palette under
the dot that recolours a tag for the whole trip (PUT /trips/:id/tags/:id was
there already, the client now reaches it through repo and store) and a
custom colour field of its own instead of the system picker. A new tag is
made in a colour picked the same way.

Description, notes and the note for one day's visit sit in one card under a
switch, each with a line on where the text shows up; the day tab is named
after its day and is only there with one visit in context. Nothing about the
three fields changes underneath.

Opening hours get a switch for open around the clock next to closed. Google
writes a place open all week as one period on Sunday that never closes; it
read as Sunday open and every other day closed, and now opens every day.

The phone form follows the desktop: category as one field with a list, tags
in one field, the texts switch, and phone, e-mail and opening hours, which it
did not have. The hours and the text tabs are shared hooks (usePlaceHours,
usePlaceTextTabs). Files and costs on the phone sheets are drawn as fields
like the rest, and the paste hint, a keyboard shortcut a phone has no use
for, is gone with its key.
2026-11-17 08:44:46 +01:00
Mariano 33d074c786 feat(places): tags on places, owned by the trip
A place keeps its one category, which decides its marker, and can now carry
any number of tags as extra labels to filter by.

- Tags belong to the trip (tags.trip_id): every member sees the same list and
  whoever may edit the trip's places adds, renames and deletes. Personal tags
  stay what they were.
- The place form picks them on desktop and mobile through one shared hook,
  usePlaceTags, and can add a new tag to the trip on the spot.
- The existing category filter of the places list lists the tags as well.
- Tags show on the place (inspector pills, phone place sheet); the marker is
  untouched.
- MCP: create_place, create_and_assign_place and update_place take `tags`,
  and list/create/update/delete_trip_tag manage a trip's tags.
- /api/trips/:tripId/tags and the tripTags:changed event keep every client
  of the trip in step.
2026-11-17 08:44:10 +01:00
Maurice e952586f60 chore(deps): keep the rollup musl pins on the installed rollup
The minor and patch group raised the Alpine pins to 4.68.3 while rollup
itself stayed on 4.62.0, so npm nested a second 4.62.0 copy under rollup
and the hoisted pins went unused. The pins follow rollup again.
2026-11-17 07:56:26 +01:00
Maurice 35c0cd35ef ci(dependabot): leave TypeScript majors to be moved by hand
TypeScript 7 is the native compiler without a JavaScript API, which
typescript-eslint, the coverage instrumenter and the Postgres probe need.
2026-11-16 21:20:33 +01:00
Maurice a1d753a975 style(i18n): format Arabic and Thai locales for Prettier 3.10
Prettier 3.10 measures the width of Arabic and Thai text differently, so
strings that fit the line now stay on it.
2026-11-16 21:20:33 +01:00
Maurice 35dbcc2103 chore(deps): bump npm dependencies, jsdom 31, isomorphic-dompurify 4 and dotenv 18
The minor and patch group of #3108 without MikroORM: 7.2.9 refuses to set
the OAuth client reference from its client_id (the Postgres probe fails on
OauthTokens.insertToken and OauthConsents.upsertGrant), so MikroORM stays on
7.2.8 and with it better-sqlite3 and kysely, which it pins exactly.

mapbox-gl 3.33 shapes right-to-left text itself and removed setRTLTextPlugin,
so the engine registers nothing any more and @mapbox/mapbox-gl-rtl-text, its
Vite alias and the registration guard go. The WebAssembly module it loads
instead comes from api.mapbox.com, which the CSP already allows.

jsdom 31 and isomorphic-dompurify 4 move together, since the sanitizer needs
jsdom 31. jsdom is also a root devDependency now: listed only in the client,
npm nests it under client/ where vitest's built-in jsdom environment does not
find it.

dotenv 18 is loaded through `import 'dotenv/config'` only, which stays quiet.

Closes #3108, closes #3109, closes #3110, closes #3112
2026-11-16 21:20:33 +01:00
Maurice 3c054fffa0 chore(plugin-sdk): bump @clack/prompts from 1.8.1 to 1.9.0
Closes #3106
2026-11-16 21:20:32 +01:00
Maurice c1bad1204c ci: bump SonarSource/sonarqube-scan-action from 8.4.1 to 8.5.0
Closes #3107
2026-11-16 21:20:32 +01:00
Maurice 9890f31d10 chore(dawarich): keep the stay naming helpers module-private 2026-11-16 16:57:03 +01:00