Commit Graph

2150 Commits

Author SHA1 Message Date
Maurice 5f372fa17f fix(db): type the integer flag and the owner literal for the server's compiler
The server compiles without strictNullChecks, where a narrowed value returned
as is keeps its full type, and two literal arms of one UNION column must share
a type.
2026-11-17 13:11:27 +01:00
Maurice 9f7efd49ea test(db): the Postgres plugin data store against a stand-in client
Covers what every call sets up, the schema check before plugin SQL runs, the
row cap, the time budget between chunks and at statement_timeout, the quota,
a script committed statement by statement, the tx and migrate shapes and one
call at a time per plugin, without a database. The Postgres lane runs the
same store on a real one.
2026-11-17 13:02:48 +01:00
Maurice 8e85fb1c2c docs(db): plugin data on Postgres, its SQL rules and one TREK per database
The plugin-sdk README says what dialect a plugin writes when TREK runs on
Postgres, what the guard refuses and how the limits hold there. The wiki's
Postgres page describes the plugin schemas, their backup and restore and
the one-instance-per-database limit, and its key rotation path now names
the file where it is.
2026-11-17 13:00:55 +01:00
Maurice b82cd03a28 fix(db): carry the plugin data schemas in Postgres backups and restores
pg_dump now names the trek_plugin_ schemas TREK's role owns beside its own
schema. A restore reads the dump's plugin schemas from its table of
contents, drops the plugin schemas the role owns, creates the dump's and
loads them with TREK's schema, all in the one psql transaction. Schemas of
another role and every other schema of the database are never read or
dropped.

The dump's migration history is read from TREK's mikro_orm_migrations only,
so a plugin table of that name cannot pass for it.
2026-11-17 13:00:55 +01:00
Maurice 3460511fa4 feat(plugins): store plugin data on the engine TREK runs on
PLUGIN_DATA binds the plugin data backend to the engine (the same
{ sqlite, postgres }[engine.id] binding the backup port uses): the SQLite
file per plugin, unchanged, or the Postgres schema per plugin. The host
opens a plugin's store through it and an uninstall that deletes data erases
through it, which drops the schema on Postgres. Nothing outside db/engine
asks which engine runs.

A plugin's SQL is passed through as written, so a plugin needs to know the
dialect: db.engine (db:own) answers 'sqlite' or 'postgres', and the SDK
exposes it as ctx.db.engine(), in the mock host (dbEngine option) and in
dev. The plugin facts are regenerated and the SDK goes to 1.10.0.

The Postgres lane's PGPLUG suite installs, uses and erases plugin data on
Postgres and shows that a plugin reaches neither TREK's users table nor
another plugin's schema.
2026-11-17 13:00:47 +01:00
Maurice b345c92493 feat(db): keep a plugin's own data in a Postgres schema of its own
A plugin with db:own got a SQLite WAL file in the data volume whatever the
engine, so a Postgres install still needed a local disk (discussion #386).
This adds the plugin data port (db/engine/plugin-data.ts) and its Postgres
side: one schema per plugin, trek_plugin_<id>, reached through TREK's pool.

Every call runs in a transaction that creates the schema if missing, puts
only it on the search_path and sets statement_timeout to the call's budget.
Reads stream through a cursor and stop at the row cap, a write that grows
the schema past the quota rolls back, and the calls of one plugin run one at
a time on at most half of the pool.

TREK's role may read every table, and a role per plugin would need
CREATEROLE, so plugin-sql-guard.ts checks each statement first: it lexes the
SQL as Postgres does and refuses another schema as a qualifier, the pg_
catalogs and functions, functions that run SQL from a string or reach an
object by a name in one, temporary tables, and every statement but DML and
DDL on tables, indexes, views, sequences and types.
2026-11-17 13:00:39 +01:00
Maurice 7a9a7c1831 refactor(db): keep the touched files within their size baselines
The users identity guard moves to repositories/_shared/user-identity.ts
(Users.repository.ts re-exports UserIdentityTakenError), the flag text pass
of the entity generator to scripts/generate-entities-flags.ts, and the
Wallet pass query takes the repository's EntityManager. The size baseline
of Users.repository.ts is lowered to what it holds now (1445 to 1421); a
few exports only their own file used are no longer exported.
2026-11-17 12:24:08 +01:00
Maurice 99c3c670a5 fix(demo): keep a demo baseline per engine and check it before every reset
The baseline was travel-baseline.db on both engines and went straight into
the swap, so a SQLite baseline left in the data directory made every hourly
reset of a Postgres demo fail inside pg_restore, and never got replaced. It
is now named after the engine's archive entry (travel-baseline.db on
SQLite, travel-baseline.pgdump on Postgres), and the backup port's new
checkCopy refuses one the engine cannot restore before the swap; the reset
then saves a fresh baseline and logs the reason instead of failing hourly.
2026-11-17 12:24:08 +01:00
Maurice 7a99190550 fix(db): reset the admin on Postgres and keep the key rotation off it
reset-admin.js opened data/travel.db with better-sqlite3 whatever the
engine, so on a Postgres install it created a stray SQLite file or reset
the admin in a leftover one. With TREK_DB_URL set it now resets the account
in Postgres through the built server's engine (its pool, SSL and write
lock, in one transaction), and refuses a TREK_DB_FILE beside the URL; the
SQLite branch is unchanged. migrate-encryption.ts cannot re-encrypt the
secrets in Postgres yet, so it now stops before it opens or writes any file
when TREK_DB_URL is set. The wiki says what each does on Postgres.
2026-11-17 12:24:08 +01:00
Maurice 7e8c74b8ff fix(db): sort NULLs where SQLite sorts them on Postgres
SQLite takes NULL as the smallest value and Postgres as the largest, so any
list ordered by a nullable column (bookings without a time, trips without
dates) came back in another order on Postgres. Instead of every ORDER BY
spelling its NULL position, the Postgres engine renders SQLite's wherever a
statement names none: a Kysely query compiler for this.kysely() statements
and a MikroORM platform for find() and the query builder. An explicit
position is kept.

A new Postgres-lane suite checks the answers the probe cannot see: the NULL
order, a stay linked as '14.0' in the public reads, the traveller owner
flag, the identity errors for a lost race, and the plugin caps under a race
on the pool.
2026-11-17 12:24:07 +01:00
Maurice 7bde1ec9d4 fix(plugins): hold the write lock for the plugin cap checks on Postgres
meta.set and scheduler.set checked their caps in a transaction of the
repository's own, which keeps check and write together on SQLite's single
writer but not on Postgres, where two pool connections both read the same
count. Both RPCs now run the capped write inside UnitOfWork.transactional,
which takes the engine's write lock; the repository's transaction becomes a
savepoint of it.
2026-11-17 12:24:07 +01:00
Maurice ec733fc927 fix(db): store addons.enabled as the integer flag on Postgres too
addons.enabled was the one flag MikroORM's boolean type declared as a
Postgres boolean, while SQLite and every other flag hold 0/1, so an addon
migration's usual SET enabled = 1 would fail there and a Kysely read would
answer true instead of 1. The column now uses IntegerFlagType: the entity
still reads a boolean, both engines store 0/1, and the regenerated Postgres
baseline declares it bigint default 0. The entity generator renders the
type for the columns in BOOLEAN_COLUMNS.
2026-11-17 12:24:07 +01:00
Maurice 20fb054d27 docs(db): Postgres 14 to 18, the database role, restore limits and the manual rollback
- Database-Postgres: supported versions, the version 18 client and the tool
  search order, sslrootcert with a client before 16, how to create a role
  that is not a superuser and move an older compose install to one, a
  current schema other than public, and the space a restore needs.
- Updating: the manual rollback clears the whole schema and loads the copy
  in one transaction, as TREK's restore does; dropping only the tables left
  the collation, the trigger functions and the sequences behind. A unit
  test keeps the printed statement equal to CLEAR_SCHEMA_SQL, and the boot's
  rollback hint points at that section.
- Troubleshooting, Install-Docker-Compose, Environment-Variables, README
  and server/CLAUDE.md follow.
2026-11-17 12:24:07 +01:00
Maurice 9c14f941ba build(docker): ship the PostgreSQL 18 client tools from the PGDG repository
A client handles every server up to its own major and refuses a newer one,
so the image needs the newest supported major, 18, which Debian 13 does not
ship. The PGDG repository is added with its signing key kept in the
repository and its component pinned to 18.
2026-11-17 12:23:51 +01:00
Maurice 6f4dfc486f fix(db): check the Postgres server and tools before a dump or a restore
- The client tools are looked for newest first, from 18 down: a client
  handles every server up to its own major and refuses a newer one. A dump
  or a restore now refuses a client older than the server up front, naming
  both versions, and the boot refuses a server outside PostgreSQL 14 to 18.
- sslrootcert=system is used only by a client from libpq 16 on; an older
  client is refused with a message that names sslrootcert and the bin dir.
- The restore's CVE-2025-8714 table covers 18.
- A restore refuses a role that is a superuser or may run programs or write
  files on the database host, since the archive's SQL runs as that role;
  the boot warns about such a role.
- pg_restore's script is streamed to its work file and stopped once it
  passes BACKUP_MAX_DECOMPRESSED_MB, so a small dump can no longer expand
  without limit on the data volume.
- A failed tool's error keeps only the summarized stderr; its cause names
  the exit code and signal instead of carrying the process error, whose
  message and stderr quoted row data into the log.
- The migration history is looked for in the current schema, where the
  tables and the backups are, so a role whose search_path is not public
  boots a second time.
2026-11-17 12:23:51 +01:00
Maurice ce1c3cf62d fix(db): read unique violations and missing tables the same way on both engines
guardIdentity matched only SQLite's message text, so on Postgres a lost
race for an email or a username escaped as a 500 instead of a 409. The new
dialect module driver-errors.ts reads a unique violation by Postgres's
SQLSTATE 23505 and constraint name, or by SQLite's message, and the users
repository maps the key names of both engines. isMissingTableError moves
there too and also knows Postgres's 42P01 for an unconverted error.
2026-11-17 12:23:50 +01:00
Maurice 28ce04d866 fix(db): return the traveller owner flag as a number on Postgres
The owner and member branches of listTravellerUsernames selected bound
values, which Postgres resolves to text in a UNION column, so the public
API reported every traveller as owner: false. The literals are written into
the statement as integers now.
2026-11-17 12:23:37 +01:00
Maurice 22973036b5 fix(db): match a stay-linked booking on Postgres in the public stay check
publicStayExists cast reservations.accommodation_id straight to integer,
which Postgres refuses for the stored '14.0', so the calendar feed and the
share links failed for every trip once any booking was linked to a stay.
The correlation now goes through accommodationIdMatches, which compares
through the new dialect helper integerTextEqualsKysely, and the Wallet pass
context uses the same helper instead of its own numeric cast.
2026-11-17 12:23:36 +01:00
Maurice 40f6efbf5b docs(db): describe the discovered Postgres lane and the 14/18 job 2026-11-17 12:05:17 +01:00
Maurice e4b990b75e ci(db): run the Postgres probe and lane on Postgres 14 and 18
TREK supports Postgres 14 to 18, and CI tested 17 only. One server-postgres
job per major now runs the probe and then the lane against the same service,
so the change keeps the job count of the two jobs it replaces. The lane's
global setup refuses a service whose major is not TREK_TEST_PG_MAJOR, and
the image's Postgres boot moves to 18, the major its client tools target.

scripts/ci/workflow-jobs.test.mjs fails when a job other than the scan is
missing from ci-ok's needs, and changed-areas pins that migrations,
repositories, the engine and the lane's files always run the job.
2026-11-17 12:04:45 +01:00
Maurice 3b6d7238fe test(db): find the Postgres lane's suites instead of listing them
The lane ran a hand-kept list of 26 suites that had already missed
trip-tags-mcp and the broadcast-parity suites, and listed two that never
touch a database. tests/postgres/lane.ts now takes every test file that
reaches src/bootstrap.ts or src/nest/app.module.ts through its imports and
does not vi.mock src/db/database, minus a named SQLITE_ONLY map with the
reason for each.

tests/unit/postgres-lane.test.ts (PGLANE, in the SQLite lane) fails on a
suite that names a boot entry point the discovery does not see, on a stale
exclusion, and on a new suite that boots the app over the SQLite snapshot
mock; the 105 that still do are held in snapshot-suites-baseline.json,
which only shrinks.
2026-11-17 12:04:45 +01:00
Maurice 16da5dd685 docs(db): the routine for a migration after the Postgres cut 2026-11-17 11:48:18 +01:00
Maurice 53d875aecb fix(db): keep check:pg-baseline speaking once the baseline is frozen
A frozen baseline still has its manifest verified, and the check now lists the
migrations that run on Postgres after the cut and what holds them. Every
problem it reports says what to do about it.
2026-11-17 11:48:18 +01:00
Maurice a748d6c554 feat(db): hold migrations after the Postgres cut to engineDdl in lint:dialect
A migration newer than the baseline's through (read from
assets/postgres/baseline.json) fails on a schema statement outside engineDdl,
a plain string column for addColumnIfMissing or a SQLite-only helper, and on a
SQLite spelling in an engineDdl postgres member. The advice names the two
steps, and the baseline cannot hold an allowance for these rules.
2026-11-17 11:48:17 +01:00
Maurice 90f1f1977d feat(db): print the Postgres side of a new migration with gen:pg-migration
The script builds a scratch database the way a boot does (the baseline, then
the migrations after its cut) up to the target migration, compares it with the
entities by PARITY-PG's rules and prints the remaining statements as the
postgres member of engineDdl. The scratch database is always dropped.
2026-11-17 11:48:17 +01:00
Maurice 8c5e138150 refactor(db): move PARITY-PG's schema comparison into the Postgres engine
Which comparator statements change nothing (drop not null on a live primary
key, spelling-only defaults and checks proven by a rolled-back fingerprint, the
hand-ported triggers) is now one module the parity test and the migration
generator share, with unit tests for the sorting and the rollback.
2026-11-17 11:48:17 +01:00
Maurice 1bf0670566 feat(db): write a migration's schema change once per engine
A migration after the Postgres baseline's cut runs on both engines, but its DDL
is not one language. engineDdl(this, { sqlite, postgres }) queues the statements
of the engine the migration runs on, like addSql, and addColumnIfMissing takes a
column definition per engine, both picked in the dialect layer so a migration
never branches on the engine itself.
2026-11-17 11:48:16 +01:00
Maurice 29cdc594d3 fix(db): cut the Postgres baseline at the newest migration again
The merge with dev brought eight migrations written for SQLite (DATETIME,
INTEGER PRIMARY KEY without identity); run verbatim on Postgres they stop the
first boot. No Postgres install has been released, so the baseline moves to
the newest migration and those tables come from the entities.
2026-11-17 11:31:04 +01:00
Maurice a2d00111ca merge dev into feat/postgres-support 2026-11-17 11:10:01 +01:00
Mariano 33d074c786 feat(places): tags on places, owned by the trip
A place keeps its one category, which decides its marker, and can now carry
any number of tags as extra labels to filter by.

- Tags belong to the trip (tags.trip_id): every member sees the same list and
  whoever may edit the trip's places adds, renames and deletes. Personal tags
  stay what they were.
- The place form picks them on desktop and mobile through one shared hook,
  usePlaceTags, and can add a new tag to the trip on the spot.
- The existing category filter of the places list lists the tags as well.
- Tags show on the place (inspector pills, phone place sheet); the marker is
  untouched.
- MCP: create_place, create_and_assign_place and update_place take `tags`,
  and list/create/update/delete_trip_tag manage a trip's tags.
- /api/trips/:tripId/tags and the tripTags:changed event keep every client
  of the trip in step.
2026-11-17 08:44:10 +01:00
Maurice 35dbcc2103 chore(deps): bump npm dependencies, jsdom 31, isomorphic-dompurify 4 and dotenv 18
The minor and patch group of #3108 without MikroORM: 7.2.9 refuses to set
the OAuth client reference from its client_id (the Postgres probe fails on
OauthTokens.insertToken and OauthConsents.upsertGrant), so MikroORM stays on
7.2.8 and with it better-sqlite3 and kysely, which it pins exactly.

mapbox-gl 3.33 shapes right-to-left text itself and removed setRTLTextPlugin,
so the engine registers nothing any more and @mapbox/mapbox-gl-rtl-text, its
Vite alias and the registration guard go. The WebAssembly module it loads
instead comes from api.mapbox.com, which the CSP already allows.

jsdom 31 and isomorphic-dompurify 4 move together, since the sanitizer needs
jsdom 31. jsdom is also a root devDependency now: listed only in the client,
npm nests it under client/ where vitest's built-in jsdom environment does not
find it.

dotenv 18 is loaded through `import 'dotenv/config'` only, which stays quiet.

Closes #3108, closes #3109, closes #3110, closes #3112
2026-11-16 21:20:33 +01:00
Maurice 9890f31d10 chore(dawarich): keep the stay naming helpers module-private 2026-11-16 16:57:03 +01:00
Maurice 04e0dc26e1 fix(dawarich): name a stay from its geocoded points when Dawarich sends a placeholder
Since 1.22 Dawarich sends a visit's display name, which is "Unknown Location"
for a visit without a place. TREK showed that as the name, although the points
recorded during the stay are reverse geocoded. A stay without a real name now
takes its place's name or address, and otherwise the city most of its points
fall in. A visit without a place also takes the middle of its points as its
coordinate.

Closes #3105
2026-11-16 16:34:47 +01:00
Maurice 2a134babbc fix(trips): keep a visit's own stay when a trip is copied
insertAssignmentCopy took stay_minutes but never wrote it, so a copied trip
fell back to the place's duration for every visit.
2026-11-16 10:45:24 +01:00
Maurice 2180175040 fix(mail-ingest): refuse an undo when the booking was moved in between
A move keeps the entry's resolution and changes only its trip and created
ids, so an undo that read them before a move landed still claimed the entry,
removed nothing and left the moved booking behind. The claim is now bound to
the trip and ids it was read with.
2026-11-16 10:45:23 +01:00
Maurice ea5542f8eb fix(plugins): require budget_edit for a plugin switching a trip's costs mode
trips.update checked trip_edit and the archive and cover permissions, but
let costs_mode through without the budget_edit check REST and MCP apply.
2026-11-16 10:45:23 +01:00
Maurice 40dea84085 fix(flight-live): leave a through flight's first sector aircraft to that sector
With two booked sectors under one flight number, the second is in its watch
window for the whole first one, and the aircraft flying the first sector was
read as the second having departed, with its departure notice hours early.
An aircraft that fits a sibling sector better is now left to it.
2026-11-16 10:45:22 +01:00
Maurice 89f7908359 fix(mail-ingest): let the pause notice job keep its cursor row
The CHECK on mail_ingest_cursor.transport did not allow 'notice', so the
hourly job that tells address holders the import is paused failed on its
lease insert every run. A new migration rebuilds the table with 'notice'
allowed, keeping every reader's position and lease.
2026-11-16 10:45:22 +01:00
Maurice 5b0412bc2c fix(mail-ingest): read a mail of bare sender lines in one pass
isBareHeaderBlock rescanned the header run below every sender line, so a
plain-text mail of nothing but "From:" lines without a subject took
quadratic time and held the event loop for seconds before the address was
even checked. The block starts are now worked out for all lines in one pass
from the bottom up, with the same rule.
2026-11-16 10:45:21 +01:00
Maurice 71510a62f3 fix(packing): take a bringer off in one transaction, and settle the last baselines
Clearing a common item's owner and dropping the bringer row now happen inside
one unit of work, so a failure cannot leave one without the other. The lint
and RTL baselines follow the rows that lost their last physical sides.
2026-11-15 12:40:01 +01:00
Maurice b8966330e0 test: keep the bringer and panel width changes within the lint and RTL ratchets
The bringer tests read their items through a small type instead of any, the
panel hook names its two ceilings after the panels rather than the sides, and
the stored-width test runs on a window wide enough for the map's share.
2026-11-15 12:22:12 +01:00
Maurice 7bbb5a9f71 fix(packing): anyone on the trip can be put down as the bringer of a shared item, and taken off again (#3098)
A common item only let you pledge yourself. Nobody else could be named,
and a bringer could never be removed, so an item that went personal and back
kept its owner's avatar for good. The item menu now lists every member to
tick on or off, a click on the avatar opens it there, and the add route takes
an optional user_id limited to the trip's members. Taking the owner off
clears the owner. The remove route now checks visibility for the caller, not
for the user named in the path.
2026-11-15 11:00:21 +01:00
Maurice c72c8dcaf7 fix(plugins): hand a visit's stay_minutes to plugins in the assignment output contract (#3092) 2026-11-14 18:03:20 +01:00
Maurice bac4599d64 fix(roadtrip): keep the stay on each visit, not on the place (#3092)
A nullable day_assignments.stay_minutes holds the stay of one visit; a visit without one keeps reading the place's duration, so existing trips drive as before. REST PUT /trips/:tripId/assignments/:id/stay and the MCP tool set_assignment_stay write it with day_edit.
2026-11-14 17:48:57 +01:00
Maurice f71acb08bd fix(assistant): send the earlier answer back to the provider in the shape it saw it (#3094)
Follow-up messages failed with HTTP 400: a whitespace-only text block, two model messages in a row where the turn had pushed for an answer, or a tool call renamed after its provider state was recorded. The rebuilt history now matches what the provider saw during the first answer.
2026-11-14 17:48:56 +01:00
Luffy ba562c9047 fix(tests): ensure email channel is registered before sending notifications 2026-11-14 16:49:32 +01:00
Luffy 6bcc49fc6f fix(notifications): link admin emails to admin notification settings 2026-11-14 16:49:32 +01:00
Maurice a76a96ccca fix(notifications): drop a tab the planner lacks from the URL, link booking mails to the reservations tab, open the chat on the phone 2026-11-14 16:49:20 +01:00
Maurice bfdbf4d811 fix(notifications): format the tests, move the open-planner tab test into its own file, lower the exhaustive-deps baseline 2026-11-14 16:49:20 +01:00
Luffy b068e593fc fix(notifications): update navigation targets for trip and vacay events 2026-11-14 16:49:19 +01:00