npm run db:move-to-postgres (dist/move-to-postgres.js in the image) copies
a SQLite file at this build's migration level into an empty Postgres
database. It refuses a file at another level, rows whose references point
at nothing and a target holding any table or plugin schema, creates the
schema with the engine's own bootstrap, then in one transaction copies every
table in foreign key order (setting a self-reference once its rows are in),
converts each value to its column's type and refuses one Postgres cannot
hold exactly, copies each plugin's plugin.db into its schema, moves the
sequences past the copied ids and compares every table's row count and an
order-free checksum over all rows before it commits. A failure drops the
schema again. The SQLite files and the uploads are never written.
Setting TREK_DB_URL on an existing SQLite install booted a new, empty
instance beside its data. The Postgres bootstrap now refuses an empty
database, or a migrated one without a single account, while the data
directory's travel.db holds accounts, and names the move command. A file
that cannot be read counts as holding them. TREK_DB_IGNORE_SQLITE_FILE=true
starts anyway, with a warning; the file is opened read-only and only when
the database holds no TREK data.
Covers what every call sets up, the schema check before plugin SQL runs, the
row cap, the time budget between chunks and at statement_timeout, the quota,
a script committed statement by statement, the tx and migrate shapes and one
call at a time per plugin, without a database. The Postgres lane runs the
same store on a real one.
pg_dump now names the trek_plugin_ schemas TREK's role owns beside its own
schema. A restore reads the dump's plugin schemas from its table of
contents, drops the plugin schemas the role owns, creates the dump's and
loads them with TREK's schema, all in the one psql transaction. Schemas of
another role and every other schema of the database are never read or
dropped.
The dump's migration history is read from TREK's mikro_orm_migrations only,
so a plugin table of that name cannot pass for it.
A plugin with db:own got a SQLite WAL file in the data volume whatever the
engine, so a Postgres install still needed a local disk (discussion #386).
This adds the plugin data port (db/engine/plugin-data.ts) and its Postgres
side: one schema per plugin, trek_plugin_<id>, reached through TREK's pool.
Every call runs in a transaction that creates the schema if missing, puts
only it on the search_path and sets statement_timeout to the call's budget.
Reads stream through a cursor and stop at the row cap, a write that grows
the schema past the quota rolls back, and the calls of one plugin run one at
a time on at most half of the pool.
TREK's role may read every table, and a role per plugin would need
CREATEROLE, so plugin-sql-guard.ts checks each statement first: it lexes the
SQL as Postgres does and refuses another schema as a qualifier, the pg_
catalogs and functions, functions that run SQL from a string or reach an
object by a name in one, temporary tables, and every statement but DML and
DDL on tables, indexes, views, sequences and types.
The users identity guard moves to repositories/_shared/user-identity.ts
(Users.repository.ts re-exports UserIdentityTakenError), the flag text pass
of the entity generator to scripts/generate-entities-flags.ts, and the
Wallet pass query takes the repository's EntityManager. The size baseline
of Users.repository.ts is lowered to what it holds now (1445 to 1421); a
few exports only their own file used are no longer exported.
reset-admin.js opened data/travel.db with better-sqlite3 whatever the
engine, so on a Postgres install it created a stray SQLite file or reset
the admin in a leftover one. With TREK_DB_URL set it now resets the account
in Postgres through the built server's engine (its pool, SSL and write
lock, in one transaction), and refuses a TREK_DB_FILE beside the URL; the
SQLite branch is unchanged. migrate-encryption.ts cannot re-encrypt the
secrets in Postgres yet, so it now stops before it opens or writes any file
when TREK_DB_URL is set. The wiki says what each does on Postgres.
SQLite takes NULL as the smallest value and Postgres as the largest, so any
list ordered by a nullable column (bookings without a time, trips without
dates) came back in another order on Postgres. Instead of every ORDER BY
spelling its NULL position, the Postgres engine renders SQLite's wherever a
statement names none: a Kysely query compiler for this.kysely() statements
and a MikroORM platform for find() and the query builder. An explicit
position is kept.
A new Postgres-lane suite checks the answers the probe cannot see: the NULL
order, a stay linked as '14.0' in the public reads, the traveller owner
flag, the identity errors for a lost race, and the plugin caps under a race
on the pool.
addons.enabled was the one flag MikroORM's boolean type declared as a
Postgres boolean, while SQLite and every other flag hold 0/1, so an addon
migration's usual SET enabled = 1 would fail there and a Kysely read would
answer true instead of 1. The column now uses IntegerFlagType: the entity
still reads a boolean, both engines store 0/1, and the regenerated Postgres
baseline declares it bigint default 0. The entity generator renders the
type for the columns in BOOLEAN_COLUMNS.
- Database-Postgres: supported versions, the version 18 client and the tool
search order, sslrootcert with a client before 16, how to create a role
that is not a superuser and move an older compose install to one, a
current schema other than public, and the space a restore needs.
- Updating: the manual rollback clears the whole schema and loads the copy
in one transaction, as TREK's restore does; dropping only the tables left
the collation, the trigger functions and the sequences behind. A unit
test keeps the printed statement equal to CLEAR_SCHEMA_SQL, and the boot's
rollback hint points at that section.
- Troubleshooting, Install-Docker-Compose, Environment-Variables, README
and server/CLAUDE.md follow.
- The client tools are looked for newest first, from 18 down: a client
handles every server up to its own major and refuses a newer one. A dump
or a restore now refuses a client older than the server up front, naming
both versions, and the boot refuses a server outside PostgreSQL 14 to 18.
- sslrootcert=system is used only by a client from libpq 16 on; an older
client is refused with a message that names sslrootcert and the bin dir.
- The restore's CVE-2025-8714 table covers 18.
- A restore refuses a role that is a superuser or may run programs or write
files on the database host, since the archive's SQL runs as that role;
the boot warns about such a role.
- pg_restore's script is streamed to its work file and stopped once it
passes BACKUP_MAX_DECOMPRESSED_MB, so a small dump can no longer expand
without limit on the data volume.
- A failed tool's error keeps only the summarized stderr; its cause names
the exit code and signal instead of carrying the process error, whose
message and stderr quoted row data into the log.
- The migration history is looked for in the current schema, where the
tables and the backups are, so a role whose search_path is not public
boots a second time.
guardIdentity matched only SQLite's message text, so on Postgres a lost
race for an email or a username escaped as a 500 instead of a 409. The new
dialect module driver-errors.ts reads a unique violation by Postgres's
SQLSTATE 23505 and constraint name, or by SQLite's message, and the users
repository maps the key names of both engines. isMissingTableError moves
there too and also knows Postgres's 42P01 for an unconverted error.
publicStayExists cast reservations.accommodation_id straight to integer,
which Postgres refuses for the stored '14.0', so the calendar feed and the
share links failed for every trip once any booking was linked to a stay.
The correlation now goes through accommodationIdMatches, which compares
through the new dialect helper integerTextEqualsKysely, and the Wallet pass
context uses the same helper instead of its own numeric cast.
TREK supports Postgres 14 to 18, and CI tested 17 only. One server-postgres
job per major now runs the probe and then the lane against the same service,
so the change keeps the job count of the two jobs it replaces. The lane's
global setup refuses a service whose major is not TREK_TEST_PG_MAJOR, and
the image's Postgres boot moves to 18, the major its client tools target.
scripts/ci/workflow-jobs.test.mjs fails when a job other than the scan is
missing from ci-ok's needs, and changed-areas pins that migrations,
repositories, the engine and the lane's files always run the job.
A frozen baseline still has its manifest verified, and the check now lists the
migrations that run on Postgres after the cut and what holds them. Every
problem it reports says what to do about it.
Which comparator statements change nothing (drop not null on a live primary
key, spelling-only defaults and checks proven by a rolled-back fingerprint, the
hand-ported triggers) is now one module the parity test and the migration
generator share, with unit tests for the sorting and the rollback.
A migration after the Postgres baseline's cut runs on both engines, but its DDL
is not one language. engineDdl(this, { sqlite, postgres }) queues the statements
of the engine the migration runs on, like addSql, and addColumnIfMissing takes a
column definition per engine, both picked in the dialect layer so a migration
never branches on the engine itself.
TREK runs on PostgreSQL when TREK_DB_URL is set, and on SQLite as before
otherwise: an engine port under src/db/engine, the dialect helpers under
src/db/dialect, a Postgres baseline for the migrations up to the cut,
pg_dump/pg_restore backups, a probe of every repository method against a
real server and a Postgres test lane in CI.
A place keeps its one category, which decides its marker, and can now carry
any number of tags as extra labels to filter by.
- Tags belong to the trip (tags.trip_id): every member sees the same list and
whoever may edit the trip's places adds, renames and deletes. Personal tags
stay what they were.
- The place form picks them on desktop and mobile through one shared hook,
usePlaceTags, and can add a new tag to the trip on the spot.
- The existing category filter of the places list lists the tags as well.
- Tags show on the place (inspector pills, phone place sheet); the marker is
untouched.
- MCP: create_place, create_and_assign_place and update_place take `tags`,
and list/create/update/delete_trip_tag manage a trip's tags.
- /api/trips/:tripId/tags and the tripTags:changed event keep every client
of the trip in step.
A move keeps the entry's resolution and changes only its trip and created
ids, so an undo that read them before a move landed still claimed the entry,
removed nothing and left the moved booking behind. The claim is now bound to
the trip and ids it was read with.
The CHECK on mail_ingest_cursor.transport did not allow 'notice', so the
hourly job that tells address holders the import is paused failed on its
lease insert every run. A new migration rebuilds the table with 'notice'
allowed, keeping every reader's position and lease.
A common item only let you pledge yourself. Nobody else could be named,
and a bringer could never be removed, so an item that went personal and back
kept its owner's avatar for good. The item menu now lists every member to
tick on or off, a click on the avatar opens it there, and the add route takes
an optional user_id limited to the trip's members. Taking the owner off
clears the owner. The remove route now checks visibility for the caller, not
for the user named in the path.
A nullable day_assignments.stay_minutes holds the stay of one visit; a visit without one keeps reading the place's duration, so existing trips drive as before. REST PUT /trips/:tripId/assignments/:id/stay and the MCP tool set_assignment_stay write it with day_edit.
The cron callbacks hand back nothing; the cursor's lookup is findCursor so it
no longer shadows the base repository's find(); the draft's suggested trip
uses the shared contract's type; endpoints spell out their optional columns;
the relay page's running count is typed. Prettier over every new file outside
the generated entities.
A plan without the import switches it off from outside the server, and an
admin can switch it off too; the plan promises a "mail import paused" note.
An hourly job sends mail_ingest_paused to each holder of a live address with
their own import on, notes the pause first, and forgets it once it is back.
The inbox pages a person's entries with the mail each came from and the trip
it suggests (named only when the reader can see it), shows a mail as plain
text, and takes an uploaded .eml, reading it at once. Move writes the booking
anew in the other trip and removes it from the first in one transaction.
The IMAP box, the webhook secret and the address template join the admin
settings form (masked, encrypted, held by the operator on a managed install),
and MAIL_INGEST_* variables override them through the mailIngestConfig token.
The audit row masks every masked key, and key rotation covers the new secrets.
Addresses per person and per trip (random base32 token, stored hashed and
encrypted), an intake every transport shares, and a job that reads each
mail once under a lease: filters without AI, machine-readable data always,
one AI request with the person's trips, the server's checks G1 to G9, then
one transaction per mail that adds what passed, keeps the rest in the inbox
with a reason, and notifies the forwarder and co-travellers. Undo within
7 days, manual apply, filing as documents, a prefilled form and a nightly
retention complete it; on a managed install the operator's switch decides.
Five tables carry the mail import: forwarding addresses (token hash plus the
encrypted token), a person's switches, the readers' cursor and lease, every
mail as it arrived, and each booking read from it with its checks and fate.
Repositories read the trips a booking may go to by column, so the import
weighs a booking without reaching into five other domains.
CreateReservationData and insertReservation take an optional external_source,
external_id and sync_enabled, so an importer can record where a booking came from;
without them every write keeps the column defaults. Client bodies on REST and the
plugin RPC have the origin stripped, so no caller can pose as an import or a link.
getReservationWithJoins now takes the trip and reads nothing outside it.
A new flight-live domain polls the open feeds adsb.lol and adsb.fi (no key,
no cost; adsb.fi only on self-hosted installs, its terms are non-commercial)
for every booked flight leg in its watch window, derives phase, take-off,
landing, delay and an arrival estimate, and pushes changes as flight:live.
GET /api/trips/:tripId/flights/live, the MCP tool get_trip_flight_status,
notifications for take-off, landing and delays, and the admin switch
flight_live_enabled (FLIGHT_LIVE_ENABLED wins) come with it.
The place and endpoint reads behind the passport tile now carry the last day
of each started trip, and getTravelStats orders the countries by their latest
visit, with countries only marked by hand after them. The country set is
unchanged; repository, parity and service tests pin the order.
A read tool that answers one list a page at a time (100 by default, up to
500) as compact one-line JSON rows, filtered by status, label or "no label",
with the list's labels, the matching total and a cursor. The cursor is the
last place's position (sort_order, created_at, id), so labelling a page
while paging through unlabelled places skips none. getCollection orders by
the same keys, id last, so both tools agree on ties.
Travellers who track what a trip costs but never split it saw an
Unfinished warning on every expense and a settle-up they had no use for.
A trip now has a costs mode (appended migration, trips.costs_mode, split
by default); totals leaves payers, shares and settling up out of Costs on
desktop and phone through the same gate as a trip of one, and saves new
expenses without a payer. PUT /api/trips/:id and update_trip take it on
budget_edit. The split chips move into their own component.
- The SSO unlink answers with a response contract (adminSsoUnlinkResponseSchema).
- It reuses linkOidcIdentity with nulls instead of a new repository method,
so UsersRepository keeps its method count and size.
- The client calls the route through apiClient, so api/client.ts does not
grow, and its confirm dialog takes no async callback.
- The pager's preview step and pagesFor keep their complexity.
A sign-in with another subject of the same provider is refused, so an
account whose provider re-issued the subject (a migrated or rebuilt IdP, a
changed sub format) was locked out: editing the email, as the wiki advised,
never touched the stored link. The edit dialog now offers "Remove SSO link"
for other accounts while SSO is set up (DELETE /api/admin/users/:id/sso,
audit admin.user_sso_unlink). The next SSO sign-in links again under the
rules of a first one, so it still needs a verified, not self-set email.
The log line names the remedy, and the wiki tells the two refusals apart.
Fixes#3065
geoBoundaries has no county layer for Ireland: its ADM1 is the four
provinces and its ADM2 the 166 local electoral areas, so the Atlas showed
Ireland as four provinces next to a UK of counties.
The Atlas builder can now take a country's regions from another source.
Ireland's come from Tailte Éireann's 26 counties (CC BY 4.0), with Dublin,
Cork, Galway and Limerick each one county, under their ISO 3166-2:IE
codes. Those outlines come at full detail, so the builder simplifies them
itself and keeps the borders two counties share identical on both sides,
so no point between them falls into neither. ATLAS_ONLY rebuilds just the
named countries without downloading the whole world.
A migration clears Ireland's cached place regions so they are worked out
again as counties, moves visited rows that name a county onto its code,
and hides each county of a province a user had hidden. Provinces a user
marked as visited are kept as they are.
Fixes#2922
Deleting a message set the flag and kept the text in the row and in every backup; only the client hid it. The delete now blanks the text with the flag, and a migration clears what earlier deletes left. The row stays for the placeholder.
The edit form fell back to the linked stay's times, the cards read only the
booking's metadata, so a hotel showed 15:00 and 11:00 when edited and no times
in the list. The reservation list now carries the stay's check-in window and
check-out, and the desktop facts and the phone card fall back to them. On the
phone they come first, so a price or a seat cannot push them off the card.
"Add to Apple Wallet" on a booking in the iOS app turns it into a TREK pass:
a boarding pass from and to for flights, trains, buses and boats, check-in and
check-out for a stay, pickup and drop-off for a car, the time for anything
else, with the confirmation code as a QR code. Wallet brings it up on the lock
screen around its time and place: times carry the offset of the endpoint's zone
(looked up from the coordinates where the endpoint has none), stays and
restaurants their place's coordinates. The labels come from the booking forms'
own strings, so nothing new to translate on the pass.
Only liketrek holds the Pass Type ID certificate, so the server sends the pass's
texts to the TREK services' signer (TREK_WALLET_PASS_URL, push.liketrek.com)
once per tap; it stores nothing, and TREK_SERVICES=off turns it off with the
other services. Texts are cleaned of links and shortened here, the answer must
be a .pkpass within 512 KB, and a user gets 20 passes a minute.
- New wallet-pass domain: GET /api/trips/:tripId/reservations/:id/wallet-pass
behind the trip guard, the booking checked to belong to the trip; trip title
and place come from the reservations domain (findPassContext)
- MWalletButton in the transport and booking sheets, only where the app binary
has the wallet feature; undated bookings and failures are told apart
- TREK_WALLET_PASS_URL in .env.example, compose, Helm and the wiki
Moves the Leaflet map's event bridges (click, context menu, viewport, POI drop
and the longitude-frame follower) out of MapView into leafletMapBridges, and
shares one POI drop target between the Leaflet and GL maps, so both map files
shrink instead of growing. The repair migration does its arithmetic in one
UPDATE per table, and the new tests drop their non-null assertions and loops.
A right-click, click or drag on a repeated world copy reported the longitude it
has on that copy (274.9 for -85.1). The place was saved there, the geocoder
found nothing, and a route to a place on the real world ran across the whole
map (#2976). Longitudes are now brought back onto [-180, 180] where the map
hands them out (Leaflet and GL), where they are stored (places, booking
endpoints, collection places, so REST, MCP, plugin RPC and importers alike)
and before reverse geocoding. A migration moves the ones already stored.
Leaflet draws a layer at its literal longitude only, so a trip from Los
Angeles to Auckland had its pins at opposite ends of the world while the
flight ran off the edge to the next copy (#2975). A trip that crosses the
antimeridian is now drawn in one longitude frame, seamed in the widest
stretch it leaves empty, so pins, routes, flights and every overlay lie on
one copy and the map opens across the Pacific. Every other trip has no frame
and draws exactly as before. The GL day fit also takes the short way.
A new install now starts with Journey and Collections switched on, next to
the trip addons (all but Road trip), Atlas, Vacay and the Naver import that
already were. Existing installs keep their addons as they are.
Closes#2975Closes#2976
An installation now asks for its TREK services key a few minutes after
it starts, and the daily check registers a missing key as well as
renewing a due one. Until now nothing happened before someone searched
for a place or opened the admin settings, so "Share place details"
stayed unavailable on a fresh or updated instance.
When an admin agrees to share place details, what users typed into
places since TREK 5.1.0 goes out too, not only what they type from then
on. The origin of every field is recorded since 5.1.0 whatever the
decision, so a catch-up walks those places page by page before each
send run: only values that are still the ones a user typed, only for
fields switched on, only for places the service can key, and only
while the queue holds fewer than 200 values. It goes on where it
stopped after a restart, never replaces a newer value a save queued,
and ends on switching off, declining or withdrawing everything. A field
switched on later catches up as well, under an agreement given with the
new prompt sentence that says so; an agreement from 5.1.0 does not.
Queuing moves into PlaceContribQueue so a save and the catch-up take
the same path.
- A restore (admin panel or RESTORE_FROM_BACKUP) now leaves
data/.trek-reclaim-reset before it swaps the database in. Boot reconcile
resets the reclaim state when it finds the file, whatever install id the
database names, and removes it afterwards; a successful reopen removes it
too. A restart after a failed reopen, or a boot restore of this
installation's own backup, no longer keeps the old switch, marks and
confirmations. The id mismatch reset stays.
- The release job skips a tick (install_mismatch) while the database names
another installation or a reset is still pending.
- An allowed owner verdict is cached for 15 minutes, a refusal for 26 hours.
The local driver reads a list prefix as a folder, so the category root is
still listed whole, at most once per category per window.
- Each tick reads every allowed category's due rows separately, an equal
share of the 200 each, so a category whose deletes keep failing cannot
starve the others.
- Rows dropped after 30 days overdue and releases refused by a full queue
are logged at warn with their counts.
A delete or a replace no longer deletes inside the request. The owner's call
queues the keys in storage_release_queue, due after RELEASE_DELAY_MS (26 h:
the assistant's 24 h approval window plus a margin), and StorageReleaseJob
takes the due rows every ten minutes, at most 200 a tick and 50 keys a
question, asks every reference source again and deletes what none names.
It refuses what the nightly pass refuses (StorageReclaimGuardService: the
ledger, the required sources, a shared location, another installation's
owner marker, cached per category), and waits while a restore or a backup
runs or another storage job holds the slot. An assistant undo or a late
write that names the old value again keeps its object.
The install id lives in data/.trek-install-id, outside any backup and never
written by a restore. At boot and after every reopen the database is
reconciled with it: a database naming another installation (a boot restore,
a restart after a failed restore, a copied travel.db) loses its switch,
confirmations, reports, marks and queue before anything reads them.
A confirmation now stores the count the admin was shown and when; it covers
up to one and a half times that count (at least 100 more) for 14 days, is
used up once the share is back under a fifth, and a count the last report
no longer holds is refused as reclaim_confirm_stale. The admin state folds
a live confirmation in, so the prompt goes at once.
A pass reads each source once for every category and drops each category's
keys once judged; right before a batch it reads the category's sources
again, whole and in memory, instead of a LIKE per key. A narrowed read holds
at most 50 LIKE terms. A pass held off by a backup reports backup_running,
and a category the ledger never reclaims reports not_reclaimed even after a
stop.
The 20 % brake skipped a category and wiped its marks every night once a
fifth of it looked unreferenced. Any user could push a small install over
that line, a backlog from #2924 started braked, and a second install on the
same bucket below the line was not caught at all. It is gone:
- Each pass leaves a trek-owner-<install id> marker at the root of every
category it judges. A category holding another install's marker is
skipped as foreign_install, one where the marker cannot be written as
owner_unknown. Backups leave the markers out.
- The install id lives in app_settings outside the storage.* prefix. The
switch is stored as that id, so a database from another install reads as
off, and every reopen (a restore) writes this install's id back, turns
deletion off and drops the marks, confirmations and reports.
- More than a fifth unreferenced still marks and reports, but deletes only
after an admin confirmed the count shown (POST .../reclaim/confirm). A
larger count later waits again.
A skip no longer touches the marks, so a failed listing or lookup does not
restart the week. Check now runs in the background and answers 202; the
admin state says running and keeps the last deleting report apart, so a
report-only check never resets what was freed. A pass that deletes writes a
system audit entry, a pass that stops or fails keeps its partial counts and
marks the rest interrupted, a backend error stops deleting in its category,
and deleting stops for the rest of the pass once the switch is off or a
backup is reading the uploads. The usage scan waits for a pass still
running.
Sources read each category once per pass and answer the pre-delete
re-check with a query narrowed to the keys' plain runs. Bare names keep
their raw spelling (a '#', '?' or trailing blank is part of an upload's
name), and local roots are resolved through their symlinks before the
overlap check. A pass lists only a category's root.
storage_reclaim_marks records since when the reclaim pass has seen an object
unreferenced, keyed by category and key. A mark is never moved forward, so
its age is how long the object has been unreferenced without a break; it goes
once the object is named again or gone. Owned by the storage domain.