The packing row tests read the TREK tooltip and the accessible name instead
of the title the badges no longer carry, the journey delete suite takes its
service from the module over a snapshot db and seeds through the row
factories, the storage notifier suite builds its subject once, the POI
layer follows the view only while it has POIs, the collections repository
reads the list's places through its own query, and the baselines the moved
code left smaller come down with it.
An assistant writing expenses through the MCP was told "e.g. Accommodation, Food,
Transport" and stored exactly that, so "Food" and "food" sat side by side in the
MCP totals while the Costs tab already read both as one. A typed category in any
case, or a common name like Hotel or Flight, is now stored as its fixed key on
every path (REST, MCP, linked booking costs), the totals the MCP reads group by
that key, and the tools name the real keys. The client reads categories through
the same shared mapping. Any other text is still kept as given.
The journal tools said that a private entry stays off the share link. It never
did: the share link shows every entry that is not a draft, whatever its
visibility, and the web editor has no visibility at all. The tools now say so
and point to is_draft, which is what keeps an entry to the contributors.
The wiki explains why the location button needs HTTPS and how to run TREK behind
an HTTPS tunnel while still using it over plain HTTP at home, and that Android's
photo picker drops a photo's location while Immich and Synology keep it.
Leaving every replaced cover, place image and deleted journey photo to the
nightly pass kept them publicly fetchable by URL for a week at least, and
for good while deletion was off. Owners now hand the values a delete or a
replace left behind to StorageReleaseService once their write committed.
It asks every registered reference source, with the same parser as the
nightly pass, and deletes at once what no row names any more: an object a
trip copy, a journey or a saved place still shows stays, which the old
inline deletes got wrong.
The release holds the pass's guards (reclaimable categories at their root
only, never an owner marker, never while a restore runs, never with a
required source missing or an overlapping location), never fails the
request and logs at debug. It runs for trip covers (update, upload, trip
delete), collection covers and saved-place images, place images (update,
bulk update, delete), journey covers and journey or gallery photos
(their thumbnail goes with a file that went), avatars (change, delete,
account deletion), trip files (permanent delete, empty trash) and collab
note and chat attachments. A trip file row now goes before its bytes, so a
failing backend no longer fails the delete. The objects of a deleted trip's
places and files are left to the nightly pass: their rows are other
domains'.
PlacesService drops the collection places repository nothing read any
more. Size, warning and boundary baselines follow the shrunk files.
Replacing a trip or list cover, changing or deleting a place or saved-place
image and deleting a journey photo removed the bytes at once, without asking
whether a trip copy, a journey, a saved place or another photo row still
named them (#2924). Those paths now leave the object in place and the reclaim
job removes it once no row names it. A journey photo the user deletes takes
its local photo row with it when nothing else links it, so its bytes become
reclaimable instead of being kept by a row nobody sees.
A place image copied from a trip file is still dropped at once when the
update that would name it fails: that copy was never anyone else's.
Size and warning baselines follow the shrunk files.
The Atlas server no longer keeps the admin-1 regions of every country in memory. Each country's regions are now a pre-gzipped file of their own, sent to the map as they are.
**Why:** `GET /addons/atlas/regions/geo?countries=…` built `admin1Store` (every country's region text, kept until restart), then joined the requested countries, `JSON.parse`d them, and Nest stringified them again. On the 512 MB – 1 GB boxes TREK runs on, the first z≥5 view could OOM the server (same shape as #1576).
**What changes:**
- `server/assets/atlas/admin1/<ISO_A2>.geojson.gz`: 198 files, 14.85 MB, the same 3,447 features as `admin1.geojson.gz`.
- New `GET /addons/atlas/regions/geo/:code` with a Zod contract (`regionGeoParamsSchema` in `@trek/shared`: two letters, any case). It sends the country's file as it is, with `Content-Encoding: gzip` and the current 1-day `Cache-Control`, as `countries/geo` does. A country without regions gets an empty FeatureCollection. Anything that is not two letters gets a 400.
- `useAtlas.ts` fetches one URL per uncached country and still rebuilds the layer once. Each country has a stable URL, so a country evicted by `REGION_CACHE_MAX` comes back from the browser cache.
- `getRegionFeatures` reads only the country it needs. `admin1Store` is removed.
- `?countries=` keeps working for cached PWA clients. It joins the per-country files as bytes, without a parse.
- The migrations are untouched. `admin1.geojson.gz` still ships, only for `Migration20200101021500` and `Migration20200101031500`. This costs ~15 MB of disk in the image and no RAM. `build-atlas-geo.mjs` writes both, and ATLAS-BUNDLE-008 keeps them equal.
**Memory** (real `atlas-geo` module, first z≥5 view over Europe = 49 country codes, JS heap + Buffers):
| | peak | kept after |
|---|---|---|
| before, first view (builds the store) | +372 MB | +61 MB |
| before, later views | +115 MB | 0 |
| after, `/regions/geo/:code` | +4.7 MB | 0 |
| after, legacy `?countries=` | +59 MB | 0 |
**Tests:**
- New: `atlas-region-files.test.ts` (moved from `atlas.service.test.ts`, plus join and dedupe cases), the `regionGeoParamsSchema` spec, ATLAS-BUNDLE-007 (the file format the byte join depends on), ATLAS-BUNDLE-008 (bundle and per-country files are equal), ATLAS-016/017/018 (the new route over HTTP, including the 400).
- The `getVisitedRegions` tests (ATLAS-UNIT-019..024) used to pass only when `getRegionGeo`'s `beforeAll` had built the store first. They now also pass on their own.
- The shared, server and client gates pass. Baselines go down only where this change removed code (`no-console`, `no-explicit-any`, `no-swallowed-catch`, file sizes).
**Trade-offs:**
- A first zoom over Europe sends ~49 small parallel requests instead of 1. Over HTTP/1.1 the browser queues them 6 at a time. The total bytes are the same (4.4 MB gz).
- The per-country URLs are unversioned and cached for a day, like `countries/geo`. After an upgrade that changes a country's regions, a browser can show the old shapes for up to 24 h.
Approved on Discord in #github-pr: <paste the thread link here>
Related: #1576, #2559
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] I have read the [Contributing Guidelines](https://github.com/liketrek/TREK/wiki/Contributing)
- [ ] My branch is [up to date with `dev`](https://github.com/liketrek/TREK/wiki/Development-environment#3-keep-your-fork-up-to-date)
- [x] This PR targets the `dev` branch, not `main` *(wiki-only PRs are exempt)*
- [x] I have tested my changes locally
- [x] I have added/updated tests that prove my fix is effective or that my feature works
- [x] I have updated documentation if needed
TREK 5.0 left most registry plugins blocked because their authors still declare
"<5.0.0", and the only way past that was TREK_PLUGINS_IGNORE_TREK_RANGE for every
plugin at once. An admin can now install or switch on one such plugin after a
warning; the consent is kept on its row for that plugin version and TREK major,
so an update or a new major asks again. The environment switch works as before.
Discover marks a plugin not confirmed for this TREK on its card, and Install opens
the warning first. Admin > Plugins reads cleaner: one row of alike facts per plugin
with each state that needs attention on its own line, type, status and order behind
one Filter button, and the containment notes in a dialog behind the shield in the
header. The row, the filter menu and the dialog moved into their own files.
Each member's amount was converted and rounded to a cent on its own, so four 34,000 ISK quarters of a 136,000 ISK bill came to 997.52 EUR against the 997.51 the payer was credited. The balances ended a few cents off zero and settle-up had no payment for them. A split whose amounts match the payers in the bill's own currency is now converted as one set by largest remainder, in the settlement and in the per-person summary alike.
update_journey_entry now answers previous, the entry's earlier values of
exactly the fields it names (a suggestion written up goes back to being
one), and reorder_journey_entries answers previous_order, the ids in the
order they stood in before. Both come from the service the REST routes
use, which now builds an entry's patch in one pure helper; the routes'
bodies are unchanged. The tools' inline zod moves into strict schemas in
@trek/shared.
The MCP tool lacked show_trip_tracks and cover_gradient, which PATCH
/api/journeys/:id has always taken. Both now read the input through one
helper, and the tool goes through a small settings service that writes the
same patch in a transaction with the stored values it replaces, answered
as previous, so a change can be written back. The subtitle and the cover
gradient take null to clear them. The tool's inline zod moves into strict
schemas in @trek/shared; the REST route and its body are unchanged.
Opening the year sent vacay:settings from inside the day's transaction, so
a client could reload the years before they were there. The rows go in
with the day, and the broadcast follows the commit. The helpers file is
formatted the way the server's Prettier wants it.
The holiday feeds' limits and patterns and the colour palette move to
vacay.helpers.ts, which holds the vacay domain's pure half already. That
keeps the service under its line budget after the missing-year fix.
Every help screen checked against the v5 redesign. Tours is documented for
the first time, as a screen of its own with twelve screens under it, and
notifications get help of their own. All pictures are taken again on a
realistic demo trip, with short loops where a still cannot show the motion.
The English is plain and every language is written naturally rather than
word for word.
The switcher shows two levels at most, a screen's overview lists its
sub-screens as cards above its guides, each wiki page is linked once, and
the help sidebar is wider.
Fixes found along the way: Save to Collection keeps a tour's route, a track
dropped on the places column opens the Tours import, tour Share on desktop,
the unsaved tour draft lives on the account, a route click no longer adds
an end point, Trip places leaves the trip's tours out, Escape closes a
stage's day menu, confirms before destructive deletes, DWD warnings sharing
one id kept apart, a booking's date moves with it, and smaller fixes across
notifications, bookings, the day plan, stays, Atlas, Collections, Vacay and
the admin screens.
GET /api/journeys/:id/map-layers answers the planned places of the linked
trips (one per place, with every day the plan puts it on and the entry
written about it), the trips' GPX routes while the track switch is on, and
the located stays entries were taken over from. It goes through
journey_trips, so a trip nobody wrote about yet shows its plan too. Every
source is read on its own: one that fails says unavailable and the others
still draw. A position that is not one is left out, never moved.
GET /api/public/journey/:token/map-layers cuts the same on the server by the
link: nothing while the map is not shared, the plan and its routes only with
share_plan, the recorded days only with share_recordings and only beside a
published entry, stays only of published entries. Migration 046190 adds both
flags to journey_share_tokens, off by default, so a journey that is public
already shows what it showed before. The share link REST route and the MCP
share tool take them alike; get_journey_map_layers reads the layers over MCP
on the same access, a route by its size only.
- Paddle as a plannable tour type: one boat (kayak, canoe, SUP, other), river or
lake, a pace that a boat change never overwrites; time is distance over pace
- Hand-drawn legs beside routed ones, drawn dotted and labelled as not routed,
kept through save, reopen and edits elsewhere
- Reverse a tour and close it back to its start, each one undo step
- GPX import shows a preview first (line, measures, duplicates, type) and only
the confirmation stores the tour
- Base maps and overlays per tour type in the planner: hike keeps topo and the
via ferrata layer, paddle gets the OpenSeaMap seamark overlay
- P2C-02: tours record their origin (planned, GPX with file name, wanderer)
instead of reading it off their shape; the detail answers a provenance
contract and a trip copy keeps a tour's import record
- MF-03: a wanderer snapshot's source can be checked on purpose, read only,
with a confirmed reimport as a new snapshot
- PR1-18: every way of putting a tour on a day shares one set of rules
- PR1-07/08: each person can hide tours from the trip map; the list shows
each tour's line colour and the picked tour stands out on both maps
An SSO login that matched no account by its subject took the account with the
same email address and linked the provider identity to it. That address proves
nothing when the account typed it in itself: anyone could sign up with, or
switch their profile to, a colleague's address and receive the colleague's
first SSO login. The provider switch branch (#2110) relinked a row on any email
match without even asking for email_verified, and also for another subject of
the same provider.
users.email_self_asserted remembers an address the account set itself: signup
(not the first account) and an email change in the profile settings set it, an
admin entering a new address clears it, and accounts the provider creates start
without it. Existing accounts keep linking, except those whose address moved
away from the registered one without an admin editing it. A login by email now
needs a verified address and an unmarked account, and is refused for another
subject of the same issuer, with email_link_blocked and a translated message on
the login screen. An email change in the profile settings writes a
user.email_change audit row.
Refs #2110
Moving a trip's start shifted the owner's vacay entries one row at a time,
checking each target against rows that were about to move themselves. A
block shifted by less than its own length kept its first days behind and
pushed the rest onto the weekend, even with weekends blocked.
The shift now decides every day before writing: a day whose target is a
blocked weekend or a whole company holiday is dropped, as is one whose
target already holds leave that is not moving; a half company holiday
leaves half a day. The rest move in the direction of travel inside one
transaction. The plan is the one the owner is active in, so a member of a
fused plan gets their days moved too, and a trip edit no longer creates a
vacay plan for an owner who has none. Open vacay views are notified after
the trip update commits.
The pure vacay helpers moved to vacay.helpers.ts and the trip window tests
to their own suite, so both files stay inside their size budgets.
Refs #2604
Tightens the two comments the rawBodyBase64 field and the type parity note
added to runtime/plugin-sdk.ts, lowers its lint:size entry, and points the
shared guide at the client files it names by their full path.
toMinor, sumMinor, splitEqualShares and currencyDecimals move to
shared/src/money. The budget service and its MCP tools call them instead of
private copies, and the share table both sides were pinned to becomes the
spec of the shared function. Stored values and every computed figure stay
the same.
Every write tool used to open with its own demo check, 196 copies across 30
tool files, most of them injecting AuthService for nothing else. The
registry now takes a toolGate (built from the container) that runs before
each registered tool handler, and TREK's gate refuses a demo account on
every tool not annotated read-only with the same canned result. The
per-tool checks, the AuthService injections, AuthService.isDemoUser and the
AuthModule imports that only fed them are gone; ESLint refuses a demo check
in a tool file, and a registry-wide test calls every write tool as the
demo account.
UserImmichRepository, UserSynologyRepository and UserAirtrailRepository hold the eighteen connection statements UsersRepository carried, on the same users table and with their statements unchanged, provided through a small module as factories over the EntityManager. ImmichService, SynologyService, AirtrailService and JourneyService inject them instead of the account repository. UserRow drops the dead immich_access_token column (the table keeps it), and a shape test pins the account repository's size.
app-settings.registry.ts lists every app_settings key the server reads by name with its shape, default, environment override and flags. The admin form's keys, its masked and encrypted handling and the public config's reads derive from it, isOidcConfigured() replaces the three issuer/client-id checks in AuthService, and OidcService resolves its values through the same rule while still requiring the client secret. Every literal getValue() read goes through readAppSetting()/resolveAppSetting(), and ESLint refuses a new one.
The ICS export reads the stops and notes of all dated days in two statements instead of two per day, the journey's full read and entry list take the source trip titles in one join instead of one read per entry, and reordering a day's stops or sorting it by time writes every position in one CASE update. Results are unchanged: each day keeps its order (the id settles a tie), a missing trip still names null, and an id listed twice keeps its last position.
The sessions domain owns user_sessions, yet AuthService injected
UserSessionsRepository next to SessionsService and AuthModule registered
the entity in its own forFeature, only so verifyJwtToken could hand the
repository to verifyJwtAndLoadUser.
verifyJwtAndLoadUser now takes a SessionLookup, the two repository
methods the session check uses (findActive, touchLastSeen).
SessionsService answers both, so AuthService passes itself through the
sessions domain's API and drops the repository and the forFeature entry.
The guards keep handing in the repository from their own entity
manager, which satisfies the same type. auth.service.ts lost lines, so
its size baseline goes down with it.
confirm() created the venue's place row in its own transaction and
broadcast place:created before the booking and its cost were written in
a second one. When that second write failed, the item was skipped but
the trip kept a place no booking pointed at, and clients had already
been told about it.
Per item, the venue and endpoint geocoding, the day lookups and the
cost's frozen rate still run first, outside any transaction. The place
row, the booking and its linked cost then go into one transaction
through the new ReservationsService.createWithCostInTx, the core that
createWithCost itself now wraps. place:created, reservation:created,
accommodation:created, the stay mirror and the cost events are sent
only after the commit. Each item keeps its own transaction, so a
failing item still leaves the others imported; confirm() is tagged
@txIndependent for that and leaves the lint:tx baseline.
TripsService.update rebased the budget onto the new currency in its own
transaction and only then wrote the trip row in a second one. When the
day grid rebuild, the booking resync or the leave shift threw after the
rebase had committed, budget items and settlements stayed pinned and
re-frozen against a currency the trip never took (#1543).
rebaseTripCurrency is split into prepareCurrencyRebase, which fetches
the rates before any transaction opens, and applyCurrencyRebase, which
reads the outgoing currency and writes the rebase. update() fetches
first and hands the plan to updateTrip, which applies it inside the
transaction that writes the row, the leave shift and the day grid.
updateTrip called without a plan (the plugin RPC host) still never
rebases, and rebaseTripCurrency keeps working as before.
The updateTrip docstring no longer claims the row is written outside
the day regeneration transaction. TripsService.update leaves the lint:tx
baseline, and the size baselines shrink with the files.
createWithCost and updateWithCost run the booking and its cost in one
transaction, but the cost sync still caught and logged its own errors
inside it. A price that could not be written therefore committed the
booking edit and left the linked expense on its old figure; only the
category re-file path actually rolled back.
The sync no longer swallows a failing cost write. Both WithCost methods
are its only callers, so the error now reaches their transaction and
the booking and its cost commit or fail together. A booking import
item whose cost cannot be written is skipped like any other failing
item instead of landing without its cost.
syncBudgetOnUpdate created the expense and then set its reservation_id in a
second statement, inside a try that logs and swallows the failure. Running
inside updateWithCost, a failed back-link committed an expense that was not
linked to its booking. The expense is now created through
linkBudgetItemToReservation, which writes the link on the insert, the same
way syncBudgetOnCreate does.
A sweep of the services for write sequences that committed step by step,
each one leaving half a change behind when the next statement failed:
- accommodations: the linked bookings' times moved after the stay's own
transaction had committed
- airports: the boot backfill wrote a flight's two endpoints one at a
time, and a flight left with only the first was never picked up again;
each flight is its own transaction now
- auth: the admin settings form and a password reset (retire the old
link, issue the new one) each land whole
- oauth: a refresh revoked the old pair before issuing the new one, so a
failure in between logged the client out; rotating a client secret and
revoking its tokens go together too
- oidc: the identity link, the role from the claims and the avatar of a
returning user are one write
- addons, notification preferences: a settings save lands whole
- places: create with its tags, update, and the GPX/KML imports with
their track colours
- trips: the trip row, the shifted leave entries and the rebuilt day grid
- vacay: a first visit provisions the plan in one transaction with a
second look against a concurrent first visit; updatePlan fetches the
public holidays between two transactions instead of inside its writes
- memories: a photo and its trip link, and the deletes with the orphan
sweep behind them
AddonsService, AirportsService and OauthService take the UnitOfWork for
this. Network I/O stays outside every transaction. Comments that only
narrated the migration plan make room under the lint:size entries, which
drop with them.
Uninstall deleted the plugin row and then up to fourteen tables of rows
about it one by one. A failure halfway left grants, scheduled tasks or
per-user settings behind for a later plugin that reuses the id to
inherit. The rows now go in one transaction, and the data directory is
removed after the commit instead of before the rows that describe it.
A registry install records the discovered row, its provenance, the
pinned author key and the cleared update block together, so a failure
never leaves a registered plugin without its pinned key. Starting an
OAuth connect drops the stale state and stores the fresh one together,
and a disconnect drops tokens and state together; PluginOAuthService
takes the UnitOfWork for that. The scheduler's per-task claims stay
separate on purpose and say so for the transaction gate.
createJourney inserted the journey, then its owner row, then linked each
trip with its skeleton entries and set the cover, every step committed on
its own: a failure halfway left a journey nobody owned. It now runs as
one transaction and broadcasts the linked trips after the commit.
addTripToJourney links inside a transaction the same way.
The photo paths wrapped only the gallery insert and left the entry link
and the skeleton promotion outside; the photo row, the gallery row, the
link and the promotion are one write now. removeTripFromJourney,
updateEntry, updatePhoto, setPhotoProvider, the photo deletes and the
place hooks write in one transaction each.
createEntry and uploadGalleryPhotos read MAX(sort_order) inside the
transaction that inserts, so two entries or uploads at once take one
position each. reconcileTripSkeletons and the photo capture helpers are
tagged as deliberately split for the transaction gate that follows.
sql-functions.ts sat at 1007 lines, held there by the size baseline, so it
had no room for the Postgres branch every helper is meant to grow. The
Kysely-expression twins are a separate family already (a Kysely Expression
and a MikroORM RawQueryFragment do not convert into each other), so they
move to src/db/dialect/kysely-functions.ts, and the platform check plus the
fail-closed unsupported() move to platform.ts, which both files share.
No helper changes behaviour or spelling. lint:dialect now skips the whole
src/db/dialect/ directory instead of the one file, and the size baseline
drops its sql-functions.ts entry.
The repositories typed their Kysely statements against 159 table interfaces
written by hand, one or more per repository. Nothing compared them with the
schema: a migration that renamed a column or changed its type left them
stale while tsc stayed green, and several had already drifted
(reservations.accommodation_id is TEXT, not INTEGER).
scripts/generate-db-types.ts migrates a throwaway database with the real
migrations, reads every table back and writes one interface per table into
src/db/kysely/ plus the DB map in db.ts. Every hand-written interface is now
Pick<DB, ...> of that type, and npm run check:db-types fails when the
committed files no longer match the migrations.
A column an insert may leave out (rowid, DEFAULT, nullable) is
InsertOptional<T>, declared as ColumnType<T, T | void, T>. The server
compiles without strictNullChecks, where Kysely's own Generated<T> still
demands the column; that is why the old code kept separate insert-only
interfaces. The ids callers pass as number | string are cast to the column
type at the bind, so every statement binds the same value as before.
Each gate came with its npm script but ran nowhere. The server job now
runs lint:size, lint:boundaries and shared's contracts:open after the
shared build; the client job runs lint:warnings in place of lint:check
(it fails on errors the same way and also holds the warning count per
rule), then lint:format, lint:layers, lint:offline, lint:skips and
lint:i18n-keys. All of them are steps of jobs ci-ok already needs.
Both size baselines are taken on the merged branch. Four server files
grew while the other changes landed (the claimed reminder, the email
uniqueness error and the generator rule for an expression index) and
four client files by two or three lines with the plural forms; one
client file shrank and is lowered.
The client has had lint:size since #2612, the server had nothing, and
the server is where the biggest files live: maps.service.ts holds 3231
lines and grew by 200 in a month, and nine more services are past 1100.
scripts/size-lint.mjs carries the client's semantics over to src/ and
scripts/: a file may hold 1000 lines, the 23 that already hold more are
pinned at today's length in scripts/size-baseline.json, and --update
only ever lowers an entry.
A line past prettier's 120 columns counts once per 120 columns it spans,
so joining lines does not make room under the limit. The check fails
closed: a missing or malformed baseline or a missing src/ stops it
instead of passing with nothing checked.
Maurice