Replacing a cover or an avatar and deleting a note file now hand the old
bytes to the release queue instead of deleting them in the request; the
integration tests check the queue row and that the bytes are still
there. Release mocks answer with a count, the reclaim e2e sends its
refused bodies one after another, and a stale-count reload that fails is
told to the admin instead of swallowed.
A delete or a replace no longer deletes inside the request. The owner's call
queues the keys in storage_release_queue, due after RELEASE_DELAY_MS (26 h:
the assistant's 24 h approval window plus a margin), and StorageReleaseJob
takes the due rows every ten minutes, at most 200 a tick and 50 keys a
question, asks every reference source again and deletes what none names.
It refuses what the nightly pass refuses (StorageReclaimGuardService: the
ledger, the required sources, a shared location, another installation's
owner marker, cached per category), and waits while a restore or a backup
runs or another storage job holds the slot. An assistant undo or a late
write that names the old value again keeps its object.
The install id lives in data/.trek-install-id, outside any backup and never
written by a restore. At boot and after every reopen the database is
reconciled with it: a database naming another installation (a boot restore,
a restart after a failed restore, a copied travel.db) loses its switch,
confirmations, reports, marks and queue before anything reads them.
A confirmation now stores the count the admin was shown and when; it covers
up to one and a half times that count (at least 100 more) for 14 days, is
used up once the share is back under a fifth, and a count the last report
no longer holds is refused as reclaim_confirm_stale. The admin state folds
a live confirmation in, so the prompt goes at once.
A pass reads each source once for every category and drops each category's
keys once judged; right before a batch it reads the category's sources
again, whole and in memory, instead of a LIKE per key. A narrowed read holds
at most 50 LIKE terms. A pass held off by a backup reports backup_running,
and a category the ledger never reclaims reports not_reclaimed even after a
stop.
Leaving every replaced cover, place image and deleted journey photo to the
nightly pass kept them publicly fetchable by URL for a week at least, and
for good while deletion was off. Owners now hand the values a delete or a
replace left behind to StorageReleaseService once their write committed.
It asks every registered reference source, with the same parser as the
nightly pass, and deletes at once what no row names any more: an object a
trip copy, a journey or a saved place still shows stays, which the old
inline deletes got wrong.
The release holds the pass's guards (reclaimable categories at their root
only, never an owner marker, never while a restore runs, never with a
required source missing or an overlapping location), never fails the
request and logs at debug. It runs for trip covers (update, upload, trip
delete), collection covers and saved-place images, place images (update,
bulk update, delete), journey covers and journey or gallery photos
(their thumbnail goes with a file that went), avatars (change, delete,
account deletion), trip files (permanent delete, empty trash) and collab
note and chat attachments. A trip file row now goes before its bytes, so a
failing backend no longer fails the delete. The objects of a deleted trip's
places and files are left to the nightly pass: their rows are other
domains'.
PlacesService drops the collection places repository nothing read any
more. Size, warning and boundary baselines follow the shrunk files.
The reclaim e2e and the source-registration check mocked src/db/database,
which lint:test-mocks holds at zero for new files; both now boot buildApp
against the worker's snapshot db. The key parser drops its non-null
assertions (the indexed reads are typed string already), and the e2e sends
its refused bodies at once instead of awaiting inside a loop, so the eslint
ratchet stays where it was.
Replacing a trip or list cover, changing or deleting a place or saved-place
image and deleting a journey photo removed the bytes at once, without asking
whether a trip copy, a journey, a saved place or another photo row still
named them (#2924). Those paths now leave the object in place and the reclaim
job removes it once no row names it. A journey photo the user deletes takes
its local photo row with it when nothing else links it, so its bytes become
reclaimable instead of being kept by a row nobody sees.
A place image copied from a trip file is still dropped at once when the
update that would name it fails: that copy was never anyone else's.
Size and warning baselines follow the shrunk files.
Domains whose rows name stored objects register a reference source with
StorageReclaimRegistryService; storage never imports them. A ledger in
storage decides per category whether it is reclaimed at all and which
sources must have answered first, and a category whose resolved location
overlaps another one is left alone.
A nightly pass at 04:05 lists the root level of files, journey, covers,
avatars and places, skips objects younger than a day or without a
modification time, asks every source in pages of 500 and marks the rest.
Only with deletion switched on, and only for marks a week old, it asks every
source about the single key again and deletes through StorageService. A
category is skipped and its marks start over when a source is missing or
fails, the listing fails, or more than a fifth of its objects look
unreferenced. A pass shares the one-job slot with backfills and migrations,
never runs on a demo instance and stops when the database is reopened.
Admins switch deletion and run a pass on demand under
/api/admin/storage/reclaim; both are audited, and GET /api/admin/storage
carries the switch and the last report. Deletion is off after install and
upgrade.
The Atlas server no longer keeps the admin-1 regions of every country in memory. Each country's regions are now a pre-gzipped file of their own, sent to the map as they are.
**Why:** `GET /addons/atlas/regions/geo?countries=…` built `admin1Store` (every country's region text, kept until restart), then joined the requested countries, `JSON.parse`d them, and Nest stringified them again. On the 512 MB – 1 GB boxes TREK runs on, the first z≥5 view could OOM the server (same shape as #1576).
**What changes:**
- `server/assets/atlas/admin1/<ISO_A2>.geojson.gz`: 198 files, 14.85 MB, the same 3,447 features as `admin1.geojson.gz`.
- New `GET /addons/atlas/regions/geo/:code` with a Zod contract (`regionGeoParamsSchema` in `@trek/shared`: two letters, any case). It sends the country's file as it is, with `Content-Encoding: gzip` and the current 1-day `Cache-Control`, as `countries/geo` does. A country without regions gets an empty FeatureCollection. Anything that is not two letters gets a 400.
- `useAtlas.ts` fetches one URL per uncached country and still rebuilds the layer once. Each country has a stable URL, so a country evicted by `REGION_CACHE_MAX` comes back from the browser cache.
- `getRegionFeatures` reads only the country it needs. `admin1Store` is removed.
- `?countries=` keeps working for cached PWA clients. It joins the per-country files as bytes, without a parse.
- The migrations are untouched. `admin1.geojson.gz` still ships, only for `Migration20200101021500` and `Migration20200101031500`. This costs ~15 MB of disk in the image and no RAM. `build-atlas-geo.mjs` writes both, and ATLAS-BUNDLE-008 keeps them equal.
**Memory** (real `atlas-geo` module, first z≥5 view over Europe = 49 country codes, JS heap + Buffers):
| | peak | kept after |
|---|---|---|
| before, first view (builds the store) | +372 MB | +61 MB |
| before, later views | +115 MB | 0 |
| after, `/regions/geo/:code` | +4.7 MB | 0 |
| after, legacy `?countries=` | +59 MB | 0 |
**Tests:**
- New: `atlas-region-files.test.ts` (moved from `atlas.service.test.ts`, plus join and dedupe cases), the `regionGeoParamsSchema` spec, ATLAS-BUNDLE-007 (the file format the byte join depends on), ATLAS-BUNDLE-008 (bundle and per-country files are equal), ATLAS-016/017/018 (the new route over HTTP, including the 400).
- The `getVisitedRegions` tests (ATLAS-UNIT-019..024) used to pass only when `getRegionGeo`'s `beforeAll` had built the store first. They now also pass on their own.
- The shared, server and client gates pass. Baselines go down only where this change removed code (`no-console`, `no-explicit-any`, `no-swallowed-catch`, file sizes).
**Trade-offs:**
- A first zoom over Europe sends ~49 small parallel requests instead of 1. Over HTTP/1.1 the browser queues them 6 at a time. The total bytes are the same (4.4 MB gz).
- The per-country URLs are unversioned and cached for a day, like `countries/geo`. After an upgrade that changes a country's regions, a browser can show the old shapes for up to 24 h.
Approved on Discord in #github-pr: <paste the thread link here>
Related: #1576, #2559
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] I have read the [Contributing Guidelines](https://github.com/liketrek/TREK/wiki/Contributing)
- [ ] My branch is [up to date with `dev`](https://github.com/liketrek/TREK/wiki/Development-environment#3-keep-your-fork-up-to-date)
- [x] This PR targets the `dev` branch, not `main` *(wiki-only PRs are exempt)*
- [x] I have tested my changes locally
- [x] I have added/updated tests that prove my fix is effective or that my feature works
- [x] I have updated documentation if needed
TREK 5.0 left most registry plugins blocked because their authors still declare
"<5.0.0", and the only way past that was TREK_PLUGINS_IGNORE_TREK_RANGE for every
plugin at once. An admin can now install or switch on one such plugin after a
warning; the consent is kept on its row for that plugin version and TREK major,
so an update or a new major asks again. The environment switch works as before.
Discover marks a plugin not confirmed for this TREK on its card, and Install opens
the warning first. Admin > Plugins reads cleaner: one row of alike facts per plugin
with each state that needs attention on its own line, type, status and order behind
one Filter button, and the containment notes in a dialog behind the shield in the
header. The row, the filter menu and the dialog moved into their own files.
Linking a gallery photo to an entry and taking it off told nobody, so other
tabs, and the journey page after an assistant change, stayed as they were.
Both now go through a photo links service that broadcasts
journey:entry:updated after the write, with the asking socket left out on
REST; a caption change through PATCH photos/:id broadcasts journey:updated
the same way. The service also moves photos from one entry to another of
the same journey in one transaction, as POST
/api/journeys/entries/:entryId/photos/move, with a strict contract. The MCP
tools link_journey_photo, unlink_journey_photo, move_journey_photos and
set_journey_photo_caption call the same methods; the caption tool answers
the caption it replaced.
The broadcast parity cases toggle a day in 2027, which the seeded plan does
not hold, so both MCP and REST now send vacay:settings for the opened year
before vacay:update.
Removing a member deleted their membership row only, and the invite link
checked nothing but its own token, so whoever kept the link walked straight
back in (GHSA-wr4c-964p-39jg).
trip_member_removals now notes who another member took off a trip. The invite
link refuses them on preview and on accept with a 403 that says so, old link
or regenerated, and the join page shows that instead of the join button.
Adding them back by name, or through a trip-bound invite or a plugin acting
for a member manager, lifts the note. Leaving a trip yourself leaves none, so
the link keeps working for whoever left and for everybody else.
A GPX file that mixes routes and tracks gives each its own text: the pieces
are matched to the reader's rows by kind, routes first as the reader gives
them, where they were paired by index and a route got a track's points. The
place pictures reach the network through a small repo instead of the api
module, the wanderer parity case and the tool catalog test allow wanderer,
and the knip and theme baselines follow the lower counts.
P6: insert_tour_stop and remove_tour_stop take a place into a saved hike or
city walk as one stop and out again, each the other's undo, with change
cards in the chat. The planner's insertion rule (poiInsertion) moved to
shared so the planner and the server place a stop the same way. "+ Tour" on
a found place holds it as a stop of the saved tour open in the planner.
P7: GPX, KML and KMZ files attached in the chat are read into their tracks
and each is held as a change that takes it in as a hike through
import_tour_gpx, undoable per track; the GPX text stays on the held call.
The assistant can now search a round hike of about a wished length from one
point and the router's other ways for one leg of a hike, with the planner's
own search, and propose each one as a hike card to compare and take. It can
also search the user's wanderer trails, preview one as a card, import it and
check a wanderer tour against its source.
The round search, its loop geometry, the test whether two lines are one way,
the other ways among a router's offers and the pins that hold a chosen way
move to @trek/shared; the planner and the road trip use them from there.
City walk changes take edit, reverse and close_loop; edits alone keep the
saved way and never ask the router. update_tour turns a hike round or closes
its loop with reshape. New MCP tools mirror the Tours tab's REST writes and
reads: update_tour_details, set_tour_route_color, copy_tour,
classify_ski_track, set_paddle_plan, decide_paddle_obstacles,
restore_recorded_track, get_tour_trail_details and find_horse_routes, with
change cards, undo where the answer keeps the values before, and broadcast
parity cases.
The assistant can now plan and save every type the Tours tab plans. A bike
tour or a run is proposed and created through the hike's own plan and save
(propose_tour, create_tour), routed on the server with the costing of its
profile, now kept once in shared and read by the planner as well. A paddle
or ski tour planned with plan_paddle_route or plan_ski_route is saved with
create_paddle_tour or create_ski_tour, and "Plan this" takes those plans,
bike tours and runs too. Every create has its card and its way back.
design_book, revise_book_design and apply_book_design work on a draft, the
apply keeps the old book as a revision for undo, skips pages changed since,
and broadcasts once after the commit. Book level tools set the page setup,
duplicate pages and save layouts.
A suite under tests/integration/broadcast-parity boots the app once per
group, runs a write tool through a real MCP session the way the
assistant attaches the registry, runs its REST twin on a twin scene, and
records every broadcast. Per case it checks the exact events in order and
count, that no MCP broadcast skips a socket (the asking tab hears its own
change), that nothing is sent inside a transaction, that every payload
parses with its registered schema, that REST sends the same sequence to
the same room and recipients, and that MCP's keys cover REST's.
A registry test lists the write tools as an MCP client sees them and
fails on a tool without a row, a stale row, or an exemption without a
reason; exemptions run too and must send nothing, and the tools still
waiting for a case are frozen in a list that may only shrink.
The new events the next changes send are registered now (member:updated,
todo:reordered, trips:changed, settings:updated, tags:changed,
notification:read-all, journey:updated, journey:deleted, atlas:changed);
the client keeps them apart as awaiting a sender, and the journey page
already leaves a journey that journey:deleted says is gone.
- the assistant keeps facts for a trip (remember_for_trip) as a change card,
applied into the user's trip notes; existing expenses can be changed
- change card fields as a few icon chips with the rest folded under Details,
the day band marks only what changes, the week grid as compact tiles
- no map button on receipts of things that have no place on the map
- tests follow the composer's accessible name, the receipt's status icon and
the GL map's container; the hanging stream helper honours an early abort
- Postgres probe: qualified columns in the upsert updates; entities restored
to the generator's output; strict ratchet reads the SDK client as untyped
Adds an in-app assistant that reads and plans the open trip. Users connect
their own provider (Sign in with ChatGPT, or an Anthropic, OpenAI, Gemini or
OpenRouter key) or use the one the admin set up, including a custom
OpenAI-compatible server. It is an addon and only shows while a connection
is usable.
The chat runs on the existing MCP tools, pinned to the trip. Every change is
held as a change card the user applies or discards, and applied changes can
be undone. On the desktop it is a floating window that can be moved and
resized; on the phone it is fullscreen. Search results show as place cards
that can go onto a day, weather lookups as a strip, proposed places as
ghost markers on the map with a before/after switch, and an answer can be
shared into the trip chat. Booking files can be dropped in and are read
into change cards.
Every help screen checked against the v5 redesign. Tours is documented for
the first time, as a screen of its own with twelve screens under it, and
notifications get help of their own. All pictures are taken again on a
realistic demo trip, with short loops where a still cannot show the motion.
The English is plain and every language is written naturally rather than
word for word.
The switcher shows two levels at most, a screen's overview lists its
sub-screens as cards above its guides, each wiki page is linked once, and
the help sidebar is wider.
Fixes found along the way: Save to Collection keeps a tour's route, a track
dropped on the places column opens the Tours import, tour Share on desktop,
the unsaved tour draft lives on the account, a route click no longer adds
an end point, Trip places leaves the trip's tours out, Escape closes a
stage's day menu, confirms before destructive deletes, DWD warnings sharing
one id kept apart, a booking's date moves with it, and smaller fixes across
notifications, bookings, the day plan, stays, Atlas, Collections, Vacay and
the admin screens.
A ride sends its full line with heights and every horse route section's
own line besides, about fifty bytes a point, so a ride along a few
thousand points of bridleway hit the API's 100kb body ceiling and the
save failed with a 413. Creating and editing a tour are now measured
against TOUR_WRITE_MAX_BYTES (2 MB), like the book and list file
routes; every other route, the tour copy among them, keeps 100kb.
The horse routes kept in memory are now bounded by section points as
well as by count (64 answers, 500k points), so panning a dense map can
no longer grow the server by hundreds of megabytes.
A search that found no connection can be asked again on the view whose
routes are shown, and a ride with a recorded time shows the horseshoe
beside its riding time instead of footprints.
The departure port needed a global binding that every suite building TripMembersModule without AppModule lacked. The roster's repository now deletes the leaver's row on the trip's journal link itself (leaveJournalLink), inside the transaction that removes the member: no edge into journey, no DI, no test stub, and the eight suites the stub touched are back as they were.
The schema inventory tests count the two new tables and their NOT NULL columns, the legacy step map lists the migration as post-legacy, and the TJL scenes give their users numbered names so cases sharing one database do not collide.
useTripJournalLink had grown past the 120-line limit for a function. The member list (candidates per journal, roles, who is new) is now useJournalMemberChoice. What loaded is kept with the journal it belongs to, so a new pick shows nothing until its own people are in, without resetting state by hand. The integration test adds its trip members with Promise.all.
A trip can now show one journal from the Journey addon as its own: any
journal its owner picks, and one journal on as many trips as they like.
The trip members chosen for the link get into the journal as viewers or
editors, see it in their own journal list and follow it live, for as long
as the link holds them and they stay on the trip.
That access is never copied into journey_contributors. It is read where
journal access is read: the contributor repository answers with the
stronger of a direct role and a link role, the journal list and the live
update recipients take the link members in, and the photo access check
goes the same way. Every one of those reads starts from linkReach, which
only counts members still on the linking trip, so unlinking, deselecting
someone and their leaving the trip all end exactly what the link gave.
A member who leaves is also taken off the link, so coming back later does
not quietly reopen the journal.
Linking or changing the members takes the journal's owner and the new
journal_link trip permission (trip owner by default); unlinking takes
either. /api/trips/:tripId/journal answers the link as each member may see
it, /candidates says who a journal would be opened to for the first time,
and four MCP tools do the same over the same service. A trip:journal:changed
event tells the trip to look again.
A new entry always went to the end of its day, so writing up 1 am after
2 am left the night out of order, and changing an entry's time never moved
it. A timed entry now lands before the first entry of its day that starts
later, on create and when its time or day changes; untimed entries still
go last, and the hand-made order of the others stays as it was. A save
that leaves time and day untouched keeps an entry where it was dragged.
Closes#2679
An SSO login that matched no account by its subject took the account with the
same email address and linked the provider identity to it. That address proves
nothing when the account typed it in itself: anyone could sign up with, or
switch their profile to, a colleague's address and receive the colleague's
first SSO login. The provider switch branch (#2110) relinked a row on any email
match without even asking for email_verified, and also for another subject of
the same provider.
users.email_self_asserted remembers an address the account set itself: signup
(not the first account) and an email change in the profile settings set it, an
admin entering a new address clears it, and accounts the provider creates start
without it. Existing accounts keep linking, except those whose address moved
away from the registered one without an admin editing it. A login by email now
needs a verified address and an unmarked account, and is refused for another
subject of the same issuer, with email_link_blocked and a translated message on
the login screen. An email change in the profile settings writes a
user.email_change audit row.
Refs #2110
Marking a region wrote its country with source 'manual', the same as ticking
the country off by hand, and removing the last region then took the country
away whatever its origin, Dawarich confirmations included. A country that
only comes in through a region is now stored as 'region', marking it visited
afterwards turns it into the user's own mark, and the cascade in unmarkRegion
only runs for a country without such a mark. Readers still see 'manual' for
those rows, so the country detail, the MCP resource and the plugin RPC answer
as before. Rows written before this change stay 'manual' and keep their
country.
REST, MCP and plugin RPC share the service. The atlas popup on desktop and
phone now asks the server whether the country is still marked before it
drops it from the map instead of guessing.
Refs #2597
Reordering handed out the dates by position, so moving a day added without a
date to the front gave it the first date and took the date of the last dated
day. The undated row stayed at the end and the move looked like it did
nothing, by drag and by the arrows alike. The dates now go to the dated days
only, in their new order; the rule lives in shared so the server, the
planner's optimistic list and the MCP tool apply the same one.
A mechanical prettier --write over server/src, server/tests, server/scripts
and shared/src with the workspaces' .prettierrc; nothing but layout and
import order changes, and every file keeps the same set of imports. shared
is formatted throughout and its format baseline is empty. The server
baseline keeps the 35 files that Prettier would grow past their lint:size
limit (they need a split first, which is not a formatting change), and the
generated entity files stay as their generator writes them.
createInvite returned { error: 'Trip not found', status: 404 } and the admin controller never looked at it: POST /api/admin/invites with an unknown trip_id answered 201 with an empty body and wrote an admin.invite_create audit row for resource 'undefined'. The service throws the DomainError now, so the route answers 404 { error: 'Trip not found' } and writes neither the invite nor the audit row.
A hand-built service gets a FakeRealtimeService where it took a RealtimeService, a Nest testing module overrides the provider, and a suite over buildApp() spies on the container's instance (spyOnRealtime). The 167 vi.mock factories on src/websocket are gone, RealtimeService reads the socket registry in ws-state.ts directly, and the re-export stub src/websocket.ts is deleted. The facade's own suite now checks what reaches a socket instead of which module function it called.
tests/setup.ts mocks src/config once with tests/helpers/test-config.ts. A suite that needs other values calls overrideTestConfig(), and the setup file restores the defaults after every case. The 191 per-file factories are gone; config.test.ts takes the real module through vi.unmock, and the few suites that signed tokens with their own literal secret read it from TEST_CONFIG.
Every domain's *.rpc.ts was written against rpc-kit, rpc-params, rpc-errors
and PluginGuards, and every domain module imported PluginGuardsModule, all
of it inside src/nest/plugins, while PluginsRuntimeModule imports those same
domains so their handlers exist at boot. That made the plugins domain part
of every cycle. The kit imports no domain but the protocol tables and
permissions/addons for the guards, so it moves to src/nest-rpc beside
src/nest-mcp, with its tests, its README and its extraction rule; the
runtime module keeps its domain imports, which the boot-time coverage check
of the wire surface needs in any subset app.
The converted specs wrote accommodation_id as a plain id, while the
services store the legacy '<id>.0' text, and wrote read-only foreign key
twins instead of their relations. The files e2e schema gains the trip
columns the entity fills, and the idempotency spec seeds a real user.
Maurice