The IMAP box, the webhook secret and the address template join the admin
settings form (masked, encrypted, held by the operator on a managed install),
and MAIL_INGEST_* variables override them through the mailIngestConfig token.
The audit row masks every masked key, and key rotation covers the new secrets.
The transport page explains what TREK follows, where it shows and what open
data cannot give; the environment page documents the switch and what leaves
the server.
- TREK_WALLET_PASS_URL read as a secret to the env check (the word PASS); it
is TREK_WALLET_SIGNER_URL now, everywhere it is documented
- findPassContext compared the TEXT accommodation_id with an integer id, which
Postgres refuses; it compares them as numbers now, as both databases accept
- the wallet-pass service test takes its subject from createTestModule instead
of building it by hand, and no longer names the DOM-only BodyInit type
- Reservations.repository.ts is formatted, so it leaves the format baseline
- the wallet button test reads the translated texts the render helper gives
"Add to Apple Wallet" on a booking in the iOS app turns it into a TREK pass:
a boarding pass from and to for flights, trains, buses and boats, check-in and
check-out for a stay, pickup and drop-off for a car, the time for anything
else, with the confirmation code as a QR code. Wallet brings it up on the lock
screen around its time and place: times carry the offset of the endpoint's zone
(looked up from the coordinates where the endpoint has none), stays and
restaurants their place's coordinates. The labels come from the booking forms'
own strings, so nothing new to translate on the pass.
Only liketrek holds the Pass Type ID certificate, so the server sends the pass's
texts to the TREK services' signer (TREK_WALLET_PASS_URL, push.liketrek.com)
once per tap; it stores nothing, and TREK_SERVICES=off turns it off with the
other services. Texts are cleaned of links and shortened here, the answer must
be a .pkpass within 512 KB, and a user gets 20 passes a minute.
- New wallet-pass domain: GET /api/trips/:tripId/reservations/:id/wallet-pass
behind the trip guard, the booking checked to belong to the trip; trip title
and place come from the reservations domain (findPassContext)
- MWalletButton in the transport and booking sheets, only where the app binary
has the wallet feature; undated bookings and failures are told apart
- TREK_WALLET_PASS_URL in .env.example, compose, Helm and the wiki
What is sent and when, what never is, how values are checked before anyone
sees them, and where an admin switches it off or takes it back. The new
TREK_PLACES_CONTRIBUTE variable is listed with the compose file and the chart.
Translate the 55 keys dev added since the branch was cut (TREK services
allowance, Valhalla routing service, PDF routing progress, overview
distances, Unsplash key hint, tour day limit) and drop the two reservation
toasts dev removed, so he matches en key for key again. Refresh the two
admin strings and the Valhalla hint whose English changed.
Check he in the placeholder spec and the RTL test, and count 28 languages
with Hebrew in the README and the Languages, Environment Variables,
Contributing and Help Center wiki pages.
Admin settings, transport, environment and MCP pages still said Transitous
is the only fallback and that Google is billed only when picked. Document
that an empty plan asks the other backend, that the Google fallback uses
only an instance key within the daily limit, and mention it in the
search_transit_routes tool description.
POST /api/routing/valhalla/route, /trace_attributes and /height ask the
Valhalla at TREK_VALHALLA_URL (default https://valhalla.liketrek.com) for a
signed-in user. The body is held to the shared contract (64 KB, at most 20
locations), every question counts against 120 a minute per user, and the
installation key goes along only to a host TREK_SERVICES_HOSTS lists.
Valhalla's answer is passed on as it came, refusals included.
The road trip router asks the same relay when the routing choice lands on
the TREK services, so the MCP road trip, city walk and hike tools share its
limits, and hands a question it cannot answer to FOSSGIS.
docker-compose.yml and the Helm chart name the switch and the key (the key in the Secret, never the ConfigMap), and the wiki explains the installation key, what it carries, where it goes and the admin view of the allowance.
The ways and routes layers now count failures on a breaker of their own, so
busy or slow layer answers no longer send place search to the fallback for a
minute, and a layer's 404 no longer clears the count of search failures.
When a later batch of /v1/ways fails, the batches that answered are kept and
only the ids still open go to the Overpass mirrors.
The wiki, the compose file and the chart now say that the way ids of a planned
tour and the horse routes box go to the TREK Places API.
The search along a tour found nothing. It asked the places index for all
kinds of a stretch at once, above the 100 rows and 200 characters of terms
the index takes, so the index refused and every stretch went to the public
Overpass mirrors, which do not answer from this server. Now the index is
asked in small circles strung along the line, one request per kind and
circle within its limits, and an empty circle counts as an answer, so a
city walk lists its sights, museums, parks, cafes, restaurants and markets
in well under a second. The kinds the index does not know (viewpoints,
huts, drinking water, toilets and the like) are asked of Overpass along the
whole line at the same time, one query per stretch of up to 50 km, three at
a time, each with a short deadline of its own; a line past 400 km is
searched along its start and says so. When the index answers no circle at
all, its kinds go to Overpass beside the others instead of after them. A
kind no source could be searched for is named in a calm notice above the
list instead of reporting nothing found, and the reason is written to the
log. Unnamed taps, toilets, shelters, picnic spots, viewpoints, summits,
slipways and repair stands stay in the list and on the map under the name
of their kind; an unnamed garden, cafe or memorial is left out. The places
nearest the line are kept when there are more than sixty, listed from the
start, with the opening hours the source has.
A walk's parks are parks and gardens only, and its museums are museums
and galleries only, no longer the beaches, summits, theatres and cinemas of
the map's explore buttons; a water park the index files under parks is not
one of them. The index also answers markets and summits. The explore
buttons keep their single request, and a search of several kinds that
would exceed the index limits is asked one kind at a time instead of being
refused. The MCP tool search_tour_pois and its wiki row describe the walk
kinds and the new unavailable list, and the OVERPASS_TIMEOUT_MS row says
the search along a tour keeps its own shorter deadline.
The OpenAI-compatible client always asked for 4096 tokens, and a document
whose bookings needed more came back as "no reservations" with nothing that
pointed at the cap. LLM_MAX_OUTPUT_TOKENS now sets the cap for every
provider; unset keeps 4096 and 8192 so a small local model's context still
fits. A finish_reason of "length" is reported like Anthropic's max_tokens
stop, naming the limit and the variable that raises it.
Closes#2677
NODE_ENV="production " passed the boot check but matched none of the
exact production comparisons, so the web UI was never served and every
page answered {"error":"Cannot GET /"}, while the banner showed a clean
"production". The boot import now trims these three variables in
process.env before anything else reads them, so Express, the plugin
children and readEnv() agree, and reports each change as a process
warning. Casing stays as it was. A padded "test" is only reported, so an
instance cannot switch to an in-memory database after an update. The
banner prints NODE_ENV as set instead of lowercasing it.
Refs #2565
The 65 s default sat above nginx's 60 s but below the 90 s Traefik keeps an
idle upstream connection, so a default Traefik deployment could still reuse a
socket Node had just closed and answer that request with a 502. 95 s clears
both, and the headers timeout one second above it stays well under Node's
300 s request timeout and the 290000 cap the schema accepts.
Also moves the stray RawEnv import in boot-derive.ts up to the other import.
Three small gaps in how the process behaves at its edges:
- An unhandled rejection or uncaught exception crashed the process with
the stack on stderr only. index.ts now logs it to trek.log and runs the
same orderly shutdown a SIGTERM gets, exiting with 1; a second fatal
error during that shutdown exits at once.
- /api/health/ready kept answering 200 while a SIGTERM was draining the
process, so an orchestrator sent traffic for another probe period. A
shutdown now marks the new ReadinessService first (and Nest's own
shutdown hook marks it too), and the probe answers 503 from then on.
- Node closes an idle keep-alive connection after 5 seconds while reverse
proxies keep upstream connections for about a minute, which shows up as
intermittent 502s. The server now keeps them for 65 seconds, configurable
through HTTP_KEEP_ALIVE_TIMEOUT_MS (owned by the httpConfig token), with
the headers timeout one second above.
Migrations only go forward, and since #2610 an older image refuses a
database a newer one migrated. Going back after a bad update therefore
needs a copy from before it, and the only copies were backups the admin
takes by hand or auto-backup, which is off by default. The first boot
on 4.4 migrates every 4.3.x install through the legacy baseline and the
new migrations without either.
Before applying pending migrations to a file database that holds
anything, the boot now writes a VACUUM INTO copy next to it, named
pre-migrate-<schema>-<time>.db (legacy-<schema_version> for an install
the positional runner left, otherwise the last recorded migration), and
keeps the newest TREK_DB_PRE_MIGRATE_SNAPSHOT_KEEP (3). The copy is
written under a temporary name and renamed, so a crash never leaves a
truncated file that looks good. If it cannot be written the boot refuses
before anything is migrated; TREK_DB_PRE_MIGRATE_SNAPSHOT=false is the
documented way past that. The newer-database refusal names the copy
taken right before the newer release migrated past this one.
A fresh install, a restart with nothing pending, an in-memory database
and a restore's re-bootstrap (the archive is the copy) take none, and
NODE_ENV=test leaves it off unless a suite sets the switch.
Place photos, link previews, webhooks and the integrations fetch through the guard's own dispatcher and so ignored the outbound proxy, which left a server that can only reach the internet through one without them. They now read HTTP_PROXY, HTTPS_PROXY and NO_PROXY themselves and connect through the proxy, after the guard has vetted the target. The Helm chart passes the three variables through. (#1754)
The 50 MB limit for files, booking attachments and note attachments becomes FILE_UPLOAD_LIMIT_MB, 50 by default. The server hands it to the client through the app config, so the upload dialogs refuse a file over it up front and say what the limit is; on desktop a file over it no longer vanishes without a word. (#1364)
Providers like Pocket ID send the full name in 'name', which became 'JaneDoe'. OIDC_USERNAME_CLAIM=preferred_username builds the username from that claim instead. Unset, or with the claim empty, the old order applies (name, preferred_username, the email's local part), and it is only read when an account is created. (#1677)
Small requests from the discussions, each built end to end: planner and
map (cycling mode, date first headings, clear a day, stay times per day,
off-route stops, map lock, compact unplanned markers, pending stays,
cable car, transit quick picks), places (location filter, category from
OSM, open in maps, geo: link, image from an attachment, enrichment on
file import, places near a pin, place name language, Amap first in
China), lists and costs (packed counter, inherited sharing, Markdown
to-dos, payment notes, bookings CSV, my plan PDF), sharing (unplanned
places, travel only, without photos), journey (photo zoom, drafts,
photo order, trip suggestion, day jump), atlas (region counter, region
bucket list, places per country), vacay half company holidays, studio
copy and paste plus the book's own layouts, and RESTORE_FROM_BACKUP for
a first start restore.
A fourth built-in channel next to email, webhook and ntfy. Browsers
register per device, the server encrypts per RFC 8291 and signs per RFC
8292 with node:crypto and sends only to known push services through the
SSRF guard. The event matrix decides per category what arrives by push;
admins switch the channel on under Notifications.
VAPID keys come from the environment or are generated once, only while
neither stored row exists, and kept encrypted. A pair that does not
decrypt or match is never overwritten and never swapped for another one:
push stays off with an error in the log and every device stays
registered until the key is fixed. The subject is VAPID_SUBJECT, then an
https APP_URL, then the project URL.
Only chat messages collapse into one notification per trip, and a
replacement still alerts. A 403 removes a device only after three
failures in a row; 404 and 410 remove it at once. Password change, reset
and an admin setting a password drop the user's devices with the other
credentials. The request log redacts subscriptions and endpoints, boot
errors print *** for every secret variable, and logout forgets the
device with a plain fetch so an expired session cannot redirect midway.
Addresses discussion #894.
Translates the keys dev gained since the branch was cut, the help center
above all, and drops the six keys dev no longer has. Registers th
everywhere a language is listed: notification emails, the rate-limit
message, the placeholder check and the wiki. The two tests that counted
the languages by hand now derive the count from SUPPORTED_LANGUAGES.
Translates the keys dev gained since the branch was cut, the help center
above all, and drops the six keys dev no longer has. Registers sk
everywhere a language is listed: notification emails, the rate-limit
message, the placeholder check and the wiki. The two tests that counted
the languages by hand now derive the count from SUPPORTED_LANGUAGES.
Translates the keys dev gained since the branch was cut, the help center
above all, and drops the six keys dev no longer has. Registers et
everywhere a language is listed: notification emails, the rate-limit
message, the placeholder check and the wiki. The two tests that counted
the languages by hand now derive the count from SUPPORTED_LANGUAGES.
Translates the keys dev gained since the branch was cut, the help center
above all, and drops the six keys dev no longer has. Registers az
everywhere a language is listed: notification emails, the rate-limit
message, the placeholder check and the wiki. The two tests that counted
the languages by hand now derive the count from SUPPORTED_LANGUAGES.
The server already put PLACES_API_KEY ahead of the key saved in the admin
panel, but nothing said so: the variable was in no template and the admin
field kept showing a stored value that no search used.
GET /me/settings now answers env_keys, the variable behind each key the
environment sets (Google, Unsplash, Amap), and null for that key's value.
The admin field turns read-only and names the variable, Test still probes
the key a search resolves to, and a save leaves those names out so the
stored fallback survives. The variable is documented in .env.example,
docker-compose.yml, the Helm chart (as a secret) and the wiki.
Addresses discussion #1881.
Every file the release touches was read once more, and what survived an
independent check against the code is fixed here, with a test for each.
Data and credentials
- A document sync connection edited to a new address no longer carries the
stored token along: a blank form under another origin is refused, the same
rule the probe route already had. Dawarich and AirTrail testConnection send
the stored key only to the host it was stored for.
- The key rotation script now covers document_connections.secrets and
trip_document_links.webhook_secret, which it had skipped.
- Turning a booked night into a pause in the road trip popup asks first and
says that the booking and its expense go with it. Deleting a place says the
same when a night is booked there, on the desktop and on the phone.
- A moved version marker over unchanged bytes no longer replaces the TREK file
and its attachments; a real revision carries the attachments over.
Planner
- A drop in the day plan names the row it lands ahead of in the full day, so a
hidden hotel stop no longer shifts it one slot early; the store sorts before
it splices. Via re-anchoring, the booked-night guard and the sidebar's day
membership read the stored day, the index space the server uses.
- A hotel stop placed mid-day re-pins the road trip vias of that day, and the
booking session receives the new stop order too. The Public API and the iCal
feed list a booked hotel once. A plugin can no longer write a stay against
another trip's day.
Road trip and map
- The route overview paces its requests to the public routers and retries a
rate limit, and it says which legs it could not route instead of quietly
reporting a shorter trip. A click on a via handle no longer drops a second
via; handles hide when zooming out. Read-only members cannot edit the driving
limits. The phone shows a way back when no day is picked and prints the night
drive departure in the chosen clock format.
Integrations and search
- Google transit keeps conventional trains when the train chip is on.
TREK_PLACES_ENABLED=false stops the details and enrichment calls too, and
accepts the whole boolean family. A caller-supplied description no longer
lands in the instance-wide enrichment cache. Nominatim searches carry a
timeout. An OSM place id is checked for shape before it reaches Overpass.
- Dawarich puts a stay on the trip whose dates hold it, the Atlas hand-off
knows what the Atlas already shows, and a host change with a new key clears
the old instance's state.
- Document sync: a webhook address pasted by hand works on the token alone,
deleting a connection unregisters its subscriptions, Nextcloud under a
sub-path connects, Synology answers a name collision before uploading and
verifies a move into the bin, Paperless titles are capped at 128 characters,
a DNS failure reads as unreachable rather than as a policy verdict, videos
from the store are accepted like uploads, and remote_missing has a way out.
Conflict buttons only for the owner, a refused choice is reported, a refused
credential gets a Reconnect button, disconnecting asks first on the desktop
as it does on the phone, and the folder picker no longer spins forever.
Release
- The server holds the release notice back from a bundle that cannot draw it,
so the previous shell served from the service worker cache after an update
no longer shows bare keys and no longer lets a click use the notice up.
- Import into an existing list gets the same body limit as a new list. Chat
images say when the limit is reached and the new controls are translated.
The transit backend switch reports a failed save. The Helm chart carries the
Amap credentials in its Secret; the shipped templates document the Places
API and Amap variables and the Overpass default. The wiki follows.
Every page was read against the code on dev. What changed:
- Pages that described 4.2 behaviour: satellite view on the vector maps,
the mobile journal timeline, Collab with its fifth tab, the Review badge,
clustered pins on the share page, the Leaflet preset list, the strict
guard on fe80::/10, the invite link tools and their scopes, the Public
API scope rules, the permission count in the plugin SDK.
- Features of this release that no page mentioned: videos in a journal
entry, receipts on an expense, the Google transit backend, routing
counters, the rating on a place, the navigation menu, car stops, the
manual school holiday catalogue and its MCP tools, the Dawarich panel in
the Atlas, the release notes modal, plugin frames going fullscreen, the
corridor call into the search provider hook.
- Statements that sent an admin the wrong way: DEFAULT_LANGUAGE is not in
the Helm ConfigMap, env.PORT cannot be changed on the chart alone, the
rotation script does not cover the document sync credentials, event
handlers cannot broadcast, there is no per user MFA reset in the admin
panel, the demo user can edit, an unparsable APP_URL stops the server.
- Updating gets a section for 4.3.0: the TREK Places API as the new
default, the new variables, the Overpass timeout, SSO sessions on an
OIDC only instance, the migration that puts booked nights on the drive.
A rootless Podman container reaches its host through 169.254.1.2, and an
identity provider behind that address failed the OIDC login with "Requests
to link-local / cloud-metadata addresses are not allowed", with no setting
to change it.
ALLOW_LINK_LOCAL_IPS names single link-local IPv4 addresses that may be
reached after all. The admin-configured lane (OIDC, AI parsing, plugin
OAuth, routing engines) reaches a listed address directly; the guard on
user-supplied URLs treats it as internal, so it also needs
ALLOW_INTERNAL_NETWORK there. 169.254.169.0/24 and 169.254.170.0/24, where
clouds serve instance metadata and credentials, cannot be listed, and an
entry that is not one link-local IPv4 aborts startup. Unset, nothing
changes.
Closes#2400
The 4.3.0 features had no pages, or pages that described an earlier
state. Three new pages and a rewrite:
- Road-Trip: the addon from enabling it to the phone tab, stays and a
stop's End as the time you leave, pinned times and the backward
schedule, daily travel times, the three driving limits, search along
the route, via points, other ways and what avoiding tolls needs,
following a track, the warning layer, MCP tools and troubleshooting.
- Document-Sync: switching stores on, binding a trip, what each store
asks for and binds, direction and delete rules, conflicts, the
mass-delete guard, renames and restores, what needs a look, limits,
private addresses, and the MCP tools.
- TREK-Places-API: what the index is, where TREK asks it, what leaves
the instance, switching it off or pointing it at a copy of your own,
and what happens when it is down.
- Dawarich: brought up to date with suggestions, the journal fold,
recorded routes, Atlas proposals and the wish list.
Places and search, documents, the addon and admin overviews, MCP
tools, environment variables, internal network access, the sidebar and
home link the new pages, and the in-app help picks them up from the
sidebar.
Follow-up to #2379.
NOMINATIM_URL arrives from a compose file or a ConfigMap with whatever
whitespace was typed around it. The schema validates the trimmed string and
calls a blank one unset, the derivation did neither, so " " passed startup as
"use the default" and then became the base URL that cannot be fetched, and a
padded value lost every geocoding call the same way. It trims now, which is
also what the places client next door does with its own override.
The chart could not pass the variable at all: the ConfigMap is an allowlist and
an unknown key is dropped without a word, which is exactly the locked-down
cluster the override exists for.
The docs said what to set and not what it costs. They now say that it replaces
the public service with no fallback, that the endpoint has to speak the
Nominatim API, and that the one-per-second spacing stays either way, so nobody
stands up their own instance expecting it to run faster.
Test IDs follow the file again.
The index and OpenStreetMap answer every search; the provider choice
only decides who fills the keyed slot next to them. Rows carry a source
mark, a place keeps its Amap id, and the journey map shares the shifted
projection.
Two reports, both about an import that fails without saying why.
#2262 — temperature: 0 sat in the shared request body, so it went out on every
attempt including both retries. Reasoning models (the gpt-5 family) reject any
explicit temperature, and the json_object retry fires on any 400, so the import
spent two doomed requests and could not succeed at all. The parameter is now
dropped once a server has explicitly said it will not take it.
Not as a third link in the chain, though. A gpt-5 rejects max_tokens AND
temperature, the API names one parameter per response, and it may name either
first — a chain of one-shot ifs converges on only one of the two orders. The
retries are a loop over what the server actually said instead: each remedy
applies at most once and they compose. Detection is a phrase, not the bare word,
because dropping temperature costs the deterministic sampling that small local
models depend on, and an unfamiliar wording still falls through to the
json_object retry exactly as before. NuExtract and Ollama are untouched: the
first attempt always carries temperature, and it is the only one they ever see.
#2261 — the bundled extractor is Debian trixie's 24.12.3, roughly eighteen
months behind upstream, and the reporter is right that this is invisible: a
stale extractor, a mistyped KITINERARY_EXTRACTOR_PATH and a genuinely
unsupported provider all look the same from the UI, which is nothing imported
and no reason given. The version now appears in the startup log and in a new
admin-only GET /api/admin/system-info, and LOG_LEVEL=debug passes the
extractor's raw stderr through — the JS ERROR and Invalid-result-type lines the
filter drops are the only evidence that a vendor script is missing or failing.
Capped at 200 lines, and opt-in because those lines can carry document
fragments.
Not bumping the package: the filter the reporter needs first ships in KDE Gear
26.08 and no Debian suite carries it, and sid's 26.04.3 wants KF6 6.22 and Qt6
6.10.2 against trixie's 6.6 and 6.8.2 — a partial upgrade of Qt6, KF6 and libc6
from unstable into the production image, on two architectures, that still would
not fix the reported carrier. It comes for free with the base-image move after
Debian 14.
Three shipped sentences became false with LLM_TIMEOUT_MS: two claiming the
addon needs no environment variables (one of them in the very file the
variable was added to), and one naming the old five-minute ceiling that the
deleted constant used to set.
The timeout for a model call was spelled three times and adjustable
nowhere: 120s in the Anthropic client, 300s in the OpenAI-compatible one,
300s in the Ollama one. None of them was the ceiling that actually
applied, either — every one of those calls goes through the pinned undici
dispatcher, and an undici Agent stops waiting for response headers after
five minutes by default, so a client constant set above that never got
the chance to fire.
- LLM_TIMEOUT_MS (default 900000) replaces the three constants and is
passed to the dispatcher as headersTimeout/bodyTimeout, so the abort
signal and undici are governed by the same setting. The default is
deliberately generous: heavier parsing work should fit without a code
change, and an admin who would rather fail fast can lower it.
- The ceiling stays on the model lane. safeFetchAdminConfigured is also
the OIDC lane (discovery, token, userinfo, JWKS) and the plugin OAuth
token exchange, and that one sends no AbortSignal of its own, so it
keeps undici's five minutes rather than inheriting a model setting.
- The derived value is floored: the schema accepts any positive number
and undici rejects a fractional headersTimeout.
* feat(plugins): TREK_PLUGINS_IGNORE_TREK_RANGE turns the version gate into a warning
A plugin whose author has not updated its `trek` range for the running TREK can
now be installed, updated and activated when the operator sets
TREK_PLUGINS_IGNORE_TREK_RANGE to a truthy value. Every gate (registry picker,
post-extraction re-check, sideload, dev-link, activation) warns instead of
refusing; a missing range is tolerated the same way; "install latest" takes the
newest published version. The plugin-API version gate is not lifted and the
Discover compatibility verdict stays truthful.
The admin is warned at every step: each bypass is logged, install/upload/link/
update responses carry a `trekRangeBypassed` marker, and both admin panels show
a "Version checks off" pill, an "Install anyway" confirm dialog before a
registry install, the same notice after a path that could not ask first, and a
persistent chip on the row while the plugin runs outside its range.
Nine new i18n keys in every locale; wiki pages updated (Environment-Variables,
Admin-Plugins, Plugin-Development, Security-Hardening).
* refactor(plugins): one useRangeBypass hook behind both admin shells
SonarQube flagged PR 2226 at 10.8% duplicated new code: the range-bypass
logic had been added to the desktop and phone plugin panels as identical
copies. The marker/warning types, the row chip, the "Install anyway" offer,
the dialog copy and the confirm-or-notice state now live in
useRangeBypass (next to useInstanceSettings, the existing shared-logic
precedent); each panel keeps only its own dialog/sheet markup.
* docs(claude): record the SonarQube gate rules and the no-English-placeholder i18n rule
The CI SonarCloud gate (Sonar way, on new code: >=80% coverage, <=3%
duplication, A ratings, hotspots reviewed) was only discoverable by waiting
20-25 minutes for the workflow. The root CLAUDE.md now states the conditions,
the desktop/mobile shell-mirror duplication trap and its shared-hook fix, and
how to check a PR with the sonarqube MCP first; client and server CLAUDE.md
point at it. shared/CLAUDE.md says every locale needs a real translation —
parity cannot tell an English placeholder from a translation.
* docs(claude): keep CLAUDE.md to commands, invariants, gates and philosophy
Verified every concrete claim in the five CLAUDE.md files against the code
and fixed the stale ones: the websocket server is the Nest realtime gateway,
MCP tools live in <domain>.mcp.ts files, the client "native shell" section
described code that was never written, offline maps also prefetch vector
basemaps, plus several renamed files and wrong table names.
Then trimmed everything that drifts on its own: dates, migration history,
counts, and inventories of stores, slices, domains, sync files and CLI
commands. What stays is the commands, the invariants with the file that
enforces each one, what CI actually runs, the reference implementations to
copy, and the rules for new code. Each file now opens with that rule.
The last pass reviewed diffs, so it never saw a new paragraph contradicting an
older one on the same page, or a long-standing claim nobody had checked.
Plugin authors were told ctx.config carries their per-user secrets and that
definePlugin takes a permissions key — neither is true, and both fail silently.
FORCE_HTTPS was said to need TRUST_PROXY, which it does not.
* docs(wiki): correct the install and operations pages for 4.0.0
Quick Start passed docker an empty ENCRYPTION_KEY: an assignment prefix is not
expanded on the same command line, so the server silently generated its own key
instead of taking the one the page told you to set. The tag examples pinned to
the 3.x line, the documented tmpfs size did not match the shipped compose file,
and several .env values were described as reaching the container when compose
never interpolates them.
* docs(wiki): match the MCP reference to the registered surface
The scope table was missing the whole Collections group and the totals said 27
scopes across 13 groups where the code registers 29 across 14. The Cloudflare
link on the setup page pointed at an anchor on its own page instead of the
heading on Troubleshooting.
* docs(wiki): correct the plugin and admin pages
Three links to Plugin-Development carried a .md suffix, which does not resolve on
the wiki. The admin pages described tabs, permissions and storage behaviour that
had moved on.
* docs(wiki): bring the feature pages back in line with the app
Costs still documented the old panel, where categories could be added, renamed,
reordered and deleted; they are fixed now and the panel is a day-grouped ledger
with settlements. Trip sharing, journeys, maps, packing, reservations and the
account pages had all drifted in the same way.
* docs(wiki): cover the pages the first pass did not reach
The MCP catalog listed 168 of the 201 registered tools; the settlement tools and
rate_place were missing here and the Collections family from the addon page.
Several resource rows claimed a trips:* scope where the handler asks for
trips:read.
* docs(wiki): tighten the corrections a review picked apart
The audit-log resource is the row id, not the credential id; an admin password
reset does not bump password_version the way the page implied; and a few of the
new paragraphs described behaviour more confidently than the code supports.
* docs(wiki): fix what a last read-through caught
Costs shows every figure in your display currency, not the trip's, and an
expense with no payer still counts against the members it is split between.
Studio's cutoff is a 1024px viewport rather than phones alone, and the
Integrations tab is gated on the providers an admin has configured.
* docs(wiki): the last few the review caught
A passkey satisfies the MFA policy everywhere except the WebSocket handshake,
which still asks for TOTP, so a passkey-only account under that policy gets no
live updates. Mobile optimizes a day from two places, not three. The ctx.db row
carried an unescaped pipe that split the table, and the Backups table linked
with a wiki link whose pipe did the same.
An adversarial review of what had already landed here turned up three that
were wrong, and they are the reason for this commit.
The CSP change removed 'unsafe-eval', which is not leftover: heic-to's
libheif build initialises embind through new Function(), and without it every
iPhone .heic upload fails in the browser. 'wasm-unsafe-eval' alone was tried
once already and was not enough (93b51a0b). Put back, with the reason written
down this time and the test pinning it rather than its absence.
The idempotency in-flight guard never fired. finalize() runs when the
handler's observable completes, but Nest writes the response - and therefore
stores the row - several microtasks later, so the waiting request always woke
up before the row existed and ran the handler a second time. It releases from
the point the row is written now; finalize is the backstop for handlers that
answer another way. The test drove the two in the opposite order from Nest,
which is why it passed either way.
And the trip-scoping check treated an id that resolves to nothing like one
belonging to someone else. accommodation_id has no foreign key, so shortening
a trip's dates leaves a booking pointing at a gap - which update() has always
healed by clearing the link, and which would have become a permanent 400.
The rest: a reply to a soft-deleted chat message is refused and the create
path blanks the quote like the list path does; the ws-token buckets are keyed
by account rather than by IP, where every user behind one NAT shared a
ceiling; the weather and vacay caps read through the stream, since a chunked
response has no content-length to check; the Google host gate takes country
domains again; abandoned response bodies are cancelled instead of left to GC;
the boot failure prints its errno; and a share link that is not yours to read
now says so instead of reporting that the journey has none.
TICKETJSON gets a real repair: the in-place GLOB fix only helps installs that
have not run it yet, so an appended migration puts back the note text the
case-insensitive LIKE cut off, for rows where what it stored is not valid
JSON. A genuine receipt always parses, so it cannot be touched.
Wiki: the audit log records the IP Express resolves after TRUST_PROXY, not
the leftmost X-Forwarded-For entry a caller can write. That is the right way
round, but it means the hop count has to be accurate - said plainly in both
places now.
* feat(storage): storage key validation, driver contract suite, LocalDriver
* feat(storage): MirrorDriver composite backend
* feat(server): add TREK_UPLOADS_DIR env var for the uploads root
* feat(storage): settings-backed registry, StorageService facade, module wiring
* test(storage): pin src/nest/storage coverage thresholds
* feat(storage): multer upload options factory
* refactor(files): spool trip-file uploads through storage
* refactor(collab): spool note attachments through storage; normalize files/ prefix
* refactor(journey): spool journey media uploads through storage
* refactor(trips): spool trip cover upload through storage
* refactor(places): spool place-image and collection uploads through storage
* refactor(auth): spool avatar upload through storage
* fix(server): consolidate allowed_file_types onto AllowedFileTypesService
* fix(storage): default defParamCharset to utf8 in the upload factory
* test(db): re-point the trailing-migration guard at the trip_files prefix strip
* fix(storage): age-gate the boot spool sweep
* test(files): key FILE-P02 residue checks on upload size
* test(storage): raise src/nest/storage coverage pin
* refactor(platform): serve /uploads routes through storage
* refactor(files): serve trip-file downloads through storage
* refactor(memories): serve photo bytes through storage
* refactor(maps): stream place-photo bytes through storage
* refactor(share): stream shared place-photo bytes through storage
* refactor(journey): serve public journey assets through storage
* fix(share): answer place-photo cache miss with 204 like the maps proxy
* test(server): close serving-swap gaps surfaced by the full tier
* test(storage): raise src/nest/storage branches pin
* refactor(place-photos): route place-photo cache through storage
* refactor(places): reclaim place images and collection covers through storage
* refactor(trips): trip covers through storage
* refactor(auth): avatar reclaim through storage
* refactor(collab): note-file deletion through storage
* refactor(files): trash deletion through storage
* refactor(memories): trek photo cache internals through storage
* refactor(memories): thumbnail service category-addressed
* refactor(memories): immich upload reads through storage; drop uploads-root
* refactor(journey): photo deletion through storage
* fix(memories): reclaim orphaned trek photo cache objects in the ttl sweep
* feat(journey): journey-thumbs orphan sweep and delete-site derived-artifact reclaim
* test(server): close slice-4 gaps surfaced by the full tier
* test(coverage): raise slice-4 domain pins
* refactor(backup): list, existence and delete through storage
* refactor(backup): download through storage sendToResponse
* refactor(backup): create backup through per-category storage enumeration
* refactor(backup): restore and upload-restore through storage
* refactor(backup): auto-backup retention through storage
* refactor(server): boot dir-creation to driver init
* test(server): close slice-5 gaps surfaced by the coverage tier
* test(coverage): raise slice-5 backup pin
* test(server): resolve async backupFileExists mocks for the tests typecheck
* test(server): pin ranged-read stat and zero-byte roundtrip in the storage driver contract
* feat(server): TREK_S3_* env wiring — schema, s3 namespace, all-or-nothing boot rule
* feat(server): S3 driver core — aws-lite client seam, stat/delete, driver-owned deadlines
* refactor(server): type the aws-lite S3 client via s3-types reference
* feat(server): S3 driver getStream and list — ranged full-object stat, iterator pagination
* feat(server): S3 driver put — bounded-peek routing, Upload inactivity deadline
* fix(server): S3 driver Upload monitoring — Transform interposition, stream cleanup
* feat(server): env-declared s3-main storage backend; settings-declared s3 rejected
* test(server): env-gated S3 driver contract suite against MinIO
* fix(server): S3 driver zero-byte put — work around aws4 duplicate Content-Length signing defect
* fix(server): scope the zero-byte put workaround to buffered bodies; bind test MinIO to localhost
* ci(server): MinIO-backed S3 driver contract job
* ci(server): align s3-contract shim comment and MinIO bind with references
* docs: S3 storage backend configuration and off-box backup mirroring
* docs: restore README formatting, keep S3 additions only
* ci(server): make live-MinIO contract runs manual-only (AIStor licensing)
MinIO's community line is discontinued entirely; the maintained successor, AIStor, requires a license key that can't safely live in a shared CI runner. The s3-contract CI job is removed; live-provider contract runs against MinIO/AIStor are now a manual, license-gated procedure documented in docker-compose.minio-test.yml's header.
* chore(server): final-review polish — stale CI refs, dockerignore key, boundary pins, zero-byte tempfile contract case
* expose admin ui
* feat(shared): storage backend type registry, wire schemas, category relocation
* feat(server): accept settings-declared s3 backends — shared-schema options, decrypt-at-build, registry preview()
* feat(server): remove TREK_S3_*/TREK_UPLOADS_DIR — tombstone boot rule, built-in override as the relocation path
* feat(server): deriveSecurity.encryptionKeySet and storage secret plumbing (gate, encrypt, unmask, mask, redact)
* chore(server): drop storage-secrets import that only Task 8's probe helper will use
* feat(server): seed-once storage-config.json boot import — fail closed on any invalid seed
* feat(server): registry snapshot() — effective storage config with provenance
* feat(server): api/admin/storage GET/PUT — masked secrets, preview-gated saves, wired reload, audits
* feat(server): api/admin/storage/test — per-target ephemeral probes, mirror targets probed individually
* test(server): e2e coverage for api/admin/storage — guards, managed refusal, envelope, probes
* test(server): give e2e harness DBs the app_settings table the storage registry reads at boot
* test(coverage): raise pins after the storage admin-config slice
* fix(server): final-review polish — encryption-gate wording, all-field sentinel guard, README/e2e notes
* test(server): storage slice-1 review follow-ups — mirror s3-replica probe decrypt, all-encrypted seed without key, test-route 401
* feat(i18n): storage admin domain + tab label in every locale
* feat(client): adminApi storage endpoints — getStorage, updateStorage, testStorageBackend
* feat(client): storage admin draft-document helpers + relative-time util
* feat(client): schema-driven storage BackendForm — fields by kind, mask echo, encryption banner
* feat(client): admin storage panel — backends list, category map, health strip, one-PUT save
* feat(client): storage tab on the admin page
* feat(client): storage admin panel and section on the phone shell
* fix(client): block renaming a storage backend onto an existing name
* feat(server): allow mirrors on any storage category — lift the backups-only rule
* feat(i18n): replicas-on-primary mirror strings in every locale
* feat(client): mirror fold/synthesize model helpers for replicas-on-primary
* feat(client): BackendForm mirror-targets composer; raw mirror type hidden
* feat(client): desktop storage panel folds mirrors into their primaries
* feat(client): phone storage panel folds mirrors into their primaries
* fix(client): strip removed backends from remaining mirror replicas — no phantom refs
* feat(storage): retire photos as a configurable category — served-only via the ServedCategory seam
* feat(i18n): descriptive storage category names and descriptions in every locale
* feat(client): descriptive category names, descriptions and id badges in the storage panel
* docs(storage): fix stale nine-category comments after the eight-category shrink
* docs: storage configuration via the admin UI and seed file — README + compose
* docs(wiki): Admin Storage page and storage cross-links
* docs(server): storage domain blueprint entry; TREK_PLACE_PHOTO_DIR tombstone clarifier
* refactor(server): drop the storage env-var tombstones — the variables never shipped
* docs: storage config is simply new in v4 — drop the removed-variables story
* feat(shared): backfill-status and usage wire schemas on the storage admin state
* feat(server): MirrorDriver.backfill — prefix-scoped, size-compared, cancellable replica catch-up
* feat(server): one-at-a-time storage backfill jobs + the storage events bridge
* feat(server): storage usage scan — nightly cron, on-demand service, persisted with computedAt
* feat(server): replica_failure admin notifications — events bridge, hourly debounce, channel registration
* feat(i18n): replica-failure notification text in every locale
* feat(server): storage backfill and stats-refresh admin endpoints
* feat(client): adminApi backfill and stats-refresh endpoints
* feat(i18n): storage sync and usage strings in every locale
* feat(client): desktop storage panel — sync now, usage display, backfill polling
* feat(client): phone storage panel — sync now, usage display, backfill polling
* fix(client): per-category usage line on the phone storage panel
* docs: sync now, usage stats and replica-failure notifications
* test(server): admin scope now includes replica_failure, not just version_available
Commit 6db135da added replica_failure to ADMIN_SCOPED_EVENTS but left
NPREF-005b asserting the admin scope contained only version_available.
Update the expectation to match.
* test(server): cover storage backfill/stats/jobs branches to hold the coverage pin
* fix(client): guard storage admin polling against stale responses racing a save
A poll GET issued before a save's PUT resolves could land after the save
applied its fresher response, overwriting it with the pre-save world and
re-deriving the draft from that stale snapshot. useStorageAdmin now tracks
a state-application sequence number and drops any refreshState() response
whose sequence has moved on, or that resolves while a save is in flight.
Also guard the fire-and-forget refreshState() calls in startBackfill/
cancelBackfill against unhandled rejections, matching the poll's style.
* fix(server): report 404 before 409 when a backfill start names an unknown mirror
startBackfill checked the global busy flag before validating that :name is
a mirror routed by some category, so POST .../backends/garbage/backfill
answered 409 instead of 404 whenever a sync was already running. Resolve
and validate the target first (BackfillTargetError -> 404), then check
busy (BackfillBusyError -> 409).
* feat(client): two-column category grid on the storage panel
* feat(server): plugin files.getContent reads through the storage layer
* feat(server): plugin files.create writes through the storage layer
* refactor(server): drop FilesService.resolveFilePath, last direct-fs seam in the plugin surface
* fix(server): map StorageInvalidKeyError to the plugin file accessibility envelope
* feat(storage): allow credentialed backends without an explicit ENCRYPTION_KEY
Secrets are still always encrypted at rest; without an explicit key the
implicit data-directory key is used. Removes the plaintext-secret refusal
from the admin PUT and the seed import, the encryptionReady GET field, the
save-blocking banner in both admin shells, and the now-consumerless
security.encryptionKeySet env derivation. Docs now recommend (not require)
an explicit key, since the implicit key rides inside backup archives.
* test(client): pin the stale-poll release to the pre-save GET in FE-ADMIN-STOR-024
The stub deferred EVERY poll GET and rebound the release handle to the
newest one, so a tick landing after the save was released with the
pre-save payload — a response no real backend can produce, which the seq
guard rightly applied. Only the first (genuinely stale) GET is held now;
later GETs answer with the world the mock server currently holds. Also
drops the encryptionReady fixture field removed from the schema in the
previous commit.
* fix(storage): backfill puts carry content-type, stat failures report op stat, progress clamps
* feat(storage): category migration job — copy, flip, delta sweep, reclaimable tally
* feat(storage): migration start/cancel endpoints and client API
* test(client): storage admin fixtures carry the new migrations field
* fix(storage): migration POST answers 200 and validates its body via the shared schema
* fix(storage): cancel running storage jobs when a config save removes their backend
* fix(notifications): in-app replica-failure row carries the suppressed count
* feat(client): migration prompt, queue, and progress on the storage panel
* fix(client): migration queue holds a synchronous single-flight lock
* feat(client): migration prompt and progress on the mobile storage panel
* fix(server): register category-migration routes in the managed-route allow list
StorageAdminController.migrationStart/migrationCancel were added under the
controller-level @ManagedForbidden() decorator but never added to
MANAGED_ROUTE_ALLOW_LIST, so validateManagedRoutes() threw "Managed surface
changed" on every buildApp() call, failing 46 test files (any test booting
the real app).
* test(server): cover category-migration copy/sweep branches to hold the storage coverage pin
Adds MIG-005 (a target object already matching size/key is skipped in the
copy phase itself, not just the delta sweep) and MIG-006 (a copy failure
during the delta sweep is counted without undoing the flip that already
happened). src/nest/storage/**'s branch coverage was 83.91%, just under the
84% pin.
* docs(storage): category migration — move existing objects on reassign
* fix(storage): close the pre-flip cancel window and cancel re-routed migrations
A cancel landing between the last copy iteration and the category flip was
silently ignored, and a config save that re-routed the migrating category to
a third, still-defined backend never stopped the in-flight migration. Also
guards assignCategory against writing an unknown backend name, and fixes a
missing `suppressed` notification param being treated as truthy suppression.
* fix(client): migrations keep mirrors, queue waits on backfills, sweep failures surface
Reassigning a category onto a mirrored primary now posts the migration to the
mirror's wire name instead of the bare primary, so replication survives the
move. The queued-migration effect also waits out a running backfill (the
server's one-job rule spans both), and a done migration with sweep failures
now surfaces a doneFailures line in both shells, translated into every locale.
* ci(server): run the S3 contract suite against MinIO
Resurrects the s3-contract job removed in 6fbff835, adapted for AIStor:
writes the MINIO_LICENSE_KEY secret to a license file, boots the licensed
container, and runs the contract suite hermetically (no --coverage) so the
storage coverage pin keeps holding from unit tests alone. Gated to same-repo
PRs since fork PRs have no access to the license secret.
* fix(server): backup archives uploads from a remote primary
createBackup() enumerated uploads through withLocalFile, which downloads a
remote object to a temp file, hands it to the callback, then deletes it
before archiver ever reads it — archiver reads entries lazily during
finalize(), so a remote (S3) primary silently backed up zero upload bytes.
Add StorageService.getLocalPathOrNull(category, name): a narrow locality
probe that returns a real path only when one exists on disk right now (the
fs-existence check is the fail-safe half of the contract — a driver-reported
path that vanished between list() and here falls back too, not just a
path-less remote driver). The enumeration loop uses it directly when
available (zero-copy, unchanged for a local primary) and otherwise streams
the object into a per-backup staging dir under the backups spool, cleaned up
in the existing finally alongside zipSpool/dbSnap.
Also wire archive.on('warning', reject): archiver treats a missing/unreadable
entry as a warning, not an error, and skips it by default — which would
silently drop the entry from an otherwise "successful" backup.
* fix(server): client aborts stop 500ing and leaking streams
storage.service.ts sendToResponse: the local res.sendFile branch now treats
ECONNABORTED/ECONNRESET/ERR_STREAM_PREMATURE_CLOSE/syscall=write as success,
mirroring res.sendFile's own default callback (the contract
storageStaticHandler already implements caller-side). The remote branch
replaces the hand-rolled pipe+Promise with pipeline(stream, res), which
destroys the source stream on any failure — fixing the leaked S3 keep-alive
socket on a client abort — while swallowing the same client-abort error set;
a real source error still rejects so callers keep their miss contracts.
This centralizes the fix for every caller that streams through
sendToResponse: files-download, journey-public, photo-resolver,
trek-photo-cache, backup downloads, and /uploads/photos.
trek-exception.filter.ts now no-ops (destroying the socket) when headers are
already sent, so no filter path ever writes onto a response in flight.
maps.controller.ts and share.controller.ts's place-photo bytes proxies
switch their direct getStream+pipe to pipeline() with the same abort-tolerant
catch, keeping the existing 204-empty fallback for pre-header failures.
* fix(storage): mirror read fallback consults replicas on primary miss
MirrorDriver.getStream/stat now fall back to replicas on a primary miss
(StorageNotFoundError / null stat), not just an infra error — binding
decision: a read succeeds if ANY member holds the object. Only a
deterministic StorageInvalidKeyError skips fallback.
stat() gets a dedicated fallback loop instead of reusing fromReplicas:
a replica returning null means "keep looking," which the generic
catch-and-continue helper would have short-circuited on.
StorageService.sendToResponse/withLocalFile stop treating composite
driver.stat() as proof the object is at localPath; both now fs-check
the path itself and fall through to the stream branch when it's
missing, which is what actually reaches a MirrorDriver's replicas.
* feat(storage): remote serving parity — content-type, validators, range
Local backends get content type, ETag/Last-Modified, conditional GET, Range
and HEAD for free from res.sendFile; the proxying branch every remote
backend (and a local primary whose file vanished) falls through to had none
of it — every request re-streamed the whole object with no cache validator
and no way to seek.
sendStreamed() reproduces the contract from ONE pre-flight driver.stat():
a miss throws before any header is set (the caller's 404/204/next()
contract), and that same stat then answers 304s, 416s, HEADs and suffix
ranges without ever opening a body. The decisions live in a new pure module,
http-serving.ts:
- validatorsFor: the driver's ETag, else express/send's weak
W/"<size hex>-<mtime hex>". mtimeMs 0 (S3 can report it) means no
Last-Modified and no derived tag — serving uncached beats a 1970 header
that reads as fresh forever.
- isNotModified: If-None-Match (weak compare, *, list) wins over
If-Modified-Since, per RFC 9110 §13.2.2.
- decideRange: single range only, inclusive clamp, suffix computed from the
pre-stat size, unsatisfiable → 416, If-Range strong-exact or full entity.
A multi-range or malformed spec is answered with the full object rather
than multipart/byteranges. If-Match/If-Unmodified-Since stay out of scope.
Range math uses the pre-stat size as the single source of truth, the ranged
getStream stat is never re-consulted, and the pipeline abort semantics are
unchanged.
S3Driver now surfaces the bucket's ETag on stat()/getStream() (on a ranged
GET it is still the full object's tag). aws-lite strips the tag's
surrounding quotes — verified live against AIStor RELEASE.2026-08-07 — and
an unquoted entity-tag is not a valid HTTP ETag, so the driver restores
exactly the quotes the SDK removed and nothing else.
* fix(server): key rotation covers storage.backends
The S3 storage backend's secretAccessKey is encrypted at rest inside the
app_settings row keyed storage.backends, but scripts/migrate-encryption.ts
didn't know about it — after a key rotation that secret stayed encrypted
under the old key and silently failed to decrypt on the next read.
The script now re-encrypts secretAccessKey for every s3 backend inside the
storage.backends JSON array, using a pinned copy of the secret-field list
(it deliberately can't import @trek/shared). The parity test pins that copy
against storageSecretFields('s3') so a new secret field can't be missed
silently.
* fix(server): restore reloads the storage registry
restoreFromZip closed and reopened the DB but never reloaded the
storage registry, so uploads rehydration ran against the STALE
pre-restore driver/category map instead of the config the restored
archive's own travel.db carries (audit #4, major).
StorageService gains reloadConfig(), a narrow passthrough to the
(unexported) registry's reload() — same shape as health(). backup.impl's
restoreFromZip now calls it right after reinitialize() and right before
the uploads wipe/rehydrateUploads/plugin staging, so rehydrated bytes
land where the restored config says. When reopen fails (reinitFailed),
reload and rehydration are skipped entirely — already reported as
"restart required", and there is no live DB handle for the registry to
read anyway.
* fix(server): category migration rewrites keys across prefix changes
Category migration copied objects to the destination backend using the
SOURCE key verbatim. A mode-A ('' prefix) source migrating onto a
prefixed backend (or vice versa) landed objects at keys the registry
could never resolve post-flip (audit #8).
Extracts StorageRegistryService.keyPrefixFor(category, backendName) from
build()'s inline conditional so startMigration can compute the
destination prefix before the flip. runMigration now maps every
destination key (destPrefix + key.slice(sourcePrefix.length)) in the
copy skip-check, the put, and the delta sweep; the reclaimable tally
stays source-keyed. Equal-prefix migrations are unaffected (identity
mapping).
* fix(plugins): await the now-async place/collection services in RPCs
PlacesService.update/remove and CollectionsService.deletePlace became
async when they started deleting storage objects, but their plugin RPC
handlers still called them synchronously: the null/false refusal check
compared against an always-truthy Promise, and places.update broadcast
a Promise instead of the resolved place. Both handlers now await the
service call; mapCollectionError is async-aware (accepts a sync-or-
async thunk) so the sync callers (create/update/savePlace/copyToTrip)
keep working through the same path. Verified the RPC host already
awaits handler results, so no change was needed there.
* fix(storage): backfill Sync now gains a deletion sweep
The Sync now backfill copied primary objects to replicas but never removed
replica objects the primary no longer has, leaving mirror read-fallback able
to serve stale data indefinitely after a failed replica delete. backfill()
now sweeps each replica post-copy: any replica key absent from the primary
key-set is re-checked against primary.stat (a race gate against the primary
regaining the key) before being deleted. Adds a deleted counter end-to-end
(driver, jobs service, shared schema, both admin shells, all 23 locales) and
a new ReplicaFailure 'list' op for a replica whose listing itself fails
during the sweep.
* feat(storage): optimistic concurrency for admin config saves
A stale admin PUT of storage settings could silently undo a category
migration's just-flipped assignment. Adds a version counter
(app_settings row storage.config_version) bumped inside the same
transaction by both writers — applyConfig and the migration flip
(assignCategory). applyConfig compares the submitted version first
and throws StorageConflictError, mapped to 409 ahead of the blanket
400. The client carries the draft's loaded version through every
local edit and maps a 409 to a distinct "settings changed underneath
you" message, refreshing state while preserving the dirty draft for
review.
* fix(storage): surface a failed config load + redact secretAccessKey
Registry load() now records the exact parse/validate failure as
lastLoadError() instead of silently falling back to last-good config or
built-in defaults; StorageAdminService exposes it as configError on the
admin state (storageAdminStateSchema extended additively), and
AdminStoragePanel renders it as a warning banner naming the error. Save
stays enabled — it's the recovery path — but an admin can no longer save
over a broken stored config without knowing.
Also adds 'secretaccesskey' to globalMiddleware's SENSITIVE_KEYS request-log
redaction list, so storage admin PUT bodies never leak the S3 secret into
debug logs.
* fix(client): admin storage panel state races
Migration prompt now recomputes computeMigrationCandidates(draft, state)
on render and again at confirm time, so a stale open-time snapshot never
drives stripCategories or the migration queue; adds a Cancel button to
the dialog. A failed startMigration POST explicitly clears the remaining
queue with a toast naming the dropped categories and refreshes state,
instead of leaving it invisibly stranded; a "Queued: X, Y" line renders
while the queue is non-empty. Testing a mirrored row now expands the
draft primary + replicas client-side and probes each individually,
instead of sending the mirror stub to the server (which resolved
targets from its own live snapshot, silently testing saved options over
unsaved draft edits). A completed/cancelled/errored backfill now renders
its terminal status alongside the Sync now button instead of hiding it,
so it can be re-run without a page reload.
* fix(storage): refuse shared replica backends so the sync sweep cannot cross categories
A mirror's "Sync now" sweep makes each replica MATCH the primary: it lists the
replica under every prefix the mirror's categories occupy and deletes the keys
the primary doesn't hold. The backups category's prefix is '' — the replica's
entire root — so making a category-serving backend (say uploads-local) a
replica of a backups mirror meant the first sync deleted every user upload.
Nothing forbade the config: validateConfig only checked that mirror targets
existed and weren't nested, and the admin picker offered every backend.
The sweep is correct, so the fix is at the config level:
- validateConfig refuses a backend that is both a mirror replica and a
category target (directly, or as the primary of a mirror a category routes
to), and refuses a backend replicating two mirrors whose swept prefixes
overlap — '' overlaps everything. Disjoint prefixes stay legal.
- assignCategory (the migration flip, the other writer of the settings rows)
refuses routing a category onto a backend that is currently a replica.
- The admin mirror-target picker (both shells) only offers backends that serve
no category; already-selected targets stay listed so they can be unchecked.
A category assigned to a replica still surfaces the server's 400 verbatim.
Existing shared-replica fixtures were rewritten onto dedicated replicas.
* fix(client): a 409 conflict save is recoverable without a page reload
After a 409 the admin storage panel was a dead end. refreshState leaves a
dirty draft's version alone on purpose (audit #7: the background poll must
never float a mid-edit draft past a migration flip it never carried), setDraft
re-attaches that same version to every later edit, and save() resubmits it —
so every retry 409s again and only a full page reload got the operator out,
while the banner told them to "review and save again".
The version block stays. Instead the conflict banner now offers the one action
that can clear it: discard the local edits and reload the saved settings —
a fresh GET applied through applyState, re-deriving the draft (and its
version) and clearing dirty/saveError. Bumping the version in refreshState,
or only in the 409 branch, was rejected: either resurrects audit #7, since a
full-document save would then revert a flip the draft never carried.
storage.saveConflict now points at that action instead of implying a plain
re-save works (all 23 locales). The escape lives in the shared hook, so the
phone panel renders it too.
* fix(client): deflake the queued-migration-waits-out-backfill panel test
FE-ADMIN-STOR-037 seeds a running backfill, so the panel polls GET from
first render — but the stub served the post-save world (files: off-box)
before the PUT had landed, which no real server would do. A poll
resolving while the confirm dialog was open made the confirm-time
candidate recompute see from === to, so nothing was queued and the
migration never POSTed.
The GET stub now stays faithful (pre-save world until the PUT lands,
usage included so candidates keep their object counts), and the test
deterministically forces the once-flaky interleaving: it waits for
polls to land while the dialog is open and asserts the candidate
survives them.
* fix(storage): surface the underlying cause in probe target errors
The admin storage Test probe reported only the driver wrapper's outer
message ("put failed for '…' on 'probe-s3'"), dropping the cause that
says WHY — connection refused vs missing bucket. Flatten the error's
cause chain into the probe result, appending an S3-style code when it
isn't already in the message.
* fix(storage): surface the underlying cause in replica-failure reports
Same defect as the probe fix, one layer over: MirrorDriver's
reportReplicaFailure captured only the wrapper's message, so
replica-failure notifications said "put failed" without the why.
Move the probe's cause-chain flattener into storage.types.ts as
describeError and use it in both call sites.
* fix(storage): strip ContentType from multipart Upload calls (MalformedXML)
Every multipart S3 put carrying a contentType — every backup zip and
photo over the 10MB threshold — failed with 400 MalformedXML. Upstream
@aws-lite/s3's Upload leaks the ContentType param into its final
CompleteMultipartUpload headers ({ ...xml, ...paramHeaders }), clobbering
application/xml, which flips @aws-lite/client's payload serializer to
JSON — so the server receives a JSON parts list where XML belongs.
Confirmed live against MinIO AIStor; any S3 refuses the same.
uploadMonitored now drops ContentType before s3.Upload. Multipart
objects are stored without a content-type, which TREK never reads back
(serving derives Content-Type from the key/caller, never object
metadata); single-request PutObject paths keep sending it.
Also closes the coverage gap that let this ship: the live contract
suite never crossed MULTIPART_THRESHOLD, so s3.Upload had never run
against a real server — it now round-trips a >10MB put carrying a
contentType.
---------
Co-authored-by: Maurice <mauriceboe@icloud.com>
The variables only take effect when Node runs with NODE_USE_ENV_PROXY=1, and
that is set in the image. This page also covers Helm, Unraid and source
installs, where nothing sets it, so following it there changed nothing.
Also recommend NO_PROXY. Without it the proxy sees every request TREK makes to
itself, including the container health check.
Maurice