Commit Graph

15 Commits

Author SHA1 Message Date
jubnl 6ff45be032 test(server): plan 4 — collapse nest service test harnesses onto the ORM snapshot 2026-09-24 04:47:29 +02:00
jubnl f337f116d9 fix(server): plan 3h close-out — new-value-wins COALESCE restores stay/reservation propagation; doc-sync state filter truthiness; request-context and parity ratchets for share, doc-sync and the integrations 2026-09-24 02:39:30 +02:00
jubnl 8bcfd92142 refactor(doc-sync): connections, reconciler and controller-SQL relocation through repositories 2026-09-24 00:40:23 +02:00
jubnl ad6e1ce8f0 fix(vacay): type the loop accumulators the async sweep introduced 2026-09-22 03:44:48 +02:00
jubnl cb177ce655 refactor(vacay, admin, doc-sync): async DB-touching methods; transactions through UnitOfWork 2026-09-22 03:29:33 +02:00
jubnl 0cf99a9939 refactor(realtime, systemNotices, addons, airports, audit, categories, permissions, query-helpers, tags, transit, trip-membership): async DB-touching methods; transactions through UnitOfWork 2026-09-21 20:48:03 +02:00
Maurice ed18c37903 fix: the findings of the final 4.3.0 audit
Every file the release touches was read once more, and what survived an
independent check against the code is fixed here, with a test for each.

Data and credentials
- A document sync connection edited to a new address no longer carries the
  stored token along: a blank form under another origin is refused, the same
  rule the probe route already had. Dawarich and AirTrail testConnection send
  the stored key only to the host it was stored for.
- The key rotation script now covers document_connections.secrets and
  trip_document_links.webhook_secret, which it had skipped.
- Turning a booked night into a pause in the road trip popup asks first and
  says that the booking and its expense go with it. Deleting a place says the
  same when a night is booked there, on the desktop and on the phone.
- A moved version marker over unchanged bytes no longer replaces the TREK file
  and its attachments; a real revision carries the attachments over.

Planner
- A drop in the day plan names the row it lands ahead of in the full day, so a
  hidden hotel stop no longer shifts it one slot early; the store sorts before
  it splices. Via re-anchoring, the booked-night guard and the sidebar's day
  membership read the stored day, the index space the server uses.
- A hotel stop placed mid-day re-pins the road trip vias of that day, and the
  booking session receives the new stop order too. The Public API and the iCal
  feed list a booked hotel once. A plugin can no longer write a stay against
  another trip's day.

Road trip and map
- The route overview paces its requests to the public routers and retries a
  rate limit, and it says which legs it could not route instead of quietly
  reporting a shorter trip. A click on a via handle no longer drops a second
  via; handles hide when zooming out. Read-only members cannot edit the driving
  limits. The phone shows a way back when no day is picked and prints the night
  drive departure in the chosen clock format.

Integrations and search
- Google transit keeps conventional trains when the train chip is on.
  TREK_PLACES_ENABLED=false stops the details and enrichment calls too, and
  accepts the whole boolean family. A caller-supplied description no longer
  lands in the instance-wide enrichment cache. Nominatim searches carry a
  timeout. An OSM place id is checked for shape before it reaches Overpass.
- Dawarich puts a stay on the trip whose dates hold it, the Atlas hand-off
  knows what the Atlas already shows, and a host change with a new key clears
  the old instance's state.
- Document sync: a webhook address pasted by hand works on the token alone,
  deleting a connection unregisters its subscriptions, Nextcloud under a
  sub-path connects, Synology answers a name collision before uploading and
  verifies a move into the bin, Paperless titles are capped at 128 characters,
  a DNS failure reads as unreachable rather than as a policy verdict, videos
  from the store are accepted like uploads, and remote_missing has a way out.
  Conflict buttons only for the owner, a refused choice is reported, a refused
  credential gets a Reconnect button, disconnecting asks first on the desktop
  as it does on the phone, and the folder picker no longer spins forever.

Release
- The server holds the release notice back from a bundle that cannot draw it,
  so the previous shell served from the service worker cache after an update
  no longer shows bare keys and no longer lets a click use the notice up.
- Import into an existing list gets the same body limit as a new list. Chat
  images say when the limit is reached and the new controls are translated.
  The transit backend switch reports a failed save. The Helm chart carries the
  Amap credentials in its Secret; the shipped templates document the Places
  API and Amap variables and the Overpass default. The wiki follows.
2026-09-20 13:54:31 +02:00
Maurice 3ad40e056d fix(docsync): an orphaned binding stays stopped on every path that starts a run 2026-09-19 21:57:04 +02:00
Maurice 233f46575c fix(docsync): make the admin switch pause a binding, and retry a failed update
The check that was meant to stand a binding down when its provider is
switched off asked AddonsService.isAddonEnabled(link.provider_id). The
addons table has no rows for paperless, papra and the other document
providers, so it answered "off" for every binding, and every run came
back disabled before it listed anything. The unit test stubbed that
answer and the e2e never runs a sync, so nothing caught it.

isSwitchedOff() now asks the two things an admin actually switches: the
Documents addon, and document_providers.enabled for the binding's
provider. dueLinks leaves such bindings out in SQL, since one that
never runs never gets a last_sync_at and twenty of them would take
every slot of a tick. A run, the webhook on arrival and when its timer
fires, "Sync now" and the MCP sync tool all refuse before anything is
written, so a paused binding keeps its state and its documents' attempt
counters and resumes where it stopped. The REST route answers 409
provider_disabled, the MCP tool reports the same code, status() carries
providerOff per link, and both shells show one line saying the binding
is paused because its provider is switched off, in all 23 languages.

A failed pull_update also left the row stuck. upsertItem wrote the
listing's version, size and time next to the old bytes, so the next run
saw no upstream change, planned touch, which leaves error alone, and
the document stayed in error on stale bytes for good, "Sync now"
included. A failed transfer now leaves the pairing describing the copy
TREK holds; a new row keeps what pairs it but no version, so it is
fetched again.
2026-09-19 15:12:47 +02:00
Maurice 843582863c fix(docsync): offer the sync button only where something is behind it
The Documents addon put a "Document sync" button on every trip's Files
tab, and the seed ships every provider switched off, so after the
upgrade each of those buttons led into "No document providers are
available". The button now shows on a trip that is already bound, to
every member, and otherwise only to somebody who may bind the trip on
an instance that offers a provider. Without it the Files toolbar is
the one from before the feature, trash flush right, on both shells.

/api/addons carries no document providers for a normal user, so the
answer comes from the docsync routes through a new online-only
docSyncRepo and is kept per session in docSyncOfferStore. A revisit
renders the last answer at once and checks it again; offline nothing
is asked and the last answer stands. Creating a binding, removing one
and removing a connection now ping the trip with docsync:changed and
zero counts, so a member already on the Files tab sees the button come
and go without leaving it.

Both shells keep listing a binding whose provider the admin has since
switched off instead of claiming there are no providers, and name its
store through one storeName() helper. A load that answers 403 or 404
clears the bindings as well, so they do not linger once the Documents
addon is gone.

The phone admin screen gets the Documents shelf the desktop one has:
the document providers as sub rows under the addon, toggled through the
same admin route and re-read after Documents itself is toggled, because
the server switches the providers off with it.

isTripOwner and the isOwner prop become canManageSync and canManage,
which is what canManageDocSync answers. Both folder pickers take
loadScopes out of the sync object before their effect, so
exhaustive-deps is satisfied without the effect running on every
render.
2026-09-19 13:36:33 +02:00
Maurice 25d4e34959 fix(docsync): keep the Synology device token across a restart, tied to its row
The DSM trusted-device token lived in process memory only. After a
restart every token was gone and the stored one-time code long spent,
so every two-factor NAS binding stopped syncing until its owner typed a
fresh code, and nothing on screen said why.

The token is now kept in the connection's encrypted secrets, together
with a digest of the instance and account it was issued for. A saved
connection's ref carries a saveSecret callback that writes through
DocSyncConfigService.saveEarnedSecret, which rereads the row inside a
transaction so a sync run holding an old ref cannot undo a password
the owner changed meanwhile. The key belongs to no form field, so
maskSecrets never reports it and mergeSecrets never touches it: the GET
and PUT payloads do not carry it, an edit keeps it, and a client cannot
plant one. It survives a password edit and a spent or cleared code. A
connection repointed at another NAS or account drops it on its next
call, and a token DSM refuses with 403 or 404 is dropped in memory and
in storage. A probe of form values and an unsaved form may use what is
stored but never write. A refused token no longer shadows a fresh code:
the code in the form gets its turn in the same login call.

In memory the slot is keyed by the connection id together with the
row's created_at. A backup restored in place or the demo reset rolls
the id sequence back without a restart, and a new connection that
reused an id must not log in on the token of the row it replaced.

The Synology adapter receives its client through DI like Paperless and
WebDAV, instead of building a second instance beside the one the module
registers.

Two readings of a failed request are settled as well:
- INVALID_PURPOSE, INVALID_CA and PATH_LENGTH_EXCEEDED count as an
  untrusted certificate again, so the settings screen offers the
  self-signed switch. Papra used to find them through the message text,
  and the shared classifier reads codes only.
- A host that never completes the TCP handshake reads as unreachable on
  every client. WebDAV said timeout before the shared transport; the
  change is now documented and tested.
2026-09-19 13:36:16 +02:00
Maurice b88b92add7 fix(docsync): a paused binding no longer resets its own settings
PATCH links validated its body with a schema derived from the create
contract through .partial(). In Zod 4 a .default() still fires inside an
optional field, so a patch naming one setting arrived with every other
one filled in: pausing a pull-only binding switched it to two-way, reset
both policies and blanked its label. The update contract is now written
out without defaults, and an e2e test drives a patch through the real
pipe, which the controller unit tests never see.

Also, the cosmetic leftovers from the review:
- POST connections/:connectionId/scopes no longer wants the connection
  in the body as well. It was required, never read, and let one request
  name two connections; a client that still sends it has the key
  stripped rather than refused.
- createLink loses the scope argument every caller passed as null.
- The controller's owner check is now assertCanManage, because it lets
  instance admins through just as canManageDocSync does on the client.
  Who may manage a binding is unchanged.
- document_provider_fields drops settings_key and payload_key. Nothing
  read them: the connect form takes its fields from /docsync/providers
  and the admin shelf sends none for document providers.
- The e2e test imports the provider classes under their new names.
2026-09-19 12:07:28 +02:00
Maurice 4fa06c3483 fix(docsync): an idle WebDAV binding read as an emptied folder, and two ids nobody checked
The audit's worst find, reproduced against a live Nextcloud before and after the
fix: the WebDAV adapters answer "nothing changed here" by returning an EMPTY
listing plus `cursorUnchanged`, and nothing read that flag — `grep -rn
cursorUnchanged src/` found it set by four adapters and consulted by none. The
core therefore saw a folder that had lost everything. Run 1 and 2 look fine
because the run's own uploads move the ETag; from run 3 on, every run of an
idle binding ends `partial` with `mass_delete_guard` and six documents reported
missing, forever, until failure_count opens the circuit and the scheduler drops
the binding entirely. That is the ordinary case — a binding nobody is touching.
planReconcile now takes `remoteUnchanged` and plans only what happened in TREK.

Two ids were authorised by trip and never checked against it:

  - resolveConflict took an item id on trust, so the owner of any trip could
    decide a conflict belonging to somebody else's. Both are plain integers and
    nothing compared them.
  - The connection test merged the STORED credential into a probe of whatever
    baseUrl the caller named — a way to have TREK post another account's API
    token at a server of the caller's choosing. Stored secrets are now reused
    only for the address they were stored against.

The phone's connect form was wrong in three ways at once, each of which the
desktop had right: it read the address as `baseUrl` while the field is
`base_url` (so every connection it saved carried an empty address), it drew
`allow_insecure_tls` as a text box and always sent false, and it built label
keys as `docsync.provider${cap(label)}` when the label column already holds the
full suffix — `docsync.providerProviderApiKey`, rendered raw on screen. Those
rules are not markup, so they now live in `useConnectForm`, which both shells
use. Its test fixture invented the field shape and so agreed with the broken
sheet; it now mirrors document_provider_fields.

Also: the scheduler read its interval once at bootstrap while this file's own
comment promised a per-tick read, so changing it did nothing until a restart;
and dueLinks ordered by a column that a successful run clears, which left every
binding tied and SQLite answering in insertion order — on a database with 170
bindings the same twenty ran on every tick and nothing created after them was
ever synced automatically.
2026-09-19 00:07:33 +02:00
Maurice 1430056d24 fix(docsync): read the retry limit back out, debounce the webhook, and cover the UI
The attempt counter and the backoff were written on every failure and never
consulted when planning the next run: rows were found in a live database at 46
attempts against a limit of 6, so a document a provider refuses — one it reads
as corrupt, a type it will not take, a full quota — was re-uploaded on every
cron tick forever. planReconcile now skips a row that is shelved or still inside
its window, and a manual run clears the counters, which is what "waits for a
person" has to mean.

The webhook had the same shape of bug: WEBHOOK_NUDGE_DEBOUNCE_SECONDS existed,
with a comment explaining it, and nothing read it. Providers fire once per
document, so twenty files dropped into a watched folder were twenty runs — the
in-flight guard answered `busy` to nineteen of them, throwing away nineteen
announcements, and the one run that did start had begun before most of the
changes landed. A burst now folds into one trailing run, and the binding is
re-read when the timer fires so a sync switched off in between does not get a
last run out of a stale row.

Four defects the new suites turned up, all fixed: the sync button lost its
accessible name below sm, where its label is hidden; the stored address was
never shown in the connect dialog although the form kept submitting it; the
allow-self-signed switch only read the connection at mount, so a stored "on"
showed as off and the next save wrote that back; and a refused folder create
escaped as an unhandled rejection with nothing said to anybody.

Also here: the phone shell gets the same feature (one sheet, four views, the
same hook as the desktop), the marks come from the product's own icon files
rather than redrawn approximations, and the 22 non-English locales carry the
new keys.

ADMIN-SVC-069 asserted that `documents` has no MCP surface. It does now, so the
test takes its example from MCP_GATED_ADDON_IDS instead of naming one.
2026-09-18 22:44:45 +02:00
Maurice 2d7cb124b3 feat(documents): two-way sync with Paperless, Papra, Nextcloud, OpenCloud and Synology
A trip can be bound to a folder, tag or space in a self-hosted document store,
and documents then move in both directions: what someone uploads in TREK appears
there, and what someone files there appears in TREK.

Three decisions shape the rest.

The connection belongs to the TRIP, not to a person. Every other integration
here is per user, and photos are shared only through an opt-in flag; neither can
deliver "everyone on the trip sees the same documents", because visibility would
depend on whose credentials fetched a file, and a member without their own
Paperless account would see nothing. So the trip owner binds the trip once, the
server talks to the provider under that one identity, and TREK's own membership
decides who sees what. The provider never learns that TREK has members. When the
credential holder leaves the trip the binding goes to `orphaned` rather than
quietly carrying on with an ex-member's token.

TREK keeps the bytes. A synced document is an ordinary trip_files row, so
download, trash, offline cache, PDF export and backup keep working untouched,
and a provider outage does not blank out a trip that starts tomorrow.

Deletion never crosses the boundary on its own. A document that disappears
upstream is recorded as missing and waits for a human, and a run that sees more
than 40 percent of its known documents vanish at once refuses the whole listing
rather than acting on it: an unmounted share answers exactly like a mass delete.

The core reconciles by enumerating both sides, because the weakest provider sets
the model. A changed-since query cannot see a deletion, Papra does not move
updatedAt when a tag changes, a Paperless bulk edit leaves `modified` alone, and
Synology FileStation has no change feed at all. A webhook is therefore only ever
"look now", never a source of truth. Echo is caught by comparing content, not
timestamps: pushed_sha256 records what TREK itself wrote, so a provider
reporting TREK's own upload is not mistaken for a stranger's edit.

Provider differences are answered by capability data rather than by branching on
an id: stable ids where they exist (Paperless document id, oc:fileid, driveId)
and a normalised path where they do not, replace-in-place where the API offers
it, and a content hash from the listing only where the provider really supplies
one.

Admins switch providers on under Admin, Addons, Documents, mirroring how photo
providers hang off Journey, including the cascade and the 409 when the parent
addon is off. The per-trip setup lives in the file manager, where the person has
the context that makes the folder choice obvious.
2026-09-18 19:23:07 +02:00