Commit Graph

1311 Commits

Author SHA1 Message Date
Maurice a67361c6bd fix(mail-ingest): read on to To when a delivery header names no address
Stalwart writes Delivered-To as the mailbox itself, its sub-address dropped,
so a forwarded mail to trek+<token>@ was refused as unknown_address: the
first delivery header won even though it carried no token. The headers are
now read in their order of trust, and the first that names a live address
decides.
2026-11-13 20:57:55 +01:00
Maurice 7a40225852 feat(llm-parse): read bookings on a ChatGPT plan
A ChatGPT sign-in only ever ran the assistant: parsing's clients call an
endpoint with a key, and a plan has none. AI connections now lends the plan
as a reader. Each document is one Responses call with the schema as the one
tool the model has to call (tool_choice required). The token is read and
refreshed per call by ChatgptAuthService, and a sign-in that has to be
renewed lends nothing.

- ResolvedLlmConfig is a union: ApiLlmConfig (what an admin configures) or
  PlanLlmConfig (provider chatgpt, a read function, never stored).
- ConnectionReadClient sends the text plus photo or scanned pages, and reads
  the filled shape, or the shape written as text; prose is unreadable.
- AiChatClientService.chatgptPlan is the one place that builds the plan's
  client, for the assistant and for parsing.
- Wiki: a ChatGPT sign-in reads bookings on its plan.
2026-11-13 20:49:08 +01:00
Maurice 29816f81a9 fix(mail-ingest): send an uploaded .eml as multipart; publish the booking origin as a contract
- The upload posted its FormData through the client's JSON default, so the
  form went as JSON and the server saw no file (400). It goes through
  postMultipart now, like every other upload.
- reservation-origin.ts becomes reservation-origin.contract.ts: the booking
  import sets an origin, and a contract is what another domain may import.
- CI: the client's strict type check holds the whole program with strict
  checks and outgrew the default ~4 GB heap; it gets 8 GB like the server's.
2026-11-13 20:49:08 +01:00
Maurice 4337969c2c fix(mail-ingest): the settings card makes the person's own address on first look
The own address was made only when asked for, and the card had no way to ask
before one existed, so it showed none. Once a template is set and the
person's import is on, the first look provisions it (idempotent; read after).
2026-11-13 20:32:31 +01:00
Maurice 6789ee1cb6 refactor(server): keep the grown files at their size baselines; centre the inbox dialog
- auth.service: how an admin form value is stored (validated, encrypted, a
  masked echo kept) moves to the app settings registry as storedAdminValue
  and isMaskEcho.
- reservations.service: a booking's origin (withoutOrigin, originColumns)
  moves to reservation-origin.ts, its input shapes to reservation-inputs.ts,
  exported on from the service so importers stay as they were.
- Reservations.repository: the joined projection, the origin lookup and the
  origin columns move to _shared/, like the Wallet pass context before.
- test-sql: MailIngestApplyService.prepare is marked as no SQL statement.
- The pending bookings dialog opens centred, and says 'nothing waiting' once.
2026-11-13 20:28:17 +01:00
Maurice 89b67a791f fix(mail-ingest): Outlook HTML forwards, the default AI limit, secret env vars, and the pipeline tests
- mime-walker: trim each line of a stripped HTML body, so the space a tag
  leaves is not read as a folded header line that swallows the booking
  under an Outlook forward's header block.
- settings: an unset daily AI limit is the default of ten, not zero
  (Number(null) is 0), which kept the AI from ever reading a mail.
- env: MAIL_INGEST_IMAP_PASS and MAIL_INGEST_WEBHOOK_SECRET are secrets
  and never printed in the boot report.
- pipeline test: the subject is kept without its forward prefixes, the
  demo move adds its booking before the second trip exists, the typed
  notification mock, and AI reading stays offered while the person
  switched it off.
- e2e: the pipeline tick runs in a request context of its own, as the
  cron registrar runs it, and the 401 checks run without awaiting in a loop.
2026-11-13 19:31:13 +01:00
Maurice 2ee413368b fix(mail-ingest): AI reading can be switched back on, and its hint names the addon
ai_available folded the person's own switch in, so switching AI reading off
greyed the switch out and it never came back. It now says only whether AI
reading works on the instance (the AI Parsing addon and a connection). When
it does not, the settings card says it needs the AI Parsing addon, by its
localized name, in all 28 locales; the wiki says the same.
2026-11-13 19:29:48 +01:00
Maurice 53833fd212 fix(mail-ingest): server gates after joining dev, and smaller admin buttons
- lint:tx: the mail transports and the intake document why their writes stand
  alone (@txIndependent, network I/O between them, one copy per address); the
  JSDoc of NotificationsService.respond is back above it after the response
  actions were registered in between.
- lint:boundaries: booking-import asks McpToolGuardsService whether the person
  may edit the trip's bookings instead of reading the Users table; its unit
  test builds the service through one named factory.
- lint:query-api: findIdByExternal reads through Kysely.
- lint:test-mocks: the mail e2e hands its own snapshot db to the ORM instead
  of mocking the database module.
- Entities regenerated (index order only).
- Client: the review row reads the live booking, the shells share
  useMailInboxShell, and the admin card's buttons are one size down.
2026-11-13 18:58:55 +01:00
Maurice bd11cd347d fix(mail-ingest): type errors and formatting after joining the flight work
The cron callbacks hand back nothing; the cursor's lookup is findCursor so it
no longer shadows the base repository's find(); the draft's suggested trip
uses the shared contract's type; endpoints spell out their optional columns;
the relay page's running count is typed. Prettier over every new file outside
the generated entities.
2026-11-13 18:03:14 +01:00
Maurice 44f669cd0e feat(mail-ingest): tell every address holder once when the import stops
A plan without the import switches it off from outside the server, and an
admin can switch it off too; the plan promises a "mail import paused" note.
An hourly job sends mail_ingest_paused to each holder of a live address with
their own import on, notes the pause first, and forgets it once it is back.
2026-11-13 17:44:55 +01:00
Maurice 1d052f725f fix(mail-ingest): send mailInbox:settings, and give both events their client place
The settings event was registered with no sender; renewing or revoking an
address and changing the switches now send it. The client lists both events
as not yet shown, which keeps the registry-parity test green until the mail
screens move them to their listener.
2026-11-13 17:43:55 +01:00
Maurice 264f352502 fix(mail-ingest): guard the public webhook and the settings that open it
The webhook refuses headers that cannot be signed right before reading a
byte, limits each address to 120 requests a minute and reads four bodies at
once. The admin save refuses a webhook secret under 32 characters, and the
MCP settings tool registers its input strict like every other tool.
2026-11-13 17:43:54 +01:00
Maurice 32291ce8b4 fix(mail-ingest): trust a sender verdict only from the IMAP box's own server
Anyone could add an Authentication-Results header with dmarc=pass. The hint
now needs a DKIM pass signed by the From domain or a DMARC pass naming it,
and reads it only for IMAP, where the receiving server's header is topmost;
the Cloud relay, the webhook and uploads never show a sender as verified.
2026-11-13 17:43:53 +01:00
Maurice c35f39add1 fix(mail-ingest): only the forwarder moves a booking, and filing claims the entry first
A co-member could move the forwarder's booking into a private trip, carrying
their mail along; undo stays open to editors. Move and filing refuse demo
accounts, and filing claims the entry before writing documents, putting it
back into the inbox when nothing could be filed.
2026-11-13 17:43:52 +01:00
Maurice 993ed4b472 fix(mail-ingest): refuse an upload while the person's import is off, and start no run in tests
An upload made an address and stored a mail the pipeline then put aside;
it now answers 409 instead. The run an upload asks for at once starts only
where the crons run, so it can no longer outlive a test's database.
2026-11-13 17:43:51 +01:00
Maurice e7db110417 fix(mail-ingest): read the Cloud inbox as it is sent, and keep mail refused for now
The relay job reads the control plane's { messages, more, next } page, checks
each mail on its own, pages on while more wait and is idle on a 404. A mail
for several addresses is stored for each, and a passing refusal (inbox full,
daily share taken) leaves it with the relay or in IMAP's TREK/deferred.
2026-11-13 17:43:51 +01:00
Maurice 011cc9f11f fix(mail-ingest): an empty IMAP folder is not asked for a uid range
Some servers refuse a FETCH range on a folder with no mails, which would turn
every quiet run into a failure on the status card.
2026-11-13 17:43:50 +01:00
Maurice 844d03a9cd feat(mail-ingest): REST routes and MCP tools for the settings card, the inbox and trip addresses
/api/mail-ingest serves the settings card, the inbox, the mail preview, the
.eml upload and every action on an entry; trip addresses and the admin status
and connection test have routes of their own. The MCP tools act through the
same services and never hand out an address, and the trip assistant is not offered them.
2026-11-13 17:43:50 +01:00
Maurice 1bfaf4ea98 feat(mail-ingest): read forwarded mails from an IMAP box, a signed webhook and the TREK Cloud relay
The IMAP job reads the box every two minutes through the SSRF guard, keeps a
uid cursor and moves read mails to TREK/processed. The webhook takes raw MIME
signed with HMAC-SHA256 over a timestamp, the recipient and the mail, and is a
404 unless it is the transport. The relay job collects a managed install's mails.
2026-11-13 17:43:49 +01:00
Maurice 6309064f40 feat(mail-ingest): the inbox a person reads, and moving an added booking to another trip
The inbox pages a person's entries with the mail each came from and the trip
it suggests (named only when the reader can see it), shows a mail as plain
text, and takes an uploaded .eml, reading it at once. Move writes the booking
anew in the other trip and removes it from the first in one transaction.
2026-11-13 17:43:49 +01:00
Maurice 7d3ed21a03 feat(mail-ingest): the transport settings in the admin form, each overridable from the environment
The IMAP box, the webhook secret and the address template join the admin
settings form (masked, encrypted, held by the operator on a managed install),
and MAIL_INGEST_* variables override them through the mailIngestConfig token.
The audit row masks every masked key, and key rotation covers the new secrets.
2026-11-13 17:43:48 +01:00
Maurice f319b3610a feat(mail-ingest): tell open clients when the inbox changes
Every new waiting entry and every decision sends mailInbox:changed with the count, so a badge follows without a reload.
2026-11-13 17:43:27 +01:00
Maurice 81d27d8e8d refactor(mail-ingest): keep the types nothing else imports private to their file
Exported types without a reader outside their file only add to the unused-export count.
2026-11-13 17:43:26 +01:00
Maurice de7de4b8de feat(mail-ingest): read forwarded booking mails and add what passes the checks to its trip
Addresses per person and per trip (random base32 token, stored hashed and
encrypted), an intake every transport shares, and a job that reads each
mail once under a lease: filters without AI, machine-readable data always,
one AI request with the person's trips, the server's checks G1 to G9, then
one transaction per mail that adds what passed, keeps the rest in the inbox
with a reason, and notifies the forwarder and co-travellers. Undo within
7 days, manual apply, filing as documents, a prefilled form and a nightly
retention complete it; on a managed install the operator's switch decides.
2026-11-13 17:43:26 +01:00
Maurice 340454cfc7 feat(notifications): events for bookings added from mail, and actions a domain registers
mail_booking_added tells the forwarder (with keep or undo) and co-travellers
of a shared trip; mail_booking_pending points at the inbox. Both are listed
only while the mail import is on, with texts in all 28 locales.
registerResponseAction lets a domain name what a yes/no tap does without
this module importing it.
2026-11-13 17:43:25 +01:00
Maurice aa84d71302 feat(llm-parse): read a forwarded booking mail and name its trip in one request
MailReadService sends one mail as one document: its subject, body and up to
three attachments inside data tags the prompt calls untrusted, with the
person's trips under references made up for the call. Answers keep only
offered trip references and known field names. BookingImportService reads
a mail's machine-readable parts and maps an AI reading to booking items.
2026-11-13 17:42:44 +01:00
Maurice 53ebc9d632 fix(llm-parse): the local router reports how its reading ended
The Ollama router catches a failed model call and answered an empty list, which
the outcome read as a document without bookings. It now returns the outcome with
the list; failureOutcome and foundOutcome move to llm-outcome.ts so the router and
the service share them without importing each other.
2026-11-13 17:40:53 +01:00
Maurice 57d0b42726 feat(llm-parse): a MIME walker that keeps a booking mail's body and attachments
walkMail() reads a forwarded mail for the import: the envelope apart, a body
without From, To, Cc or forwarding header blocks, message/rfc822 and quoted
forwards unwrapped up to depth 10, and the PDF, pkpass, ics, JPEG, PNG and WebP
attachments whose bytes match their type. Nothing is rendered or fetched.
2026-11-13 17:40:52 +01:00
Maurice d557f08e28 feat(llm-parse): a typed outcome for every AI reading instead of warning text
LlmParseService.parse() now says how a reading ended (ok, empty, no_text,
quota_exhausted, ai_disabled, user_cap, provider_error, too_large). The clients throw
LlmRequestError with status and body code, so the TREK Cloud gate's 429 QUOTA_SPENT
and HARD_CAP read as a spent allowance. The preview reports the outcome per file and
the assistant reads its empty reason from it rather than matching warning strings.
2026-11-13 17:40:29 +01:00
Maurice 5dc1de0e5b feat(booking-import): report each unconfirmed item and check the right in confirm()
confirm() now answers { created, failed: [{ index, reason }] } instead of logging
and dropping the items it could not write, and checks the trip and reservation_edit
itself so the import route and the assistant get one answer. An item with a booking
reference records it as its origin (sync off) and a repeat import is a duplicate.
2026-11-13 17:40:27 +01:00
Maurice d2b12cd3dd feat(reservations): record an import's origin and scope the joined read to its trip
CreateReservationData and insertReservation take an optional external_source,
external_id and sync_enabled, so an importer can record where a booking came from;
without them every write keeps the column defaults. Client bodies on REST and the
plugin RPC have the origin stripped, so no caller can pose as an import or a link.
getReservationWithJoins now takes the trip and reads nothing outside it.
2026-11-13 17:40:27 +01:00
Maurice 154563f34d feat(flight-live): follow booked flights live from keyless ADS-B feeds
A new flight-live domain polls the open feeds adsb.lol and adsb.fi (no key,
no cost; adsb.fi only on self-hosted installs, its terms are non-commercial)
for every booked flight leg in its watch window, derives phase, take-off,
landing, delay and an arrival estimate, and pushes changes as flight:live.
GET /api/trips/:tripId/flights/live, the MCP tool get_trip_flight_status,
notifications for take-off, landing and delays, and the admin switch
flight_live_enabled (FLIGHT_LIVE_ENABLED wins) come with it.
2026-11-13 15:29:03 +01:00
Maurice c2c0535d52 fix(ci): list the packing share pair, and read a stored costs mode through its enum
The whole-list sharing runs on usePackingCategorySharing in the packing panel
and the phone tab, its sheet is markup fed by the tab. The trips service reads
a stored costs_mode as one of TripsCostsMode, which the export check wants used.
2026-11-13 13:30:37 +01:00
Maurice d6799d1205 fix(ci): the calendar query DTO as the contract it publishes, and two typed tests
feeds and trips import the calendar query DTO, so it lives in
calendar.contract.ts rather than as calendar's internal dto file. The places
controller tests pass the KML service in every call, and the revisit pill
test builds a whole list.
2026-11-13 13:13:13 +01:00
Maurice 71cc97822a fix(ci): the second round of the quick-win checks on the server
oidc_auto_redirect is managed-locked in the registry too, costs_mode is read
as the mode it is, the days of a dated trip are read through the factories,
and the dashboard's country order moves out of the atlas service into a
module of its own, so the service keeps its size.
2026-11-13 12:57:56 +01:00
Maurice 721a610a70 fix(ci): green the quick-win round on the server, the shared contracts and the plugin facts
The trips entity takes costs_mode as the generator writes it, the plugin
output contract publishes it, and the hand-built test schemas carry it. The
new MCP fields come from @trek/shared, the link tool tests join the collab
file that already mocks the database, oidc_auto_redirect is the operator's
on a managed install, and the tests follow the KML service, the dated trip
days and the zod pipe.
2026-11-13 12:40:15 +01:00
Maurice 74af74fb5e feat(atlas): show the most recently visited countries first on the dashboard (#2648)
The place and endpoint reads behind the passport tile now carry the last day
of each started trip, and getTravelStats orders the countries by their latest
visit, with countries only marked by hand after them. The country set is
unchanged; repository, parity and service tests pin the order.
2026-11-13 10:17:26 +01:00
Maurice 924b4e0787 feat(auth): let admins keep the login page on OIDC-only instances (#2925)
A new app setting oidc_auto_redirect (default on) is part of the admin form
and the public app config. Off, an instance without password login no longer
sends visitors straight to the IdP: the login page shows its SSO button,
also while the cached config is read. Admin toggle on desktop and phone.
2026-11-13 10:17:26 +01:00
Maurice d5d9428cf2 feat(calendar): let a feed leave out the per-day summary events (#2602)
Both feed routes and the one-off export.ics take ?days=0, validated by a
shared query contract, and then skip the all-day "Day N" event per day; the
default is unchanged. The subscribe dialog gets a Day overview events switch
that adds the parameter to its links, export_trip_ics takes dayEvents and the
feed tools explain the parameter.
2026-11-13 10:17:25 +01:00
Maurice b9231ffb53 feat(mcp): manage the trip's shared links over MCP (#3076)
list_collab_links, create_collab_link, update_collab_link and
delete_collab_link run the Links panel's REST routes as tools: the shared
body contracts, trip access, collab_edit on writes and the collab🔗*
broadcasts, behind the links sub-feature flag. Unit, e2e and broadcast
parity cases cover them; the wiki lists the tools.
2026-11-13 10:17:15 +01:00
Maurice 489ba93c0c feat(mcp): let update_display_settings set the dashboard world clocks (#3044)
dashboard_timezones joins the display-preference allow-list as a list of
IANA zone ids, each checked with Intl like the widget does, without repeats
and capped at 24. The write goes through saveMany, so open tabs refresh, and
both tool descriptions name the widget's fallback zones.
2026-11-13 10:17:15 +01:00
Maurice 3e98938c95 feat(export): places as KML for Google My Maps (#1866)
Google Maps takes no list from outside, but My Maps imports a KML layer.
The trip export dialog and the phone export sheet get "Places for Google
My Maps" (GET places/export.kml, MCP export_trip_kml), and a list's export
menu gets KML next to GPX. The KML codec gains a pin writer; the list file
mapping moves to collections.helpers to keep the service within its size.
2026-11-13 10:17:12 +01:00
Maurice 7283b75cac feat(mcp): list_collection_places pages a long list compactly (#2647)
A read tool that answers one list a page at a time (100 by default, up to
500) as compact one-line JSON rows, filtered by status, label or "no label",
with the list's labels, the matching total and a cursor. The cursor is the
last place's position (sort_order, created_at, id), so labelling a page
while paging through unlabelled places skips none. getCollection orders by
the same keys, id last, so both tools agree on ties.
2026-11-13 10:17:11 +01:00
Maurice cae04e83dd feat(collections): edit a saved place's provider ids, website, phone and price (#2646)
The update contract takes google_place_id, google_ftid, osm_id, website,
phone, price and currency like saving does, website with the same
normalisation and refusal. REST and update_collection_place share it, so an
agent can correct a place instead of re-saving it and losing its labels,
ratings, position and id. The assistant's change card names the new fields.
2026-11-13 10:17:11 +01:00
Maurice 72e8b1f8d9 feat(collections): "want to revisit" status for saved places (#2567)
A fourth status after visited, for a place already seen that the user wants
to go back to. It joins the one-tap cycle, the filters, the pickers, the
assistant cards and the MCP status tool, and counts as visited wherever
visits are counted (isVisitedCollectionStatus), so "visited everywhere"
keeps it. The status pickers go two by two where four would not fit.
2026-11-13 10:17:11 +01:00
Maurice a70eea5bf2 feat(costs): let a trip keep its costs as totals only (#1714)
Travellers who track what a trip costs but never split it saw an
Unfinished warning on every expense and a settle-up they had no use for.
A trip now has a costs mode (appended migration, trips.costs_mode, split
by default); totals leaves payers, shares and settling up out of Costs on
desktop and phone through the same gate as a trip of one, and saves new
expenses without a payer. PUT /api/trips/:id and update_trip take it on
budget_edit. The split chips move into their own component.
2026-11-13 10:17:03 +01:00
github-actions[bot] 2bcfc1bfcb chore: bump version to 5.1.4 [skip ci] 2026-11-12 19:51:06 +00:00
Maurice b6e3c9bf39 feat(native): the app needs TREK 5.1.4 on the server
nativeApi goes to 3 with 5.1.4, whose UI follows a widget or Live Activity
tap that starts the app. The apps refuse an older server with the hint to
update, and check a server they remember once more.
2026-11-12 20:21:16 +01:00
Maurice ae26a6c61c refactor(airtrail): save the settings in one transaction
The address and the key it may no longer carry now change together, as
they do for Dawarich, so the transaction ratchet drops the AirTrail entry.
The tests that pinned the old no-transaction shape now pin the new one.
2026-11-12 17:31:56 +01:00
Maurice 74d243d8e0 fix(airtrail): a new host does not inherit the stored API key
Saving the AirTrail settings with a different URL and the key field left
masked or blank kept the stored key, so the next sync sent the old
instance's key to the new host. The key now goes with the host, as it
already did for the connection test and for Dawarich: a new origin without
a new key clears it, a path edit on the same origin keeps it. The key is
cleared before the new address is written.
2026-11-12 16:58:24 +01:00