Commit Graph

17 Commits

Author SHA1 Message Date
jubnl 6a718fccea feat(import): selective GPX/KML element import and performance improvements
Add type-selector UI in the file import modal letting users choose which
GPX elements (waypoints, routes, tracks) or KML/KMZ elements (points,
paths) to import. KML LineString placemarks are now imported as path
places with route_geometry.

Performance improvements:
- Extract MemoPlaceRow with React.memo and contentVisibility:auto to cut
  unnecessary re-renders in PlacesSidebar
- Add weatherQueue to cap concurrent weather fetches at 3
- Replace sequential per-place deletes with a single bulkDelete API call
  (new DELETE /places/bulk endpoint + deletePlacesMany service)
- Memoize atlas/photo/weather service calls to avoid redundant requests
- Add multi-select mode to PlacesSidebar for bulk operations

Add large GPX/KML/KMZ fixtures for integration/perf testing and two
profiler analysis scripts under scripts/.
2026-04-18 01:28:37 +02:00
jubnl 875c91e5ff feat(places): unified file import modal with drag-and-drop and deduplication
- Replace separate GPX and KML/KMZ import buttons with a single "Import
  file" modal accepting all three formats, with a drag-and-drop drop zone
- Support dragging files directly onto the Places sidebar panel; overlay
  appears on hover and pre-loads the file into the modal on drop
- Fix [object Object] description bug in KML imports caused by
  fast-xml-parser returning mixed-content nodes as objects; add stopNodes
  config and object guard in asTrimmedString
- Fix CDATA sections leaking into descriptions (e.g. "text.]]>") by
  unwrapping CDATA markers before tag stripping
- Add import deduplication across all import paths (GPX, KML/KMZ, Google
  list, Naver list): reimporting skips places already in the trip by name
  (case-insensitive) or by coordinates (within ~11 m tolerance), with
  intra-batch dedup so duplicate placemarks within the same file are
  also collapsed
- Fix KML route returning 400 "No valid Placemarks found" when all
  placemarks were valid but deduplicated; 400 now only fires when the
  file contains zero placemarks
- Show a warning toast "All places were already in the trip" instead of
  a misleading success toast when a reimport produces zero new places
  (GPX, KML/KMZ, Google list, Naver list)
- Add 8 new i18n keys across all 14 locales; remove 11 keys made unused
  by the modal consolidation
2026-04-15 06:07:26 +02:00
jubnl 801ffbfb7b fix(kml-import): address PR #488 review issues
- Strip BOM (U+FEFF) from 14 translation files injected by editor
- Guard KMZ unpack against zip-bomb: check entry.uncompressedSize against
  50 MB cap (KMZ_DECOMPRESSED_SIZE_LIMIT) before calling .buffer();
  limit is an exported constant so tests can override it
- Fix non-BMP HTML entity decoding: replace String.fromCharCode with
  String.fromCodePoint + 0x10FFFF bounds check so emoji like 😀
  round-trip correctly
- Switch KML namespace stripping from regex to fast-xml-parser's
  removeNSPrefix option; XMLValidator accepts namespaced XML natively,
  making the pre-strip step unnecessary
- Remove dead skippedCount overwrite after transaction; per-loop
  increment already tracks it alongside per-item error messages
- Type multer req.file as Express.Multer.File on both /import/gpx
  and /import/map routes instead of (req as any).file
- Add unit tests: emoji entity decoding (decimal + hex), KMZ zip-bomb
  rejection, KMZ-with-no-KML rejection
2026-04-15 05:16:47 +02:00
jubnl a1a7795945 Merge PR #488: KMZ/KML place import
Resolves conflicts with Naver list import (PR #662) — kept both unified
list-import dialog and new KMZ/KML dialog. Dropped duplicate react-dom
import and unused CustomSelect import from PlacesSidebar.
2026-04-15 05:09:45 +02:00
jubnl 9789c51d4f fix(naver-import): address PR #495 review issues
- SSRF: validate user-supplied URLs with checkSsrf() before fetch in
  both importNaverList and importGoogleList; upgrade naver.me substring
  check to exact hostname comparison to prevent bypass
- i18n: add missing places.importNaverList key to de.ts and es.ts
- migration: switch Naver addon seed to INSERT OR IGNORE to preserve
  admin customizations on re-runs; restore budget_category_order
  CREATE TABLE to its original formatting
- route: remove redundant cast after type-narrowing guard in naver-list handler
- component: hoist provider ternary above try/catch in handleListImport
- tests: add four new Naver import cases (502, empty list, no-coords,
  canonical URL skipping redirect fetch)
2026-04-15 04:48:39 +02:00
Marco Sadowski 18da5aed39 Merge branch 'dev' into feature/naver-support 2026-04-13 10:04:28 +02:00
Maurice 344b769583 fix: deduplicate places on Google Maps list re-import (#543)
Skip places that already exist in the trip (same name + coordinates
within ~10m) when re-importing a Google Maps list. Only new places
are added, preventing duplicates on repeated imports.
2026-04-12 01:45:32 +02:00
Marco Sadowski f82f00216b Merge remote-tracking branch 'origin/dev' into naver-list-import 2026-04-10 15:35:16 +02:00
unknown 6aeec0ead1 fix: add osm_id to update_place 2026-04-09 12:45:12 +02:00
unknown 978df648eb feat(mcp): add list_places assignment filter for orphan activities 2026-04-09 12:45:09 +02:00
Marco Sadowski d9d389d090 feat: naver list import
Added Naver List Import in a similar style like the Google List Import. To keep the frontend clean I combined both list options.
2026-04-07 14:04:27 +02:00
Yannis Biasutti aacfd24b58 refactor(places): merge KML/KMZ routes into single POST /import/map endpoint 2026-04-06 21:35:01 +02:00
Yannis Biasutti 8c8bd5bc37 refactor(server): consolidate KML import utilities 2026-04-06 21:27:10 +02:00
Yannis Biasutti 5271463064 feat(server): add KML and KMZ place import pipeline 2026-04-06 18:31:47 +02:00
jubnl 5cc81ae4b0 refactor(server): replace node-fetch with native fetch + undici, fix photo integrations
Replace node-fetch v2 with Node 22's built-in fetch API across the entire server.
Add undici as an explicit dependency to provide the dispatcher API needed for
DNS pinning (SSRF rebinding prevention) in ssrfGuard.ts. All seven service files
that used a plain `import fetch from 'node-fetch'` are updated to use the global.
The ssrfGuard safeFetch/createPinnedAgent is rewritten as createPinnedDispatcher
using an undici Agent, with correct handling of the `all: true` lookup callback
required by Node 18+. The collabService dynamic require() and notifications agent
option are updated to use the dispatcher pattern. Test mocks are migrated from
vi.mock('node-fetch') to vi.stubGlobal('fetch'), and streaming test fixtures are
updated to use Web ReadableStream instead of Node Readable.

Fix several bugs in the Synology and Immich photo integrations:
- pipeAsset: guard against setting headers after stream has already started
- _getSynologySession: clear stale SID and re-login when decrypt_api_key returns null
  instead of propagating success(null) downstream
- _requestSynologyApi: return retrySession error (not stale session) on retry failure;
  also retry on error codes 106 (timeout) and 107 (duplicate login), not only 119
- searchSynologyPhotos: fix incorrect total field type (Synology list_item returns no
  total); hasMore correctly uses allItems.length === limit
- _splitPackedSynologyId: validate cache_key format before use; callers return 400
- getImmichCredentials / _getSynologyCredentials: treat null from decrypt_api_key as
  a missing-credentials condition rather than casting null to string
- Synology size param: enforce allowlist ['sm', 'm', 'xl'] per API documentation
2026-04-05 21:12:51 +02:00
jubnl 6ba08352ed fix(gpx): replace regex parsing with fast-xml-parser and import tracks alongside waypoints
GPX files containing both <wpt> and <trk> elements would only import
waypoints, silently discarding track geometry. The fallback chain only
parsed <trkpt> when no waypoints were found.

Replaced all regex-based XML parsing helpers with fast-xml-parser for
correctness (namespaces, CDATA, attribute ordering). Tracks are now
always parsed independently of waypoints, with each <trk> element
becoming its own place with route geometry. Fixes #427.
2026-04-05 15:54:42 +02:00
Maurice 979322025d refactor: extract business logic from routes into reusable service modules 2026-04-02 17:14:53 +02:00