Commit Graph

4 Commits

Author SHA1 Message Date
jubnl 8077ffab34 fix(maps): bound place-photo cache growth (Wikimedia + Google) (#1174)
The place-photo cache (uploads/photos/google) grew unbounded: a Wikimedia
geosearch path cached full-res originals despite requesting a 400px thumb,
the writer applied no size guard, nothing reclaimed orphaned files, and
backups archived the whole re-derivable cache verbatim.

- Prefer the scaled `thumburl` over the full-res `info.url` in the Commons
  geosearch fallback.
- Downscale any cached image to <=800px JPEG via the existing jimp dep,
  with a safe fallback to the original bytes on decode failure.
- Add sweepOrphans() (orphaned meta rows + stray files) wired into the
  scheduler (startup + nightly), and removeIfUnreferenced() called on
  place delete for prompt reclamation.
- Exclude the re-derivable photo/trek caches from backups; restores
  self-heal as the cache dirs are recreated at startup.
2026-06-14 23:31:02 +02:00
jubnl 2aad8f465c fix(maps): prevent server crash when legacy Google photo URLs are stored as placeIds
Migration 107 only rewrote image_url rows matching /places/%/photos/%; URLs using
the /place-photos/ or /places/<opaque> paths survived the upgrade and were passed
verbatim to the Places API, producing a malformed request whose empty/HTML response
body threw SyntaxError before detailsRes.ok was checked. The resulting rejection was
leaked by placePhotoCache.setInFlight via an unhandled .finally() chain, triggering
Node 22's default unhandledRejection=throw and terminating the process.

- placePhotoCache: add .catch() after .finally() to prevent unhandled rejection crash
- mapsService: reject URL-shaped placeIds early; read response as text before JSON.parse
- migrations: add migration to rewrite remaining googleusercontent/places.googleapis URLs
- MapView/MapViewGL: prefer stable proxy URL form of image_url before google_place_id

Fixes #770
2026-04-21 00:13:35 +02:00
jubnl 6a718fccea feat(import): selective GPX/KML element import and performance improvements
Add type-selector UI in the file import modal letting users choose which
GPX elements (waypoints, routes, tracks) or KML/KMZ elements (points,
paths) to import. KML LineString placemarks are now imported as path
places with route_geometry.

Performance improvements:
- Extract MemoPlaceRow with React.memo and contentVisibility:auto to cut
  unnecessary re-renders in PlacesSidebar
- Add weatherQueue to cap concurrent weather fetches at 3
- Replace sequential per-place deletes with a single bulkDelete API call
  (new DELETE /places/bulk endpoint + deletePlacesMany service)
- Memoize atlas/photo/weather service calls to avoid redundant requests
- Add multi-select mode to PlacesSidebar for bulk operations

Add large GPX/KML/KMZ fixtures for integration/perf testing and two
profiler analysis scripts under scripts/.
2026-04-18 01:28:37 +02:00
jubnl 9c2decb095 fix(maps): reduce Google Places API quota usage with persistent caching
P0 — stop the bleeding:
- Honor place.image_url in MapView and TripPlannerPage to skip redundant fetchPhoto calls
- Trim Place Details field mask (drop reviews/editorialSummary from default; new getPlaceDetailsExpanded for inspector)
- Admin toggle places_photos_enabled (default ON) to kill Google photo fetches under quota pressure; Wikimedia unaffected
- Return { photoUrl: null } instead of 204 so client handles disabled state cleanly

P1 — structural fix:
- New placePhotoCache service: persistent disk cache at uploads/photos/google/<sha1>.jpg, atomic writes, stampede dedup via in-flight Map
- Migrations 105-107: google_place_photo_meta table, place_details_cache table, backfill signed Google URLs to stable proxy URLs
- getPlacePhoto rewrites to fetch image bytes directly, store on disk, return /api/maps/place-photo/:id/bytes proxy URL
- Stable proxy URLs written to places.image_url — survive container restarts, no expiry
- New GET /api/maps/place-photo/:placeId/bytes route serving cached files with long-lived Cache-Control
- Place Details DB row cache with 7-day TTL; ?refresh=1 escape hatch
- photoService fast-path: proxy URLs bypass the mapsApi round-trip and go straight to urlToBase64

Bug fixes:
- MapView now requests base64 thumbs for places with proxy image_url (markers were showing color fallback)
- createPlaceIcon accepts /api/maps/place-photo/ URLs as interim fallback while thumb generates
- setSelectedAssignmentId ReferenceError in mobile day-detail handler (use selectAssignment)
- Remove redundant decodeURIComponent on already-decoded Express route param
- Use SHA1 hash for disk filenames to prevent coords:lat:lng pseudo-ID collisions
- Add checkSsrf guard to Wikimedia byte fetch
- Tighten migration 107 LIKE filter to avoid rewriting manually-pasted Google image URLs
- Validate enabled is boolean on PUT /admin/places-photos
- Drop aggressive iconCache.clear() on every thumb arrival

Observability:
- googleFetch() wrapper counts and debug-logs every outbound Google API call with running total
2026-04-17 19:07:39 +02:00