mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-06-20 22:01:45 +00:00
fix(maps): reduce Google Places API quota usage with persistent caching
P0 — stop the bleeding:
- Honor place.image_url in MapView and TripPlannerPage to skip redundant fetchPhoto calls
- Trim Place Details field mask (drop reviews/editorialSummary from default; new getPlaceDetailsExpanded for inspector)
- Admin toggle places_photos_enabled (default ON) to kill Google photo fetches under quota pressure; Wikimedia unaffected
- Return { photoUrl: null } instead of 204 so client handles disabled state cleanly
P1 — structural fix:
- New placePhotoCache service: persistent disk cache at uploads/photos/google/<sha1>.jpg, atomic writes, stampede dedup via in-flight Map
- Migrations 105-107: google_place_photo_meta table, place_details_cache table, backfill signed Google URLs to stable proxy URLs
- getPlacePhoto rewrites to fetch image bytes directly, store on disk, return /api/maps/place-photo/:id/bytes proxy URL
- Stable proxy URLs written to places.image_url — survive container restarts, no expiry
- New GET /api/maps/place-photo/:placeId/bytes route serving cached files with long-lived Cache-Control
- Place Details DB row cache with 7-day TTL; ?refresh=1 escape hatch
- photoService fast-path: proxy URLs bypass the mapsApi round-trip and go straight to urlToBase64
Bug fixes:
- MapView now requests base64 thumbs for places with proxy image_url (markers were showing color fallback)
- createPlaceIcon accepts /api/maps/place-photo/ URLs as interim fallback while thumb generates
- setSelectedAssignmentId ReferenceError in mobile day-detail handler (use selectAssignment)
- Remove redundant decodeURIComponent on already-decoded Express route param
- Use SHA1 hash for disk filenames to prevent coords:lat:lng pseudo-ID collisions
- Add checkSsrf guard to Wikimedia byte fetch
- Tighten migration 107 LIKE filter to avoid rewriting manually-pasted Google image URLs
- Validate enabled is boolean on PUT /admin/places-photos
- Drop aggressive iconCache.clear() on every thumb arrival
Observability:
- googleFetch() wrapper counts and debug-logs every outbound Google API call with running total
This commit is contained in:
@@ -1634,6 +1634,45 @@ function runMigrations(db: Database.Database): void {
|
||||
try { db.exec('ALTER TABLE trip_album_links ADD COLUMN passphrase TEXT DEFAULT NULL'); } catch (err: any) { if (!err.message?.includes('duplicate column name')) throw err; }
|
||||
try { db.exec('ALTER TABLE trek_photos ADD COLUMN passphrase TEXT DEFAULT NULL'); } catch (err: any) { if (!err.message?.includes('duplicate column name')) throw err; }
|
||||
},
|
||||
// Migration 105: Persistent Google place photo disk cache registry
|
||||
() => {
|
||||
db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS google_place_photo_meta (
|
||||
place_id TEXT PRIMARY KEY,
|
||||
attribution TEXT,
|
||||
fetched_at INTEGER NOT NULL,
|
||||
error_at INTEGER
|
||||
)
|
||||
`);
|
||||
},
|
||||
// Migration 106: Persistent Place Details row cache
|
||||
() => {
|
||||
db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS place_details_cache (
|
||||
place_id TEXT NOT NULL,
|
||||
lang TEXT NOT NULL DEFAULT '',
|
||||
expanded INTEGER NOT NULL DEFAULT 0,
|
||||
payload_json TEXT NOT NULL,
|
||||
fetched_at INTEGER NOT NULL,
|
||||
PRIMARY KEY (place_id, lang, expanded)
|
||||
)
|
||||
`);
|
||||
},
|
||||
// Migration 107: Backfill expired signed Google photo URLs to stable proxy URLs
|
||||
{ raw: () => {
|
||||
db.exec(`
|
||||
UPDATE places
|
||||
SET image_url = '/api/maps/place-photo/' || google_place_id || '/bytes',
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
WHERE google_place_id IS NOT NULL
|
||||
AND image_url IS NOT NULL
|
||||
AND image_url != ''
|
||||
AND (
|
||||
(image_url LIKE '%googleusercontent.com%' AND image_url LIKE '%/places/%/photos/%')
|
||||
OR (image_url LIKE '%places.googleapis.com%' AND image_url LIKE '%/places/%/photos/%')
|
||||
)
|
||||
`);
|
||||
}},
|
||||
];
|
||||
|
||||
if (currentVersion < migrations.length) {
|
||||
|
||||
Reference in New Issue
Block a user