serve each country's regions from its own file, so the server no longer holds them all

The Atlas server no longer keeps the admin-1 regions of every country in memory. Each country's regions are now a pre-gzipped file of their own, sent to the map as they are.

**Why:** `GET /addons/atlas/regions/geo?countries=…` built `admin1Store` (every country's region text, kept until restart), then joined the requested countries, `JSON.parse`d them, and Nest stringified them again. On the 512 MB – 1 GB boxes TREK runs on, the first z≥5 view could OOM the server (same shape as #1576).

**What changes:**
- `server/assets/atlas/admin1/<ISO_A2>.geojson.gz`: 198 files, 14.85 MB, the same 3,447 features as `admin1.geojson.gz`.
- New `GET /addons/atlas/regions/geo/:code` with a Zod contract (`regionGeoParamsSchema` in `@trek/shared`: two letters, any case). It sends the country's file as it is, with `Content-Encoding: gzip` and the current 1-day `Cache-Control`, as `countries/geo` does. A country without regions gets an empty FeatureCollection. Anything that is not two letters gets a 400.
- `useAtlas.ts` fetches one URL per uncached country and still rebuilds the layer once. Each country has a stable URL, so a country evicted by `REGION_CACHE_MAX` comes back from the browser cache.
- `getRegionFeatures` reads only the country it needs. `admin1Store` is removed.
- `?countries=` keeps working for cached PWA clients. It joins the per-country files as bytes, without a parse.
- The migrations are untouched. `admin1.geojson.gz` still ships, only for `Migration20200101021500` and `Migration20200101031500`. This costs ~15 MB of disk in the image and no RAM. `build-atlas-geo.mjs` writes both, and ATLAS-BUNDLE-008 keeps them equal.

**Memory** (real `atlas-geo` module, first z≥5 view over Europe = 49 country codes, JS heap + Buffers):

| | peak | kept after |
|---|---|---|
| before, first view (builds the store) | +372 MB | +61 MB |
| before, later views | +115 MB | 0 |
| after, `/regions/geo/:code` | +4.7 MB | 0 |
| after, legacy `?countries=` | +59 MB | 0 |

**Tests:**
- New: `atlas-region-files.test.ts` (moved from `atlas.service.test.ts`, plus join and dedupe cases), the `regionGeoParamsSchema` spec, ATLAS-BUNDLE-007 (the file format the byte join depends on), ATLAS-BUNDLE-008 (bundle and per-country files are equal), ATLAS-016/017/018 (the new route over HTTP, including the 400).
- The `getVisitedRegions` tests (ATLAS-UNIT-019..024) used to pass only when `getRegionGeo`'s `beforeAll` had built the store first. They now also pass on their own.
- The shared, server and client gates pass. Baselines go down only where this change removed code (`no-console`, `no-explicit-any`, `no-swallowed-catch`, file sizes).

**Trade-offs:**
- A first zoom over Europe sends ~49 small parallel requests instead of 1. Over HTTP/1.1 the browser queues them 6 at a time. The total bytes are the same (4.4 MB gz).
- The per-country URLs are unversioned and cached for a day, like `countries/geo`. After an upgrade that changes a country's regions, a browser can show the old shapes for up to 24 h.

Approved on Discord in #github-pr: <paste the thread link here>
Related: #1576, #2559

- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update

- [ ] I have read the [Contributing Guidelines](https://github.com/liketrek/TREK/wiki/Contributing)
- [ ] My branch is [up to date with `dev`](https://github.com/liketrek/TREK/wiki/Development-environment#3-keep-your-fork-up-to-date)
- [x] This PR targets the `dev` branch, not `main` *(wiki-only PRs are exempt)*
- [x] I have tested my changes locally
- [x] I have added/updated tests that prove my fix is effective or that my feature works
- [x] I have updated documentation if needed
This commit is contained in:
Vini
2026-10-26 20:08:57 +01:00
committed by Maurice
parent 70f27085b9
commit 7277e35a96
220 changed files with 360 additions and 185 deletions
+2 -1
View File
@@ -91,7 +91,8 @@ ENV KITINERARY_EXTRACTOR_PATH=/usr/local/bin/kitinerary-extractor
COPY --chown=node:node --from=server-builder /app/server/dist ./server/dist
# Runtime data assets read from server/assets at runtime: airports.json (flight
# transport search) and atlas/*.geojson.gz (Atlas country/region map). The build
# transport search) and atlas/*.geojson.gz + atlas/admin1/*.geojson.gz (Atlas
# country/region map). The build
# only emits dist, so these must be copied explicitly or the features silently
# degrade to empty in the image.
COPY --chown=node:node --from=server-builder /app/server/assets ./server/assets
+3 -2
View File
@@ -5,8 +5,9 @@ TREK bundles and uses third-party data that requires attribution.
## geoBoundaries — country & sub-national boundaries
The Atlas map's administrative boundaries (admin-0 countries and admin-1
provinces/counties), shipped at `server/assets/atlas/admin0.geojson.gz` and
`server/assets/atlas/admin1.geojson.gz` and generated by
provinces/counties), shipped at `server/assets/atlas/admin0.geojson.gz`,
`server/assets/atlas/admin1.geojson.gz` and `server/assets/atlas/admin1/<ISO_A2>.geojson.gz`
(the same regions, one file per country) and generated by
`server/scripts/build-atlas-geo.mjs`, are derived from **geoBoundaries**.
> Runfola, D. et al. (2020) geoBoundaries: A global database of political
+1 -1
View File
@@ -5,7 +5,7 @@
"no-console": 88,
"complexity": 218,
"@typescript-eslint/no-explicit-any": 144,
"trek/no-swallowed-catch": 129,
"trek/no-swallowed-catch": 128,
"max-params": 32,
"trek/store-without-selector": 67,
"react-hooks/refs": 555,
+1 -1
View File
@@ -22,7 +22,7 @@
"src/pages/JourneyDetailPage.test.tsx": 4030,
"src/pages/LoginPage.tsx": 1348,
"src/pages/TripPlannerPage.tsx": 1128,
"src/pages/atlas/useAtlas.ts": 1211,
"src/pages/atlas/useAtlas.ts": 1210,
"src/pages/tripPlanner/useTripPlanner.roadtrip.test.tsx": 3518,
"src/pages/tripPlanner/useTripPlanner.test.tsx": 3270,
"src/styles/studio.css": 1918
+3 -3
View File
@@ -215,7 +215,7 @@ function useDefaultAtlasHandlers() {
// country features override this handler via server.use(...).
http.get('/api/addons/atlas/countries/geo', () => HttpResponse.json({ type: 'FeatureCollection', features: [] })),
// Handler for region GeoJSON fetch (triggered by loadRegionsForViewport when intersects=true)
http.get('/api/addons/atlas/regions/geo', () => HttpResponse.json({ features: [] })),
http.get('/api/addons/atlas/regions/geo/:code', () => HttpResponse.json({ features: [] })),
);
}
@@ -1589,7 +1589,7 @@ describe('AtlasPage', () => {
);
server.use(
http.get('/api/addons/atlas/regions/geo', () => HttpResponse.json({ features: [] })),
http.get('/api/addons/atlas/regions/geo/:code', () => HttpResponse.json({ features: [] })),
);
render(<AtlasPage />);
@@ -1746,7 +1746,7 @@ describe('AtlasPage', () => {
type: 'FeatureCollection',
features: [{ type: 'Feature', properties: { ISO_A2: 'FR', ADM0_A3: 'FRA', ISO_A3: 'FRA', NAME: 'France', ADMIN: 'France' }, geometry: null }],
})),
http.get('/api/addons/atlas/regions/geo', () => HttpResponse.json({
http.get('/api/addons/atlas/regions/geo/:code', () => HttpResponse.json({
type: 'FeatureCollection',
features: [{ type: 'Feature', properties: { iso_a2: 'FR', iso_3166_2: 'FR-IDF', name: 'Île-de-France', name_en: 'Île-de-France', admin: 'France' }, geometry: null }],
})),
+3 -3
View File
@@ -274,7 +274,7 @@ function installAtlasHandlers(over: Partial<Record<string, unknown>> = {}) {
http.get('/api/addons/atlas/bucket-list', () => HttpResponse.json({ items: over.bucket ?? [] })),
http.get('/api/addons/atlas/regions', () => HttpResponse.json({ regions: over.regions ?? {} })),
http.get('/api/addons/atlas/countries/geo', () => HttpResponse.json(over.geo ?? { type: 'FeatureCollection', features: [] })),
http.get('/api/addons/atlas/regions/geo', () => HttpResponse.json(over.regionGeo ?? { features: [] })),
http.get('/api/addons/atlas/regions/geo/:code', () => HttpResponse.json(over.regionGeo ?? { features: [] })),
http.get('/api/atlas-layers', () => HttpResponse.json({ layers: over.layers ?? [] })),
);
}
@@ -1053,8 +1053,8 @@ describe('useAtlas', () => {
await waitFor(() => expect(atlas.atlas_country_options.length).toBe(seen.length));
const requested: string[] = [];
server.use(http.get('/api/addons/atlas/regions/geo', ({ request }) => {
requested.push(new URL(request.url).searchParams.get('countries') ?? '');
server.use(http.get('/api/addons/atlas/regions/geo/:code', ({ params }) => {
requested.push(String(params.code));
return HttpResponse.json(regionGeo);
}));
+10 -10
View File
@@ -317,18 +317,18 @@ export function useAtlas() {
}
if (!toLoad.length) return
for (const code of toLoad) pendingRegionCodes.current.add(code)
apiClient.get(`/addons/atlas/regions/geo?countries=${toLoad.join(',')}`)
// One stable URL per country, so a country evicted by REGION_CACHE_MAX or seen on an earlier
// visit comes back from the browser cache. Every request settles; the layer rebuilds once.
void Promise.all(toLoad.map(c => apiClient.get(`/addons/atlas/regions/geo/${c}`)
.then(geoRes => {
const geo = geoRes.data
if (!geo?.features) return
let added = false
for (const c of toLoad) {
const features = geo.features.filter((f: any) => f.properties?.iso_a2?.toUpperCase() === c)
if (features.length > 0) { regionGeoCache.current[c] = { type: 'FeatureCollection', features }; touchRegionCode(c); added = true }
}
if (added) { evictRegionCache(); setRegionGeoLoaded(v => v + 1) }
const features = (geoRes.data?.features || []).filter((f: any) => f.properties?.iso_a2?.toUpperCase() === c)
if (features.length === 0) return false
regionGeoCache.current[c] = { type: 'FeatureCollection', features }
touchRegionCode(c)
return true
})
.catch(() => {})
.catch(() => false)))
.then(added => { if (added.some(Boolean)) { evictRegionCache(); setRegionGeoLoaded(v => v + 1) } })
.finally(() => { for (const code of toLoad) pendingRegionCodes.current.delete(code) })
}
loadRegionsForViewportRef.current = loadRegionsForViewport
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More