Vini
7277e35a96
The Atlas server no longer keeps the admin-1 regions of every country in memory. Each country's regions are now a pre-gzipped file of their own, sent to the map as they are. **Why:** `GET /addons/atlas/regions/geo?countries=…` built `admin1Store` (every country's region text, kept until restart), then joined the requested countries, `JSON.parse`d them, and Nest stringified them again. On the 512 MB – 1 GB boxes TREK runs on, the first z≥5 view could OOM the server (same shape as #1576). **What changes:** - `server/assets/atlas/admin1/<ISO_A2>.geojson.gz`: 198 files, 14.85 MB, the same 3,447 features as `admin1.geojson.gz`. - New `GET /addons/atlas/regions/geo/:code` with a Zod contract (`regionGeoParamsSchema` in `@trek/shared`: two letters, any case). It sends the country's file as it is, with `Content-Encoding: gzip` and the current 1-day `Cache-Control`, as `countries/geo` does. A country without regions gets an empty FeatureCollection. Anything that is not two letters gets a 400. - `useAtlas.ts` fetches one URL per uncached country and still rebuilds the layer once. Each country has a stable URL, so a country evicted by `REGION_CACHE_MAX` comes back from the browser cache. - `getRegionFeatures` reads only the country it needs. `admin1Store` is removed. - `?countries=` keeps working for cached PWA clients. It joins the per-country files as bytes, without a parse. - The migrations are untouched. `admin1.geojson.gz` still ships, only for `Migration20200101021500` and `Migration20200101031500`. This costs ~15 MB of disk in the image and no RAM. `build-atlas-geo.mjs` writes both, and ATLAS-BUNDLE-008 keeps them equal. **Memory** (real `atlas-geo` module, first z≥5 view over Europe = 49 country codes, JS heap + Buffers): | | peak | kept after | |---|---|---| | before, first view (builds the store) | +372 MB | +61 MB | | before, later views | +115 MB | 0 | | after, `/regions/geo/:code` | +4.7 MB | 0 | | after, legacy `?countries=` | +59 MB | 0 | **Tests:** - New: `atlas-region-files.test.ts` (moved from `atlas.service.test.ts`, plus join and dedupe cases), the `regionGeoParamsSchema` spec, ATLAS-BUNDLE-007 (the file format the byte join depends on), ATLAS-BUNDLE-008 (bundle and per-country files are equal), ATLAS-016/017/018 (the new route over HTTP, including the 400). - The `getVisitedRegions` tests (ATLAS-UNIT-019..024) used to pass only when `getRegionGeo`'s `beforeAll` had built the store first. They now also pass on their own. - The shared, server and client gates pass. Baselines go down only where this change removed code (`no-console`, `no-explicit-any`, `no-swallowed-catch`, file sizes). **Trade-offs:** - A first zoom over Europe sends ~49 small parallel requests instead of 1. Over HTTP/1.1 the browser queues them 6 at a time. The total bytes are the same (4.4 MB gz). - The per-country URLs are unversioned and cached for a day, like `countries/geo`. After an upgrade that changes a country's regions, a browser can show the old shapes for up to 24 h. Approved on Discord in #github-pr: <paste the thread link here> Related: #1576, #2559 - [x] Bug fix - [ ] New feature - [ ] Breaking change - [ ] Documentation update - [ ] I have read the [Contributing Guidelines](https://github.com/liketrek/TREK/wiki/Contributing) - [ ] My branch is [up to date with `dev`](https://github.com/liketrek/TREK/wiki/Development-environment#3-keep-your-fork-up-to-date) - [x] This PR targets the `dev` branch, not `main` *(wiki-only PRs are exempt)* - [x] I have tested my changes locally - [x] I have added/updated tests that prove my fix is effective or that my feature works - [x] I have updated documentation if needed
6.0 KiB
Third-party data & attributions
TREK bundles and uses third-party data that requires attribution.
geoBoundaries — country & sub-national boundaries
The Atlas map's administrative boundaries (admin-0 countries and admin-1
provinces/counties), shipped at server/assets/atlas/admin0.geojson.gz,
server/assets/atlas/admin1.geojson.gz and server/assets/atlas/admin1/<ISO_A2>.geojson.gz
(the same regions, one file per country) and generated by
server/scripts/build-atlas-geo.mjs, are derived from geoBoundaries.
Runfola, D. et al. (2020) geoBoundaries: A global database of political administrative boundaries. PLoS ONE 15(4): e0231866. https://doi.org/10.1371/journal.pone.0231866
geoBoundaries is licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Source: https://www.geoboundaries.org/
The bundled files are simplified (coordinate-quantized) and re-tagged with the
property names TREK consumes. Country borders (admin0) derive from the geoBoundaries
CGAZ composite; sub-national regions (admin1) derive from the per-country open
(gbOpen) release.
OpenStreetMap — geocoding, place details & descriptions
Atlas reverse-geocodes places via the Nominatim service, and place search, details and descriptions come from Nominatim and Overpass. Data is © OpenStreetMap contributors, licensed under the Open Database License (ODbL). https://www.openstreetmap.org/copyright
Wikimedia Commons & Wikipedia — place pictures & descriptions
Pictures offered for a place come from Wikimedia Commons (geosearch by
coordinate) and are cached locally rather than hotlinked. Descriptions can come
from Wikipedia, resolved from the place's OpenStreetMap wikipedia tag.
Commons files carry their own licence, most often CC BY or CC BY-SA. TREK shows the author, the licence and a link to the file description page next to every picture while it is being chosen, and keeps the author and licence with the cached file so the credit stays visible afterwards. Wikipedia article text is CC BY-SA 4.0. https://commons.wikimedia.org/wiki/Commons:Licensing
Google Places — optional place data
When an administrator configures a Google Maps API key, place search, details, pictures and editorial summaries can also come from the Google Places API, subject to Google's terms. Photos carry the author attribution Google supplies and no reusable licence; TREK displays that attribution and claims nothing further. This path is off unless a key is configured, and each part of it has its own switch in the admin panel. https://cloud.google.com/maps-platform/terms
OurAirports — airport reference data
server/assets/airports.json is built from OurAirports
(https://ourairports.com/data/), released into the public domain.
Typefaces: Studio books and the app
Every typeface a book can be set in, and the app's own (Poppins, MuseoModerno,
Geist), is self-hosted from the @fontsource npm packages, which redistribute
the families published on Google Fonts split into subsets by script. The catalogue of book faces is
shared/src/book/book-fonts.data.ts. Each package carries its licence and the
copyright notice of the family's authors in its LICENSE file.
Both licences allow a font to be bundled with software and embedded in a document made with it, so a printed or exported book may carry the faces it is set in. No family is renamed, redrawn or sold on its own. SIL OFL 1.1: https://openfontlicense.org. Apache 2.0: https://www.apache.org/licenses/LICENSE-2.0
| Family | Package | Licence |
|---|---|---|
| Abril Fatface | @fontsource/abril-fatface |
SIL Open Font License 1.1 |
| Alfa Slab One | @fontsource/alfa-slab-one |
SIL Open Font License 1.1 |
| Allura | @fontsource/allura |
SIL Open Font License 1.1 |
| Anton | @fontsource/anton |
SIL Open Font License 1.1 |
| Archivo Black | @fontsource/archivo-black |
SIL Open Font License 1.1 |
| Bebas Neue | @fontsource/bebas-neue |
SIL Open Font License 1.1 |
| Big Shoulders Display | @fontsource/big-shoulders-display |
SIL Open Font License 1.1 |
| Bodoni Moda | @fontsource/bodoni-moda |
SIL Open Font License 1.1 |
| Bricolage Grotesque | @fontsource/bricolage-grotesque |
SIL Open Font License 1.1 |
| Caveat | @fontsource/caveat |
SIL Open Font License 1.1 |
| Cormorant Garamond | @fontsource/cormorant-garamond |
SIL Open Font License 1.1 |
| Dancing Script | @fontsource/dancing-script |
SIL Open Font License 1.1 |
| DM Mono | @fontsource/dm-mono |
SIL Open Font License 1.1 |
| DM Serif Display | @fontsource/dm-serif-display |
SIL Open Font License 1.1 |
| EB Garamond | @fontsource/eb-garamond |
SIL Open Font License 1.1 |
| Fraunces | @fontsource/fraunces |
SIL Open Font License 1.1 |
| Great Vibes | @fontsource/great-vibes |
SIL Open Font License 1.1 |
| Homemade Apple | @fontsource/homemade-apple |
Apache License 2.0 |
| Inter | @fontsource/inter |
SIL Open Font License 1.1 |
| Kaushan Script | @fontsource/kaushan-script |
SIL Open Font License 1.1 |
| League Gothic | @fontsource/league-gothic |
SIL Open Font License 1.1 |
| Lora | @fontsource/lora |
SIL Open Font License 1.1 |
| Montserrat | @fontsource/montserrat |
SIL Open Font License 1.1 |
| MuseoModerno | @fontsource/museomoderno |
SIL Open Font License 1.1 |
| Oswald | @fontsource/oswald |
SIL Open Font License 1.1 |
| Pacifico | @fontsource/pacifico |
SIL Open Font License 1.1 |
| Parisienne | @fontsource/parisienne |
SIL Open Font License 1.1 |
| Permanent Marker | @fontsource/permanent-marker |
Apache License 2.0 |
| Playfair Display | @fontsource/playfair-display |
SIL Open Font License 1.1 |
| Poppins | @fontsource/poppins |
SIL Open Font License 1.1 |
| Sacramento | @fontsource/sacramento |
SIL Open Font License 1.1 |
| Satisfy | @fontsource/satisfy |
Apache License 2.0 |
| Shrikhand | @fontsource/shrikhand |
SIL Open Font License 1.1 |
| Yellowtail | @fontsource/yellowtail |
Apache License 2.0 |
| Geist Sans | @fontsource/geist-sans |
SIL Open Font License 1.1 |