mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-11-20 19:16:08 +00:00
feat(db): choose the database engine from TREK_DB_URL
A port for the engine TREK runs on, without changing what it runs on. src/db/engine/ names the engine (SqliteEngine, PostgresEngine) and builds the MikroORM config from it; src/mikro-orm.config.ts exports that config for the CLI and the Nest container alike. Without TREK_DB_URL the config is the one it was: the shared SQLite driver on TREK_DB_FILE or data/travel.db. - TREK_DB_URL takes a postgres:// or postgresql:// URL and is masked in the boot report; TREK_DB_POOL_MAX sizes the pool (1 to 100, default 10). TREK_DB_URL beside TREK_DB_FILE, TREK_DB_JOURNAL_MODE or TREK_DB_SYNCHRONOUS refuses the boot as an ambiguous configuration. - MikroORM builds the pg pool from the URL's host, port, user, password and database and drops the rest of the query, so the engine reads sslmode and sslrootcert itself. require, verify-ca and verify-full all verify the server's certificate, as the pg driver reads them; a mode that would not verify, or a misspelt one, refuses. - Postgres cannot run TREK yet, so a boot with TREK_DB_URL is refused with that reason, in the boot check and again where the config is built, before anything connects. - @mikro-orm/postgresql moves to dependencies; pg is plain JavaScript. - lint:engine-dispatch (CI) holds the engine checks outside src/db/dialect/ and src/db/engine/ (isSqlite(, isPostgres(, an instanceof on a platform, a comparison on the engine's name) to a baseline that starts and stays empty.
This commit is contained in:
@@ -367,6 +367,12 @@ jobs:
|
||||
# so the code stays portable to a second engine.
|
||||
run: cd server && npm run lint:dialect
|
||||
|
||||
- name: Engine checks may only shrink
|
||||
# isSqlite(/isPostgres( and platform instanceof outside src/db/dialect/
|
||||
# and src/db/engine/ are held to scripts/engine-dispatch-baseline.json,
|
||||
# so what differs between engines stays in those two places.
|
||||
run: cd server && npm run lint:engine-dispatch
|
||||
|
||||
- name: Writes outside a transaction
|
||||
# A method that writes twice does it inside one uow.transactional.
|
||||
# Today's exceptions sit in scripts/tx-writes-baseline.json and may
|
||||
|
||||
@@ -51,6 +51,7 @@ cd client && npm run lint:skips # no .only; skipped tests may
|
||||
cd client && npm run lint:i18n-keys # every key the client names exists in en; unused en keys may only shrink (CI gate)
|
||||
cd server && npm run lint:boundaries # no new import cycles or cross-domain reach-ins (CI gate)
|
||||
cd server && npm run lint:dialect # SQLite-only SQL spellings may only shrink (CI gate; rules in server/CLAUDE.md)
|
||||
cd server && npm run lint:engine-dispatch # engine checks outside src/db/dialect/ and src/db/engine/ may only shrink (CI gate)
|
||||
cd server && npm run probe:pg # dialect helpers + the repository statements it reaches against Postgres (CI job; needs TREK_PG_PROBE_URL)
|
||||
cd server && npm run lint:tx # no new method writing twice outside one transaction (CI gate)
|
||||
cd server && npm run lint:test-sql # raw SQL fixtures in tests/ may only shrink; seed through tests/helpers/factories (CI gate)
|
||||
@@ -134,7 +135,7 @@ These principles come out of a full-repo audit and shape all new code:
|
||||
|
||||
## Quality gates in CI
|
||||
|
||||
`.github/workflows/test.yml` runs, per package: typecheck (server also `typecheck:tests`, client also the strict-mode ratchet `lint:strict`), lint (client and server through `lint:warnings`, which also holds the warning count per rule), `lint:pages`, `lint:rtl`, `lint:size` (client and server), the server's `lint:strict`, `lint:boundaries`, `lint:tx`, `lint:dialect`, `lint:test-sql`, `lint:test-mocks`, `lint:test-new-service`, `lint:query-api`, `lint:mcp-zod`, `lint:service-http` and `lint:response-contracts`, the client's and server's `lint:format`, `lint:layers`, `lint:offline`, `lint:dup`, `lint:pairs`, `lint:skips`, `lint:i18n-keys` and `theme:lint`, shared's `contracts:open`, the repo-wide `lint:knip` (in server-lint-checks), `check:plugin-facts`, `i18n:parity:strict` (with the untranslated-strings ratchet), the S3 contract suite, the Postgres probe (`postgres-probe`: every dialect helper on SQLite and Postgres, and the statements every repository method reaches with sample arguments against a Postgres service; the SQL errors per method and the list of methods it cannot measure are held to a baseline that only shrinks and fails while it is still unmeasured; see `server/CLAUDE.md`), the browser end-to-end flows (Chromium and WebKit), a client build with `check:gl-split`, coverage for all four packages, then a SonarCloud scan. It also builds the Docker image and boots it until `/api/health/ready` answers (`docker-image`), and lints and renders the Helm chart against the Kubernetes schemas and runs `docker compose config` on every compose file (`deploy-manifests`). `lint-prettier.yml` additionally runs `lint` + `lint:format` on `shared/`, both check-only. `plugin-sdk` has no lint script and is outside the eslint gates. **Not in CI**: the screenshot and help-media Playwright projects; run them locally. There are no git hooks; nothing runs before a commit except you. Other workflows build Docker images, scan them, and publish `trek-plugin-sdk` on `plugin-sdk-v*` tags.
|
||||
`.github/workflows/test.yml` runs, per package: typecheck (server also `typecheck:tests`, client also the strict-mode ratchet `lint:strict`), lint (client and server through `lint:warnings`, which also holds the warning count per rule), `lint:pages`, `lint:rtl`, `lint:size` (client and server), the server's `lint:strict`, `lint:boundaries`, `lint:tx`, `lint:dialect`, `lint:engine-dispatch`, `lint:test-sql`, `lint:test-mocks`, `lint:test-new-service`, `lint:query-api`, `lint:mcp-zod`, `lint:service-http` and `lint:response-contracts`, the client's and server's `lint:format`, `lint:layers`, `lint:offline`, `lint:dup`, `lint:pairs`, `lint:skips`, `lint:i18n-keys` and `theme:lint`, shared's `contracts:open`, the repo-wide `lint:knip` (in server-lint-checks), `check:plugin-facts`, `i18n:parity:strict` (with the untranslated-strings ratchet), the S3 contract suite, the Postgres probe (`postgres-probe`: every dialect helper on SQLite and Postgres, and the statements every repository method reaches with sample arguments against a Postgres service; the SQL errors per method and the list of methods it cannot measure are held to a baseline that only shrinks and fails while it is still unmeasured; see `server/CLAUDE.md`), the browser end-to-end flows (Chromium and WebKit), a client build with `check:gl-split`, coverage for all four packages, then a SonarCloud scan. It also builds the Docker image and boots it until `/api/health/ready` answers (`docker-image`), and lints and renders the Helm chart against the Kubernetes schemas and runs `docker compose config` on every compose file (`deploy-manifests`). `lint-prettier.yml` additionally runs `lint` + `lint:format` on `shared/`, both check-only. `plugin-sdk` has no lint script and is outside the eslint gates. **Not in CI**: the screenshot and help-media Playwright projects; run them locally. There are no git hooks; nothing runs before a commit except you. Other workflows build Docker images, scan them, and publish `trek-plugin-sdk` on `plugin-sdk-v*` tags.
|
||||
|
||||
- **Job names read "package: what"** (`Server: Types`, `Client: Tests 3/7`), and the list at the top of `test.yml` says what each job holds. A new job follows the scheme and joins that list, and the job-count note there says when a long job needs a slot freed first.
|
||||
- **Tests run beside the gates, in shards.** The client and server suites run in slices on parallel runners (`client-tests`, `server-tests`, through each package's `vitest.shard.config.ts`), and `client-coverage`/`server-coverage` merge their blob reports and check the thresholds against the whole. A red slice is usually a failing test, a red merge job usually a threshold.
|
||||
|
||||
Generated
+1
-20
@@ -5306,7 +5306,6 @@
|
||||
"version": "7.2.1",
|
||||
"resolved": "https://registry.npmjs.org/@mikro-orm/postgresql/-/postgresql-7.2.1.tgz",
|
||||
"integrity": "sha512-PQ1qe4Bj/hQSHYJjR02l06FfEDwniAvncduwZSCLSKuaCUddHc5AN2Utj3yo4xRIVRY8pOa7DFitEQPBozTMJA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@mikro-orm/sql": "7.2.1",
|
||||
@@ -19487,7 +19486,6 @@
|
||||
"version": "8.23.0",
|
||||
"resolved": "https://registry.npmjs.org/pg/-/pg-8.23.0.tgz",
|
||||
"integrity": "sha512-Ip2EQCngowJLGOfCwkFhPXU7/ljlhn6Rxlmy4XYfL2Y+vyRM59+8uR2xqRWKdYmbXmxCFOAmKxBuSUCdF34qLg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"pg-connection-string": "^2.14.0",
|
||||
@@ -19515,7 +19513,6 @@
|
||||
"version": "1.4.1",
|
||||
"resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.4.1.tgz",
|
||||
"integrity": "sha512-6PQbsFWZcp9EmJEwy5cGQ2La+AMWpP46lgbb8X+U/XsHIUweYDNCpeuKck5RxL2MdVFi7krbbEi5nX4Zh7JhrQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"optional": true
|
||||
},
|
||||
@@ -19523,14 +19520,12 @@
|
||||
"version": "2.14.1",
|
||||
"resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.14.1.tgz",
|
||||
"integrity": "sha512-qR3kGNPBLpCNtz0evbKA0Y/MRFXwSSdT+pTJvYp/bXTcReZbvX1kzF0IyTc1QnxqF7AZbOeBhNL8R5mYQZV/MA==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/pg-cursor": {
|
||||
"version": "2.22.0",
|
||||
"resolved": "https://registry.npmjs.org/pg-cursor/-/pg-cursor-2.22.0.tgz",
|
||||
"integrity": "sha512-knzXLKqarTjOvb3qDSW0JiGsazmxwEKXrqHfWRte7XUsOYccQRafn3BLnQobWwInkzFJSyOej8y8cQRh2z3kGw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peerDependencies": {
|
||||
"pg": "^8"
|
||||
@@ -19540,7 +19535,6 @@
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",
|
||||
"integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",
|
||||
"dev": true,
|
||||
"license": "ISC",
|
||||
"engines": {
|
||||
"node": ">=4.0.0"
|
||||
@@ -19550,7 +19544,6 @@
|
||||
"version": "3.14.0",
|
||||
"resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.14.0.tgz",
|
||||
"integrity": "sha512-gKtPkFdQPU3DksooVLi9LsjZxrsBUZIpa+7aVx+LV5pNh0KzP4Zleud2po+ConrxbuXGBJ6Hfer6hdgpIBpBaw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peerDependencies": {
|
||||
"pg": ">=8.0"
|
||||
@@ -19560,14 +19553,12 @@
|
||||
"version": "1.16.1",
|
||||
"resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.16.1.tgz",
|
||||
"integrity": "sha512-p9VOFMiHB/ZbJATetbg+99PxssTVSQRnyuPSQ67mN1+1KBOjZaZ83ZQzltnxPhJwSsC3nwVjJ10DVJlerbFzLg==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/pg-types": {
|
||||
"version": "2.2.0",
|
||||
"resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",
|
||||
"integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"pg-int8": "1.0.1",
|
||||
@@ -19584,7 +19575,6 @@
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",
|
||||
"integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=4"
|
||||
@@ -19594,7 +19584,6 @@
|
||||
"version": "1.0.7",
|
||||
"resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",
|
||||
"integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=0.10.0"
|
||||
@@ -19604,7 +19593,6 @@
|
||||
"version": "1.2.0",
|
||||
"resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",
|
||||
"integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"xtend": "^4.0.0"
|
||||
@@ -19617,7 +19605,6 @@
|
||||
"version": "1.0.5",
|
||||
"resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",
|
||||
"integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"split2": "^4.1.0"
|
||||
@@ -19921,7 +19908,6 @@
|
||||
"version": "3.0.4",
|
||||
"resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-3.0.4.tgz",
|
||||
"integrity": "sha512-nAUSGfSDGOaOAEGwqsRY27GPOea7CNipJPOA7lPbdEpx5Kg3qzdP0AaWC5MlhTWV9s4hFX39nomVZ+C4tnGOJQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
@@ -19931,7 +19917,6 @@
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz",
|
||||
"integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=0.10.0"
|
||||
@@ -19941,7 +19926,6 @@
|
||||
"version": "2.1.0",
|
||||
"resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-2.1.0.tgz",
|
||||
"integrity": "sha512-K7Juri8gtgXVcDfZttFKVmhglp7epKb1K4pgrkLxehjqkrgPhfG6OO8LHLkfaqkbpjNRnra018XwAr1yQFWGcA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
@@ -19951,7 +19935,6 @@
|
||||
"version": "4.1.0",
|
||||
"resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-4.1.0.tgz",
|
||||
"integrity": "sha512-nJLACcOHi2iyRuMJeW6CuIA0Dc0i1YEpRr26XDyHAfxPjYUJOFxwZjmOc5oFFctTTh/OlRAMlnlXEJvr170fJg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
@@ -22020,7 +22003,6 @@
|
||||
"version": "4.2.0",
|
||||
"resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
|
||||
"integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",
|
||||
"dev": true,
|
||||
"license": "ISC",
|
||||
"engines": {
|
||||
"node": ">= 10.x"
|
||||
@@ -24829,7 +24811,6 @@
|
||||
"version": "4.0.2",
|
||||
"resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
|
||||
"integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=0.4"
|
||||
@@ -25061,6 +25042,7 @@
|
||||
"@mikro-orm/core": "^7.2.1",
|
||||
"@mikro-orm/migrations": "^7.2.1",
|
||||
"@mikro-orm/nestjs": "^7.1.0",
|
||||
"@mikro-orm/postgresql": "^7.2.1",
|
||||
"@mikro-orm/seeder": "^7.2.1",
|
||||
"@mikro-orm/sql": "^7.2.1",
|
||||
"@mikro-orm/sqlite": "^7.2.1",
|
||||
@@ -25114,7 +25096,6 @@
|
||||
"@aws-lite/s3-types": "^0.2.8",
|
||||
"@eslint/js": "^10.0.1",
|
||||
"@mikro-orm/entity-generator": "^7.2.1",
|
||||
"@mikro-orm/postgresql": "^7.2.1",
|
||||
"@nestjs/testing": "^11.2.5",
|
||||
"@swc/core": "^1.16.2",
|
||||
"@trivago/prettier-plugin-sort-imports": "^6.0.2",
|
||||
|
||||
@@ -52,6 +52,8 @@ DEMO_MODE=false # Demo mode: resets data hourly
|
||||
# BACKUP_MAX_DECOMPRESSED_MB=5120 # Max size (MB) a restore archive may unpack to, on both restore paths: the zip-bomb guard. Raise it if a large backup is refused with "Backup exceeds the maximum decompressed size". (default: 5120)
|
||||
# BACKUP_UPLOAD_LIMIT_MB=500 # Max size (MB) of a backup archive you can upload when restoring. Raise it if your backup exceeds 500 MB. If you sit behind a reverse proxy, raise its upload limit too (e.g. nginx client_max_body_size).
|
||||
|
||||
# TREK_DB_URL=postgres://trek:password@db:5432/trek # Run on a Postgres database instead of the SQLite file in data/. Not supported yet: a boot with it set is refused. Cannot be combined with TREK_DB_FILE, TREK_DB_JOURNAL_MODE or TREK_DB_SYNCHRONOUS.
|
||||
# TREK_DB_POOL_MAX=10 # Most connections TREK opens to the Postgres database (1 to 100). Unused on SQLite. (default: 10)
|
||||
# TREK_DB_JOURNAL_MODE=WAL # SQLite journal mode (DELETE, TRUNCATE, PERSIST, MEMORY, WAL, OFF). WAL is fastest and the right choice on local disk. Set DELETE if the data directory lives on network storage (Azure App Service, an SMB/NFS volume on a NAS) — WAL coordinates writers through a shared-memory file that those filesystems do not implement reliably. The mode is stored inside the database file, so it survives a restart; changing this applies on the next boot.
|
||||
# TREK_DB_PRE_MIGRATE_SNAPSHOT=true # Copy the database to data/pre-migrate-<schema>-<time>.db before a startup migrates it, so the previous image can run on the copy again. A copy that cannot be written stops the startup; false migrates without one.
|
||||
# TREK_DB_PRE_MIGRATE_SNAPSHOT_KEEP=3 # How many pre-migrate copies to keep, newest first (1 to 100). (default: 3)
|
||||
|
||||
@@ -25,6 +25,7 @@ npm run db:call-graph -- --sync --tx [--domains nest/days,…] # sync DB-touchi
|
||||
npm run lint:size # no file in src/ or scripts/ past 1000 lines, none in tests/ past 2000, longer ones only shrink (CI gate)
|
||||
npm run lint:boundaries # import cycles and cross-domain reach-ins may only shrink (CI gate)
|
||||
npm run lint:dialect # SQLite-only SQL spellings under src/ may only shrink (CI gate)
|
||||
npm run lint:engine-dispatch # isSqlite(/isPostgres(/platform instanceof outside src/db/dialect/ and src/db/engine/ may only shrink (CI gate)
|
||||
npm run probe:pg # dialect helpers + the repository statements it reaches against Postgres (CI job postgres-probe; needs TREK_PG_PROBE_URL)
|
||||
npm run lint:tx # methods writing more than once outside one transaction may only shrink (CI gate)
|
||||
npm run lint:test-sql # raw better-sqlite3 statements in tests/ may only shrink (CI gate)
|
||||
|
||||
+2
-1
@@ -21,6 +21,7 @@
|
||||
"lint:strict": "node scripts/strict-lint.mjs",
|
||||
"lint:boundaries": "node scripts/import-boundaries.mjs",
|
||||
"lint:dialect": "node scripts/dialect-lint.mjs",
|
||||
"lint:engine-dispatch": "node scripts/engine-dispatch-ratchet.mjs",
|
||||
"lint:tx": "node scripts/tx-writes.mjs",
|
||||
"lint:test-sql": "node scripts/test-sql-ratchet.mjs",
|
||||
"lint:test-mocks": "node scripts/test-mocks-ratchet.mjs",
|
||||
@@ -63,6 +64,7 @@
|
||||
"@mikro-orm/core": "^7.2.1",
|
||||
"@mikro-orm/migrations": "^7.2.1",
|
||||
"@mikro-orm/nestjs": "^7.1.0",
|
||||
"@mikro-orm/postgresql": "^7.2.1",
|
||||
"@mikro-orm/seeder": "^7.2.1",
|
||||
"@mikro-orm/sql": "^7.2.1",
|
||||
"@mikro-orm/sqlite": "^7.2.1",
|
||||
@@ -116,7 +118,6 @@
|
||||
"@aws-lite/s3-types": "^0.2.8",
|
||||
"@eslint/js": "^10.0.1",
|
||||
"@mikro-orm/entity-generator": "^7.2.1",
|
||||
"@mikro-orm/postgresql": "^7.2.1",
|
||||
"@nestjs/testing": "^11.2.5",
|
||||
"@swc/core": "^1.16.2",
|
||||
"@trivago/prettier-plugin-sort-imports": "^6.0.2",
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"counts": {}
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
#!/usr/bin/env node
|
||||
/*
|
||||
* lint:engine-dispatch: code outside the dialect layer and the engine port
|
||||
* never asks which database it runs on.
|
||||
*
|
||||
* What differs between SQLite and Postgres has two homes: the SQL helpers in
|
||||
* src/db/dialect/ and the engine implementations in src/db/engine/ (the
|
||||
* driver, the schema bootstrap, maintenance, backups). Anywhere else, an
|
||||
* `isSqlite(`/`isPostgres(` check, an `instanceof` on a platform class or a
|
||||
* comparison on the engine's name (`readEnv().db.engine === 'postgres'`) is a
|
||||
* branch a third engine would have to find by searching, so each file is held
|
||||
* to its entry in scripts/engine-dispatch-baseline.json and a file without one
|
||||
* holds none (scripts/lib/count-ratchet.mjs has the rules). Comments do not
|
||||
* count.
|
||||
*
|
||||
* npm run lint:engine-dispatch check against the baseline (CI)
|
||||
* npm run lint:engine-dispatch -- --update lower the counts; never raises or adds an entry
|
||||
*/
|
||||
import { cliMain, stripComments } from './lib/count-ratchet.mjs';
|
||||
|
||||
const DISPATCH = new RegExp(
|
||||
[
|
||||
/\bis(?:Sqlite|Postgres)\s*\(/.source,
|
||||
/\binstanceof\s+(?:Sqlite|BasePostgreSql|PostgreSql)Platform\b/.source,
|
||||
// A comparison on the engine's name: `db.engine === 'postgres'`, `engine.id !== 'sqlite'`.
|
||||
/\.(?:engine|id)\s*[!=]==?\s*['"](?:sqlite|postgres)['"]/.source,
|
||||
].join('|'),
|
||||
'g',
|
||||
);
|
||||
|
||||
/** Where an engine may be named: the dialect helpers and the engine port. */
|
||||
const ENGINE_HOMES = ['src/db/dialect/', 'src/db/engine/'];
|
||||
|
||||
/** The engine checks in one file's text, comments left out. */
|
||||
export function countEngineDispatch(text) {
|
||||
return stripComments(text).match(DISPATCH)?.length ?? 0;
|
||||
}
|
||||
|
||||
export const check = {
|
||||
name: 'engine-dispatch',
|
||||
script: 'lint:engine-dispatch',
|
||||
root: 'src',
|
||||
baseline: 'scripts/engine-dispatch-baseline.json',
|
||||
unit: 'engine check(s)',
|
||||
advice: 'Dispatch through a helper in src/db/dialect/ or the DatabaseEngine in src/db/engine/ instead.',
|
||||
include: (file) => file.endsWith('.ts') && !ENGINE_HOMES.some((home) => file.startsWith(home)),
|
||||
count: (text) => countEngineDispatch(text),
|
||||
};
|
||||
|
||||
cliMain(check, import.meta.url);
|
||||
@@ -338,7 +338,14 @@ export function deriveBackup(raw: RawEnv) {
|
||||
|
||||
export function deriveDb(raw: RawEnv) {
|
||||
const durability = resolveDurability(raw.TREK_DB_JOURNAL_MODE, raw.TREK_DB_SYNCHRONOUS);
|
||||
const url = raw.TREK_DB_URL?.trim() || undefined;
|
||||
return {
|
||||
/** Which database this process runs on: Postgres when TREK_DB_URL names one, the SQLite file otherwise. */
|
||||
engine: (url ? 'postgres' : 'sqlite') as 'sqlite' | 'postgres',
|
||||
/** The Postgres connection URL; it carries the password, so never log it (db/engine/redact-url.ts). */
|
||||
url,
|
||||
/** The largest number of connections the Postgres pool opens. */
|
||||
poolMax: Math.min(positiveIntOr(raw.TREK_DB_POOL_MAX, 10), 100),
|
||||
trekDbFile: raw.TREK_DB_FILE,
|
||||
/** Resolved journal_mode — WAL unless the operator asked for something else (network storage needs DELETE/TRUNCATE). */
|
||||
journalMode: durability.journalMode,
|
||||
|
||||
@@ -51,6 +51,14 @@ const url = optionalWith((v) => {
|
||||
return false;
|
||||
}
|
||||
}, 'must be a valid URL (with protocol)');
|
||||
/** A Postgres connection URL: TREK_DB_URL names no other engine. */
|
||||
const postgresUrl = optionalWith((v) => {
|
||||
try {
|
||||
return ['postgres:', 'postgresql:'].includes(new URL(v).protocol);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}, 'must be a postgres:// or postgresql:// URL');
|
||||
const duration = optionalWith((v) => parseDurationMs(v) != null, 'must be a duration like "1h", "7d" or "30d"');
|
||||
const oneOf = (values: string[]) =>
|
||||
optionalWith((v) => values.includes(v.toLowerCase()), `must be one of: ${values.join(', ')}`);
|
||||
@@ -156,6 +164,10 @@ export const envSchema = z.object({
|
||||
Path: anyString,
|
||||
|
||||
// Data / paths
|
||||
// A Postgres database instead of the SQLite file. Its value is a secret (it
|
||||
// carries the password), so it is masked in the boot report.
|
||||
TREK_DB_URL: postgresUrl,
|
||||
TREK_DB_POOL_MAX: integer(1, 100),
|
||||
TREK_DB_FILE: anyString,
|
||||
// The two SQLite pragmas are deliberately unvalidated: a typo here must fall
|
||||
// back and warn rather than abort boot, because reset-admin.js — the way out
|
||||
|
||||
@@ -33,6 +33,7 @@ export const SECRET_ENV_KEYS: ReadonlySet<string> = new Set([
|
||||
'MAPBOX_ACCESS_TOKEN',
|
||||
'CARTO_API_KEY',
|
||||
'DEMO_ADMIN_PASS',
|
||||
'TREK_DB_URL',
|
||||
]);
|
||||
|
||||
/** How a variable's value appears in the boot report. */
|
||||
@@ -95,7 +96,7 @@ export function validateEnvAtBoot(raw: RawEnv = process.env as RawEnv): void {
|
||||
const key = String(issue.path[0]);
|
||||
return ` - ${key}=${reportedValue(key, raw[key])}: ${issue.message}`;
|
||||
});
|
||||
lines.push(...managedPreconditions(raw));
|
||||
lines.push(...databasePreconditions(raw), ...managedPreconditions(raw));
|
||||
if (lines.length === 0) return;
|
||||
console.error(`Invalid environment configuration:\n${lines.join('\n')}`);
|
||||
throw new Error(
|
||||
@@ -104,6 +105,33 @@ export function validateEnvAtBoot(raw: RawEnv = process.env as RawEnv): void {
|
||||
);
|
||||
}
|
||||
|
||||
/** Why a boot with TREK_DB_URL is refused until the Postgres engine can run TREK. */
|
||||
export const POSTGRES_NOT_READY =
|
||||
'Postgres support is not complete in this release; unset TREK_DB_URL to run on the SQLite file.';
|
||||
|
||||
/** The SQLite settings that mean nothing next to TREK_DB_URL. */
|
||||
const SQLITE_ONLY_DB_KEYS = ['TREK_DB_FILE', 'TREK_DB_JOURNAL_MODE', 'TREK_DB_SYNCHRONOUS'] as const;
|
||||
|
||||
/**
|
||||
* Cross-field rules for the database engine. TREK_DB_URL beside a SQLite
|
||||
* setting names two databases, and either guess would leave the operator on
|
||||
* the one they did not mean: it refuses instead (fail closed, see
|
||||
* server/CLAUDE.md). The URL itself never reaches the report.
|
||||
*/
|
||||
function databasePreconditions(raw: RawEnv): string[] {
|
||||
if (!raw.TREK_DB_URL?.trim()) return [];
|
||||
const problems: string[] = [];
|
||||
const sqliteKeys = SQLITE_ONLY_DB_KEYS.filter((key) => raw[key]?.trim());
|
||||
if (sqliteKeys.length > 0) {
|
||||
problems.push(
|
||||
` - ambiguous database configuration: TREK_DB_URL is set together with ${sqliteKeys.join(', ')}, ` +
|
||||
'which only apply to the SQLite file. Unset one side.',
|
||||
);
|
||||
}
|
||||
problems.push(` - TREK_DB_URL: ${POSTGRES_NOT_READY}`);
|
||||
return problems;
|
||||
}
|
||||
|
||||
/**
|
||||
* Cross-field rules that only apply to a centrally administered install.
|
||||
*
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
import type { Options } from '@mikro-orm/core';
|
||||
|
||||
/** The databases TREK can be pointed at. MySQL/MariaDB are out of scope (server/CLAUDE.md). */
|
||||
type EngineId = 'sqlite' | 'postgres';
|
||||
|
||||
/**
|
||||
* One database engine: what the ORM needs to reach it and how to name it in a
|
||||
* log line. Code that differs by engine dispatches through this, or through
|
||||
* the helpers in `db/dialect/`, never through its own platform check
|
||||
* (`lint:engine-dispatch`).
|
||||
*
|
||||
* Free of Nest like every other file under `db/`: `src/db` does not import
|
||||
* `src/nest` (`lint:boundaries`).
|
||||
*/
|
||||
export interface DatabaseEngine {
|
||||
readonly id: EngineId;
|
||||
/** Whether this release can run TREK on the engine; `buildOrmConfig` refuses one that cannot. */
|
||||
readonly ready: boolean;
|
||||
/** For log lines and errors: the file path, or the URL without its password. */
|
||||
describe(): string;
|
||||
/** What MikroORM needs to reach the database: the driver and its target, the pool, driver options. */
|
||||
ormOptions(): Partial<Options>;
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
import { readEnv } from '../../app-config';
|
||||
import { POSTGRES_NOT_READY } from '../../app-config/env';
|
||||
import { ALL_ENTITIES } from '../entities';
|
||||
import type { DatabaseEngine } from './engine';
|
||||
import { PostgresEngine } from './postgres/postgres-engine';
|
||||
import { SqliteEngine } from './sqlite/sqlite-engine';
|
||||
import { defineConfig, type Options } from '@mikro-orm/core';
|
||||
import { Migrator } from '@mikro-orm/migrations';
|
||||
import { SeedManager } from '@mikro-orm/seeder';
|
||||
|
||||
export type { DatabaseEngine } from './engine';
|
||||
|
||||
/**
|
||||
* The engine this process runs on, from `TREK_DB_URL` (`readEnv().db`): the
|
||||
* SQLite file unless the URL names a Postgres database. Read afresh on every
|
||||
* call, like everything `readEnv()` answers, so nothing caches a choice a test
|
||||
* or a restart changes.
|
||||
*/
|
||||
export function resolveEngine(db: ReturnType<typeof readEnv>['db'] = readEnv().db): DatabaseEngine {
|
||||
if (db.engine === 'postgres' && db.url) return new PostgresEngine({ url: db.url, poolMax: db.poolMax });
|
||||
return new SqliteEngine();
|
||||
}
|
||||
|
||||
/**
|
||||
* The MikroORM config for `engine`: what every engine shares (entities,
|
||||
* migrations, seeders) and what the engine adds. `src/mikro-orm.config.ts`
|
||||
* exports it for the CLI and the Nest container, so both reach the same
|
||||
* database. An engine this release cannot run is refused here, before
|
||||
* anything connects; the boot check refuses it earlier with the same words.
|
||||
*/
|
||||
export function buildOrmConfig(engine: DatabaseEngine): Options {
|
||||
if (!engine.ready) throw new Error(`${POSTGRES_NOT_READY} (${engine.describe()})`);
|
||||
return defineConfig({
|
||||
entities: ALL_ENTITIES,
|
||||
extensions: [Migrator, SeedManager],
|
||||
migrations: {
|
||||
path: 'dist/db/migrations',
|
||||
pathTs: 'src/db/migrations',
|
||||
snapshot: false,
|
||||
},
|
||||
seeder: {
|
||||
path: 'dist/db/seeders',
|
||||
pathTs: 'src/db/seeders',
|
||||
defaultSeeder: 'DatabaseSeeder',
|
||||
},
|
||||
...engine.ormOptions(),
|
||||
}) as Options;
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
import type { DatabaseEngine } from '../engine';
|
||||
import { redactUrl } from '../redact-url';
|
||||
import { PostgreSqlDriver } from '@mikro-orm/postgresql';
|
||||
|
||||
import { readFileSync } from 'node:fs';
|
||||
|
||||
/** What `TREK_DB_URL` and `TREK_DB_POOL_MAX` resolve to (`readEnv().db`). */
|
||||
export interface PostgresSettings {
|
||||
url: string;
|
||||
poolMax: number;
|
||||
}
|
||||
|
||||
/** The `ssl` option of a `pg` pool, as `sslOptionFor` builds it. */
|
||||
export type PgSslOption = false | { rejectUnauthorized: true; ca?: string };
|
||||
|
||||
/** The sslmode values that verify the server, read the way the `pg` driver reads them. */
|
||||
const VERIFYING_MODES = new Set(['require', 'verify-ca', 'verify-full']);
|
||||
|
||||
/**
|
||||
* The `ssl` option for the pool, from the URL's `sslmode` and `sslrootcert`.
|
||||
*
|
||||
* MikroORM builds the pool from the URL's host, port, user, password and
|
||||
* database alone and drops the rest of the query, so an `sslmode` in
|
||||
* `TREK_DB_URL` would be ignored without this. `require`, `verify-ca` and
|
||||
* `verify-full` all verify the server's certificate, as the `pg` driver itself
|
||||
* treats them; `disable` turns SSL off, and no `sslmode` leaves the driver's
|
||||
* default (no SSL). A mode that would connect without verifying (`prefer`,
|
||||
* `allow`, `no-verify`) or that is misspelt refuses, rather than quietly
|
||||
* connecting with less than was asked. `sslrootcert` names the CA file a host
|
||||
* that signs its own certificates needs (Supabase's, say).
|
||||
*/
|
||||
export function sslOptionFor(url: string): PgSslOption | undefined {
|
||||
const params = new URL(url).searchParams;
|
||||
const mode = params.get('sslmode');
|
||||
const rootCert = params.get('sslrootcert');
|
||||
if (mode === null) {
|
||||
if (rootCert !== null) throw new Error('TREK_DB_URL names an sslrootcert without sslmode=verify-full.');
|
||||
return undefined;
|
||||
}
|
||||
if (mode === 'disable') return false;
|
||||
if (!VERIFYING_MODES.has(mode)) {
|
||||
throw new Error(
|
||||
`TREK_DB_URL has sslmode "${mode}"; use disable, require, verify-ca or verify-full (all but disable verify the server).`,
|
||||
);
|
||||
}
|
||||
if (rootCert === null) return { rejectUnauthorized: true };
|
||||
try {
|
||||
return { rejectUnauthorized: true, ca: readFileSync(rootCert, 'utf8') };
|
||||
} catch (error) {
|
||||
throw new Error(`TREK_DB_URL names sslrootcert ${rootCert}, which cannot be read.`, { cause: error });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The Postgres engine: MikroORM's own `pg` driver on `TREK_DB_URL`, with a
|
||||
* pool of at most `TREK_DB_POOL_MAX` connections that keeps none open while
|
||||
* idle. Not ready in this release: `buildOrmConfig` refuses it until the
|
||||
* engine can bring a database to the schema.
|
||||
*/
|
||||
export class PostgresEngine implements DatabaseEngine {
|
||||
readonly id = 'postgres';
|
||||
readonly ready = false;
|
||||
|
||||
constructor(private readonly settings: PostgresSettings) {}
|
||||
|
||||
describe(): string {
|
||||
return `Postgres ${redactUrl(this.settings.url)}`;
|
||||
}
|
||||
|
||||
ormOptions() {
|
||||
const ssl = sslOptionFor(this.settings.url);
|
||||
return {
|
||||
driver: PostgreSqlDriver,
|
||||
clientUrl: this.settings.url,
|
||||
pool: { min: 0, max: this.settings.poolMax },
|
||||
driverOptions: ssl === undefined ? {} : { ssl },
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
/**
|
||||
* A connection URL fit for a log line: the password masked and the query
|
||||
* dropped, since libpq reads a password from the query too
|
||||
* (`?password=...`). A value that does not parse is not echoed at all.
|
||||
*/
|
||||
export function redactUrl(url: string): string {
|
||||
let parsed: URL;
|
||||
try {
|
||||
parsed = new URL(url);
|
||||
} catch {
|
||||
return '<unparseable URL>';
|
||||
}
|
||||
if (parsed.password) parsed.password = '***';
|
||||
parsed.search = '';
|
||||
parsed.hash = '';
|
||||
return parsed.toString();
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
import { resolveDbPath } from '../../db-path';
|
||||
import { SharedSqliteDriver } from '../../orm-driver';
|
||||
import type { DatabaseEngine } from '../engine';
|
||||
|
||||
/**
|
||||
* The SQLite file engine, TREK's default: one better-sqlite3 handle that
|
||||
* `db/database.ts` owns and the ORM's bound driver shares (`db/orm-driver.ts`),
|
||||
* on `TREK_DB_FILE` or `data/travel.db` (`:memory:` under test).
|
||||
*/
|
||||
export class SqliteEngine implements DatabaseEngine {
|
||||
readonly id = 'sqlite';
|
||||
readonly ready = true;
|
||||
|
||||
describe(): string {
|
||||
return `SQLite file ${resolveDbPath()}`;
|
||||
}
|
||||
|
||||
ormOptions() {
|
||||
return { driver: SharedSqliteDriver, dbName: resolveDbPath() };
|
||||
}
|
||||
}
|
||||
@@ -1,30 +1,8 @@
|
||||
import { resolveDbPath } from './db/db-path';
|
||||
import { ALL_ENTITIES } from './db/entities';
|
||||
import { SharedSqliteDriver } from './db/orm-driver';
|
||||
import { Migrator } from '@mikro-orm/migrations';
|
||||
import { SeedManager } from '@mikro-orm/seeder';
|
||||
import { defineConfig } from '@mikro-orm/sqlite';
|
||||
import { buildOrmConfig, resolveEngine } from './db/engine';
|
||||
|
||||
export default defineConfig({
|
||||
entities: ALL_ENTITIES,
|
||||
extensions: [Migrator, SeedManager],
|
||||
// Reuses the connection db/database.ts owns rather than opening a second one.
|
||||
// See orm-driver.ts — with `:memory:` in tests, a second connection would be a
|
||||
// second, empty database.
|
||||
driver: SharedSqliteDriver,
|
||||
migrations: {
|
||||
path: 'dist/db/migrations',
|
||||
pathTs: 'src/db/migrations',
|
||||
snapshot: false,
|
||||
},
|
||||
seeder: {
|
||||
path: 'dist/db/seeders',
|
||||
pathTs: 'src/db/seeders',
|
||||
defaultSeeder: 'DatabaseSeeder',
|
||||
},
|
||||
// Same resolution database.ts uses: `:memory:` under test, TREK_DB_FILE when
|
||||
// set, else data/travel.db. The driver above ignores it for the connection
|
||||
// itself, but the CLI and the migrator's logging still read it, and naming a
|
||||
// different file here than the one actually open is a trap.
|
||||
dbName: resolveDbPath(),
|
||||
});
|
||||
/**
|
||||
* The one MikroORM config, for the CLI and the Nest container alike: the
|
||||
* engine `TREK_DB_URL` names (the SQLite file by default) and what every
|
||||
* engine shares (`db/engine/index.ts`).
|
||||
*/
|
||||
export default buildOrmConfig(resolveEngine());
|
||||
|
||||
@@ -371,6 +371,24 @@ describe('deriveDb: the pre-migration snapshot', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('deriveDb: the engine', () => {
|
||||
it('DB-ENGINE-001: SQLite unless TREK_DB_URL names a database', () => {
|
||||
expect(deriveDb({})).toMatchObject({ engine: 'sqlite', url: undefined });
|
||||
expect(deriveDb({ TREK_DB_URL: ' ' })).toMatchObject({ engine: 'sqlite', url: undefined });
|
||||
expect(deriveDb({ TREK_DB_URL: ' postgres://u:p@db:5432/trek ' })).toMatchObject({
|
||||
engine: 'postgres',
|
||||
url: 'postgres://u:p@db:5432/trek',
|
||||
});
|
||||
});
|
||||
|
||||
it('DB-ENGINE-002: a pool of ten connections unless told otherwise, from 1 to 100', () => {
|
||||
expect(deriveDb({}).poolMax).toBe(10);
|
||||
expect(deriveDb({ TREK_DB_POOL_MAX: '25' }).poolMax).toBe(25);
|
||||
expect(deriveDb({ TREK_DB_POOL_MAX: '0' }).poolMax).toBe(10);
|
||||
expect(deriveDb({ TREK_DB_POOL_MAX: '500' }).poolMax).toBe(100);
|
||||
});
|
||||
});
|
||||
|
||||
describe('deriveNet', () => {
|
||||
it('ALLOW_INTERNAL_NETWORK coerces the boolean-like family (ssrfGuard)', () => {
|
||||
expect(deriveNet({ ALLOW_INTERNAL_NETWORK: 'True' }).allowInternalNetwork).toBe(true);
|
||||
|
||||
@@ -341,3 +341,43 @@ describe('validateEnvAtBoot: the pre-migration snapshot switches', () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('validateEnvAtBoot: the database engine', () => {
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it('DB-BOOT-001: refuses a TREK_DB_URL that is not a postgres:// or postgresql:// URL, without printing it', () => {
|
||||
for (const value of ['mysql://u:secret@db/trek', 'not a url', 'sqlite:///data/travel.db']) {
|
||||
const { printed, thrown } = bootReport({ TREK_DB_URL: value });
|
||||
expect(thrown, value).toMatch(/Invalid environment configuration/);
|
||||
expect(printed, value).toContain('TREK_DB_URL=***');
|
||||
expect(printed, value).not.toContain('secret');
|
||||
}
|
||||
});
|
||||
|
||||
it('DB-BOOT-002: masks TREK_DB_URL like every other secret', () => {
|
||||
expect(SECRET_ENV_KEYS.has('TREK_DB_URL')).toBe(true);
|
||||
});
|
||||
|
||||
it('DB-BOOT-003: refuses TREK_DB_URL together with a SQLite setting, naming both', () => {
|
||||
for (const key of ['TREK_DB_FILE', 'TREK_DB_JOURNAL_MODE', 'TREK_DB_SYNCHRONOUS']) {
|
||||
const { thrown, printed } = bootReport({ TREK_DB_URL: 'postgres://u:p@db/trek', [key]: 'x' });
|
||||
expect(thrown, key).toMatch(/Invalid environment configuration/);
|
||||
expect(printed, key).toContain('ambiguous database configuration');
|
||||
expect(printed, key).toContain(key);
|
||||
}
|
||||
});
|
||||
|
||||
it('DB-BOOT-004: refuses a Postgres database while this release cannot run on one', () => {
|
||||
const { thrown, printed } = bootReport({ TREK_DB_URL: 'postgresql://u:p@db:5432/trek?sslmode=require' });
|
||||
expect(thrown).toMatch(/Invalid environment configuration/);
|
||||
expect(printed).toContain('Postgres support is not complete in this release');
|
||||
expect(printed).not.toContain('u:p@');
|
||||
});
|
||||
|
||||
it('DB-BOOT-005: refuses a pool size outside 1 to 100', () => {
|
||||
expect(bootReport({ TREK_DB_POOL_MAX: '0' }).thrown).toMatch(/Invalid environment configuration/);
|
||||
expect(() => validateEnvAtBoot({ TREK_DB_POOL_MAX: '20' })).not.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,110 @@
|
||||
import { deriveDb } from '../../../../src/app-config/derive';
|
||||
import { POSTGRES_NOT_READY } from '../../../../src/app-config/env';
|
||||
import { buildOrmConfig, resolveEngine } from '../../../../src/db/engine';
|
||||
import { sslOptionFor } from '../../../../src/db/engine/postgres/postgres-engine';
|
||||
import { redactUrl } from '../../../../src/db/engine/redact-url';
|
||||
import { SharedSqliteDriver } from '../../../../src/db/orm-driver';
|
||||
import { PostgreSqlDriver } from '@mikro-orm/postgresql';
|
||||
|
||||
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
const PG_URL = 'postgresql://trek:s3cret@db.example:5432/trek';
|
||||
|
||||
describe('resolveEngine', () => {
|
||||
it('ENGINE-001: SQLite when no TREK_DB_URL is set, which is the test environment', () => {
|
||||
const engine = resolveEngine();
|
||||
expect(engine.id).toBe('sqlite');
|
||||
expect(engine.ready).toBe(true);
|
||||
expect(engine.ormOptions()).toEqual({ driver: SharedSqliteDriver, dbName: ':memory:' });
|
||||
expect(engine.describe()).toBe('SQLite file :memory:');
|
||||
});
|
||||
|
||||
it('ENGINE-002: Postgres when TREK_DB_URL names a database, with the pool size and no SSL by default', () => {
|
||||
const engine = resolveEngine(deriveDb({ TREK_DB_URL: PG_URL, TREK_DB_POOL_MAX: '7' }));
|
||||
expect(engine.id).toBe('postgres');
|
||||
expect(engine.ready).toBe(false);
|
||||
expect(engine.ormOptions()).toEqual({
|
||||
driver: PostgreSqlDriver,
|
||||
clientUrl: PG_URL,
|
||||
pool: { min: 0, max: 7 },
|
||||
driverOptions: {},
|
||||
});
|
||||
});
|
||||
|
||||
it('ENGINE-003: describes a Postgres database without its password', () => {
|
||||
const engine = resolveEngine(deriveDb({ TREK_DB_URL: `${PG_URL}?sslmode=require&password=other` }));
|
||||
expect(engine.describe()).toBe('Postgres postgresql://trek:***@db.example:5432/trek');
|
||||
expect(engine.describe()).not.toContain('s3cret');
|
||||
expect(engine.describe()).not.toContain('other');
|
||||
});
|
||||
});
|
||||
|
||||
describe('sslOptionFor', () => {
|
||||
it('ENGINE-010: maps sslmode onto the pg pool option (MikroORM drops the URL query), always verifying the certificate', () => {
|
||||
// As the pg driver itself reads them: require and verify-ca are verify-full.
|
||||
expect(sslOptionFor(PG_URL)).toBeUndefined();
|
||||
expect(sslOptionFor(`${PG_URL}?sslmode=disable`)).toBe(false);
|
||||
for (const mode of ['require', 'verify-ca', 'verify-full']) {
|
||||
expect(sslOptionFor(`${PG_URL}?sslmode=${mode}`), mode).toEqual({ rejectUnauthorized: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('ENGINE-011: refuses an sslmode it does not know or that would skip verification', () => {
|
||||
expect(() => sslOptionFor(`${PG_URL}?sslmode=requrie`)).toThrow(/sslmode "requrie"/);
|
||||
expect(() => sslOptionFor(`${PG_URL}?sslmode=no-verify`)).toThrow(/sslmode "no-verify"/);
|
||||
expect(() => sslOptionFor(`${PG_URL}?sslmode=prefer`)).toThrow(/sslmode "prefer"/);
|
||||
});
|
||||
|
||||
it('ENGINE-012: passes the SSL option to the pool when the URL asks for one', () => {
|
||||
const engine = resolveEngine(deriveDb({ TREK_DB_URL: `${PG_URL}?sslmode=require` }));
|
||||
expect(engine.ormOptions().driverOptions).toEqual({ ssl: { rejectUnauthorized: true } });
|
||||
});
|
||||
|
||||
it('ENGINE-013: trusts the CA in sslrootcert (a managed host that signs its own certificates), and refuses a missing file', () => {
|
||||
const dir = mkdtempSync(path.join(tmpdir(), 'trek-ca-'));
|
||||
try {
|
||||
const ca = path.join(dir, 'root.crt');
|
||||
writeFileSync(ca, '-----BEGIN CERTIFICATE-----\nprobe\n-----END CERTIFICATE-----\n');
|
||||
expect(sslOptionFor(`${PG_URL}?sslmode=verify-full&sslrootcert=${encodeURIComponent(ca)}`)).toEqual({
|
||||
rejectUnauthorized: true,
|
||||
ca: readFileSync(ca, 'utf8'),
|
||||
});
|
||||
expect(() =>
|
||||
sslOptionFor(`${PG_URL}?sslmode=require&sslrootcert=${encodeURIComponent(path.join(dir, 'no.crt'))}`),
|
||||
).toThrow(/sslrootcert/);
|
||||
expect(() => sslOptionFor(`${PG_URL}?sslrootcert=${encodeURIComponent(ca)}`)).toThrow(
|
||||
/sslrootcert without sslmode/,
|
||||
);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('redactUrl', () => {
|
||||
it('ENGINE-020: hides the password and drops the query, which can carry one too', () => {
|
||||
expect(redactUrl('postgres://u:p@h/d?password=x&sslmode=require')).toBe('postgres://u:***@h/d');
|
||||
expect(redactUrl('postgres://u@h:5433/d')).toBe('postgres://u@h:5433/d');
|
||||
expect(redactUrl('not a url')).toBe('<unparseable URL>');
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildOrmConfig', () => {
|
||||
it('ENGINE-030: the SQLite config carries the entities, migrations and seeders with the engine options', () => {
|
||||
const config = buildOrmConfig(resolveEngine());
|
||||
expect(config.driver).toBe(SharedSqliteDriver);
|
||||
expect(config.dbName).toBe(':memory:');
|
||||
expect(config.migrations).toMatchObject({ pathTs: 'src/db/migrations', snapshot: false });
|
||||
expect(config.seeder).toMatchObject({ defaultSeeder: 'DatabaseSeeder' });
|
||||
expect(Array.isArray(config.entities) && config.entities.length).toBeGreaterThan(100);
|
||||
});
|
||||
|
||||
it('ENGINE-031: refuses an engine this release cannot run, before anything connects, without the password', () => {
|
||||
const engine = resolveEngine(deriveDb({ TREK_DB_URL: PG_URL }));
|
||||
expect(() => buildOrmConfig(engine)).toThrow(POSTGRES_NOT_READY);
|
||||
expect(() => buildOrmConfig(engine)).not.toThrow(/s3cret/);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,88 @@
|
||||
/**
|
||||
* scripts/engine-dispatch-ratchet.mjs (lint:engine-dispatch): an engine check
|
||||
* outside src/db/dialect/ and src/db/engine/ may only shrink. Runs as the real
|
||||
* script against throwaway server roots (--dir) and once on this repository,
|
||||
* so the committed baseline is checked by the unit suite.
|
||||
*/
|
||||
import { countEngineDispatch } from '../../../scripts/engine-dispatch-ratchet.mjs';
|
||||
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { afterEach, describe, expect, it } from 'vitest';
|
||||
|
||||
const SCRIPT = path.join(__dirname, '../../../scripts/engine-dispatch-ratchet.mjs');
|
||||
const SERVER_ROOT = path.join(__dirname, '../../..');
|
||||
const BASELINE = 'scripts/engine-dispatch-baseline.json';
|
||||
|
||||
const roots: string[] = [];
|
||||
|
||||
function serverRoot(files: Record<string, string>, counts: Record<string, number> = {}): string {
|
||||
const dir = mkdtempSync(path.join(tmpdir(), 'trek-engine-dispatch-'));
|
||||
roots.push(dir);
|
||||
mkdirSync(path.join(dir, 'src'), { recursive: true });
|
||||
mkdirSync(path.join(dir, 'scripts'), { recursive: true });
|
||||
for (const [file, text] of Object.entries(files)) {
|
||||
mkdirSync(path.dirname(path.join(dir, file)), { recursive: true });
|
||||
writeFileSync(path.join(dir, file), text);
|
||||
}
|
||||
writeFileSync(path.join(dir, BASELINE), JSON.stringify({ counts }));
|
||||
return dir;
|
||||
}
|
||||
|
||||
function run(dir: string, ...args: string[]): { status: number | null; out: string } {
|
||||
try {
|
||||
const stdout = execFileSync(process.execPath, [SCRIPT, `--dir=${dir}`, ...args], {
|
||||
encoding: 'utf8',
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
});
|
||||
return { status: 0, out: stdout };
|
||||
} catch (err) {
|
||||
const { status, stdout, stderr } = err as { status: number | null; stdout: string; stderr: string };
|
||||
return { status, out: `${stdout}${stderr}` };
|
||||
}
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
for (const dir of roots.splice(0)) rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// Spelled in pieces so this file does not read as an engine check itself.
|
||||
const IS_SQLITE = ['isSql', 'ite(platform)'].join('');
|
||||
const IS_POSTGRES = ['isPost', 'gres(p)'].join('');
|
||||
const INSTANCE = ['x instanceof ', 'SqlitePlatform'].join('');
|
||||
|
||||
describe('lint:engine-dispatch', () => {
|
||||
it('ENGDISP-001: counts the platform checks and instanceof tests, not the ones in comments', () => {
|
||||
expect(countEngineDispatch(`if (${IS_SQLITE}) a(); else if (${IS_POSTGRES}) b();`)).toBe(2);
|
||||
expect(countEngineDispatch(`const y = ${INSTANCE};`)).toBe(1);
|
||||
const BY_NAME = ["if (readEnv().db.engine === 'post", 'gres\') a(); if (engine.id !== "sql', 'ite") b();'].join('');
|
||||
expect(countEngineDispatch(BY_NAME)).toBe(2);
|
||||
expect(countEngineDispatch("if (user.id === 'admin') a();")).toBe(0);
|
||||
expect(countEngineDispatch(`// ${IS_SQLITE}\n/* ${INSTANCE} */\nconst ok = 1;`)).toBe(0);
|
||||
});
|
||||
|
||||
it('ENGDISP-002: the dialect layer and the engine port may dispatch, any other file may not', () => {
|
||||
const text = `export const x = ${IS_SQLITE};\n`;
|
||||
expect(run(serverRoot({ 'src/db/dialect/a.ts': text, 'src/db/engine/sqlite/b.ts': text })).status).toBe(0);
|
||||
const domain = run(serverRoot({ 'src/nest/trips/trips.service.ts': text }));
|
||||
expect(domain.status).toBe(1);
|
||||
expect(domain.out).toContain('src/nest/trips/trips.service.ts');
|
||||
});
|
||||
|
||||
it('ENGDISP-003: a baselined file is held at its count, and --update only lowers it', () => {
|
||||
const text = `export const x = ${IS_SQLITE};\n`;
|
||||
const file = 'src/db/repositories/A.repository.ts';
|
||||
expect(run(serverRoot({ [file]: text }, { [file]: 1 })).status).toBe(0);
|
||||
expect(run(serverRoot({ [file]: `${text}${text}` }, { [file]: 1 })).status).toBe(1);
|
||||
const stale = serverRoot({ [file]: 'export const x = 1;\n' }, { [file]: 1 });
|
||||
expect(run(stale).status).toBe(1);
|
||||
expect(run(stale, '--update').status).toBe(0);
|
||||
expect(JSON.parse(readFileSync(path.join(stale, BASELINE), 'utf8'))).toEqual({ counts: {} });
|
||||
});
|
||||
|
||||
it('ENGDISP-004: the repository holds its committed baseline', () => {
|
||||
expect(run(SERVER_ROOT).status).toBe(0);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user