mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-10-18 21:16:47 +00:00
bf26175724
The pre-migrate snapshot docstring said an older release refuses a database a newer TREK migrated. Only images from 5.0 on do; 4.3.x and earlier start on it without a warning. server/CLAUDE.md said anything the legacy baseline cannot map refuses the boot, which holds only above step 258.
150 lines
47 KiB
Markdown
150 lines
47 KiB
Markdown
# CLAUDE.md
|
|
|
|
Scope: the **`@trek/server`** workspace. See the repo-root `CLAUDE.md` for the monorepo picture and the philosophy. This file holds the server's commands, invariants and pitfalls; the code and `src/nest/README.md` hold the inventory.
|
|
|
|
## Commands (run from `server/`)
|
|
|
|
```bash
|
|
npm run dev # scripts/dev.mjs: tsc -w then node --watch dist/index.js (server on :3001)
|
|
npm run build # scripts/build.mjs — emits even on type errors (see below)
|
|
npm run typecheck # tsc --noEmit — the real type gate
|
|
npm run typecheck:tests # tsc over tests/ — CI runs this too; vitest green does NOT mean typed mocks compile
|
|
npm run lint # eslint, no fix
|
|
npm run lint:fix # eslint --fix
|
|
npm run lint:warnings # eslint errors fail, warnings per rule may only shrink (CI gate, scripts/eslint-baseline.json)
|
|
npm run lint:format # Prettier over src/ and tests/ outside scripts/format-baseline.json, which only shrinks (CI gate)
|
|
npm run lint:strict # strict type errors (strictNullChecks + noImplicitAny) per file may only shrink; a new file has none (CI gate)
|
|
npm run test # vitest run; also test:unit / test:integration / test:ws / test:e2e
|
|
npm run test:coverage # istanbul coverage; per-domain ratchet over src/nest/**
|
|
npm run gen:plugin-facts # regenerate the plugin-protocol tables into plugin-sdk/ + shared/
|
|
npm run check:plugin-facts # CI gate — fails if the generated copies drifted
|
|
npm run gen:db-types # regenerate the Kysely table types in src/db/kysely/ from the migrated schema
|
|
npm run check:db-types # CI gate: fails when src/db/kysely/ no longer matches the migrations
|
|
node scripts/coverage-thresholds.mjs # after test:coverage — prints the ratchet block for vitest.config.ts
|
|
npm run db:call-graph -- --sync --tx [--domains nest/days,…] # sync DB-touching methods + transaction sites (exit 1 if any)
|
|
npm run lint:size # no file in src/ or scripts/ past 1000 lines, none in tests/ past 2000, longer ones only shrink (CI gate)
|
|
npm run lint:boundaries # import cycles and cross-domain reach-ins may only shrink (CI gate)
|
|
npm run lint:dialect # SQLite-only SQL spellings under src/ may only shrink (CI gate)
|
|
npm run probe:pg # dialect helpers + the repository statements it reaches against Postgres (CI job postgres-probe; needs TREK_PG_PROBE_URL)
|
|
npm run lint:tx # methods writing more than once outside one transaction may only shrink (CI gate)
|
|
npm run lint:test-sql # raw better-sqlite3 statements in tests/ may only shrink (CI gate)
|
|
npm run lint:test-mocks # vi.mock of src/config, src/nest/realtime and src/db in tests/ may only shrink (CI gate)
|
|
npm run lint:test-new-service # new XService(/new XRepository( in tests/ may only shrink; zero-argument classes are allowed (CI gate)
|
|
npm run lint:query-api # MikroORM QueryBuilder calls in the repositories may only shrink (CI gate)
|
|
npm run lint:mcp-zod # inline z. calls per *.mcp.ts may only shrink; tool fields derive from @trek/shared (CI gate)
|
|
npm run lint:service-http # error-envelope idioms (status: 4xx in domain files, branches on result.error in callers) may only shrink (CI gate)
|
|
npm run lint:response-contracts # route handlers without @ResponseContract may only shrink (CI gate)
|
|
npm run contracts:dto-open # open shapes in the createZodDto schemas may only shrink (runs in the unit tests)
|
|
```
|
|
|
|
**`lint:boundaries`** (`scripts/import-boundaries.mjs`) parses every import under `src/` and holds seven lists (`src/nest-rpc`, `src/nest-mcp` and `src/mcp` count as one domain each, so moving a file out of `src/nest` does not take its edges out of the count) in `scripts/import-boundaries-baseline.json`: load-time file cycles (type-only and lazy imports, and the MikroORM entity relations, do not count), dependencies between two `nest/<domain>` folders that already depend on each other, shared-kernel files (`nest/common`, `nest/database`, `nest/app-config`, `nest/auth-core`) importing a domain, a domain importing another domain's internals, `src/db` importing `src/nest`, a file under a domain's `providers/` folder importing that domain's orchestrator (`<domain>.service.ts`) or `MapsService` (a provider knows how to ask one source; the orchestrator decides which one answers), and a domain reaching (`@InjectRepository(X)` or `em.getRepository(X)`) the repository of an entity another domain owns (`foreignRepositories`; the owner of every injected entity is listed in `scripts/repository-owners.json`, and an entity missing there stops the run). For trip visibility, inject `TripAccessService` (trip-membership) instead of `TripsRepository`. A domain's public surface is its `*.module`, `*.service`, `*.guard`, `*.decorator`, `*.types`, `*.interface`, `*.contract`, `*.constants` and `*.logger` files plus the short `PUBLIC_FILES` list in the script; anything else is an internal. A new entry fails the check, and so does a baseline entry that no longer occurs, so the fix lands with `npm run lint:boundaries -- --update`, which drops such entries and never adds one.
|
|
|
|
**`contracts:dto-open`** (`tests/unit/contracts/dto-open-shapes.test.ts`, wrapped by `scripts/dto-open-shapes.mjs`) finds every `export class X extends createZodDto(...)` under `src/`, imports it and counts the open shapes (loose object, record of unknown, `z.unknown()`/`z.any()`, `z.custom()`) in the schema the Zod pipe validates, with shared's `contracts:open` walk. Each DTO class is held to its entry in `scripts/dto-open-shapes-baseline.json`, and a new one may hold none, whatever its schema is called and wherever it is declared. A `createZodDto` call that is not an exported class, or two DTO classes with one name, fail the check. After narrowing a schema, `npm run contracts:dto-open -- --update` lowers the baseline (it never raises or adds an entry). It needs `shared/` built.
|
|
|
|
**`lint:tx`** (`scripts/tx-writes.mjs`) reads every class under `src/` and counts, per method, the writes it issues outside a `.transactional(...)` callback: a repository call that builds an INSERT/UPDATE/DELETE, a call to a method that writes (of its own class or of a constructor-injected one), a transaction holding writes as one. A write in a loop counts twice, of two branches only the larger counts, and a helper whose every call sits inside a transaction is not reported. A method at two or more is held to its entry in `scripts/tx-writes-baseline.json`; a new one fails, and so does an entry above what its method holds now, so a fix lands with `npm run lint:tx -- --update` (it never raises or adds an entry). `--explain` lists the line of every call that counted. Two JSDoc tags mark the deliberate cases, each with its reason in the tag text: `@txStandalone` on a method whose write stands on its own whatever its caller does (an audit row, a notification, a non-fatal derived refresh, an idempotent lazy provision) makes it count for nothing at its callers, and `@txIndependent` on a method whose writes are complete one by one on purpose (network I/O between them, a retention sweep in passes, one transaction per item of a batch) keeps it out of the report.
|
|
|
|
**`lint:size`** (`scripts/size-lint.mjs`) counts a line longer than 120 columns once per 120 columns it spans, so joining lines does not make room. Files under `tests/` are held the same way at 2000 lines against `scripts/test-size-baseline.json`. Files that were already longer are held at their length in `scripts/size-baseline.json`; a file that needs to grow past its entry is split by concern instead. An entry above what its file holds now, or for a file that is gone, fails as well, so after a split `npm run lint:size -- --update` lowers the baseline in the same change (it never raises or adds an entry).
|
|
|
|
**`lint:strict`** (`scripts/strict-lint.mjs`) runs tsc over `tsconfig.strict.json`, the build config with `strictNullChecks` and `noImplicitAny` on, and counts the errors per file against `scripts/strict-baseline.json`. A file may hold at most its entry and a file without one, a new file in particular, none; an entry above what its file holds now, or for a file that is gone, fails as well, so the change that fixes errors runs `npm run lint:strict -- --update` (it never raises or adds an entry). It is a ratchet only: the build stays on `strict: false` until the baseline is empty.
|
|
|
|
**`lint:test-sql`** (`scripts/test-sql-ratchet.mjs`) counts every `.prepare(` under `tests/` per file against `scripts/test-sql-baseline.json`: a file may hold at most its entry, a file without one none. Growth fails, and so does an entry above what its file holds now, so a conversion lands with `npm run lint:test-sql -- --update` (it only lowers and drops entries). Files whose subject is raw SQL (the dialect functions, PRAGMA introspection, the legacy upgrade fixtures) sit in the baseline's `exempt` list with a reason; a single statement that must stay raw carries a `test-sql-allow: <reason>` comment on its line or the line above. `tests/unit/scripts/test-sql-ratchet.test.ts` also runs it on the tree.
|
|
|
|
**`lint:test-mocks`** (`scripts/test-mocks-ratchet.mjs`) and **`lint:test-new-service`** (`scripts/test-new-service-ratchet.mjs`) share the count rules of `scripts/lib/count-ratchet.mjs` (`scripts/test-mocks-baseline.json`, `scripts/test-new-service-baseline.json`). The first counts every `vi.mock`/`vi.doMock` of `src/config`, `src/nest/realtime/**` and `src/db/**` per file under `tests/`: the setup file fixes the config (`tests/helpers/test-config.ts`, `overrideTestConfig`), broadcasts go to a `FakeRealtimeService` or `spyOnRealtime(app)`. The second counts `new <X>Service(` and `new <X>Repository(` under `tests/`, leaving out classes whose constructor takes nothing (derived from `src/` and `tests/helpers/`); a suite takes its subject from `createTestModule` (`tests/helpers/test-module.ts`) or from a booted container (`tests/helpers/test-app.ts`).
|
|
|
|
**`lint:query-api`** (`scripts/query-api-ratchet.mjs`) counts every `.qb(` and `createQueryBuilder(` under `src/db/repositories/` per file (the `TrekRepository` base that defines `qb` is left out) against `scripts/query-api-baseline.json`: a file may hold at most its entry, a file without one none, and an entry above what its file holds now fails until `npm run lint:query-api -- --update` lowers it (it never raises or adds one). The rule it holds is "Which query API" under SQL dialect below.
|
|
|
|
**`lint:mcp-zod`** (`scripts/mcp-inline-zod.mjs`) and **`lint:service-http`** (`scripts/service-http-ratchet.mjs`) share the per-file count rules of `scripts/lib/count-ratchet.mjs`: a file may hold at most its entry in its baseline (`scripts/mcp-inline-zod-baseline.json`, `scripts/service-http-baseline.json`), a file without one none, and an entry above what its file holds now fails until `-- --update` lowers it (never raises or adds). `lint:mcp-zod` counts every `z.` in a `*.mcp.ts`: a tool field derives from the request schema in `@trek/shared` or a shared primitive (`idSchema`); a bound REST deliberately lacks stays inline and counted. `lint:service-http` counts, in every file under `src/nest` but the HTTP layer (controllers, guards, pipes, filters, interceptors, DTOs and `domain-error.ts` itself), the `status: 4xx`/`5xx` literals of a returned `{ error, status }` and every `new HttpException(`, and in a `*.controller.ts`, `*.mcp.ts` or `*.rpc.ts` every branch on a returned error (`if (result.error)`, `if (!r.error)`, `'error' in r`, `r.error === 'not_found'`, `r?.error && ...`, `r.error ? a : b`). A service refuses by throwing a `DomainError` (`src/nest/common/domain-error.ts`) instead; `src/nest/README.md` ("Refusals") has the shape.
|
|
|
|
**`lint:response-contracts`** (`scripts/response-contract-ratchet.mjs`) reads every class under `src/nest/` with the TypeScript parser and counts, per file, the route handlers (`@Get`/`@Post`/`@Put`/`@Patch`/`@Delete`/`@All`/`@Head`/`@Options`) that carry no `@ResponseContract(...)` against `scripts/response-contract-baseline.json`: a file may hold at most its entry, a file without one none, and an entry above what its file holds now fails until `npm run lint:response-contracts -- --update` lowers it (it never raises or adds one). A handler that writes through `@Res()` (a stream, a redirect) has no return value to check and is marked with a `response-contract-exempt: <reason>` comment above it instead; the marker excuses only such a handler, so one that returns a body, a third party's answer passed through included, stays counted until it declares a schema. The run prints the covered share.
|
|
|
|
Single test: `npx vitest run tests/unit/nest/weather.controller.test.ts`, or `npx vitest run -t "returns 401 without cookie"`.
|
|
|
|
**Build emits even with type errors** (`scripts/build.mjs` catches `tsc` failures and warns). `npm run build` succeeding does **not** mean the code typechecks — run `npm run typecheck`.
|
|
|
|
**`lint:dialect`** (`scripts/dialect-lint.mjs`) reads every string and template literal under `src/` (comments do not count) and every `.insertId` read, and counts the SQLite-only spellings per file and rule: `INSERT OR IGNORE`/`REPLACE`, `datetime()`/`julianday()`/`unixepoch()` (also as the bare name handed to `eb.fn`), `strftime`, `json_extract` and its siblings, `GLOB`, `last_insert_rowid`, `AUTOINCREMENT`, a `PRAGMA` statement outside the `src/db/*.ts` lifecycle files and the migrations, and `||` concatenation in SQL. The counts are held to `scripts/dialect-baseline.json`; a new hit fails, and so does an entry above what its file holds now, so the change that removes one runs `npm run lint:dialect -- --update` (it never raises or adds an entry). The dialect layer `src/db/dialect/` is not scanned, and neither are the migrations up to `FROZEN_MIGRATIONS_THROUGH` in the script; a newer migration is held to the DML rules, while its DDL stays engine specific.
|
|
|
|
**`probe:pg`** (`scripts/pg-probe.ts`, CI job `postgres-probe`) measures what of the SQL runs on Postgres. Given an empty database (`TREK_PG_PROBE_URL`, or `--url=`; it refuses one holding tables it did not create) it derives a Postgres schema from the entities with MikroORM's schema generator (timestamps stay TEXT, SQLite-only defaults and `COLLATE NOCASE` get their Postgres spelling, no foreign keys), runs every dialect helper on an in-memory SQLite and on Postgres and checks both against the same expectation (`scripts/pg-probe/helper-cases.ts`; every case must pass), then calls every public method of every class in `src/db/repositories/*.ts` once. The arguments come from the parameter types, read with the TypeScript checker (`pg-probe/arg-samples.ts`), and the calls go through a MikroORM Postgres driver that runs each statement alone in a rolled-back transaction with a statement timeout and records whether Postgres accepted it (`pg-probe/recorder.ts`), so every statement meets empty tables and nothing persists. What it measures is the statements each method reaches with those sample arguments against empty tables, not every statement the code can send. The distinct statements Postgres refuses as SQL (SQLSTATE class 42 or 0A: syntax, an unknown function or column, a datatype mismatch, an unsupported feature) are held per `Class.method` to `failing` in `scripts/pg-probe-baseline.json`: a method may fail no more than its entry, and an entry above what it fails now fails too. A refusal over a sample value (class 22, say a `probe` string in a date column) or the empty tables is reported in the log and the step summary but not held, so renaming a parameter cannot turn a sample into dialect debt. The methods whose SQL the probe did not see (it could not build their arguments, they sent no SQL because they threw first or never reach the database, or they timed out) are held to the same file's `uncovered` list: a method outside the list that goes unmeasured fails the run, and so does a listed method that is measured now or gone, so the list only shrinks and a new method the sampler cannot reach does not land unchecked. Fix such a method by giving it parameters the sampler can build or by making it reach its SQL with sample values. The job uploads the measured baseline and a JSON report (`pg-probe-baseline` artifact), so the change that fixes a statement copies the lowered baseline from there or runs `npm run probe:pg -- --update` against a local Postgres (`docker run --rm -p 5432:5432 -e POSTGRES_PASSWORD=probe postgres:17`); `--update` never raises or adds an entry in either part. While the baseline's `failing` or `uncovered` is `null` it has never been measured, and the run fails (outside `--update`) until the first measurement from the artifact is committed, so the gate cannot land on `dev` holding nothing. A run that called no repository method fails too, so a broken plan can neither pass nor seed an empty baseline. The unit tests under `tests/unit/scripts/pg-probe/` cover the probe's logic, including every helper case on SQLite.
|
|
|
|
## Architecture invariants
|
|
|
|
Nest owns everything. Every domain is a DI module under `src/nest/<domain>/` (`controller` + `service` + `module`, registered in `src/nest/app.module.ts`); business logic lives in the injectable `.service.ts`, pure halves in `.helpers.ts`, SQL in a `.repository.ts` where one exists. There is no legacy service layer — ESLint errors on any `services/` import. **`weather/` is the reference module shape; `collections.controller.ts` is the reference for body validation and enumeration-safe access checks.**
|
|
|
|
- **`src/bootstrap.ts` — `buildApp()`** is the single builder for production and the test harness. Everything on the raw Express instance (global middleware, uploads, discovery metadata, statics, the named `jsonParser`/`urlencodedParser` wrappers) is registered **before `app.init()`**, because Nest throws `NotFoundException` on unmatched routes and nothing after init is reachable. The wrappers exempt `/mcp` so the MCP SDK reads the raw stream — **never `@Body()` a `/mcp` route**. Don't add routes to that pre-init shell; it bypasses every global guard, interceptor and filter.
|
|
- **Realtime**: `/ws` is a Nest gateway in `src/nest/realtime/`. Inject `RealtimeService`; tests hand in a `FakeRealtimeService` (`tests/helpers/fake-realtime.ts`) or spy on the container's instance instead of mocking the transport module. Trip mutations forward `X-Socket-Id` so the originating client doesn't echo its own change.
|
|
- **MCP**: `src/nest-mcp/` is the decorator/registry layer (`@McpController`/`@Tool`/`@Resource`/`@Prompt`; see its `README.md`); tools live in `<domain>.mcp.ts` files beside each module; the HTTP transport is `src/nest/mcp-transport/`; `src/mcp/` holds process-wide state only (scopes, session manager, boot policy). Every tool invocation writes an audit row through the registry's `onInvoke` seam. The demo-mode write block is the registry's `toolGate` (`trekDemoToolGate`): every tool not annotated read-only refuses a demo account there (except the write tools in `DEMO_UNGATED_WRITE_TOOLS`, which never had a demo check), so a tool never checks demo mode itself (ESLint refuses it in `*.mcp.ts`). MCP tools are adapters over the same services and permission checks as REST — a change on one side lands on the other in the same change.
|
|
- **Scheduling**: `src/nest/scheduling/` (`CronRegistrarService`) is the one way to schedule a cron. Jobs are `*.job.ts` providers in their owning domain. The registrar refuses to schedule under `NODE_ENV=test` (an integration test pins it). **Never use a `@Cron` decorator**: it bypasses that gate. Every tick first takes the job's row in `scheduler_leases` (`SchedulerLeasesRepository.acquire`, one conditional upsert that decides the race), renews it while it runs and keeps it for a short settle window afterwards, so two processes on one database never run the same tick; a job needs nothing of its own for this. Claims on individual rows (the reminder jobs' `claimReminder`) stay where they are: the lease decides who runs a tick, the claim what a tick may send.
|
|
- **Plugins** (`src/nest/plugins/`): the child-process runtime (`supervisor/`, `runtime/plugin-host-entry.ts`, `protocol/envelope.ts`) is **intentionally not Nest**; it is the security boundary. `host/rpc-host.ts` registers an RPC method at spawn only if the plugin holds the unlocking permission (registration = authorization). Install-time gates live in `install/`; the manifest's `trek` version range fails closed, and `TREK_PLUGINS_IGNORE_TREK_RANGE` downgrades that to a warning. The protocol tables in `protocol/envelope.ts` (+ `src/plugin-event-sink.ts`) are **generated** into `plugin-sdk/` and `shared/`: run `gen:plugin-facts` after touching them and commit the output. What a plugin gets back is `protocol/output-contract.ts`: every wire method has an entry, and an entity result (`trips.getById`, `todos.list`, `vacay.mine`, ...) is cut down to its published fields in `rpc-host.ts` on the way out, together with every child row it carries whose field is listed in `PLUGIN_ENTITY_NESTED` (a day's `notes_items`, a reservation's `endpoints`). A new column on any whole-row table behind those entities, child tables included, fails `tests/unit/plugins/output-contract.schema.test.ts` until it is listed as published or withheld (the SDK's `PLUGIN_ENTITY_FIELDS` and `PLUGIN_ENTITY_NESTED` are generated from it); a method left as `readModel` must return a value built field by field from named columns. A plugin's own SQL (`host/plugin-data.service.ts`) is held to a row cap, and `query`, `tx` and an `exec` script to a time budget checked between rows and statements (`migrate` is deliberately not timed). `runtime/egress-policy.ts` is a hand-kept mirror guarded by a parity test in `plugin-sdk/`; change both sides together.
|
|
- **Storage**: `src/nest/storage/` is the upload driver abstraction (local / S3 / mirror behind one interface, with an admin surface and a usage job). Domain services never touch the filesystem or the S3 SDK directly. Where the data lives (`data/`, `uploads/`, backups, logs, key files, the DB file) is resolved in one place, `src/app-config/data-paths.ts`; a provider that reads one injects `DataPathsService`, plain helper modules shared with code outside the container call `resolveDataPaths()` (the list is in `src/app-config/README.md`), and nothing anchors a data path on its own `__dirname` or on `process.cwd()`.
|
|
- **DB** (`src/db/`): `better-sqlite3`, WAL, FK on. `database.ts` only holds the connection and applies pragmas. It does not build a schema, and importing it opens nothing (the first reader does). The connection's lifecycle belongs to the `DatabaseLifecycle` provider (`nest/database/database-lifecycle.service.ts`): `buildApp()` opens it through `open()` (which binds the ORM to later swaps and runs the schema bootstrap below), the shutdown closes it through `close()`, and a restore or the demo reset swaps the file only through the database backup port, `DATABASE_BACKUP` (`nest/database/database-backup.interface.ts`; the SQLite implementation is `nest/backup/sqlite-database-backup.ts`: `VACUUM INTO` snapshot, close, atomic file swap, `reopen()`). The port works on `DatabaseLifecycle.file`, the one resolved path (`TREK_DB_FILE` or `data/travel.db`), and a snapshot that fails fails the backup rather than archiving the live file. `MaintenanceRepository` is provided only by `MaintenanceModule`'s factory and injected everywhere else; no consumer constructs it. A provider that has to follow every reopen (the demo reset job saves the first demo baseline after a restore's re-bootstrap seeded the example trips) registers with `DatabaseLifecycle.onReopened()`, which runs after the reopen's schema bootstrap; a listener contains its own failures, since one that throws fails the reopen. MikroORM's migrations (`db/migrations/`) are the only schema source, **including for tests**: the hand-kept `schema.ts`/`migrations.ts`/`seeds.ts` builders and the parity test that guarded them are gone. Seeding is the MikroORM seeders in `db/seeders/`, applied by `db/orm.ts` from `buildApp()` before anything reads. An install the retired positional runner migrated (a `schema_version` row, no `mikro_orm_migrations` rows) is baselined by `db/legacy-baseline.ts`: the migrations whose `Legacy migration step N` docstring is ≤ its version are recorded as executed (never replayed) in the same transaction as the migrator run, while the baseline migration itself still runs first, as the legacy runner's `createTables()` did on every boot. It refuses the boot when `schema_version` is not exactly one row holding an integer ≥ 1, or is above the last mapped step (258); a value from 1 to 258 is read as dev's steps of those numbers, so a pre-ORM feature-branch database whose steps differ is not detected (the Tours branch is the one case it cuts back). Recorded `mikro_orm_migrations` names this build does not ship refuse it too (`refuseNewerDatabase`), whether a newer release or a development build wrote them. 4.3.x and earlier have no such check and start on a database 5.x migrated. Before a boot applies pending migrations to a file database that holds anything, `db/pre-migrate-snapshot.ts` writes a `VACUUM INTO` copy next to it (`pre-migrate-<schema>-<time>.db`, `<schema>` being `legacy-<schema_version>` or the last recorded migration's timestamp; the newest `TREK_DB_PRE_MIGRATE_SNAPSHOT_KEEP` stay) and the newer-database refusal names the matching copy; a copy that cannot be written refuses the boot unless `TREK_DB_PRE_MIGRATE_SNAPSHOT=false`. It is off under `NODE_ENV=test` unless a suite sets that variable to true, and a restore's re-bootstrap skips it (the archive is the copy). After the migrator, `db/email-case-index.ts` creates the case-insensitive unique index on `users.email` that its migration had to defer on an install holding two accounts whose emails differ only in case (flag `email_case_index_pending`). `db/orm-driver.ts` binds the ORM to that same connection instead of opening a second one — with `:memory:` under test, a second connection is a second, empty database. Kysely statements (`this.kysely<...>()`) type against `DB` from `db/kysely/db.ts`, narrowed with `Pick<DB, 'trips' | 'days'>`; those files are generated from the migrated schema by `scripts/generate-db-types.ts`, so a migration that adds or changes a column is followed by `npm run gen:db-types`, never by a hand-written table interface. Every read and write goes through a repository in `db/repositories/` (each extends the shared `TrekRepository` base in `db/repositories/_shared/`, which validates the request context before delegating to MikroORM) and, for multi-statement writes, `UnitOfWork.transactional(...)` (`nest/database/unit-of-work.ts`, provided by the global `OrmModule`) — there is no `DatabaseService`/`DatabaseModule` to inject any more. Entrypoints outside an HTTP request — an MCP tool call, a plugin RPC dispatch, a WebSocket message, a cron job, boot-time seeding, a backup restore — open one explicitly with `withRequestContext()` (`nest/database/request-context.ts`), since only the pathless `mikroOrmRequestContext` middleware `buildApp()` mounts (with `@mikro-orm/nestjs`'s own `registerRequestContext` turned off in `app.module.ts`, UPLOADS-P16) forks an `EntityManager` per HTTP request. `NODE_ENV=test` ⇒ isolated `:memory:` DB per vitest worker, opened from a snapshot `tests/global-setup.ts` migrates once per run; `TREK_DB_FILE` ⇒ explicit file DB (Playwright harness).
|
|
|
|
## Rules for new code
|
|
|
|
- **Thin controller → injectable provider.** Never resurrect a plain function-module service; don't grow god services — split a domain into several modules when it outgrows one class (trips and auth already are).
|
|
- **Inject, don't reach for module globals.** No imports of the global `db` proxy from `nest/` layers at all any more — the last production importer converted onto a repository, and the export now survives only so ~190 test files' `vi.mock('…/db/database', …)` factories keep typechecking; no new code depending on the `ws` singleton or event-sink globals. There is no `DatabaseService` to inject either — take the repository(ies) you need plus `UnitOfWork` for transactions. Modules must be importable without side effects.
|
|
- **Transactions are not optional.** Multi-statement writes go in `await this.uow.transactional(async () => …)` (`UnitOfWork` from `nest/database`, provided by the global `OrmModule`); never hand-roll `BEGIN`/`COMMIT` and never call `db.transaction()` (the Phase 1 async sweep removes it). The sequence lives in the service, never in a controller, MCP tool or RPC handler, so every surface calls the one method that holds the transaction (`ReservationsService.createWithCost` is the shape: side effects such as broadcasts come back to the caller and go out after the commit). A position or id taken as `MAX(...) + 1` is read inside the same transaction as the insert that uses it: the transaction holds the connection, so two requests at once cannot read the same MAX. `lint:tx` holds the rest. Every DB-touching method is `async`; `@typescript-eslint/no-floating-promises` and `no-misused-promises` are errors, so an unawaited call or a `Promise` in a condition fails lint. Bind values with `?`; identifiers only via a literal allow-list. In `src/db/repositories/` and `src/db/seeders/`, ESLint refuses raw SQL in any spelling: `raw()`, the `sql` tag, and SQL text handed to `execute`/`run`/`all`/`get`. Build the statement with the query builder; `MaintenanceRepository`, `AdminSeeder` and `CategorySeeder` are the named exceptions in `eslint.config.mjs`, and `tests/unit/eslint-rules/restricted-syntax-selectors.test.ts` pins how many SQL strings each still holds, so they can only lose SQL, never gain it. An insert that needs the new id reads it with `.returning('id')`: `InsertResult.insertId` is SQLite's `last_insert_rowid` and undefined on Postgres, and ESLint refuses it in `src/db/repositories/` and `src/db/seeders/`.
|
|
- **Migrations are append-only.** A migration is identified by its class name in `mikro_orm_migrations`, and files sort by the timestamp in that name. Never rename or delete one that has shipped, and never edit its body; add a new one. No env interpolation in SQL. (The old positional `schema_version` array and the hand-kept `db/migrations.ts` test builder are both gone; the numbered migrations are the only schema history, for tests too.) A new foreign key comes with an index that leads with its column(s), or deleting the parent scans the child table (a partial index counts only when its WHERE clause is `IS NOT NULL` on those columns); `tests/unit/db/foreign-key-indexes.test.ts` fails on one without.
|
|
- **Every write endpoint validates** through the Zod pipe with a schema from `@trek/shared` — no `Record<string, unknown>` + ad-hoc checks. Also verify every referenced id exists and belongs to the same trip.
|
|
- **Every outbound `fetch`** gets a timeout, a response-size cap and boundary validation: no `as`-casting provider responses, no silent `catch → null`. User-influenced URLs go through the SSRF guard. Never fall back to another user's API key. ESLint requires an init with a `signal` on `fetch`, `safeFetch`, `safeFetchFollow` and `safeFetchAdminConfigured` alike, or a literal `timeoutMs` in the options object of `safeFetch`/`safeFetchFollow`; an init spelled `undefined`, `null` or `void 0` counts as none (`safeFetchLlm` takes `LLM_TIMEOUT_MS` inside). Behind that, the guarded wrappers (`utils/ssrfGuard.ts`) cap every body at `DEFAULT_MAX_RESPONSE_BYTES` (64 MB) and, for a call without a signal, wait `DEFAULT_RESPONSE_TIMEOUT_MS` (30 s) for the headers. Pass a tighter `maxBytes` when you buffer a small answer, `timeoutMs` for a deadline over the whole exchange, and `maxBytes: null` only where the body is streamed on or counted by the caller (`pipeAsset`, the doc-sync transfers, the Ollama pull).
|
|
- **Fail closed.** Unrecognized flag values mean OFF; sandbox/permission opt-outs refuse in production; never pick a less-safe path based on the *absence* of a file. GET handlers must not write.
|
|
- **Don't fork canonical paths.** Auth verification is `verifyJwtAndLoadUser` (with the `password_version` gate and the session gate); reuse it. A session token is minted only by `SessionsService.issue` (`nest/sessions/`, reached through `AuthService.generateToken`), which writes the `user_sessions` row its `jti` names; a token signed anywhere else carries no session and cannot be listed or revoked. An action that should sign a user out everywhere calls `SessionsService.revokeAll` inside the same transaction as the write that warrants it.
|
|
- **Money**: integer cents; largest-remainder splits (`splitEqualShares` in `@trek/shared` (`shared/src/money/`) is the reference, with `toMinor`/`sumMinor` for every sum and comparison); FX frozen at entry; explicit flags, never sentinel values; custom splits must reconcile to zero before persisting.
|
|
|
|
## SQL dialect
|
|
|
|
SQLite is the engine today and Postgres is the planned second one. Code under `src/` is written so that a Postgres port only adds a branch to the dialect layer, never a sweep through the repositories:
|
|
|
|
- **Build statements, don't spell them.** Use the entity methods or `this.kysely<Pick<DB, ...>>()` on the generated table types. Repositories and seeders may not hold SQL text at all (ESLint), and `lint:dialect` holds every other file to its baseline.
|
|
- **Which query API.** New repository code picks by the shape of the statement, not by what the file already uses:
|
|
- **Kysely** (`this.kysely<Pick<DB, ...>>()`) for every read that joins, aggregates, groups or projects columns across tables, and for set-based writes (a statement over many rows, a subquery, `RETURNING`). Its types come from the migrated schema, so a renamed column fails `tsc`.
|
|
- **The entity methods** (`findOne`/`find`/`count`/`insert`/`nativeUpdate`/`nativeDelete`/`upsert`) for single-table CRUD by key or by a plain filter. They carry the `TrekRepository` request-context check and the identity-map default.
|
|
- **No MikroORM QueryBuilder** (`this.qb(...)`, `createQueryBuilder(...)`) in new code. The existing calls stay until their method is touched for another reason, then move to one of the two above; there is no sweep. `lint:query-api` holds the count per file.
|
|
- **Insert or ignore / upsert**: `.onConflict((oc) => oc.columns([...]).doNothing())` or `.doUpdateSet({...})`, or `em.upsert`. Name the conflict target where there is one. Never `INSERT OR IGNORE`/`INSERT OR REPLACE`.
|
|
- **The new id**: `.returning('id').executeTakeFirstOrThrow()`. Never `InsertResult.insertId` or `last_insert_rowid()`; both are SQLite's rowid and undefined on Postgres.
|
|
- **Time**: `currentTimestamp`/`currentTimestampKysely`, `nowMinusDays`, `nowMinusHours`, `nowPlusSeconds`/`nowPlusSecondsKysely`, `nowDateOffset`, `dateOf`, `dateAdd`, `unixEpochToIsoKysely` from `src/db/dialect/sql-functions.ts` (the MikroORM `raw()` helpers) and `src/db/dialect/kysely-functions.ts` (their Kysely-expression twins). Never `datetime('now')`, `strftime` or `julianday`.
|
|
- **Strings and casts**: `concat`/`concatKysely` instead of `||`, `substring`/`substringKysely`, `lower`/`lowerTrim`, `collateNoCase`, `castInteger`/`castIntegerKysely`, and `startsWithIsoDate`/`startsWithIsoDateKysely` instead of `GLOB`.
|
|
- **JSON columns** are read whole and parsed in the service; no `json_extract` in SQL. Each one is declared once in `src/db/json-columns.ts` (its Zod shape and the fallback a reader gets for an empty or broken row) and read with `decodeJson`/`decodeJsonResult`, written from a server-built value with `encodeJson` (`src/utils/json-column.ts`), never with a bare `JSON.parse` at the call site: one column, one answer to a broken row, logged.
|
|
- **Timestamps** in a DATETIME column are the canonical `YYYY-MM-DD HH:MM:SS` UTC text, never `toISOString()`: in SQL the two spellings do not compare. A value written from JS is a `DbTimestamp` from `dbNow()` (`src/db/types`), and a stored value is read back as an instant with `parseDbTimestamp()` (a bare `new Date()` reads the canonical text as local time). Where a response has always carried the ISO spelling, convert on the way out (`utcSuffix`), not in the column.
|
|
- **PRAGMA** belongs to the `src/db/` lifecycle files (`database.ts`, `connection.ts`, `durability.ts`, the migrations). `MaintenanceRepository` is the baselined SQLite maintenance adapter (`wal_checkpoint`, `VACUUM INTO`).
|
|
- **A missing helper** goes into `sql-functions.ts` (or `kysely-functions.ts` for a Kysely expression), dispatching on the live platform through `src/db/dialect/platform.ts`: a SQLite branch and a Postgres branch (one spelling under `isKnownPlatform` where both engines read it alike), `unsupported(platform)` for anything else. The Postgres branch keeps the SQLite storage model, so a clock helper renders the same `YYYY-MM-DD HH:MM:SS` UTC text. Pin the Postgres text in `tests/unit/db/dialect/sql-functions.postgres.test.ts` and add a case to `scripts/pg-probe/helper-cases.ts` so the probe runs it on both engines. `src/db/dialect/` is the only place allowed to spell an engine's own SQL.
|
|
- **Migrations** that have shipped are frozen as they are. A new migration's data statements follow the rules above; its DDL is written for the engine it runs on.
|
|
- **Storage model**: the generated Kysely types describe what SQLite stores (0/1 integers for flags, TEXT timestamps, JSON in TEXT). Convert at the service boundary, as the contracts already require.
|
|
- **MySQL/MariaDB are out of scope.** The repositories rely on `INSERT ... ON CONFLICT` (Kysely's `onConflict`, often a target-less `doNothing()`) and on `RETURNING`, and MySQL has neither: it would need an insert-ignore/upsert wrapper in the dialect layer and a different way to read new ids before it could be considered.
|
|
|
|
## Configuration (app-config)
|
|
|
|
**Never read `process.env` in `src/**`** — ESLint errors on it. All env access goes through `src/app-config/`:
|
|
|
|
- Every variable has exactly one owner. Nest classes inject a boot-stable namespace token from `src/nest/app-config/` (`httpConfig`, `storageConfig`, `transitConfig`, `kitineraryConfig`) for values frozen per app build, or use `RuntimeEnvService` / `readEnv()` for everything else (DEMO_MODE, NODE_ENV, OIDC_*, ...). A token-owned variable is derived in `boot-derive.ts` and never by `readEnv()`; `tests/unit/app-config/config-ownership.test.ts` fails on a variable read through both, and on a token nothing injects.
|
|
- `server/.env.example` documents every variable `env.schema.ts` validates; `env-reference.test.ts` fails on drift either way.
|
|
- Everything else calls `readEnv()` — live, uncached, per call.
|
|
- Validation is fail-fast at boot only (`boot-validate.ts`, imported by `index.ts` right after dotenv): malformed values abort startup; unset/blank defaults. Never wire validation into `buildApp()` or `ConfigModule.forRoot`, and never `cache`/snapshot a runtime-toggled value (it breaks the env-mutating tests).
|
|
- Booleans go through `parseBool` (true/1/on/yes vs false/0/off/no). Everything else pins its exact legacy coercion — quirks and the ESLint exemption list are in `src/app-config/README.md`.
|
|
- **Instance settings** (`app_settings`) are named through the typed register `src/nest/common/app-settings.registry.ts`: one entry per key with its Zod shape, default, environment override and flags (admin form, masked, encrypted, managed-locked, public config). Read with `readAppSetting(repo, key)` (the stored text, exactly what `getValue` answers) or `resolveAppSetting(repo, key)` (a non-empty environment value first); ask `isOidcConfigured()` rather than spelling the issuer/client-id check. The admin form's key lists derive from the flags. ESLint refuses `getValue('<literal>')` outside the register; a key built from parts (`admin_notif_pref_*`) is a family the register does not list.
|
|
|
|
## Cross-cutting pieces
|
|
|
|
- **`src/nest/common/`**: the global exception filter (error envelope), the Zod validation pipe, the idempotency interceptor (`X-Idempotency-Key` replay), and `validate-route-guards.ts` (boot-time ratchet refusing any public route not on its allow-list).
|
|
- **Response contracts** (`src/nest/common/response-contract.ts`): `@ResponseContract(schema)` names the `@trek/shared` response schema a handler answers with. Under `NODE_ENV=test` it adds an interceptor that parses every response of that handler against the schema and turns a body the schema does not describe into a logged 500 naming the handler and the first issues, so every e2e and integration test that reaches the route is a drift check; in production it only records the schema and adds nothing to the request path. The parse never replaces the body. When a check fails, fix the schema to describe what the route really sends, or the test stub to return what the service really returns, never the response. Rows keep their storage spelling in these schemas (0/1 flags, TEXT or epoch times) wherever the route has always answered that way.
|
|
- **Auth is default-deny**: the global auth guard and the MFA policy guard run as `APP_GUARD`s, followed by the managed-install guard (`managed.guard.ts`), which refuses the routes a centrally administered install withholds from its own admin — it runs last so strangers still get a 401, never a route-revealing 403. Routes opt out with `@Public()`/`@OptionalAuth()`. The global guard stands down for routes declaring their own guard chain, so `@UseGuards(JwtAuthGuard, TripAccessGuard)` + `@RequirePermission` stacks work. Session is a cookie (`trek_session`); `@CurrentUser()` reads the user. Every session token carries a `jti` naming its `user_sessions` row, and `verifyJwtAndLoadUser` refuses one whose row is revoked, expired or gone (logout, `/api/auth/sessions`, a password change or reset, an MFA disable all end rows; `session-purge.job.ts` deletes the expired ones nightly; a revoked row waits for its own expiry, because for a session renewed from a pre-`jti` token, whose id is derived from that token, the revoked row is all that keeps the token from inserting it again). A token without a `jti` predates the table and passes until it expires, so an upgrade signs nobody out. The `token` field in the login-type JSON bodies and in the SSO exchange (`GET /api/auth/oidc/exchange`, which also answers `success: true`) is deprecated: kept for API clients, read by nothing of ours. **Never gate a multipart upload with a guard** — guards run before the parser and the client sees ECONNRESET instead of your 403; check in the handler.
|
|
- **Logging**: `src/nest/audit/audit-log.logger.ts` (`logInfo`/`logWarn`/`logError`/`logDebug`) prints each line to the console and queues it for `data/logs/trek.log`, which `log-file.ts` appends in batches off the event loop and rotates at 10 MB over five files; `index.ts` flushes what is queued on exit. Never write the log file synchronously from a request path.
|
|
- **Request correlation** (`src/nest/common/request-correlation.ts`): every HTTP request runs under an id (a well-formed incoming `X-Request-Id`, else a fresh UUID) that the response carries in `X-Request-Id`, and every log line written inside it is tagged `[http <id>]`. The exception filter leaves a 5xx on `res.locals` so the access log writes the request line and the stack as one entry. The other entry points (MCP tool/resource/prompt calls via the registry's `around` seam, WebSocket messages in `TrekWsAdapter`, plugin RPC dispatches in the supervisor, cron ticks in `CronRegistrarService`) each run under their own id through `traceEntry()` (`nest/audit/entry-trace.logger.ts`), which logs one line per call: debug on success, debug as `skipped` when the call declines its work on purpose (a cron tick whose lease another process holds), warn or error with the reason on failure. A new entry point goes through `traceEntry()` too.
|
|
- **Process safety nets** (`src/index.ts`, `src/shutdown.ts`): `unhandledRejection` and `uncaughtException` are logged to trek.log and end in the same orderly shutdown as SIGTERM, with exit code 1; a second fatal error during it exits at once. A shutdown first marks `ReadinessService` (`nest/health/`) as draining, so `/api/health/ready` answers 503 for the whole drain. The HTTP server keeps idle keep-alive connections for `HTTP_KEEP_ALIVE_TIMEOUT_MS` (95 s, above the 60 s of nginx and the 90 s of Traefik) with the headers timeout one second above it (`applyProxyTimeouts` in `bootstrap.ts`).
|
|
- **Process-local state sits behind injectable ports**, each with an in-memory implementation that keeps today's behaviour: `RateLimitStore` (`nest/common/rate-limit.store.ts`, provided by `RateLimitModule`), `PendingCodeStore` (`nest/oauth/oauth.pending-codes.ts`), `OidcFlowStore` (`nest/oidc/oidc-flow.store.ts`), `PermissionsCacheStore` (`nest/permissions/permissions-cache.ts`) and `RoomRegistry` (`nest/realtime/ws-state.ts`). The owning module provides the implementation. Every port a shared store could back returns promises (a store in the database or on the network is async); `RoomRegistry` stays synchronous because membership holds live sockets, and several processes would need a fan-out in front of `broadcast()` instead. The process-wide ports (`PendingCodeStore`, `PermissionsCacheStore`, `RoomRegistry`) also have readers outside the container (the pending-code sweep, the backup restore's cache flush, the broadcast functions, hand-built consumers), so their module installs the store the container resolved in a `ProcessStoreSlot` (`nest/common/process-store-slot.ts`) those readers use: overriding the provider changes every reader. New process-local state follows the same shape; TREK still runs as one process.
|
|
- **`src/middleware/globalMiddleware.ts`** is the only Express middleware left (helmet/CSP, CORS, HSTS, forced-HTTPS, logging, cookies).
|
|
|
|
## Tests
|
|
|
|
`tests/` is split into `unit/` (mirrors `src/`), `integration/`, `e2e/` (one `<domain>.e2e.test.ts` per module, booting the real guards against a temp DB via `tests/e2e/harness.ts`) and `websocket/`; helpers in `tests/helpers/`, fixtures in `tests/fixtures/`.
|
|
|
|
- **Seed and read through the ORM factories** in `tests/helpers/factories/` (`makeUser`, `makeTrip`, `makePlace`, … plus the typed `findRow`/`findRows`/`countRows`/`updateRows`/`deleteRows` readers), handed the app's ORM (`app.get(MikroORM)`) or a `createTestOrm(db)`. They go through MikroORM in a request context of their own, the way the app does, instead of raw SQLite on the sync handle; the README there has the before/after patterns. The raw `tests/helpers/factories.ts` and the `.prepare()` fixtures are legacy that `lint:test-sql` only lets shrink.
|
|
|
|
- **Upgrade fixtures**: `tests/fixtures/legacy/legacy-v*.sql` are databases the retired positional runner built; `tests/integration/legacy-upgrade-v*.test.ts` boot `buildApp()` on each. `legacy-v244` is the v4.3.3 release with rows across the main tables, rebuilt reproducibly by `node scripts/build-legacy-fixture-v244.mjs` (archives the tag into a temp dir, boots that release's runner, seeds, pins timestamps, dumps).
|
|
|
|
- **vitest uses the SWC plugin**, not esbuild, because Nest's DI needs emitted decorator metadata. Keep that in `vitest.config.ts`. Pool is `forks` (isolated DB per worker). The config also aliases `@modelcontextprotocol/sdk/*` to its CJS dist because the SDK's exports map is unresolvable.
|
|
- **Coverage is a per-domain ratchet over `src/nest/**`** (≥80% floor, most domains pinned higher). Regenerate the block with `scripts/coverage-thresholds.mjs` when coverage rises; never lower a threshold to land a change. Every folder under `src/nest/` needs its own entry (a new domain too): `tests/unit/coverage-thresholds.test.ts` fails on one that falls through to the pooled `src/nest/**` catch-all.
|
|
- CI also runs SonarCloud on the PR's new lines — see the root `CLAUDE.md`.
|
|
|
|
## Parity discipline (when adding or changing routes)
|
|
|
|
Routes are **byte-identical** for the client: same URL, method, query/body, status, `Set-Cookie` and JSON body, including bespoke error strings reproduced in the controller. Nest defaults POST to **201** — add `@HttpCode(200)` where the contract returns 200. Declare static sub-routes (`/reorder`, `/in-app/all`) **before** `:id` param routes. Trip-scoped handlers verify trip access (404) and permission (403).
|