Backup, restore and the demo reset each handled the SQLite file themselves: backup.impl closed and reinitialized the connection, built MaintenanceRepository with new, and assumed the database was data/travel.db. That tied the whole chain to one local file and hid two bugs. They now go through DATABASE_BACKUP, a port declared in the shared database kernel (snapshot, verify, keep a copy before a restore, replace). SqliteDatabaseBackup implements it with what the code did before: VACUUM INTO for a consistent copy, and a close, copy, sidecar drop, rename and reopen through DatabaseLifecycle for a restore. BackupService injects it and hands it to backup.impl. The demo reset and the baseline save use the same port; the first baseline is saved by DemoResetJob once the app is up, because the boot-time seed runs before anything can be injected. Another engine can plug in a logical dump behind the port. Fixes on the way: - The backup and the restore ignored TREK_DB_FILE. The backup looked for data/travel.db and, with the database elsewhere, silently left it out of the archive; the restore wrote data/travel.db and never replaced the live database. Both now use DatabaseLifecycle.file, the path the connection runs on, and the pre-restore copy sits next to it. - A failed VACUUM INTO fell back to archiving the live WAL-mode file without a word, the torn copy the snapshot exists to prevent. The backup now fails: nothing is archived or committed, the existing backups are untouched, a partial snapshot is removed, and the log says why. A missing database file and a failed checkpoint are logged too.
19 KiB
CLAUDE.md
Scope: the @trek/server workspace. See the repo-root CLAUDE.md for the monorepo picture and the philosophy. This file holds the server's commands, invariants and pitfalls; the code and src/nest/README.md hold the inventory.
Commands (run from server/)
npm run dev # scripts/dev.mjs: tsc -w then node --watch dist/index.js (server on :3001)
npm run build # scripts/build.mjs — emits even on type errors (see below)
npm run typecheck # tsc --noEmit — the real type gate
npm run typecheck:tests # tsc over tests/ — CI runs this too; vitest green does NOT mean typed mocks compile
npm run lint # eslint --fix
npm run lint:check # eslint, no fix (CI)
npm run test # vitest run; also test:unit / test:integration / test:ws / test:e2e
npm run test:coverage # istanbul coverage; per-domain ratchet over src/nest/**
npm run gen:plugin-facts # regenerate the plugin-protocol tables into plugin-sdk/ + shared/
npm run check:plugin-facts # CI gate — fails if the generated copies drifted
node scripts/coverage-thresholds.mjs # after test:coverage — prints the ratchet block for vitest.config.ts
npm run db:call-graph -- --sync --tx [--domains nest/days,…] # sync DB-touching methods + transaction sites (exit 1 if any)
npm run lint:size # no file in src/ or scripts/ past 1000 lines, longer ones only shrink (CI gate)
npm run lint:boundaries # import cycles and cross-domain reach-ins may only shrink (CI gate)
npm run contracts:dto-open # open shapes in the createZodDto schemas may only shrink (runs in the unit tests)
lint:boundaries (scripts/import-boundaries.mjs) parses every import under src/ and holds five lists in scripts/import-boundaries-baseline.json: load-time file cycles (type-only and lazy imports, and the MikroORM entity relations, do not count), dependencies between two nest/<domain> folders that already depend on each other, shared-kernel files (nest/common, nest/database, nest/app-config) importing a domain, a domain importing another domain's internals, and src/db importing src/nest. A domain's public surface is its *.module, *.service, *.guard, *.decorator, *.types, *.interface, *.contract, *.constants and *.logger files plus the short PUBLIC_FILES list in the script; anything else is an internal. A new entry fails the check, and so does a baseline entry that no longer occurs, so the fix lands with npm run lint:boundaries -- --update, which drops such entries and never adds one.
contracts:dto-open (tests/unit/contracts/dto-open-shapes.test.ts, wrapped by scripts/dto-open-shapes.mjs) finds every export class X extends createZodDto(...) under src/, imports it and counts the open shapes (loose object, record of unknown, z.unknown()/z.any(), z.custom()) in the schema the Zod pipe validates, with shared's contracts:open walk. Each DTO class is held to its entry in scripts/dto-open-shapes-baseline.json, and a new one may hold none, whatever its schema is called and wherever it is declared. A createZodDto call that is not an exported class, or two DTO classes with one name, fail the check. After narrowing a schema, npm run contracts:dto-open -- --update lowers the baseline (it never raises or adds an entry). It needs shared/ built.
lint:size (scripts/size-lint.mjs) counts a line longer than 120 columns once per 120 columns it spans, so joining lines does not make room. Files that were already longer are held at their length in scripts/size-baseline.json; a file that needs to grow past its entry is split by concern instead. An entry above what its file holds now, or for a file that is gone, fails as well, so after a split npm run lint:size -- --update lowers the baseline in the same change (it never raises or adds an entry).
Single test: npx vitest run tests/unit/nest/weather.controller.test.ts, or npx vitest run -t "returns 401 without cookie".
Build emits even with type errors (scripts/build.mjs catches tsc failures and warns). npm run build succeeding does not mean the code typechecks — run npm run typecheck.
Architecture invariants
Nest owns everything. Every domain is a DI module under src/nest/<domain>/ (controller + service + module, registered in src/nest/app.module.ts); business logic lives in the injectable .service.ts, pure halves in .helpers.ts, SQL in a .repository.ts where one exists. There is no legacy service layer — ESLint errors on any services/ import. weather/ is the reference module shape; collections.controller.ts is the reference for body validation and enumeration-safe access checks.
src/bootstrap.ts—buildApp()is the single builder for production and the test harness. Everything on the raw Express instance (global middleware, uploads, discovery metadata, statics, the namedjsonParser/urlencodedParserwrappers) is registered beforeapp.init(), because Nest throwsNotFoundExceptionon unmatched routes and nothing after init is reachable. The wrappers exempt/mcpso the MCP SDK reads the raw stream — never@Body()a/mcproute. Don't add routes to that pre-init shell; it bypasses every global guard, interceptor and filter.- Realtime:
/wsis a Nest gateway insrc/nest/realtime/. InjectRealtimeService;src/websocket.tsis a re-export stub for old call sites. Trip mutations forwardX-Socket-Idso the originating client doesn't echo its own change. - MCP:
src/nest-mcp/is the decorator/registry layer (@McpController/@Tool/@Resource/@Prompt; see itsREADME.md); tools live in<domain>.mcp.tsfiles beside each module; the HTTP transport issrc/nest/mcp-transport/;src/mcp/holds process-wide state only (scopes, session manager, boot policy). Every tool invocation writes an audit row through the registry'sonInvokeseam. MCP tools are adapters over the same services and permission checks as REST — a change on one side lands on the other in the same change. - Scheduling:
src/nest/scheduling/(CronRegistrarService) is the one way to schedule a cron. Jobs are*.job.tsproviders in their owning domain. The registrar refuses to schedule underNODE_ENV=test(an integration test pins it). Never use a@Crondecorator — it bypasses that gate. - Plugins (
src/nest/plugins/): the child-process runtime (supervisor/,runtime/plugin-host-entry.ts,protocol/envelope.ts) is intentionally not Nest — it is the security boundary.host/rpc-host.tsregisters an RPC method at spawn only if the plugin holds the unlocking permission (registration = authorization). Install-time gates live ininstall/; the manifest'strekversion range fails closed, andTREK_PLUGINS_IGNORE_TREK_RANGEdowngrades that to a warning. The protocol tables inprotocol/envelope.ts(+src/plugin-event-sink.ts) are generated intoplugin-sdk/andshared/— rungen:plugin-factsafter touching them and commit the output.runtime/egress-policy.tsis a hand-kept mirror guarded by a parity test inplugin-sdk/; change both sides together. - Storage:
src/nest/storage/is the upload driver abstraction (local / S3 / mirror behind one interface, with an admin surface and a usage job). Domain services never touch the filesystem or the S3 SDK directly. - DB (
src/db/):better-sqlite3, WAL, FK on.database.tsonly holds the connection and applies pragmas. It does not build a schema, and importing it opens nothing (the first reader does). The connection's lifecycle belongs to theDatabaseLifecycleprovider (nest/database/database-lifecycle.service.ts):buildApp()opens it throughopen()(which binds the ORM to later swaps and runs the schema bootstrap below), the shutdown closes it throughclose(), and a restore or the demo reset swaps the file only through the database backup port,DATABASE_BACKUP(nest/database/database-backup.interface.ts; the SQLite implementation isnest/backup/sqlite-database-backup.ts:VACUUM INTOsnapshot, close, atomic file swap,reopen()). The port works onDatabaseLifecycle.file, the one resolved path (TREK_DB_FILEordata/travel.db), and a snapshot that fails fails the backup rather than archiving the live file.MaintenanceRepositoryis a provider (MaintenanceModule), injected, never built withnewinsrc/. MikroORM's migrations (db/migrations/) are the only schema source, including for tests: the hand-keptschema.ts/migrations.ts/seeds.tsbuilders and the parity test that guarded them are gone. Seeding is the MikroORM seeders indb/seeders/, applied bydb/orm.tsfrombuildApp()before anything reads. An install the retired positional runner migrated (aschema_versionrow, nomikro_orm_migrationsrows) is baselined bydb/legacy-baseline.ts: the migrations whoseLegacy migration step Ndocstring is ≤ its version are recorded as executed (never replayed) in the same transaction as the migrator run, while the baseline migration itself still runs first, as the legacy runner'screateTables()did on every boot; anything it cannot map refuses the boot. Before a boot applies pending migrations to a file database that holds anything,db/pre-migrate-snapshot.tswrites aVACUUM INTOcopy next to it (pre-migrate-<schema>-<time>.db,<schema>beinglegacy-<schema_version>or the last recorded migration's timestamp; the newestTREK_DB_PRE_MIGRATE_SNAPSHOT_KEEPstay) and the newer-database refusal names the matching copy; a copy that cannot be written refuses the boot unlessTREK_DB_PRE_MIGRATE_SNAPSHOT=false. It is off underNODE_ENV=testunless a suite sets that variable to true, and a restore's re-bootstrap skips it (the archive is the copy). After the migrator,db/email-case-index.tscreates the case-insensitive unique index onusers.emailthat its migration had to defer on an install holding two accounts whose emails differ only in case (flagemail_case_index_pending).db/orm-driver.tsbinds the ORM to that same connection instead of opening a second one — with:memory:under test, a second connection is a second, empty database. Every read and write goes through a repository indb/repositories/(each extends the sharedTrekRepositorybase indb/repositories/_shared/, which validates the request context before delegating to MikroORM) and, for multi-statement writes,UnitOfWork.transactional(...)(nest/database/unit-of-work.ts, provided by the globalOrmModule) — there is noDatabaseService/DatabaseModuleto inject any more. Entrypoints outside an HTTP request — an MCP tool call, a plugin RPC dispatch, a WebSocket message, a cron job, boot-time seeding, a backup restore — open one explicitly withwithRequestContext()(nest/database/request-context.ts), since only the pathlessmikroOrmRequestContextmiddlewarebuildApp()mounts (with@mikro-orm/nestjs's ownregisterRequestContextturned off inapp.module.ts, UPLOADS-P16) forks anEntityManagerper HTTP request.NODE_ENV=test⇒ isolated:memory:DB per vitest worker, opened from a snapshottests/global-setup.tsmigrates once per run;TREK_DB_FILE⇒ explicit file DB (Playwright harness).
Rules for new code
- Thin controller → injectable provider. Never resurrect a plain function-module service; don't grow god services — split a domain into several modules when it outgrows one class (trips and auth already are).
- Inject, don't reach for module globals. No imports of the global
dbproxy fromnest/layers at all any more — the last production importer converted onto a repository, and the export now survives only so ~190 test files'vi.mock('…/db/database', …)factories keep typechecking; no new code depending on thewssingleton or event-sink globals. There is noDatabaseServiceto inject either — take the repository(ies) you need plusUnitOfWorkfor transactions. Modules must be importable without side effects. - Transactions are not optional. Multi-statement writes go in
await this.uow.transactional(async () => …)(UnitOfWorkfromnest/database, provided by the globalOrmModule); never hand-rollBEGIN/COMMITand never calldb.transaction()(the Phase 1 async sweep removes it). Every DB-touching method isasync;@typescript-eslint/no-floating-promisesandno-misused-promisesare errors, so an unawaited call or aPromisein a condition fails lint. Bind values with?; identifiers only via a literal allow-list. Insrc/db/repositories/andsrc/db/seeders/, ESLint refuses raw SQL in any spelling:raw(), thesqltag, and SQL text handed toexecute/run/all/get. Build the statement with the query builder;MaintenanceRepository,DemoRepositoryand the six existing seeders are the named exceptions ineslint.config.mjs, andtests/unit/eslint-rules/restricted-syntax-selectors.test.tspins how many SQL strings each still holds, so they can only lose SQL, never gain it. - Migrations are append-only. A migration is identified by its class name in
mikro_orm_migrations, and files sort by the timestamp in that name. Never rename or delete one that has shipped, and never edit its body; add a new one. No env interpolation in SQL. (The old positionalschema_versionarray and the hand-keptdb/migrations.tstest builder are both gone; the numbered migrations are the only schema history, for tests too.) A new foreign key comes with an index that leads with its column(s), or deleting the parent scans the child table (a partial index counts only when its WHERE clause isIS NOT NULLon those columns);tests/unit/db/foreign-key-indexes.test.tsfails on one without. - Every write endpoint validates through the Zod pipe with a schema from
@trek/shared— noRecord<string, unknown>+ ad-hoc checks. Also verify every referenced id exists and belongs to the same trip. - Every outbound
fetchgets a timeout, a response-size cap and boundary validation: noas-casting provider responses, no silentcatch → null. User-influenced URLs go through the SSRF guard. Never fall back to another user's API key. ESLint requires an init with asignalonfetch,safeFetch,safeFetchFollowandsafeFetchAdminConfiguredalike, or a literaltimeoutMsin the options object ofsafeFetch/safeFetchFollow; an init spelledundefined,nullorvoid 0counts as none (safeFetchLlmtakesLLM_TIMEOUT_MSinside). Behind that, the guarded wrappers (utils/ssrfGuard.ts) cap every body atDEFAULT_MAX_RESPONSE_BYTES(64 MB) and, for a call without a signal, waitDEFAULT_RESPONSE_TIMEOUT_MS(30 s) for the headers. Pass a tightermaxByteswhen you buffer a small answer,timeoutMsfor a deadline over the whole exchange, andmaxBytes: nullonly where the body is streamed on or counted by the caller (pipeAsset, the doc-sync transfers, the Ollama pull). - Fail closed. Unrecognized flag values mean OFF; sandbox/permission opt-outs refuse in production; never pick a less-safe path based on the absence of a file. GET handlers must not write.
- Don't fork canonical paths. Auth verification is
verifyJwtAndLoadUser(with thepassword_versiongate) — reuse it. - Money: integer cents; largest-remainder splits (
BudgetService.splitEqualSharesis the reference); FX frozen at entry; explicit flags, never sentinel values; custom splits must reconcile to zero before persisting.
Configuration (app-config)
Never read process.env in src/** — ESLint errors on it. All env access goes through src/app-config/:
- Nest classes inject a boot-stable namespace token from
src/nest/app-config/for values frozen per app build, or useRuntimeEnvService/readEnv()for runtime-toggled values (DEMO_MODE, NODE_ENV, OIDC_*, …) that tests mutate mid-lifetime. - Everything else calls
readEnv()— live, uncached, per call. - Validation is fail-fast at boot only (
boot-validate.ts, imported byindex.tsright after dotenv): malformed values abort startup; unset/blank defaults. Never wire validation intobuildApp()orConfigModule.forRoot, and nevercache/snapshot a runtime-toggled value (it breaks the env-mutating tests). - Booleans go through
parseBool(true/1/on/yes vs false/0/off/no). Everything else pins its exact legacy coercion — quirks and the ESLint exemption list are insrc/app-config/README.md.
Cross-cutting pieces
src/nest/common/: the global exception filter (error envelope), the Zod validation pipe, the idempotency interceptor (X-Idempotency-Keyreplay), andvalidate-route-guards.ts(boot-time ratchet refusing any public route not on its allow-list).- Auth is default-deny: the global auth guard and the MFA policy guard run as
APP_GUARDs, followed by the managed-install guard (managed.guard.ts), which refuses the routes a centrally administered install withholds from its own admin — it runs last so strangers still get a 401, never a route-revealing 403. Routes opt out with@Public()/@OptionalAuth(). The global guard stands down for routes declaring their own guard chain, so@UseGuards(JwtAuthGuard, TripAccessGuard)+@RequirePermissionstacks work. Session is a cookie (trek_session);@CurrentUser()reads the user. Never gate a multipart upload with a guard — guards run before the parser and the client sees ECONNRESET instead of your 403; check in the handler. src/middleware/globalMiddleware.tsis the only Express middleware left (helmet/CSP, CORS, HSTS, forced-HTTPS, logging, cookies).
Tests
tests/ is split into unit/ (mirrors src/), integration/, e2e/ (one <domain>.e2e.test.ts per module, booting the real guards against a temp DB via tests/e2e/harness.ts) and websocket/; helpers in tests/helpers/, fixtures in tests/fixtures/.
-
Upgrade fixtures:
tests/fixtures/legacy/legacy-v*.sqlare databases the retired positional runner built;tests/integration/legacy-upgrade-v*.test.tsbootbuildApp()on each.legacy-v244is the v4.3.3 release with rows across the main tables, rebuilt reproducibly bynode scripts/build-legacy-fixture-v244.mjs(archives the tag into a temp dir, boots that release's runner, seeds, pins timestamps, dumps). -
vitest uses the SWC plugin, not esbuild, because Nest's DI needs emitted decorator metadata. Keep that in
vitest.config.ts. Pool isforks(isolated DB per worker). The config also aliases@modelcontextprotocol/sdk/*to its CJS dist because the SDK's exports map is unresolvable. -
Coverage is a per-domain ratchet over
src/nest/**(≥80% floor, most domains pinned higher). Regenerate the block withscripts/coverage-thresholds.mjswhen coverage rises; never lower a threshold to land a change. Every folder undersrc/nest/needs its own entry (a new domain too):tests/unit/coverage-thresholds.test.tsfails on one that falls through to the pooledsrc/nest/**catch-all. -
CI also runs SonarCloud on the PR's new lines — see the root
CLAUDE.md.
Parity discipline (when adding or changing routes)
Routes are byte-identical for the client: same URL, method, query/body, status, Set-Cookie and JSON body, including bespoke error strings reproduced in the controller. Nest defaults POST to 201 — add @HttpCode(200) where the contract returns 200. Declare static sub-routes (/reorder, /in-app/all) before :id param routes. Trip-scoped handlers verify trip access (404) and permission (403).