diff --git a/server/CLAUDE.md b/server/CLAUDE.md index 22f06324c..c2e093d94 100644 --- a/server/CLAUDE.md +++ b/server/CLAUDE.md @@ -20,8 +20,9 @@ npm run gen:plugin-facts # regenerate the plugin-protocol tables into plugin-sd npm run check:plugin-facts # CI gate — fails if the generated copies drifted npm run gen:db-types # regenerate the Kysely table types in src/db/kysely/ from the migrated schema npm run check:db-types # CI gate: fails when src/db/kysely/ no longer matches the migrations -npm run gen:pg-baseline # regenerate the Postgres baseline in assets/postgres/ from the entities -npm run check:pg-baseline # CI gate: fails when the baseline drifted from the entities, its manifest from the migrations, or a SQLite trigger has no Postgres port +npm run gen:pg-baseline # regenerate the Postgres baseline in assets/postgres/ from the entities (never for a migration after its cut) +npm run check:pg-baseline # CI gate: fails when the manifest drifted from the migrations; until a migration follows the cut, also when the baseline drifted from the entities or a SQLite trigger has no Postgres port; lists the migrations after the cut +npm run gen:pg-migration # print the Postgres DDL of the newest migration (--migration= for another) for its engineDdl call; needs TREK_TEST_DB_URL or --url node scripts/coverage-thresholds.mjs # after test:coverage — prints the ratchet block for vitest.config.ts npm run db:call-graph -- --sync --tx [--domains nest/days,…] # sync DB-touching methods + transaction sites (exit 1 if any) npm run lint:size # no file in src/ or scripts/ past 1000 lines, none in tests/ past 2000, longer ones only shrink (CI gate) @@ -65,7 +66,7 @@ Single test: `npx vitest run tests/unit/nest/weather.controller.test.ts`, or `np **Build emits even with type errors** (`scripts/build.mjs` catches `tsc` failures and warns). `npm run build` succeeding does **not** mean the code typechecks — run `npm run typecheck`. -**`lint:dialect`** (`scripts/dialect-lint.mjs`) reads every string and template literal under `src/` (comments do not count) and every `.insertId` read, and counts the SQLite-only spellings per file and rule: `INSERT OR IGNORE`/`REPLACE`, `datetime()`/`julianday()`/`unixepoch()` (also as the bare name handed to `eb.fn`), `strftime`, `json_extract` and its siblings, `GLOB`, `last_insert_rowid`, `AUTOINCREMENT`, a `PRAGMA` statement outside the `src/db/*.ts` lifecycle files and the migrations, and `||` concatenation in SQL. The counts are held to `scripts/dialect-baseline.json`; a new hit fails, and so does an entry above what its file holds now, so the change that removes one runs `npm run lint:dialect -- --update` (it never raises or adds an entry). The dialect layer `src/db/dialect/` is not scanned, and neither are the migrations up to `FROZEN_MIGRATIONS_THROUGH` in the script; a newer migration is held to the DML rules, while its DDL stays engine specific. +**`lint:dialect`** (`scripts/dialect-lint.mjs`) reads every string and template literal under `src/` (comments do not count) and every `.insertId` read, and counts the SQLite-only spellings per file and rule: `INSERT OR IGNORE`/`REPLACE`, `datetime()`/`julianday()`/`unixepoch()` (also as the bare name handed to `eb.fn`), `strftime`, `json_extract` and its siblings, `GLOB`, `last_insert_rowid`, `AUTOINCREMENT`, a `PRAGMA` statement outside the `src/db/*.ts` lifecycle files and the migrations, and `||` concatenation in SQL. The counts are held to `scripts/dialect-baseline.json`; a new hit fails, and so does an entry above what its file holds now, so the change that removes one runs `npm run lint:dialect -- --update` (it never raises or adds an entry). The dialect layer `src/db/dialect/` is not scanned, and neither are the migrations up to `FROZEN_MIGRATIONS_THROUGH` in the script; a newer migration is held to the DML rules. A migration after the Postgres baseline's cut (`through` in `assets/postgres/baseline.json`, read on every run) is also held to two rules no baseline entry can excuse: `ddl` (a CREATE/ALTER/DROP of a table, index, trigger or view, or a RENAME, outside `engineDdl`; a plain string as `addColumnIfMissing`'s column; a call to the SQLite-only `tableSql` or `createPlaceRegionsFollowPlaceTrigger`) and `sqlite-ddl` (DATETIME, AUTOINCREMENT, INTEGER PRIMARY KEY, WITHOUT ROWID, PRAGMA or a SQLite-only function in an `engineDdl` postgres member). The `sqlite` member is SQLite's own and not scanned. The routine under SQL dialect, Migrations, is what satisfies them. **`probe:pg`** (`scripts/pg-probe.ts`, CI job `postgres-probe`) measures what of the SQL runs on Postgres. Given an empty database (`TREK_PG_PROBE_URL`, or `--url=`; it refuses one holding tables it did not create) it derives a Postgres schema from the entities with MikroORM's schema generator (timestamps stay TEXT, integers are `bigint`, SQLite-only defaults and `COLLATE NOCASE` get their Postgres spelling, no foreign keys), runs every dialect helper on an in-memory SQLite and on Postgres and checks both against the same expectation (`scripts/pg-probe/helper-cases.ts`; every case must pass), then calls every public method of every class in `src/db/repositories/*.ts` once. The arguments come from the parameter types, read with the TypeScript checker (`pg-probe/arg-samples.ts`), and the calls go through a MikroORM Postgres driver that runs each statement alone in a rolled-back transaction with a statement timeout and records whether Postgres accepted it (`pg-probe/recorder.ts`), so every statement meets empty tables and nothing persists. What it measures is the statements each method reaches with those sample arguments against empty tables, not every statement the code can send. The distinct statements Postgres refuses as SQL or over a value's type (SQLSTATE class 42, 0A or 22: syntax, an unknown function or column, a datatype mismatch, an unsupported feature, a string reaching an integer or date column) are held per `Class.method` to `failing` in `scripts/pg-probe-baseline.json`: a method may fail no more than its entry, and an entry above what it fails now fails too. Class 22 counts because the sampler sends what each parameter declares (a `number` is 1, a `string` named like a date is a date, the `present` flag of a partial update's pair is true), so a value of the wrong type reaching a column is the code's: a parameter typed `string` bound to an integer column compares on SQLite and is refused on Postgres; type it as the column is and convert at the service (`toRowId`). A constraint the sample values break (class 23) or the empty tables is reported in the log and the step summary but not held. The methods whose SQL the probe did not see (it could not build their arguments, they sent no SQL because they threw first or never reach the database, or they timed out) are held to the same file's `uncovered` list: a method outside the list that goes unmeasured fails the run, and so does a listed method that is measured now or gone, so the list only shrinks and a new method the sampler cannot reach does not land unchecked. Fix such a method by giving it parameters the sampler can build or by making it reach its SQL with sample values. The job uploads the measured baseline and a JSON report (`pg-probe-baseline` artifact), so the change that fixes a statement copies the lowered baseline from there or runs `npm run probe:pg -- --update` against a local Postgres (`docker run --rm -p 5432:5432 -e POSTGRES_PASSWORD=probe postgres:17`); `--update` never raises or adds an entry in either part. While the baseline's `failing` or `uncovered` is `null` it has never been measured, and the run fails (outside `--update`) until the first measurement from the artifact is committed, so the gate cannot land on `dev` holding nothing. A run that called no repository method fails too, so a broken plan can neither pass nor seed an empty baseline. The unit tests under `tests/unit/scripts/pg-probe/` cover the probe's logic, including every helper case on SQLite. @@ -80,7 +81,7 @@ Nest owns everything. Every domain is a DI module under `src/nest//` (`c - **Plugins** (`src/nest/plugins/`): the child-process runtime (`supervisor/`, `runtime/plugin-host-entry.ts`, `protocol/envelope.ts`) is **intentionally not Nest**; it is the security boundary. `host/rpc-host.ts` registers an RPC method at spawn only if the plugin holds the unlocking permission (registration = authorization). Install-time gates live in `install/`; the manifest's `trek` version range fails closed, and `TREK_PLUGINS_IGNORE_TREK_RANGE` downgrades that to a warning. The protocol tables in `protocol/envelope.ts` (+ `src/plugin-event-sink.ts`) are **generated** into `plugin-sdk/` and `shared/`: run `gen:plugin-facts` after touching them and commit the output. What a plugin gets back is `protocol/output-contract.ts`: every wire method has an entry, and an entity result (`trips.getById`, `todos.list`, `vacay.mine`, ...) is cut down to its published fields in `rpc-host.ts` on the way out, together with every child row it carries whose field is listed in `PLUGIN_ENTITY_NESTED` (a day's `notes_items`, a reservation's `endpoints`). A new column on any whole-row table behind those entities, child tables included, fails `tests/unit/plugins/output-contract.schema.test.ts` until it is listed as published or withheld (the SDK's `PLUGIN_ENTITY_FIELDS` and `PLUGIN_ENTITY_NESTED` are generated from it); a method left as `readModel` must return a value built field by field from named columns. A plugin's own SQL (`host/plugin-data.service.ts`) is held to a row cap, and `query`, `tx` and an `exec` script to a time budget checked between rows and statements (`migrate` is deliberately not timed). `runtime/egress-policy.ts` is a hand-kept mirror guarded by a parity test in `plugin-sdk/`; change both sides together. - **Storage**: `src/nest/storage/` is the upload driver abstraction (local / S3 / mirror behind one interface, with an admin surface and a usage job). Domain services never touch the filesystem or the S3 SDK directly. Where the data lives (`data/`, `uploads/`, backups, logs, key files, the DB file) is resolved in one place, `src/app-config/data-paths.ts`; a provider that reads one injects `DataPathsService`, plain helper modules shared with code outside the container call `resolveDataPaths()` (the list is in `src/app-config/README.md`), and nothing anchors a data path on its own `__dirname` or on `process.cwd()`. - **DB** (`src/db/`): `better-sqlite3`, WAL, FK on. `database.ts` only holds the connection and applies pragmas. It does not build a schema, and importing it opens nothing (the first reader does). The connection's lifecycle belongs to the `DatabaseLifecycle` provider (`nest/database/database-lifecycle.service.ts`): `buildApp()` opens it through `open()` (which binds the ORM to later swaps and runs the schema bootstrap below), the shutdown closes it through `close()`, and a restore or the demo reset swaps the file only through the database backup port, `DATABASE_BACKUP` (`nest/database/database-backup.interface.ts`; the SQLite implementation is `nest/backup/sqlite-database-backup.ts`: `VACUUM INTO` snapshot, close, atomic file swap, `reopen()`). The port works on `DatabaseLifecycle.file`, the one resolved path (`TREK_DB_FILE` or `data/travel.db`), and a snapshot that fails fails the backup rather than archiving the live file. `MaintenanceRepository` is provided only by `MaintenanceModule`'s factory and injected everywhere else; no consumer constructs it. A provider that has to follow every reopen (the demo reset job saves the first demo baseline after a restore's re-bootstrap seeded the example trips) registers with `DatabaseLifecycle.onReopened()`, which runs after the reopen's schema bootstrap; a listener contains its own failures, since one that throws fails the reopen. MikroORM's migrations (`db/migrations/`) are the only schema source, **including for tests**: the hand-kept `schema.ts`/`migrations.ts`/`seeds.ts` builders and the parity test that guarded them are gone. Seeding is the MikroORM seeders in `db/seeders/`, applied by `db/orm.ts` from `buildApp()` before anything reads. An install the retired positional runner migrated (a `schema_version` row, no `mikro_orm_migrations` rows) is baselined by `db/legacy-baseline.ts`: the migrations whose `Legacy migration step N` docstring is ≤ its version are recorded as executed (never replayed) in the same transaction as the migrator run, while the baseline migration itself still runs first, as the legacy runner's `createTables()` did on every boot. It refuses the boot when `schema_version` is not exactly one row holding an integer ≥ 1, or is above the last mapped step (258); a value from 1 to 258 is read as dev's steps of those numbers, so a pre-ORM feature-branch database whose steps differ is not detected (the Tours branch is the one case it cuts back). Recorded `mikro_orm_migrations` names this build does not ship refuse it too (`refuseNewerDatabase`), whether a newer release or a development build wrote them. 4.3.x and earlier have no such check and start on a database 5.x migrated. Before a boot applies pending migrations to a file database that holds anything, `db/pre-migrate-snapshot.ts` writes a `VACUUM INTO` copy next to it (`pre-migrate--