diff --git a/server/CLAUDE.md b/server/CLAUDE.md index 6de4ca3e3..7f2314742 100644 --- a/server/CLAUDE.md +++ b/server/CLAUDE.md @@ -63,7 +63,7 @@ Single test: `npx vitest run tests/unit/nest/weather.controller.test.ts`, or `np **`lint:dialect`** (`scripts/dialect-lint.mjs`) reads every string and template literal under `src/` (comments do not count) and every `.insertId` read, and counts the SQLite-only spellings per file and rule: `INSERT OR IGNORE`/`REPLACE`, `datetime()`/`julianday()`/`unixepoch()` (also as the bare name handed to `eb.fn`), `strftime`, `json_extract` and its siblings, `GLOB`, `last_insert_rowid`, `AUTOINCREMENT`, a `PRAGMA` statement outside the `src/db/*.ts` lifecycle files and the migrations, and `||` concatenation in SQL. The counts are held to `scripts/dialect-baseline.json`; a new hit fails, and so does an entry above what its file holds now, so the change that removes one runs `npm run lint:dialect -- --update` (it never raises or adds an entry). The dialect layer `src/db/dialect/` is not scanned, and neither are the migrations up to `FROZEN_MIGRATIONS_THROUGH` in the script; a newer migration is held to the DML rules, while its DDL stays engine specific. -**`probe:pg`** (`scripts/pg-probe.ts`, CI job `postgres-probe`) measures what of the SQL runs on Postgres. Given an empty database (`TREK_PG_PROBE_URL`, or `--url=`; it refuses one holding tables it did not create) it derives a Postgres schema from the entities with MikroORM's schema generator (timestamps stay TEXT, SQLite-only defaults and `COLLATE NOCASE` get their Postgres spelling, no foreign keys), runs every dialect helper on an in-memory SQLite and on Postgres and checks both against the same expectation (`scripts/pg-probe/helper-cases.ts`; every case must pass), then calls every public method of every class in `src/db/repositories/*.ts` once. The arguments come from the parameter types, read with the TypeScript checker (`pg-probe/arg-samples.ts`), and the calls go through a MikroORM Postgres driver that runs each statement alone in a rolled-back transaction with a statement timeout and records whether Postgres accepted it (`pg-probe/recorder.ts`), so every statement meets empty tables and nothing persists. What it measures is the statements each method reaches with those sample arguments against empty tables, not every statement the code can send. The distinct statements Postgres refuses as SQL (SQLSTATE class 42 or 0A: syntax, an unknown function or column, a datatype mismatch, an unsupported feature) are held per `Class.method` to `failing` in `scripts/pg-probe-baseline.json`: a method may fail no more than its entry, and an entry above what it fails now fails too. A refusal over a sample value (class 22, say a `probe` string in a date column) or the empty tables is reported in the log and the step summary but not held, so renaming a parameter cannot turn a sample into dialect debt. The methods whose SQL the probe did not see (it could not build their arguments, they sent no SQL because they threw first or never reach the database, or they timed out) are held to the same file's `uncovered` list: a method outside the list that goes unmeasured fails the run, and so does a listed method that is measured now or gone, so the list only shrinks and a new method the sampler cannot reach does not land unchecked. Fix such a method by giving it parameters the sampler can build or by making it reach its SQL with sample values. The job uploads the measured baseline and a JSON report (`pg-probe-baseline` artifact), so the change that fixes a statement copies the lowered baseline from there or runs `npm run probe:pg -- --update` against a local Postgres (`docker run --rm -p 5432:5432 -e POSTGRES_PASSWORD=probe postgres:17`); `--update` never raises or adds an entry in either part. While the baseline's `failing` or `uncovered` is `null` it has never been measured, and the run fails (outside `--update`) until the first measurement from the artifact is committed, so the gate cannot land on `dev` holding nothing. A run that called no repository method fails too, so a broken plan can neither pass nor seed an empty baseline. The unit tests under `tests/unit/scripts/pg-probe/` cover the probe's logic, including every helper case on SQLite. +**`probe:pg`** (`scripts/pg-probe.ts`, CI job `postgres-probe`) measures what of the SQL runs on Postgres. Given an empty database (`TREK_PG_PROBE_URL`, or `--url=`; it refuses one holding tables it did not create) it derives a Postgres schema from the entities with MikroORM's schema generator (timestamps stay TEXT, SQLite-only defaults and `COLLATE NOCASE` get their Postgres spelling, no foreign keys), runs every dialect helper on an in-memory SQLite and on Postgres and checks both against the same expectation (`scripts/pg-probe/helper-cases.ts`; every case must pass), then calls every public method of every class in `src/db/repositories/*.ts` once. The arguments come from the parameter types, read with the TypeScript checker (`pg-probe/arg-samples.ts`), and the calls go through a MikroORM Postgres driver that runs each statement alone in a rolled-back transaction with a statement timeout and records whether Postgres accepted it (`pg-probe/recorder.ts`), so every statement meets empty tables and nothing persists. What it measures is the statements each method reaches with those sample arguments against empty tables, not every statement the code can send. The distinct statements Postgres refuses as SQL or over a value's type (SQLSTATE class 42, 0A or 22: syntax, an unknown function or column, a datatype mismatch, an unsupported feature, a string reaching an integer or date column) are held per `Class.method` to `failing` in `scripts/pg-probe-baseline.json`: a method may fail no more than its entry, and an entry above what it fails now fails too. Class 22 counts because the sampler sends what each parameter declares (a `number` is 1, a `string` named like a date is a date, the `present` flag of a partial update's pair is true), so a value of the wrong type reaching a column is the code's: a parameter typed `string` bound to an integer column compares on SQLite and is refused on Postgres; type it as the column is and convert at the service (`toRowId`). A constraint the sample values break (class 23) or the empty tables is reported in the log and the step summary but not held. The methods whose SQL the probe did not see (it could not build their arguments, they sent no SQL because they threw first or never reach the database, or they timed out) are held to the same file's `uncovered` list: a method outside the list that goes unmeasured fails the run, and so does a listed method that is measured now or gone, so the list only shrinks and a new method the sampler cannot reach does not land unchecked. Fix such a method by giving it parameters the sampler can build or by making it reach its SQL with sample values. The job uploads the measured baseline and a JSON report (`pg-probe-baseline` artifact), so the change that fixes a statement copies the lowered baseline from there or runs `npm run probe:pg -- --update` against a local Postgres (`docker run --rm -p 5432:5432 -e POSTGRES_PASSWORD=probe postgres:17`); `--update` never raises or adds an entry in either part. While the baseline's `failing` or `uncovered` is `null` it has never been measured, and the run fails (outside `--update`) until the first measurement from the artifact is committed, so the gate cannot land on `dev` holding nothing. A run that called no repository method fails too, so a broken plan can neither pass nor seed an empty baseline. The unit tests under `tests/unit/scripts/pg-probe/` cover the probe's logic, including every helper case on SQLite. ## Architecture invariants @@ -104,7 +104,7 @@ SQLite is the engine today and Postgres is the planned second one. Code under `s - **Strings and casts**: `concat`/`concatKysely` instead of `||`, `substring`/`substringKysely`, `lower`/`lowerTrim`, `collateNoCase`, `castInteger`/`castIntegerKysely`, and `startsWithIsoDate`/`startsWithIsoDateKysely` instead of `GLOB`. - **JSON columns** are read whole and parsed in the service; no `json_extract` in SQL. Each one is declared once in `src/db/json-columns.ts` (its Zod shape and the fallback a reader gets for an empty or broken row) and read with `decodeJson`/`decodeJsonResult`, written from a server-built value with `encodeJson` (`src/utils/json-column.ts`), never with a bare `JSON.parse` at the call site: one column, one answer to a broken row, logged. - **Timestamps** in a DATETIME column are the canonical `YYYY-MM-DD HH:MM:SS` UTC text, never `toISOString()`: in SQL the two spellings do not compare. A value written from JS is a `DbTimestamp` from `dbNow()` (`src/db/types`), and a stored value is read back as an instant with `parseDbTimestamp()` (a bare `new Date()` reads the canonical text as local time). Where a response has always carried the ISO spelling, convert on the way out (`utcSuffix`), not in the column. -- **PRAGMA** belongs to the `src/db/` lifecycle files (`database.ts`, `connection.ts`, `durability.ts`, the migrations). `MaintenanceRepository` is the baselined SQLite maintenance adapter (`wal_checkpoint`, `VACUUM INTO`). +- **PRAGMA** belongs to the `src/db/` lifecycle files (`database.ts`, `connection.ts`, `durability.ts`, the migrations) and to the SQLite engine's folder, `src/db/engine/sqlite/`, where `SqliteMaintenance` holds the file's own maintenance (`wal_checkpoint`, `VACUUM INTO`) for the backup port. `MaintenanceRepository` keeps only what runs on every engine (the readiness ping, the plugin-table erasure). - **A missing helper** goes into `sql-functions.ts` (or `kysely-functions.ts` for a Kysely expression), dispatching on the live platform through `src/db/dialect/platform.ts`: a SQLite branch and a Postgres branch (one spelling under `isKnownPlatform` where both engines read it alike), `unsupported(platform)` for anything else. The Postgres branch keeps the SQLite storage model, so a clock helper renders the same `YYYY-MM-DD HH:MM:SS` UTC text. Pin the Postgres text in `tests/unit/db/dialect/sql-functions.postgres.test.ts` and add a case to `scripts/pg-probe/helper-cases.ts` so the probe runs it on both engines. `src/db/dialect/` is the only place allowed to spell an engine's own SQL. - **Migrations** that have shipped are frozen as they are. A new migration's data statements follow the rules above; its DDL is written for the engine it runs on. - **Storage model**: the generated Kysely types describe what SQLite stores (0/1 integers for flags, TEXT timestamps, JSON in TEXT). Convert at the service boundary, as the contracts already require. diff --git a/server/scripts/dialect-baseline.json b/server/scripts/dialect-baseline.json index a10b22214..c9ef31ada 100644 --- a/server/scripts/dialect-baseline.json +++ b/server/scripts/dialect-baseline.json @@ -26,9 +26,6 @@ "src/db/entities/PluginUserErasureQueue.entity.ts": { "datetime": 1 }, - "src/db/repositories/MaintenanceRepository.ts": { - "pragma": 1 - }, "src/nest/backup/backup-archive.ts": { "pragma": 1 } diff --git a/server/scripts/dialect-lint.mjs b/server/scripts/dialect-lint.mjs index 72681c47b..0f207376e 100644 --- a/server/scripts/dialect-lint.mjs +++ b/server/scripts/dialect-lint.mjs @@ -124,9 +124,16 @@ export function classify(file) { return 'source'; } -/** PRAGMA is fine in src/db/*.ts (connection, durability, baseline, snapshot) and the migrations. */ +/** + * PRAGMA is fine in src/db/*.ts (connection, durability, baseline, snapshot), + * the migrations and the SQLite engine's own folder (src/db/engine/sqlite/). + */ export function pragmaAllowed(file) { - return /^src\/db\/[^/]+\.ts$/.test(file) || file.startsWith('src/db/migrations/'); + return ( + /^src\/db\/[^/]+\.ts$/.test(file) || + file.startsWith('src/db/migrations/') || + file.startsWith('src/db/engine/sqlite/') + ); } function isSqlTag(node) { diff --git a/server/scripts/pg-probe-baseline.json b/server/scripts/pg-probe-baseline.json index c4916d571..60193d1ed 100644 --- a/server/scripts/pg-probe-baseline.json +++ b/server/scripts/pg-probe-baseline.json @@ -1,21 +1,4 @@ { - "failing": { - "DayAccommodationsRepository.listForTripWithPlaceAndBooking": 1, - "DayAccommodationsRepository.listRoadtripStays": 1, - "GoogleApiUsageRepository.recordCall": 1, - "MaintenanceRepository.vacuumInto": 1, - "MaintenanceRepository.walCheckpoint": 1, - "ReservationDayPositionsRepository.upsertScoped": 1, - "ReservationsRepository.findWithJoins": 1, - "ReservationsRepository.listForCalendar": 1, - "ReservationsRepository.listForTrip": 1, - "ReservationsRepository.listPublicStaysForCalendar": 1, - "RouteUsageDailyRepository.record": 1 - }, - "uncovered": [ - "BucketListRepository.update", - "BudgetItemsRepository.update", - "PackingBagsRepository.update", - "TodoItemsRepository.update" - ] + "failing": {}, + "uncovered": [] } diff --git a/server/scripts/pg-probe.ts b/server/scripts/pg-probe.ts index 3d9cb2baf..b54225689 100644 --- a/server/scripts/pg-probe.ts +++ b/server/scripts/pg-probe.ts @@ -15,10 +15,11 @@ * (pg-probe/arg-samples.ts), through a MikroORM Postgres driver that * runs each statement in its own rolled-back transaction and records * whether Postgres accepted it (pg-probe/recorder.ts); - * 4. holds the statements Postgres refuses as SQL (SQLSTATE class 42 or - * 0A), per method, to scripts/pg-probe-baseline.json, which may only - * shrink (pg-probe/baseline.ts); a refusal over a sample value - * (class 22) or the empty tables is reported, not held; + * 4. holds the statements Postgres refuses as SQL or over a value's type + * (SQLSTATE class 42, 0A or 22), per method, to + * scripts/pg-probe-baseline.json, which may only shrink + * (pg-probe/baseline.ts); a constraint over a sample value (class 23) or + * the empty tables is reported, not held; * 5. holds the methods whose SQL it did not see (it could not call them, * they sent no SQL, or they timed out) to the same file's `uncovered` * list, which may only shrink too, so a new method the probe cannot diff --git a/server/scripts/pg-probe/arg-samples.ts b/server/scripts/pg-probe/arg-samples.ts index e6273f1f4..1e3848005 100644 --- a/server/scripts/pg-probe/arg-samples.ts +++ b/server/scripts/pg-probe/arg-samples.ts @@ -4,7 +4,8 @@ * The probe has no fixtures: it reads every repository class with the * TypeScript checker and builds one sample argument list per public method * from the declared parameter types (a number is 1, a string is "probe" or a - * date-shaped string when the name says so, an object gets every property + * date-shaped string when the name says so, a boolean false unless it is the + * `present` flag of a partial update's pair, an object gets every property * it can sample, an array one element). The values only have to get the * method to send its SQL; what the probe measures is whether Postgres * accepts that SQL against an empty schema. A sample the column cannot take @@ -24,6 +25,12 @@ export type MethodPlan = | { className: string; file: string; method: string; unprobeable: string }; const MAX_DEPTH = 4; +/** + * The label of the flag in the repositories' `[present: boolean, value]` + * pairs (`presenceSet`): a patch whose pairs are all absent sends no SQL, so + * this one boolean is sampled true. Every other boolean stays false. + */ +const PRESENCE_LABEL = 'present'; const SAMPLE_DATE = '2026-01-01'; const SAMPLE_TIMESTAMP = '2026-01-01 00:00:00'; @@ -68,14 +75,14 @@ export function sampleFor(checker: ts.TypeChecker, type: ts.Type, name: string, if (type.isUnion()) { const members = type.types.filter((member) => !isNullish(member)); if (members.length === 0) return OK(null); - if (members.every((member) => member.flags & ts.TypeFlags.BooleanLiteral)) return OK(false); + if (members.every((member) => member.flags & ts.TypeFlags.BooleanLiteral)) return OK(name === PRESENCE_LABEL); const numeric = members.find((member) => member.flags & ts.TypeFlags.NumberLike); return sampleFor(checker, numeric ?? members[0]!, name, depth + 1); } if (type.isStringLiteral()) return OK(type.value); if (type.isNumberLiteral()) return OK(type.value); if (flags & ts.TypeFlags.BooleanLiteral) return OK(checker.typeToString(type) === 'true'); - if (flags & ts.TypeFlags.BooleanLike) return OK(false); + if (flags & ts.TypeFlags.BooleanLike) return OK(name === PRESENCE_LABEL); if (flags & ts.TypeFlags.BigIntLike) return OK(BigInt(1)); if (flags & ts.TypeFlags.NumberLike) return OK(1); if (flags & (ts.TypeFlags.StringLike | ts.TypeFlags.TemplateLiteral)) return OK(sampleString(name)); @@ -88,8 +95,10 @@ export function sampleFor(checker: ts.TypeChecker, type: ts.Type, name: string, function sampleObjectType(checker: ts.TypeChecker, type: ts.ObjectType, name: string, depth: number): Sample { if (checker.isTupleType(type)) { const values: unknown[] = []; + const labels = ((type as ts.TypeReference).target as ts.TupleType).labeledElementDeclarations; for (const [i, element] of checker.getTypeArguments(type as ts.TypeReference).entries()) { - const sample = sampleFor(checker, element, `${name}[${i}]`, depth + 1); + const label = labels?.[i]?.name; + const sample = sampleFor(checker, element, label && ts.isIdentifier(label) ? label.text : `${name}[${i}]`, depth + 1); if (!sample.ok) return sample; values.push(sample.value); } diff --git a/server/scripts/pg-probe/baseline.ts b/server/scripts/pg-probe/baseline.ts index 879604355..f3590daff 100644 --- a/server/scripts/pg-probe/baseline.ts +++ b/server/scripts/pg-probe/baseline.ts @@ -11,9 +11,9 @@ * `--update` lowers entries and drops the ones that reach zero or whose * method is gone; it never raises or adds one. * - * Only statements Postgres refuses as SQL count here (SQLSTATE classes 42 - * and 0A, see `countsTowardRatchet`). A value the sample arguments got wrong - * (class 22, a sample string in a date column) says nothing about the + * Statements Postgres refuses as SQL or over a value's type count here + * (SQLSTATE classes 42, 0A and 22, see `countsTowardRatchet`). A constraint + * the sample values break (class 23) or a timeout says nothing about the * dialect and is reported without being held. * * `uncovered` lists the methods whose SQL the probe did not see: the ones it @@ -56,12 +56,16 @@ export interface BaselineVerdict { /** * Whether a refused statement counts toward the ratchet: SQLSTATE class 42 * (syntax error, unknown function, operator or column, datatype mismatch) - * and class 0A (feature not supported) are what a SQLite-only spelling gets. - * Anything else (a data exception such as 22P02 or 22007, a constraint, a - * timeout) comes from the sample values or the empty tables, not the SQL. + * and class 0A (feature not supported) are what a SQLite-only spelling gets, + * and class 22 (a data exception such as 22P02 or 22007) is a value of the + * wrong type reaching a column, which SQLite compares where Postgres refuses + * (a query-string id bound to an integer column). The sampler sends what + * each parameter declares, so a class 22 refusal is the code's, not the + * sample's. A constraint (class 23) or a timeout comes from the sample values + * or the empty tables, not the SQL. */ export function countsTowardRatchet(code: string): boolean { - return /^(42|0A)/.test(code); + return /^(42|0A|22)/.test(code); } const METHOD_KEY = /^[A-Za-z_$][\w$]*\.[A-Za-z_$][\w$]*$/; diff --git a/server/scripts/pg-probe/report.ts b/server/scripts/pg-probe/report.ts index 10feb0ea7..75c9a2965 100644 --- a/server/scripts/pg-probe/report.ts +++ b/server/scripts/pg-probe/report.ts @@ -22,7 +22,7 @@ export interface FailedStatement { code: string; message: string; sql: string; - /** A dialect refusal (SQLSTATE class 42 or 0A) the ratchet holds; anything else is reported only. */ + /** A refusal the ratchet holds (SQLSTATE class 42, 0A or 22); anything else is reported only. */ counted: boolean; } @@ -38,8 +38,9 @@ export interface ProbeSummary { /** Every distinct statement Postgres refused, counted toward the ratchet or not. */ failedStatements: FailedStatement[]; /** - * `Class.method` → distinct statements Postgres refused as SQL (SQLSTATE - * class 42 or 0A); every called method appears, zeros included. + * `Class.method` → distinct statements Postgres refused as SQL or over a + * value's type (SQLSTATE class 42, 0A or 22); every called method appears, + * zeros included. */ failingByMethod: Map; /** @@ -192,7 +193,8 @@ export function formatConsole(input: ReportInput): string { const refusals = splitRefusals(summary); lines.push( `Statements: ${summary.statements} distinct, ${summary.failedStatements.length} refused by Postgres ` + - `(${refusals.counted.length} as SQL, held to the baseline; ${refusals.reported.length} over a sample value or the empty tables, reported only).`, + `(${refusals.counted.length} as SQL or over a value's type, held to the baseline; ` + + `${refusals.reported.length} over a constraint, a timeout or the empty tables, reported only).`, ); for (const [code, count] of [...summary.failuresByCode.entries()].sort((a, b) => b[1] - a[1])) { lines.push(` ${code}: ${count}${countsTowardRatchet(code) ? '' : ' (not held)'}`); @@ -254,8 +256,8 @@ export function formatMarkdown(input: ReportInput): string { ); out.push(`| Methods whose SQL the probe did not see | ${summary.uncovered.size} |`); out.push(`| Distinct statements | ${summary.statements} |`); - out.push(`| Refused by Postgres as SQL (held) | ${refusals.counted.length} |`); - out.push(`| Refused over a sample value or the empty tables (reported only) | ${refusals.reported.length} |`); + out.push(`| Refused by Postgres as SQL or over a value's type (held) | ${refusals.counted.length} |`); + out.push(`| Refused over a constraint, a timeout or the empty tables (reported only) | ${refusals.reported.length} |`); out.push(''); if (probedNothing(summary)) out.push('No repository method was called, so there is nothing to hold to the baseline.', ''); diff --git a/server/scripts/size-baseline.json b/server/scripts/size-baseline.json index 2cde2f810..13cd417b2 100644 --- a/server/scripts/size-baseline.json +++ b/server/scripts/size-baseline.json @@ -2,7 +2,7 @@ "scripts/db-call-graph.mjs": 1036, "scripts/generate-entities.ts": 2259, "src/db/repositories/Places.repository.ts": 1437, - "src/db/repositories/Reservations.repository.ts": 1604, + "src/db/repositories/Reservations.repository.ts": 1563, "src/db/repositories/Trips.repository.ts": 1121, "src/db/repositories/Users.repository.ts": 1445, "src/nest/atlas/atlas.service.ts": 1177, @@ -12,7 +12,7 @@ "src/nest/doc-sync/doc-sync.service.ts": 1060, "src/nest/doc-sync/providers/synology-drive.client.ts": 1160, "src/nest/journey/journey-domain.service.ts": 1659, - "src/nest/places/places.service.ts": 1549, + "src/nest/places/places.service.ts": 1548, "src/nest/plugins/plugin-runtime.service.ts": 1460, "src/nest/plugins/runtime/plugin-sdk.ts": 1093, "src/nest/reservations/reservations.service.ts": 1237, diff --git a/server/scripts/test-new-service-baseline.json b/server/scripts/test-new-service-baseline.json index b818f531b..7f445dd1a 100644 --- a/server/scripts/test-new-service-baseline.json +++ b/server/scripts/test-new-service-baseline.json @@ -143,7 +143,6 @@ "tests/unit/nest/sessions/sessions.service.test.ts": 1, "tests/unit/nest/settings.service.test.ts": 1, "tests/unit/nest/share.service.test.ts": 3, - "tests/unit/nest/sqlite-database-backup.test.ts": 2, "tests/unit/nest/storage/storage-admin.service.test.ts": 5, "tests/unit/nest/storage/storage-jobs.service.test.ts": 15, "tests/unit/nest/storage/storage-registry.service.test.ts": 3, diff --git a/server/src/db/engine/sqlite/sqlite-maintenance.ts b/server/src/db/engine/sqlite/sqlite-maintenance.ts new file mode 100644 index 000000000..e151c9339 --- /dev/null +++ b/server/src/db/engine/sqlite/sqlite-maintenance.ts @@ -0,0 +1,49 @@ +import type { EntityManager } from '@mikro-orm/core'; + +/** + * The SQLite engine's whole-file maintenance: the two statements a backup, + * a restore and the demo reset need that are about the database file rather + * than any table (`PRAGMA wal_checkpoint`, `VACUUM INTO`). Neither has a + * Postgres spelling, so they live under the engine that owns the file instead + * of in `MaintenanceRepository`, whose remaining statements run on every + * engine (and which the Postgres probe calls method by method). + * + * Provided through Nest DI by `MaintenanceModule` (`nest/database/`), a + * factory over the context-resolving global `EntityManager`, and injected by + * the SQLite backup port (`SqliteDatabaseBackup`). Free of Nest decorators + * like every other file under `db/`, so a test can build one over its own + * `EntityManager`. + */ +export class SqliteMaintenance { + constructor(private readonly em: EntityManager) {} + + /** + * MikroORM's own validating `getContext()`: throws outside a request + * context, the fail-closed check `TrekRepository` applies to every + * repository, since a raw `connection.execute()` has no such guard. + */ + private validateRequestContext(): void { + this.em.getContext(); + } + + /** + * `PRAGMA wal_checkpoint(TRUNCATE)`: flushes the WAL into the main file + * before a snapshot or a swap. Reached through the backup port's + * `checkpoint()`; every caller treats it as best effort. + */ + async walCheckpoint(): Promise { + this.validateRequestContext(); + await this.em.getConnection().execute('PRAGMA wal_checkpoint(TRUNCATE)'); + } + + /** + * `` VACUUM INTO '' ``: a consistent point-in-time copy of the file + * under concurrent writers, reached through the backup port's + * `snapshot()`. Single quotes in `path` are escaped by doubling. + */ + async vacuumInto(path: string): Promise { + this.validateRequestContext(); + const escaped = path.replaceAll("'", "''"); + await this.em.getConnection().execute(`VACUUM INTO '${escaped}'`); + } +} diff --git a/server/src/db/repositories/DayAccommodations.repository.ts b/server/src/db/repositories/DayAccommodations.repository.ts index 403d6cd1b..ba843c1f5 100644 --- a/server/src/db/repositories/DayAccommodations.repository.ts +++ b/server/src/db/repositories/DayAccommodations.repository.ts @@ -1,3 +1,4 @@ +import { castIntegerKysely } from '../dialect/kysely-functions'; import { coalesceOverride, columnRef } from '../dialect/sql-functions'; import type { DayAccommodations } from '../entities/DayAccommodations.entity'; import type { DB } from '../kysely/db'; @@ -249,10 +250,15 @@ export class DayAccommodationsRepository extends TrekRepository { + const platform = this.getEntityManager().getPlatform(); const rows = await this.kysely() .selectFrom('day_accommodations as a') .leftJoin('places as p', 'p.id', 'a.place_id') - .leftJoin('reservations as r', 'r.accommodation_id', 'a.id') + // TEXT `accommodation_id` against INTEGER `a.id`: cast, as Postgres refuses + // to compare the two (42883) where SQLite coerces. + .leftJoin('reservations as r', (join) => + join.on((eb) => eb(castIntegerKysely(platform, eb, 'r.accommodation_id'), '=', eb.ref('a.id'))), + ) .selectAll('a') .select([ 'p.name as place_name', @@ -469,6 +475,7 @@ export class DayAccommodationsRepository extends TrekRepository { + const platform = this.getEntityManager().getPlatform(); const rows = await this.kysely() .selectFrom('day_accommodations as a') .leftJoin('places as p', 'p.id', 'a.place_id') @@ -485,7 +492,7 @@ export class DayAccommodationsRepository extends TrekRepository eb2.fn.min('r.id').as('m')) - .whereRef('r.accommodation_id', '=', 'a.id') + .where((eb2) => eb2(castIntegerKysely(platform, eb2, 'r.accommodation_id'), '=', eb2.ref('a.id'))) .as('reservation_id'), ]) .where('a.trip_id', '=', trip_id) diff --git a/server/src/db/repositories/GoogleApiUsage.repository.ts b/server/src/db/repositories/GoogleApiUsage.repository.ts index 47932a7bc..835fe83da 100644 --- a/server/src/db/repositories/GoogleApiUsage.repository.ts +++ b/server/src/db/repositories/GoogleApiUsage.repository.ts @@ -43,7 +43,11 @@ export class GoogleApiUsageRepository extends TrekRepository { await this.kysely() .insertInto('google_api_usage') .values({ day, calls: 1 }) - .onConflict((oc) => oc.column('day').doUpdateSet({ calls: (eb) => eb('calls', '+', 1) })) + .onConflict((oc) => + // Qualified: Postgres reads a bare `calls` here as ambiguous between the + // target row and `excluded` (42702); SQLite accepts both spellings. + oc.column('day').doUpdateSet({ calls: (eb) => eb(eb.ref('google_api_usage.calls'), '+', 1) }), + ) .execute(); } diff --git a/server/src/db/repositories/MaintenanceRepository.ts b/server/src/db/repositories/MaintenanceRepository.ts index 93a82b412..89fc10dee 100644 --- a/server/src/db/repositories/MaintenanceRepository.ts +++ b/server/src/db/repositories/MaintenanceRepository.ts @@ -6,22 +6,20 @@ import type { EntityManager } from '@mikro-orm/core'; * MaintenanceRepository/DemoRepository, with a comment)"). * * Not a `TrekRepository` — it backs no table and no entity of its - * own. It exists purely as the one place two whole-database-file statements - * (a `PRAGMA` and a `VACUUM INTO`) run against the ORM's own bound driver - * connection instead of scattering raw `better-sqlite3` calls across - * `backup.impl.ts`/`demo-reset.ts`. `find/findOne → create/assign/flush → - * nativeUpdate/nativeDelete → em.upsert → qb() → em.getKysely()` all fail - * for these two statements — neither is expressible against an entity at - * all — so `connection.execute()` (the last tier in the program's query-API - * order) is the correct tool, not a shortcut around it. + * own. It holds the statements no entity can express that run on every + * engine: the readiness ping and the plugin-table erasure. `find/findOne → + * create/assign/flush → nativeUpdate/nativeDelete → em.upsert → qb() → + * em.getKysely()` all fail for them, so `connection.execute()` (the last + * tier in the program's query-API order) is the correct tool, not a + * shortcut around it. The SQLite file's own maintenance (`PRAGMA + * wal_checkpoint`, `VACUUM INTO`) is `SqliteMaintenance`, under + * `db/engine/sqlite/`. * * Provided through Nest DI by `MaintenanceModule` (`nest/database/`), a * factory over the context-resolving global `EntityManager`; its consumers - * inject it: the readiness probe, `UserCleanupService` and the SQLite backup - * port (`SqliteDatabaseBackup`), which the backup, the restore and the demo - * reset all go through. The class stays free of Nest decorators like every - * other file under `db/`, so a test can still build one over its own - * `EntityManager`. + * inject it: the readiness probe and `UserCleanupService`. The class stays + * free of Nest decorators like every other file under `db/`, so a test can + * still build one over its own `EntityManager`. */ export class MaintenanceRepository { constructor(private readonly em: EntityManager) {} @@ -30,7 +28,7 @@ export class MaintenanceRepository { * MikroORM's own validating `getContext()` — throws * `ValidationError.cannotUseGlobalContext()` outside a request context (the * production default), the same fail-closed ratchet `TrekRepository` - * applies to every entity-bound repository. Called first by both methods + * applies to every entity-bound repository. Called first by every method * below, since raw `connection.execute()` has no such guard of its own. */ private validateRequestContext(): void { @@ -43,32 +41,6 @@ export class MaintenanceRepository { await this.em.getConnection().execute('SELECT 1'); } - /** - * `PRAGMA wal_checkpoint(TRUNCATE)` — rendered text pinned, identical to - * the legacy `db.exec('PRAGMA wal_checkpoint(TRUNCATE)')` it replaces. - * Reached through the backup port's `checkpoint()`, which the backup - * (BK1) and the demo reset (DMR3) call. Flushes the WAL into the main db - * file before a snapshot or a swap; every caller treats it as best effort. - */ - async walCheckpoint(): Promise { - this.validateRequestContext(); - await this.em.getConnection().execute('PRAGMA wal_checkpoint(TRUNCATE)'); - } - - /** - * `` VACUUM INTO '' `` — rendered text pinned, identical to the - * legacy `` db.exec(`VACUUM INTO '${dbSnap.replaceAll("'", "''")}'`) `` - * (BK2, reached through the backup port's `snapshot()`): single quotes in `path` are - * escaped by doubling, the same escaping the legacy statement used — a - * consistent point-in-time snapshot under concurrent writers, with no - * MikroORM/Kysely equivalent. - */ - async vacuumInto(path: string): Promise { - this.validateRequestContext(); - const escaped = path.replaceAll("'", "''"); - await this.em.getConnection().execute(`VACUUM INTO '${escaped}'`); - } - /** * Plan 4 Task 4 — UC1 (`UserCleanupService.erasePluginUserData`): a * best-effort per-user delete across the three plugin-held tables @@ -77,13 +49,13 @@ export class MaintenanceRepository { * narrowed by Plan 4 Task 8a to just this trio — UC2/UC3 already moved to * `PluginsRepository`/`PluginUserErasureQueueRepository`), not an entity * managed by any repository, so `connection.execute()` (rule 4's last - * tier) is the correct tool here, same shape as `walCheckpoint`/ - * `vacuumInto` above — with ONE difference those two don't need: this is + * tier) is the correct tool here, same shape as `ping` above and + * `SqliteMaintenance`'s statements — with ONE difference those don't need: this is * the first `MaintenanceRepository` caller ever invoked from INSIDE an * open `UnitOfWork.transactional(...)` (`deleteUserCompletely`'s * transaction). `Connection#execute(query, params, method, ctx)`'s `ctx` * defaults to the plain (non-transactional) Kysely client when omitted — - * `walCheckpoint`/`vacuumInto` always omit it because their callers + * `SqliteMaintenance`'s statements always omit it because their callers * (the backup port) never run inside a transaction, but * omitting it here deadlocked: this repo's `better-sqlite3` connection is * single-connection/single-writer, the open transaction holds it for its diff --git a/server/src/db/repositories/Places.repository.ts b/server/src/db/repositories/Places.repository.ts index e9fe5e761..4941a1582 100644 --- a/server/src/db/repositories/Places.repository.ts +++ b/server/src/db/repositories/Places.repository.ts @@ -659,8 +659,8 @@ export class PlacesRepository extends TrekRepository { */ async listForTrip(trip_id: string, filters: { searchPattern?: string; - category?: string; - tag?: string; + category?: number; + tag?: number; assignment?: 'all' | 'unassigned' | 'assigned'; }): Promise { const platform = this.getEntityManager().getPlatform(); @@ -685,10 +685,10 @@ export class PlacesRepository extends TrekRepository { [filters.searchPattern, filters.searchPattern, filters.searchPattern], ); } - if (filters.category) { + if (filters.category !== undefined) { qb.andWhere('p.category_id = ?', [filters.category]); } - if (filters.tag) { + if (filters.tag !== undefined) { qb.andWhere('p.id IN (SELECT place_id FROM place_tags WHERE tag_id = ?)', [filters.tag]); } if (filters.assignment === 'unassigned') { diff --git a/server/src/db/repositories/ReservationDayPositions.repository.ts b/server/src/db/repositories/ReservationDayPositions.repository.ts index 99259e16f..0b6ca570e 100644 --- a/server/src/db/repositories/ReservationDayPositions.repository.ts +++ b/server/src/db/repositories/ReservationDayPositions.repository.ts @@ -52,10 +52,14 @@ export class ReservationDayPositionsRepository extends TrekRepository { await this.kysely() .insertInto('reservation_day_positions') - .orReplace() .columns(['reservation_id', 'day_id', 'position']) .expression((eb) => eb @@ -79,6 +82,9 @@ export class ReservationDayPositionsRepository extends TrekRepository ({ position: eb.ref('excluded.position') })), + ) .execute(); } } diff --git a/server/src/db/repositories/Reservations.repository.ts b/server/src/db/repositories/Reservations.repository.ts index 87f824934..f4c450962 100644 --- a/server/src/db/repositories/Reservations.repository.ts +++ b/server/src/db/repositories/Reservations.repository.ts @@ -5,6 +5,7 @@ import { coalesceOverride, columnRef, dayDistance } from '../dialect/sql-functio import { castIntegerKysely, concatKysely, startsWithIsoDateKysely, substringKysely } from '../dialect/kysely-functions'; import { publicReservationExpr, publicStayExists, type ReservationVisibilityKyselyDB } from './_shared/reservation-visibility'; import type { DayAssignmentRow } from './day-assignment-rows'; +import { selectReservationsWithJoins, type ReservationJoinKyselyDB, type ReservationJoinRow } from './reservation-join'; import { TrekRepository } from './_shared/trek-repository'; import type { DB } from '../kysely/db'; import { ROADTRIP_RESERVATION_TYPES } from '@trek/shared'; @@ -18,47 +19,15 @@ import type { PublicApiUnscheduledReservationRow, } from './reservation-public-api-rows'; -// The public API row types moved to their own module; re-exported so callers keep one import path. +// The joined and public API row types moved to their own modules; re-exported so callers keep one import path. export type { + ReservationJoinRow, PublicApiAccommodationRow, PublicApiScheduledReservationRow, PublicApiUnplannedPlaceRow, PublicApiUnscheduledReservationRow, }; -/** - * RS18/RR1's shared joined projection — `SELECT r.*, d.day_number, p.name as - * place_name, r.assignment_id, ap.place_id as accommodation_place_id, - * acc_p.name as accommodation_name, ap.start_day_id as - * accommodation_start_day_id, ap.end_day_id as accommodation_end_day_id FROM - * reservations r LEFT JOIN days d ON r.day_id = d.id LEFT JOIN places p ON - * r.place_id = p.id LEFT JOIN day_accommodations ap ON r.accommodation_id = - * ap.id LEFT JOIN places acc_p ON ap.place_id = acc_p.id`. The service - * folds `accommodation_id` (`Math.trunc(Number(x))`), `day_positions`, - * `endpoints` and `travelers` on afterward (unchanged from the legacy split). - */ -export interface ReservationJoinRow extends ReservationAllColumnsRow { - day_number: number | null; - place_name: string | null; - accommodation_place_id: number | null; - accommodation_name: string | null; - accommodation_start_day_id: number | null; - accommodation_end_day_id: number | null; -} - -/** - * Kysely typing for the RS18/RR1 join (`joinedQuery` below) — `reservations - * .accommodation_id` is a bare `p.text().nullable()` column with no FK to - * `day_accommodations` (§18.1 of the inventory), so the QueryBuilder's - * relation-path `.join()` cannot express `LEFT JOIN day_accommodations ap ON - * r.accommodation_id = ap.id` at all — the same escape hatch - * `DayAssignmentsRepository.effectiveStart`/`TripsRepository.tripSelectQuery` - * use. The `id`/`trip_id` the callers pass stay `number | string` (raw-bind, - * D4's T5 seam) and are cast to the column type only at the bind, so this - * join keeps the legacy statement's exact raw-bind flexibility. - */ -type ReservationJoinKyselyDB = Pick; - /** * `reservations` as the two aliases below bind it: `accommodation_id` is a * TEXT column, widened to `string | number | null` so the plain `number` @@ -380,21 +349,7 @@ type FlightsMissingEndpointsKyselyDB = Pick { private joinedQuery() { - return this.kysely() - .selectFrom('reservations as r') - .leftJoin('days as d', 'd.id', 'r.day_id') - .leftJoin('places as p', 'p.id', 'r.place_id') - .leftJoin('day_accommodations as ap', 'ap.id', 'r.accommodation_id') - .leftJoin('places as acc_p', 'acc_p.id', 'ap.place_id') - .selectAll('r') - .select([ - 'd.day_number as day_number', - 'p.name as place_name', - 'ap.place_id as accommodation_place_id', - 'acc_p.name as accommodation_name', - 'ap.start_day_id as accommodation_start_day_id', - 'ap.end_day_id as accommodation_end_day_id', - ]); + return selectReservationsWithJoins(this.kysely(), this.getEntityManager().getPlatform()); } /** @@ -890,10 +845,13 @@ export class ReservationsRepository extends TrekRepository { /** CL2 — see {@link CalendarReservationRow}'s docstring. */ async listForCalendar(trip_id: number | string): Promise { + const platform = this.getEntityManager().getPlatform(); const rows = await this.kysely() .selectFrom('reservations as r') .leftJoin('places as pl', 'pl.id', 'r.place_id') - .leftJoin('day_accommodations as a', 'a.id', 'r.accommodation_id') + .leftJoin('day_accommodations as a', (join) => + join.on((eb) => eb(castIntegerKysely(platform, eb, 'r.accommodation_id'), '=', eb.ref('a.id'))), + ) .leftJoin('days as sd', 'sd.id', 'a.start_day_id') .leftJoin('days as ed', 'ed.id', 'a.end_day_id') .leftJoin('days as rd', 'rd.id', 'r.day_id') @@ -909,7 +867,7 @@ export class ReservationsRepository extends TrekRepository { eb .selectFrom('reservations as r2') .select((eb2) => eb2.fn.min('r2.id').as('c')) - .whereRef('r2.accommodation_id', '=', 'a.id') + .where((eb2) => eb2(castIntegerKysely(platform, eb2, 'r2.accommodation_id'), '=', eb2.ref('a.id'))) .as('stay_first_reservation_id'), 'rd.date as day_date', 'red.date as end_day_date', @@ -947,6 +905,7 @@ export class ReservationsRepository extends TrekRepository { /** CL7 — see {@link CalendarStayRow}'s docstring. */ async listPublicStaysForCalendar(trip_id: number | string): Promise { + const platform = this.getEntityManager().getPlatform(); const rows = await this.kysely() .selectFrom('day_accommodations as a') .where('a.trip_id', '=', trip_id as number) @@ -968,7 +927,7 @@ export class ReservationsRepository extends TrekRepository { eb .selectFrom('reservations as r') .select('r.title') - .whereRef('r.accommodation_id', '=', 'a.id') + .where((eb2) => eb2(castIntegerKysely(platform, eb2, 'r.accommodation_id'), '=', eb2.ref('a.id'))) .where((eb2) => publicReservationExpr(eb2, 'r.ingest_state')) .orderBy('r.id', 'asc') .limit(1) diff --git a/server/src/db/repositories/RouteUsageDaily.repository.ts b/server/src/db/repositories/RouteUsageDaily.repository.ts index ce011685f..38cc7470d 100644 --- a/server/src/db/repositories/RouteUsageDaily.repository.ts +++ b/server/src/db/repositories/RouteUsageDaily.repository.ts @@ -81,11 +81,13 @@ export class RouteUsageDailyRepository extends TrekRepository { .insertInto('route_usage_daily') .values(entry) .onConflict((oc) => + // The target row's columns are qualified: Postgres reads a bare name + // as ambiguous between it and `excluded` (42702). oc.columns(['day', 'profile', 'surface', 'self_hosted']).doUpdateSet({ - requests: (eb) => eb('requests', '+', eb.ref('excluded.requests')), - waypoints: (eb) => eb('waypoints', '+', eb.ref('excluded.waypoints')), - km: (eb) => eb('km', '+', eb.ref('excluded.km')), - failed: (eb) => eb('failed', '+', eb.ref('excluded.failed')), + requests: (eb) => eb(eb.ref('route_usage_daily.requests'), '+', eb.ref('excluded.requests')), + waypoints: (eb) => eb(eb.ref('route_usage_daily.waypoints'), '+', eb.ref('excluded.waypoints')), + km: (eb) => eb(eb.ref('route_usage_daily.km'), '+', eb.ref('excluded.km')), + failed: (eb) => eb(eb.ref('route_usage_daily.failed'), '+', eb.ref('excluded.failed')), }), ) .execute(); diff --git a/server/src/db/repositories/reservation-join.ts b/server/src/db/repositories/reservation-join.ts new file mode 100644 index 000000000..32501b60e --- /dev/null +++ b/server/src/db/repositories/reservation-join.ts @@ -0,0 +1,71 @@ +/** + * RS18/RR1's joined read of reservations: the row shape, the Kysely tables it + * binds and the statement. It lives beside the reservations repository so + * that file stays readable; the repository re-exports the row type, so + * callers keep importing it from there. + */ +import { castIntegerKysely } from '../dialect/kysely-functions'; +import type { DB } from '../kysely/db'; +import type { ReservationAllColumnsRow } from './Reservations.repository'; +import type { Platform } from '@mikro-orm/core'; + +import type { Kysely } from 'kysely'; + +/** + * RS18/RR1's shared joined projection — `SELECT r.*, d.day_number, p.name as + * place_name, r.assignment_id, ap.place_id as accommodation_place_id, + * acc_p.name as accommodation_name, ap.start_day_id as + * accommodation_start_day_id, ap.end_day_id as accommodation_end_day_id FROM + * reservations r LEFT JOIN days d ON r.day_id = d.id LEFT JOIN places p ON + * r.place_id = p.id LEFT JOIN day_accommodations ap ON r.accommodation_id = + * ap.id LEFT JOIN places acc_p ON ap.place_id = acc_p.id`. The service + * folds `accommodation_id` (`Math.trunc(Number(x))`), `day_positions`, + * `endpoints` and `travelers` on afterward (unchanged from the legacy split). + */ +export interface ReservationJoinRow extends ReservationAllColumnsRow { + day_number: number | null; + place_name: string | null; + accommodation_place_id: number | null; + accommodation_name: string | null; + accommodation_start_day_id: number | null; + accommodation_end_day_id: number | null; +} + +/** + * Kysely typing for the RS18/RR1 join — `reservations.accommodation_id` is a + * bare `p.text().nullable()` column with no FK to `day_accommodations` (§18.1 + * of the inventory), so the QueryBuilder's relation-path `.join()` cannot + * express `LEFT JOIN day_accommodations ap ON r.accommodation_id = ap.id` at + * all — the same escape hatch `DayAssignmentsRepository.effectiveStart`/ + * `TripsRepository.tripSelectQuery` use. The `id`/`trip_id` the callers pass + * stay `number | string` (raw-bind, D4's T5 seam) and are cast to the column + * type only at the bind, so this join keeps the legacy statement's exact + * raw-bind flexibility. + */ +export type ReservationJoinKyselyDB = Pick; + +/** + * The joined projection, without a WHERE or ORDER BY: each caller adds its + * own. `accommodation_id` is TEXT and `ap.id` INTEGER, so the text side is + * cast for the join (SQLite coerces the bare comparison, Postgres refuses it + * with 42883). + */ +export function selectReservationsWithJoins(db: Kysely, platform: Platform) { + return db + .selectFrom('reservations as r') + .leftJoin('days as d', 'd.id', 'r.day_id') + .leftJoin('places as p', 'p.id', 'r.place_id') + .leftJoin('day_accommodations as ap', (join) => + join.on((eb) => eb(castIntegerKysely(platform, eb, 'r.accommodation_id'), '=', eb.ref('ap.id'))), + ) + .leftJoin('places as acc_p', 'acc_p.id', 'ap.place_id') + .selectAll('r') + .select([ + 'd.day_number as day_number', + 'p.name as place_name', + 'ap.place_id as accommodation_place_id', + 'acc_p.name as accommodation_name', + 'ap.start_day_id as accommodation_start_day_id', + 'ap.end_day_id as accommodation_end_day_id', + ]); +} diff --git a/server/src/nest/backup/sqlite-database-backup.ts b/server/src/nest/backup/sqlite-database-backup.ts index ec10b1107..7e8c6d68c 100644 --- a/server/src/nest/backup/sqlite-database-backup.ts +++ b/server/src/nest/backup/sqlite-database-backup.ts @@ -1,4 +1,4 @@ -import { MaintenanceRepository } from '../../db/repositories/MaintenanceRepository'; +import { SqliteMaintenance } from '../../db/engine/sqlite/sqlite-maintenance'; import { logInfo, logWarn } from '../audit/audit-log.logger'; import { DatabaseConnectionLostError, @@ -35,7 +35,7 @@ export class SqliteDatabaseBackup implements DatabaseBackupStrategy { constructor( private readonly lifecycle: DatabaseLifecycle, - private readonly maintenance: MaintenanceRepository, + private readonly maintenance: SqliteMaintenance, ) {} location(): string { diff --git a/server/src/nest/database/maintenance.module.ts b/server/src/nest/database/maintenance.module.ts index 57bb8bbf3..d24d2e7aa 100644 --- a/server/src/nest/database/maintenance.module.ts +++ b/server/src/nest/database/maintenance.module.ts @@ -1,14 +1,16 @@ +import { SqliteMaintenance } from '../../db/engine/sqlite/sqlite-maintenance'; import { MaintenanceRepository } from '../../db/repositories/MaintenanceRepository'; import { EntityManager } from '@mikro-orm/core'; import { Module } from '@nestjs/common'; /** * Provides `MaintenanceRepository`, the home of the few whole-database - * statements no entity can express (the readiness ping, the WAL checkpoint, - * `VACUUM INTO`, the plugin-table erasure). + * statements no entity can express (the readiness ping, the plugin-table + * erasure), and `SqliteMaintenance`, the SQLite file's own (the WAL + * checkpoint, `VACUUM INTO`). * - * A factory over the context-resolving global `EntityManager`, the same one - * Nest injects into every service, so the repository resolves the request or + * Factories over the context-resolving global `EntityManager`, the same one + * Nest injects into every service, so each resolves the request or * transaction fork that is active when a method runs. The class itself stays * free of Nest, like every other file under `db/`. */ @@ -19,7 +21,12 @@ import { Module } from '@nestjs/common'; useFactory: (em: EntityManager) => new MaintenanceRepository(em), inject: [EntityManager], }, + { + provide: SqliteMaintenance, + useFactory: (em: EntityManager) => new SqliteMaintenance(em), + inject: [EntityManager], + }, ], - exports: [MaintenanceRepository], + exports: [MaintenanceRepository, SqliteMaintenance], }) export class MaintenanceModule {} diff --git a/server/src/nest/places/places.service.ts b/server/src/nest/places/places.service.ts index e0673b6ac..f3aab83b8 100644 --- a/server/src/nest/places/places.service.ts +++ b/server/src/nest/places/places.service.ts @@ -251,12 +251,10 @@ export class PlacesService { // the coercion stays here (the service), the repository binds the // already-built pattern verbatim (PL3). const searchPattern = filters.search ? `%${escapeLikePattern(filters.search)}%` : undefined; - const places: PlaceListRow[] = await this.placesRepo.listForTrip(tripId, { - searchPattern, - category: filters.category, - tag: filters.tag, - assignment: filters.assignment, - }); + // A category or tag that is no canonical id (toRowId) matches nothing; bound as text, Postgres would refuse it. + const [category, tag] = [filters.category, filters.tag].map((id) => (id ? toRowId(id) : undefined)); + if (category === null || tag === null) return []; + const places: PlaceListRow[] = await this.placesRepo.listForTrip(tripId, { searchPattern, category, tag, assignment: filters.assignment }); const placeIds = places.map(p => p.id); const tagsByPlaceId = await this.queryHelpers.loadTagsByPlaceIds(placeIds); diff --git a/server/tests/integration/places.test.ts b/server/tests/integration/places.test.ts index 4ed094955..6fde8aece 100644 --- a/server/tests/integration/places.test.ts +++ b/server/tests/integration/places.test.ts @@ -468,6 +468,23 @@ describe('Search places', () => { expect(res.body.places).toHaveLength(1); expect(res.body.places[0].name).toBe('Scenic Place'); }); + + it('PLACE-017c — GET ?category= or ?tag= with an id that is not a number matches nothing', async () => { + const { user } = createUser(testDb); + const trip = createTrip(testDb, user.id); + createPlace(testDb, trip.id, { name: 'Somewhere' }); + + const queries = ['category=abc', 'tag=abc', 'category=1.5', 'tag=-1', 'category=0', 'tag=0']; + const responses = await Promise.all( + queries.map((query) => + request(app).get(`/api/trips/${trip.id}/places?${query}`).set('Cookie', authCookie(user.id)), + ), + ); + for (const [i, res] of responses.entries()) { + expect(res.status, queries[i]).toBe(200); + expect(res.body.places, queries[i]).toEqual([]); + } + }); }); // ───────────────────────────────────────────────────────────────────────────── diff --git a/server/tests/unit/auto-backup.test.ts b/server/tests/unit/auto-backup.test.ts index 3da9d6374..70861b687 100644 --- a/server/tests/unit/auto-backup.test.ts +++ b/server/tests/unit/auto-backup.test.ts @@ -8,7 +8,7 @@ * on another install. The run now goes through backupService.createBackup(), * which snapshots the DB with VACUUM INTO and bundles the key. */ -import type { MaintenanceRepository } from '../../src/db/repositories/MaintenanceRepository'; +import type { SqliteMaintenance } from '../../src/db/engine/sqlite/sqlite-maintenance'; import { AutoBackupJob } from '../../src/nest/backup/auto-backup.job'; import { createBackup } from '../../src/nest/backup/backup.impl'; import type { BackupService } from '../../src/nest/backup/backup.service'; @@ -78,7 +78,7 @@ const liveDb = path.join(__dirname, '../../data', 'travel.db'); /** The backup port BackupService injects: the real SQLite one over stubs. */ const database = new SqliteDatabaseBackup( { file: liveDb, close: vi.fn(), reopen: vi.fn() } as unknown as DatabaseLifecycle, - maintenanceStub as unknown as MaintenanceRepository, + maintenanceStub as unknown as SqliteMaintenance, ); // createBackup receives StorageService and the database port as parameters diff --git a/server/tests/unit/db/engine/sqlite/sqlite-maintenance.test.ts b/server/tests/unit/db/engine/sqlite/sqlite-maintenance.test.ts new file mode 100644 index 000000000..6374f0188 --- /dev/null +++ b/server/tests/unit/db/engine/sqlite/sqlite-maintenance.test.ts @@ -0,0 +1,109 @@ +import { SqliteMaintenance } from '../../../../../src/db/engine/sqlite/sqlite-maintenance'; +import { createSnapshotTestDb } from '../../../../helpers/db-mock'; +import { createTestOrm, type TestOrm } from '../../../../helpers/test-orm'; + +import Database from 'better-sqlite3'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from 'vitest'; + +const testDb = createSnapshotTestDb(); +let t: TestOrm; +let maintenance: SqliteMaintenance; + +beforeAll(async () => { + t = await createTestOrm(testDb); + maintenance = new SqliteMaintenance(t.em); +}); +afterAll(async () => { + await t.close(); + testDb.close(); +}); + +/** + * The first SQL text handed to the connection after the spy is set (the ORM + * may read its own metadata on the same connection afterwards). + */ +function spyOnExecute(): { first: () => string; restore: () => void } { + const connection = t.em.getConnection(); + const spy = vi.spyOn(connection, 'execute'); + return { + first: () => String(spy.mock.calls[0]?.[0]), + restore: () => spy.mockRestore(), + }; +} + +/** Whether the SQLite file at `file` holds a table named `table`. */ +function holdsTable(file: string, table: string): boolean { + const snap = new Database(file, { readonly: true }); + try { + return (snap.pragma(`table_info(${table})`) as unknown[]).length > 0; + } finally { + snap.close(); + } +} + +describe('SqliteMaintenance', () => { + it('SQLMAINT-001: walCheckpoint sends PRAGMA wal_checkpoint(TRUNCATE)', async () => { + const execute = spyOnExecute(); + try { + await expect(maintenance.walCheckpoint()).resolves.toBeUndefined(); + expect(execute.first()).toBe('PRAGMA wal_checkpoint(TRUNCATE)'); + } finally { + execute.restore(); + } + }); + + describe('vacuumInto', () => { + let scratchDir: string; + + afterEach(() => { + if (scratchDir) fs.rmSync(scratchDir, { recursive: true, force: true }); + }); + + it("SQLMAINT-002: sends `VACUUM INTO ''` and leaves a readable copy at the path", async () => { + scratchDir = fs.mkdtempSync(path.join(os.tmpdir(), 'sqlite-maint-')); + const target = path.join(scratchDir, 'snap.db'); + const execute = spyOnExecute(); + try { + await maintenance.vacuumInto(target); + expect(execute.first()).toBe(`VACUUM INTO '${target}'`); + } finally { + execute.restore(); + } + expect(holdsTable(target, 'users')).toBe(true); + }); + + it('SQLMAINT-003: doubles single quotes in the path', async () => { + scratchDir = fs.mkdtempSync(path.join(os.tmpdir(), "sqlite-maint-o'brien-")); + const target = path.join(scratchDir, 'snap.db'); + const execute = spyOnExecute(); + try { + await maintenance.vacuumInto(target); + expect(execute.first()).toBe(`VACUUM INTO '${target.replaceAll("'", "''")}'`); + } finally { + execute.restore(); + } + expect(holdsTable(target, 'users')).toBe(true); + }); + }); + + it('SQLMAINT-004: both methods throw outside a request context (fail closed, as TrekRepository)', async () => { + const { MikroORM } = await import('@mikro-orm/core'); + const fresh = await MikroORM.init({ + entities: [], + driver: (await import('../../../../../src/db/orm-driver')).createBoundSqliteDriver(() => testDb), + dbName: ':memory:', + allowGlobalContext: false, + discovery: { warnWhenNoEntities: false }, + }); + try { + const unwrapped = new SqliteMaintenance(fresh.em); + await expect(unwrapped.walCheckpoint()).rejects.toThrow(); + await expect(unwrapped.vacuumInto(path.join(os.tmpdir(), 'never.db'))).rejects.toThrow(); + } finally { + await fresh.close(false); + } + }); +}); diff --git a/server/tests/unit/db/repositories/MaintenanceRepository.test.ts b/server/tests/unit/db/repositories/MaintenanceRepository.test.ts index 15fb2c0aa..a10ca59c0 100644 --- a/server/tests/unit/db/repositories/MaintenanceRepository.test.ts +++ b/server/tests/unit/db/repositories/MaintenanceRepository.test.ts @@ -2,11 +2,7 @@ import { MaintenanceRepository } from '../../../../src/db/repositories/Maintenan import { createSnapshotTestDb } from '../../../helpers/db-mock'; import { createTestOrm, type TestOrm } from '../../../helpers/test-orm'; -import Database from 'better-sqlite3'; -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; -import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest'; +import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest'; const testDb = createSnapshotTestDb(); let t: TestOrm; @@ -22,61 +18,18 @@ afterAll(async () => { }); describe('MaintenanceRepository', () => { - it('MAINTREPO-001: walCheckpoint renders PRAGMA wal_checkpoint(TRUNCATE) — pinned text, equivalent to the legacy raw call', async () => { - // Rendered-SQL-equivalence: the exact text a direct statement on the - // handle executes without throwing, string-for-string what the legacy - // `db.exec('PRAGMA wal_checkpoint(TRUNCATE)')` issued. - const RENDERED = 'PRAGMA wal_checkpoint(TRUNCATE)'; - // test-sql-allow: the pragma text itself is under test, run as the raw statement it renders to. - expect(() => testDb.prepare(RENDERED).run()).not.toThrow(); - await expect(maintenance.walCheckpoint()).resolves.toBeUndefined(); + it('MAINTREPO-001: ping sends SELECT 1 and resolves while the database answers', async () => { + const spy = vi.spyOn(t.em.getConnection(), 'execute'); + try { + await expect(maintenance.ping()).resolves.toBeUndefined(); + // The first statement sent; the ORM may read its own metadata afterwards. + expect(spy.mock.calls[0]?.[0]).toBe('SELECT 1'); + } finally { + spy.mockRestore(); + } }); - describe('vacuumInto', () => { - let scratchPath: string; - - afterEach(() => { - if (scratchPath && fs.existsSync(scratchPath)) fs.rmSync(scratchPath, { force: true }); - }); - - it("MAINTREPO-002: vacuumInto renders `VACUUM INTO ''` — pinned text, produces a readable snapshot at the given path", async () => { - scratchPath = path.join(os.tmpdir(), `maintrepo-vacuum-${Date.now()}-${Math.random().toString(36).slice(2)}.db`); - await maintenance.vacuumInto(scratchPath); - expect(fs.existsSync(scratchPath)).toBe(true); - - const snap = new Database(scratchPath, { readonly: true }); - try { - // test-sql-allow: a snapshot file no ORM is bound to, read through sqlite_master. - const row = snap.prepare('SELECT name FROM sqlite_master WHERE type = ? AND name = ?').get('table', 'users'); - expect(row).toBeTruthy(); - } finally { - snap.close(); - } - }); - - it('MAINTREPO-003: single quotes in the path are escaped by doubling, matching the legacy `dbSnap.replaceAll("\'", "\'\'")` shape', async () => { - const dirWithQuote = path.join(os.tmpdir(), `maintrepo-o'brien-${Date.now()}`); - fs.mkdirSync(dirWithQuote, { recursive: true }); - scratchPath = path.join(dirWithQuote, 'snap.db'); - try { - await maintenance.vacuumInto(scratchPath); - expect(fs.existsSync(scratchPath)).toBe(true); - - // Rendered-SQL-equivalence: the SAME escaped text, run directly - // against another target path, succeeds identically. - const directPath = path.join(dirWithQuote, 'snap-direct.db'); - const escaped = directPath.replaceAll("'", "''"); - // test-sql-allow: the VACUUM INTO text itself is under test, run as the raw statement it renders to. - expect(() => testDb.prepare(`VACUUM INTO '${escaped}'`).run()).not.toThrow(); - expect(fs.existsSync(directPath)).toBe(true); - fs.rmSync(directPath, { force: true }); - } finally { - fs.rmSync(dirWithQuote, { recursive: true, force: true }); - } - }); - }); - - it('MAINTREPO-004: both methods throw outside a request context (fail-closed, matching TrekRepository)', async () => { + it('MAINTREPO-004: ping throws outside a request context (fail-closed, matching TrekRepository)', async () => { const { MikroORM } = await import('@mikro-orm/core'); const fresh = await MikroORM.init({ entities: [], @@ -87,7 +40,7 @@ describe('MaintenanceRepository', () => { }); try { const unwrapped = new MaintenanceRepository(fresh.em); - await expect(unwrapped.walCheckpoint()).rejects.toThrow(); + await expect(unwrapped.ping()).rejects.toThrow(); } finally { await fresh.close(false); } diff --git a/server/tests/unit/db/repositories/Places.repository.test.ts b/server/tests/unit/db/repositories/Places.repository.test.ts index e75f963bb..363e1f964 100644 --- a/server/tests/unit/db/repositories/Places.repository.test.ts +++ b/server/tests/unit/db/repositories/Places.repository.test.ts @@ -640,7 +640,7 @@ describe('PlacesRepository.listForTrip (PL3) — filter fragments', () => { const inCat = createPlace(testDb, trip.id, { category_id: cat.id }); const outCat = createPlace(testDb, trip.id); await deleteRows(t.orm, Categories, { id: { $ne: cat.id } }); - const rows = await places.listForTrip(String(trip.id), { category: String(cat.id) }); + const rows = await places.listForTrip(String(trip.id), { category: cat.id }); expect(rows.map((r) => r.id)).toEqual([inCat.id]); void outCat; }); @@ -652,7 +652,7 @@ describe('PlacesRepository.listForTrip (PL3) — filter fragments', () => { const tagged = createPlace(testDb, trip.id, { name: 'Tagged' }); const untagged = createPlace(testDb, trip.id, { name: 'Untagged' }); await tagPlace(t.orm, tagged.id, [tag.id]); - const rows = await places.listForTrip(String(trip.id), { tag: String(tag.id) }); + const rows = await places.listForTrip(String(trip.id), { tag: tag.id }); expect(rows.map((r) => r.id)).toEqual([tagged.id]); void untagged; }); @@ -684,7 +684,7 @@ describe('PlacesRepository.listForTrip (PL3) — filter fragments', () => { await tagPlace(t.orm, other.id, [tag.id]); // `other` is left unassigned, so `assignment: 'assigned'` alone should exclude it. const rows = await places.listForTrip(String(trip.id), { - searchPattern: '%Central Park%', category: String(cat.id), tag: String(tag.id), assignment: 'assigned', + searchPattern: '%Central Park%', category: cat.id, tag: tag.id, assignment: 'assigned', }); expect(rows.map((r) => r.id)).toEqual([winner.id]); }); @@ -703,7 +703,7 @@ describe('PlacesRepository.listForTrip (PL3) — filter fragments', () => { describe('PlacesRepository.listForTrip (PL3) — toEqual(legacy) over all 32 filter combinations', () => { function legacyListForTrip( tripId: string, - filters: { searchPattern?: string; category?: string; tag?: string; assignment?: 'all' | 'unassigned' | 'assigned' }, + filters: { searchPattern?: string; category?: number | string; tag?: number | string; assignment?: 'all' | 'unassigned' | 'assigned' }, ): unknown[] { let query = ` SELECT DISTINCT p.*, t.place_id AS tour_place_id, c.name as category_name, c.color as category_color, c.icon as category_icon, @@ -767,8 +767,8 @@ describe('PlacesRepository.listForTrip (PL3) — toEqual(legacy) over all 32 fil createPlace(testDb, otherTrip.id, { name: 'Central Perk' }); const searchOptions: (string | undefined)[] = [undefined, '%Central%']; - const categoryOptions: (string | undefined)[] = [undefined, String(cat.id)]; - const tagOptions: (string | undefined)[] = [undefined, String(tag.id)]; + const categoryOptions: (number | undefined)[] = [undefined, cat.id]; + const tagOptions: (number | undefined)[] = [undefined, tag.id]; const assignmentOptions: (('all' | 'unassigned' | 'assigned') | undefined)[] = [undefined, 'all', 'assigned', 'unassigned']; let combinations = 0; diff --git a/server/tests/unit/db/repositories/ReservationDayPositions.repository.test.ts b/server/tests/unit/db/repositories/ReservationDayPositions.repository.test.ts index c2d0e83dc..b3df54695 100644 --- a/server/tests/unit/db/repositories/ReservationDayPositions.repository.test.ts +++ b/server/tests/unit/db/repositories/ReservationDayPositions.repository.test.ts @@ -2,14 +2,15 @@ * ReservationDayPositionsRepository — parity test (Plan 3d Task 7 whole-plan * review, item 9/"missing parity tests": this repository had no standalone * test file). ONE seeded world, one `toEqual()` test for the - * read method (RS19); RS31 (`upsertScoped`, a write) already has coverage - * through `reservations.service.test.ts`'s `updatePositions` cases. + * read method (RS19); RS31 (`upsertScoped`, a write) has coverage through + * `reservations.service.test.ts`'s `updatePositions` cases and, for the + * overwrite and the trip scoping its ON CONFLICT clause holds, below. */ import { ReservationDayPositions } from '../../../../src/db/entities/ReservationDayPositions.entity'; import type { ReservationDayPositionsRepository } from '../../../../src/db/repositories/ReservationDayPositions.repository'; import { createSnapshotTestDb } from '../../../helpers/db-mock'; import { createDay, createReservation, createTrip, createUser } from '../../../helpers/factories'; -import { insertRow } from '../../../helpers/factories/rows'; +import { findRows, insertRow } from '../../../helpers/factories/rows'; import { resetTestDb } from '../../../helpers/test-db'; import { createTestOrm, type TestOrm } from '../../../helpers/test-orm'; @@ -86,3 +87,30 @@ describe('ReservationDayPositionsRepository — fully seeded world', () => { expect(await repo.listForReservation(999999)).toEqual([]); }); }); + +describe('ReservationDayPositionsRepository.upsertScoped', () => { + it('RDP-UPSERT-001: a second write to the same reservation and day overwrites its position', async () => { + const { user } = createUser(testDb); + const trip = createTrip(testDb, user.id); + const day = createDay(testDb, trip.id); + const reservation = createReservation(testDb, trip.id, { day_id: day.id }); + + await repo.upsertScoped(trip.id, reservation.id, day.id, 1.5); + await repo.upsertScoped(trip.id, reservation.id, day.id, 4); + + const rows = await findRows(t.orm, ReservationDayPositions, { reservation_id: reservation.id }); + expect(rows.map((r) => [r.day_id, r.position])).toEqual([[day.id, 4]]); + }); + + it('RDP-UPSERT-002: a reservation of another trip writes no row', async () => { + const { user } = createUser(testDb); + const trip = createTrip(testDb, user.id); + const otherTrip = createTrip(testDb, user.id); + const day = createDay(testDb, trip.id); + const foreign = createReservation(testDb, otherTrip.id); + + await repo.upsertScoped(trip.id, foreign.id, day.id, 2); + + expect(await findRows(t.orm, ReservationDayPositions, { reservation_id: foreign.id })).toEqual([]); + }); +}); diff --git a/server/tests/unit/demo/demo-reset-path.test.ts b/server/tests/unit/demo/demo-reset-path.test.ts index 62f94b047..99af90f70 100644 --- a/server/tests/unit/demo/demo-reset-path.test.ts +++ b/server/tests/unit/demo/demo-reset-path.test.ts @@ -11,7 +11,7 @@ * cases assert is which file each step names. * * The port is the real `SqliteDatabaseBackup` over a lifecycle stand-in (the - * file, close, reopen) and a MaintenanceRepository stand-in (checkpoint, + * file, close, reopen) and a SqliteMaintenance stand-in (checkpoint, * VACUUM INTO). The file system is spied rather than written to, because the * baseline path is fixed at data/travel-baseline.db and a developer running a * demo instance has a real one sitting there. @@ -21,7 +21,7 @@ * `withRequestContext` over a real (empty) test database: the reads find * nothing and the restore-writes are skipped. */ -import type { MaintenanceRepository } from '../../../src/db/repositories/MaintenanceRepository'; +import type { SqliteMaintenance } from '../../../src/db/engine/sqlite/sqlite-maintenance'; import { resetDemoUser, saveBaseline } from '../../../src/demo/demo-reset'; import { SqliteDatabaseBackup } from '../../../src/nest/backup/sqlite-database-backup'; import { DatabaseConnectionLostError } from '../../../src/nest/database/database-backup.interface'; @@ -65,7 +65,7 @@ describe('demo-reset DB path', () => { maintenance = { walCheckpoint: vi.fn(async () => {}), vacuumInto: vi.fn(async () => {}) }; database = new SqliteDatabaseBackup( lifecycle as unknown as DatabaseLifecycle, - maintenance as unknown as MaintenanceRepository, + maintenance as unknown as SqliteMaintenance, ); }); diff --git a/server/tests/unit/eslint-rules/restricted-syntax-selectors.test.ts b/server/tests/unit/eslint-rules/restricted-syntax-selectors.test.ts index 9a258b1c6..a26d99f24 100644 --- a/server/tests/unit/eslint-rules/restricted-syntax-selectors.test.ts +++ b/server/tests/unit/eslint-rules/restricted-syntax-selectors.test.ts @@ -140,7 +140,7 @@ declare const repo: { run(sql: string, params?: unknown[]): Promise; ge // text slipping in under the exemption, a lower one means the pin (and at 0 // the ignores entry) has to follow. const EXEMPT_SQL_STRINGS: Record = { - 'src/db/repositories/MaintenanceRepository.ts': 4, + 'src/db/repositories/MaintenanceRepository.ts': 2, 'src/db/seeders/AdminSeeder.ts': 2, 'src/db/seeders/CategorySeeder.ts': 2, }; diff --git a/server/tests/unit/nest/backup.impl.test.ts b/server/tests/unit/nest/backup.impl.test.ts index 1eb858602..13c8ea386 100644 --- a/server/tests/unit/nest/backup.impl.test.ts +++ b/server/tests/unit/nest/backup.impl.test.ts @@ -2,7 +2,7 @@ * Unit tests for backupService. * Covers BACKUP-031 to BACKUP-060. */ -import type { MaintenanceRepository } from '../../../src/db/repositories/MaintenanceRepository'; +import type { SqliteMaintenance } from '../../../src/db/engine/sqlite/sqlite-maintenance'; import { formatSize, parseIntField, @@ -78,9 +78,9 @@ const dbMock = vi.hoisted(() => ({ })); // The database port's SQLite implementation is built for real below with this -// stand-in for the MaintenanceRepository Nest injects into it: the WAL +// stand-in for the SqliteMaintenance Nest injects into it: the WAL // checkpoint and the VACUUM INTO snapshot are what a backup and a restore ask -// of it. The real statements are MaintenanceRepository.test.ts's coverage. +// of it. The real statements are sqlite-maintenance.test.ts's coverage. const maintenanceRepoMock = vi.hoisted(() => ({ walCheckpoint: vi.fn().mockResolvedValue(undefined), vacuumInto: vi.fn().mockResolvedValue(undefined), @@ -121,7 +121,7 @@ const lifecycleStub = { function deps(storage: StorageService) { return { storage, - database: new SqliteDatabaseBackup(lifecycleStub, maintenanceRepoMock as unknown as MaintenanceRepository), + database: new SqliteDatabaseBackup(lifecycleStub, maintenanceRepoMock as unknown as SqliteMaintenance), }; } diff --git a/server/tests/unit/nest/sqlite-database-backup.test.ts b/server/tests/unit/nest/sqlite-database-backup.test.ts index eb557b2fa..0b6ce85dc 100644 --- a/server/tests/unit/nest/sqlite-database-backup.test.ts +++ b/server/tests/unit/nest/sqlite-database-backup.test.ts @@ -2,12 +2,12 @@ * SqliteDatabaseBackup: the database backup port for a SQLite file. * * The snapshot and the swap run for real here, against a database file in a - * temp directory: a VACUUM INTO through the real MaintenanceRepository, and the + * temp directory: a VACUUM INTO through the real SqliteMaintenance, and the * copy, sidecar removal and rename of a restore. Only the connection lifecycle * is a stand-in, because closing and reopening the process's own connection is * DatabaseLifecycle's business (tests/unit/nest/database/). */ -import { MaintenanceRepository } from '../../../src/db/repositories/MaintenanceRepository'; +import { SqliteMaintenance } from '../../../src/db/engine/sqlite/sqlite-maintenance'; import { SqliteDatabaseBackup } from '../../../src/nest/backup/sqlite-database-backup'; import { DatabaseConnectionLostError } from '../../../src/nest/database/database-backup.interface'; import type { DatabaseLifecycle } from '../../../src/nest/database/database-lifecycle.service'; @@ -40,7 +40,7 @@ function maintenanceStub() { function portWith(maintenance: { walCheckpoint: unknown; vacuumInto: unknown }): SqliteDatabaseBackup { return new SqliteDatabaseBackup( lifecycle as unknown as DatabaseLifecycle, - maintenance as unknown as MaintenanceRepository, + maintenance as unknown as SqliteMaintenance, ); } @@ -116,7 +116,7 @@ describe('SqliteDatabaseBackup', () => { // test-sql-allow: the marker table is a bare fixture no entity maps; the write has to land in the WAL of this file. live.prepare('UPDATE marker SET label = ?').run('after'); const target = path.join(dir, 'copy.db'); - const port = portWith(new MaintenanceRepository(orm.em)); + const port = portWith(new SqliteMaintenance(orm.em)); await withRequestContext(orm.orm, () => port.snapshot(target)); @@ -125,7 +125,7 @@ describe('SqliteDatabaseBackup', () => { it('SQLBK-005: a VACUUM INTO that fails rejects and leaves nothing at the target', async () => { const target = path.join(dir, 'missing-dir', 'copy.db'); - const port = portWith(new MaintenanceRepository(orm.em)); + const port = portWith(new SqliteMaintenance(orm.em)); await expect(withRequestContext(orm.orm, () => port.snapshot(target))).rejects.toThrow(); diff --git a/server/tests/unit/scripts/dialect-lint.test.ts b/server/tests/unit/scripts/dialect-lint.test.ts index d01520ea8..c52d49d8f 100644 --- a/server/tests/unit/scripts/dialect-lint.test.ts +++ b/server/tests/unit/scripts/dialect-lint.test.ts @@ -128,6 +128,8 @@ describe('dialect-lint.mjs', () => { it('DIALECT-006: PRAGMA is allowed in the src/db lifecycle files, not in a repository or a domain', () => { const pragma = "export const p = 'PRAGMA wal_checkpoint(TRUNCATE)';\n"; expect(run(serverRoot({ 'src/db/durability.ts': pragma })).status).toBe(0); + expect(run(serverRoot({ 'src/db/engine/sqlite/sqlite-maintenance.ts': pragma })).status).toBe(0); + expect(run(serverRoot({ 'src/db/engine/postgres/maintenance.ts': pragma })).status).toBe(1); expect(run(serverRoot({ [REPO]: pragma })).status).toBe(1); expect(run(serverRoot({ 'src/nest/backup/probe.ts': pragma })).status).toBe(1); }); diff --git a/server/tests/unit/scripts/pg-probe/arg-samples.test.ts b/server/tests/unit/scripts/pg-probe/arg-samples.test.ts index 672c207a9..dfcecab5e 100644 --- a/server/tests/unit/scripts/pg-probe/arg-samples.test.ts +++ b/server/tests/unit/scripts/pg-probe/arg-samples.test.ts @@ -47,6 +47,7 @@ export class ThingsRepository { async record(r: Record): Promise {} async literals(n: bigint, flag: true): Promise {} async destructured({ id, label }: { id: number; label: string }): Promise {} + async presence(write: { name?: readonly [present: boolean, value: string]; flag?: readonly [boolean, number] }): Promise {} } export abstract class AbstractRepository { async skipped(): Promise {} @@ -102,6 +103,7 @@ describe('pg-probe argument samples', () => { 'literals', 'nullable', 'optionalCallback', + 'presence', 'record', 'rest', 'tuple', @@ -145,6 +147,13 @@ describe('pg-probe argument samples', () => { expect(argsOf('collections')).toEqual([new Set([1]), new Map([['1', 1]])]); }); + it('PGPROBE-037: a presence pair is present, so a patch built from it reaches its SQL', () => { + // The repositories' partial updates take `[present: boolean, value]` pairs + // and send nothing when no pair is present; a false flag would leave the + // method unmeasured. Only the element labelled `present` turns true. + expect(argsOf('presence')).toEqual([{ name: [true, 'probe'], flag: [false, 1] }]); + }); + it('PGPROBE-035: a required callback makes the method unprobeable, an optional one is left off', () => { expect(planOf('withCallback')).toMatchObject({ unprobeable: 'fn: a function parameter' }); expect(argsOf('optionalCallback')).toEqual([1]); diff --git a/server/tests/unit/scripts/pg-probe/baseline.test.ts b/server/tests/unit/scripts/pg-probe/baseline.test.ts index 99db25f11..8dc572ff4 100644 --- a/server/tests/unit/scripts/pg-probe/baseline.test.ts +++ b/server/tests/unit/scripts/pg-probe/baseline.test.ts @@ -130,11 +130,13 @@ describe('pg-probe baseline', () => { }); }); - it('PGPROBE-010: only syntax and feature errors count toward the ratchet, not data errors from the sample values', () => { - for (const code of ['42601', '42883', '42804', '42P01', '42703', '0A000']) + it('PGPROBE-010: syntax, feature and data errors count toward the ratchet; constraints and timeouts do not', () => { + // A data exception (class 22) is a value of the wrong type reaching a + // column: a string the SQLite comparison let through where Postgres + // refuses it, so it counts once the sampler sends the declared types. + for (const code of ['42601', '42883', '42804', '42P01', '42703', '0A000', '22P02', '22007', '22008']) expect(countsTowardRatchet(code)).toBe(true); - for (const code of ['22P02', '22007', '22008', '23505', '57014', 'NOSQLSTATE']) - expect(countsTowardRatchet(code)).toBe(false); + for (const code of ['23505', '23514', '57014', 'NOSQLSTATE']) expect(countsTowardRatchet(code)).toBe(false); }); it('PGPROBE-008: the checked-in baseline parses and formats back to itself', () => { diff --git a/server/tests/unit/scripts/pg-probe/report.test.ts b/server/tests/unit/scripts/pg-probe/report.test.ts index ab290e797..c71b99bf8 100644 --- a/server/tests/unit/scripts/pg-probe/report.test.ts +++ b/server/tests/unit/scripts/pg-probe/report.test.ts @@ -47,7 +47,7 @@ const RESULTS: MethodResult[] = [ { key: 'F.thrower', statements: [], threw: 'no trip 1' }, { key: 'G.sampled', - statements: [refused('select $1::date', '22007', 'invalid input syntax for type date: "probe"'), ok('select 5')], + statements: [refused('insert into t values ($1)', '23514', 'violates check constraint "t_check"'), ok('select 5')], }, ]; @@ -84,26 +84,30 @@ describe('pg-probe report', () => { new Map([ ['42883', 1], ['42601', 1], - ['22007', 1], + ['23514', 1], ]), ); expect(summary.failedStatements.map((f) => `${f.method} ${f.code} ${f.counted}`)).toEqual([ 'B.broken 42883 true', 'B.broken 42601 true', - 'G.sampled 22007 false', + 'G.sampled 23514 false', ]); }); - it('PGPROBE-068: a data error from a sample value is reported but not held to the baseline', () => { + it('PGPROBE-068: a data error is held to the baseline, a constraint over a sample value is only reported', () => { const summary = summarize([ { key: 'G.sampled', - statements: [refused('select $1::date', '22P02', 'bad'), refused('select x', '0A000', 'nope')], + statements: [ + refused('select $1::date', '22P02', 'bad'), + refused('select x', '0A000', 'nope'), + refused('insert into t values ($1)', '23514', 'check'), + ], }, ]); - expect(summary.failingByMethod.get('G.sampled')).toBe(1); - expect(splitRefusals(summary).counted.map((f) => f.code)).toEqual(['0A000']); - expect(splitRefusals(summary).reported.map((f) => f.code)).toEqual(['22P02']); + expect(summary.failingByMethod.get('G.sampled')).toBe(2); + expect(splitRefusals(summary).counted.map((f) => f.code)).toEqual(['22P02', '0A000']); + expect(splitRefusals(summary).reported.map((f) => f.code)).toEqual(['23514']); }); it('PGPROBE-069: names every method whose SQL the probe did not see, and why', () => { @@ -187,9 +191,9 @@ describe('pg-probe report', () => { expect(text).toContain('FAIL [postgres] dateOf: expected 1, got 2'); expect(text).toContain('collation "NOCASE" does not exist'); expect(text).toContain( - 'Statements: 8 distinct, 3 refused by Postgres (2 as SQL, held to the baseline; 1 over a sample value', + "Statements: 8 distinct, 3 refused by Postgres (2 as SQL or over a value's type, held to the baseline; 1 over a constraint", ); - expect(text).toContain(' 22007: 1 (not held)'); + expect(text).toContain(' 23514: 1 (not held)'); expect(text).toContain(' F.thrower: threw before any statement: no trip 1'); expect(text).toContain( 'FAIL F.thrower threw before any statement: no trip 1, and the baseline does not list it as uncovered.', @@ -217,9 +221,9 @@ describe('pg-probe report', () => { expect(md).toContain("| postgres | a\\|b | got 'x' |"); expect(md).toContain('| 42883 | 1 |'); expect(md).toContain('| E.skipped | could not be called: fn: a function parameter |'); - expect(md).toContain('| Refused by Postgres as SQL (held) | 2 |'); - expect(md).toContain('| Refused over a sample value or the empty tables (reported only) | 1 |'); - expect(md).toContain('| G.sampled | 22007 | no | invalid input syntax for type date: "probe" |'); + expect(md).toContain("| Refused by Postgres as SQL or over a value's type (held) | 2 |"); + expect(md).toContain('| Refused over a constraint, a timeout or the empty tables (reported only) | 1 |'); + expect(md).toContain('| G.sampled | 23514 | no | violates check constraint "t_check" |'); const ratchet = formatMarkdown( input({ verdict: { ...CLEAN, newlyUncovered: ['C.silent'], nowCovered: ['Z.gone'] } }), );