refactor(db): open the database through a DatabaseLifecycle provider

db/database.ts opened the connection as a side effect of being imported,
so the moment anything reached it the database was open. index.ts had to
defer loading the app until a first-boot restore was done, and nothing
in the container decided when the connection opened or closed.

The module now opens the connection on first use and opens nothing at
import. A connection closed on purpose (a restore, the shutdown) stays
closed until it is reopened, as before. The new DatabaseLifecycle
provider owns the decisions: buildApp() opens through it, which also
binds the ORM to later swaps (the hook attachOrm used to register) and
runs the schema bootstrap in the same order as before, and the shutdown
closes through it. Its file property is the one resolved database path.

The module keeps every export, so the test suites that replace it with
vi.mock keep working; buildDbMock gains the new openDb.
This commit is contained in:
Maurice
2026-10-08 15:21:05 +02:00
parent 5aa9ac8529
commit 0a4c6fbace
18 changed files with 456 additions and 96 deletions
+1 -1
View File
@@ -42,7 +42,7 @@ Nest owns everything. Every domain is a DI module under `src/nest/<domain>/` (`c
- **Scheduling**: `src/nest/scheduling/` (`CronRegistrarService`) is the one way to schedule a cron. Jobs are `*.job.ts` providers in their owning domain. The registrar refuses to schedule under `NODE_ENV=test` (an integration test pins it). **Never use a `@Cron` decorator** — it bypasses that gate.
- **Plugins** (`src/nest/plugins/`): the child-process runtime (`supervisor/`, `runtime/plugin-host-entry.ts`, `protocol/envelope.ts`) is **intentionally not Nest** — it is the security boundary. `host/rpc-host.ts` registers an RPC method at spawn only if the plugin holds the unlocking permission (registration = authorization). Install-time gates live in `install/`; the manifest's `trek` version range fails closed, and `TREK_PLUGINS_IGNORE_TREK_RANGE` downgrades that to a warning. The protocol tables in `protocol/envelope.ts` (+ `src/plugin-event-sink.ts`) are **generated** into `plugin-sdk/` and `shared/` — run `gen:plugin-facts` after touching them and commit the output. `runtime/egress-policy.ts` is a hand-kept mirror guarded by a parity test in `plugin-sdk/`; change both sides together.
- **Storage**: `src/nest/storage/` is the upload driver abstraction (local / S3 / mirror behind one interface, with an admin surface and a usage job). Domain services never touch the filesystem or the S3 SDK directly.
- **DB** (`src/db/`): `better-sqlite3`, WAL, FK on. `database.ts` only opens the connection and applies pragmas — it does not build a schema. MikroORM's migrations (`db/migrations/`) are the only schema source, **including for tests**: the hand-kept `schema.ts`/`migrations.ts`/`seeds.ts` builders and the parity test that guarded them are gone. Seeding is the MikroORM seeders in `db/seeders/`, applied by `db/orm.ts` from `buildApp()` before anything reads. An install the retired positional runner migrated (a `schema_version` row, no `mikro_orm_migrations` rows) is baselined by `db/legacy-baseline.ts`: the migrations whose `Legacy migration step N` docstring is ≤ its version are recorded as executed (never replayed) in the same transaction as the migrator run, while the baseline migration itself still runs first, as the legacy runner's `createTables()` did on every boot; anything it cannot map refuses the boot. Before a boot applies pending migrations to a file database that holds anything, `db/pre-migrate-snapshot.ts` writes a `VACUUM INTO` copy next to it (`pre-migrate-<schema>-<time>.db`, `<schema>` being `legacy-<schema_version>` or the last recorded migration's timestamp; the newest `TREK_DB_PRE_MIGRATE_SNAPSHOT_KEEP` stay) and the newer-database refusal names the matching copy; a copy that cannot be written refuses the boot unless `TREK_DB_PRE_MIGRATE_SNAPSHOT=false`. It is off under `NODE_ENV=test` unless a suite sets that variable to true, and a restore's re-bootstrap skips it (the archive is the copy). After the migrator, `db/email-case-index.ts` creates the case-insensitive unique index on `users.email` that its migration had to defer on an install holding two accounts whose emails differ only in case (flag `email_case_index_pending`). `db/orm-driver.ts` binds the ORM to that same connection instead of opening a second one — with `:memory:` under test, a second connection is a second, empty database. Every read and write goes through a repository in `db/repositories/` (each extends the shared `TrekRepository` base in `db/repositories/_shared/`, which validates the request context before delegating to MikroORM) and, for multi-statement writes, `UnitOfWork.transactional(...)` (`nest/database/unit-of-work.ts`, provided by the global `OrmModule`) — there is no `DatabaseService`/`DatabaseModule` to inject any more. Entrypoints outside an HTTP request — an MCP tool call, a plugin RPC dispatch, a WebSocket message, a cron job, boot-time seeding, a backup restore — open one explicitly with `withRequestContext()` (`nest/database/request-context.ts`), since only the pathless `mikroOrmRequestContext` middleware `buildApp()` mounts (with `@mikro-orm/nestjs`'s own `registerRequestContext` turned off in `app.module.ts`, UPLOADS-P16) forks an `EntityManager` per HTTP request. `NODE_ENV=test` ⇒ isolated `:memory:` DB per vitest worker, opened from a snapshot `tests/global-setup.ts` migrates once per run; `TREK_DB_FILE` ⇒ explicit file DB (Playwright harness).
- **DB** (`src/db/`): `better-sqlite3`, WAL, FK on. `database.ts` only holds the connection and applies pragmas. It does not build a schema, and importing it opens nothing (the first reader does). The connection's lifecycle belongs to the `DatabaseLifecycle` provider (`nest/database/database-lifecycle.service.ts`): `buildApp()` opens it through `open()` (which binds the ORM to later swaps and runs the schema bootstrap below), the shutdown closes it through `close()`, and `reopen()` swaps in a new handle and migrates it. `DatabaseLifecycle.file` is the one resolved path (`TREK_DB_FILE` or `data/travel.db`). MikroORM's migrations (`db/migrations/`) are the only schema source, **including for tests**: the hand-kept `schema.ts`/`migrations.ts`/`seeds.ts` builders and the parity test that guarded them are gone. Seeding is the MikroORM seeders in `db/seeders/`, applied by `db/orm.ts` from `buildApp()` before anything reads. An install the retired positional runner migrated (a `schema_version` row, no `mikro_orm_migrations` rows) is baselined by `db/legacy-baseline.ts`: the migrations whose `Legacy migration step N` docstring is ≤ its version are recorded as executed (never replayed) in the same transaction as the migrator run, while the baseline migration itself still runs first, as the legacy runner's `createTables()` did on every boot; anything it cannot map refuses the boot. Before a boot applies pending migrations to a file database that holds anything, `db/pre-migrate-snapshot.ts` writes a `VACUUM INTO` copy next to it (`pre-migrate-<schema>-<time>.db`, `<schema>` being `legacy-<schema_version>` or the last recorded migration's timestamp; the newest `TREK_DB_PRE_MIGRATE_SNAPSHOT_KEEP` stay) and the newer-database refusal names the matching copy; a copy that cannot be written refuses the boot unless `TREK_DB_PRE_MIGRATE_SNAPSHOT=false`. It is off under `NODE_ENV=test` unless a suite sets that variable to true, and a restore's re-bootstrap skips it (the archive is the copy). After the migrator, `db/email-case-index.ts` creates the case-insensitive unique index on `users.email` that its migration had to defer on an install holding two accounts whose emails differ only in case (flag `email_case_index_pending`). `db/orm-driver.ts` binds the ORM to that same connection instead of opening a second one — with `:memory:` under test, a second connection is a second, empty database. Every read and write goes through a repository in `db/repositories/` (each extends the shared `TrekRepository` base in `db/repositories/_shared/`, which validates the request context before delegating to MikroORM) and, for multi-statement writes, `UnitOfWork.transactional(...)` (`nest/database/unit-of-work.ts`, provided by the global `OrmModule`) — there is no `DatabaseService`/`DatabaseModule` to inject any more. Entrypoints outside an HTTP request — an MCP tool call, a plugin RPC dispatch, a WebSocket message, a cron job, boot-time seeding, a backup restore — open one explicitly with `withRequestContext()` (`nest/database/request-context.ts`), since only the pathless `mikroOrmRequestContext` middleware `buildApp()` mounts (with `@mikro-orm/nestjs`'s own `registerRequestContext` turned off in `app.module.ts`, UPLOADS-P16) forks an `EntityManager` per HTTP request. `NODE_ENV=test` ⇒ isolated `:memory:` DB per vitest worker, opened from a snapshot `tests/global-setup.ts` migrates once per run; `TREK_DB_FILE` ⇒ explicit file DB (Playwright harness).
## Rules for new code
+1 -1
View File
@@ -4,7 +4,7 @@
* index.ts imports this immediately after `import 'dotenv/config'` — as a
* side-effect import (not a function call) so validation runs BEFORE the rest
* of the import graph executes its module-load side effects (config.ts key
* resolution, db/database.ts initDb, ...). A plain call in index.ts's body
* resolution, ...). A plain call in index.ts's body
* would run after all of those, because imports hoist.
*/
import { validateEnvAtBoot } from './env';
+15 -7
View File
@@ -1,4 +1,3 @@
import { attachOrm, runSchemaBootstrap } from './db/orm';
import { applyGlobalMiddleware, routingCspOrigins } from './middleware/globalMiddleware';
import { readEnv } from './app-config';
import { nestLogLevels } from './app-config/nest-log-levels';
@@ -16,6 +15,7 @@ import { SettingsService } from './nest/settings/settings.service';
import { StorageService } from './nest/storage/storage.service';
import { MikroORM } from '@mikro-orm/core';
import { withRequestContext } from './nest/database/request-context';
import { DatabaseLifecycle } from './nest/database/database-lifecycle.service';
import type { INestApplication } from '@nestjs/common';
import type { ConfigType } from '@nestjs/config';
import { NestFactory } from '@nestjs/core';
@@ -60,6 +60,13 @@ import nodeHttp from 'node:http';
*/
let boundHttpServer: nodeHttp.Server | null = null;
/**
* The provider that owns the database connection, for index.ts to close it on
* shutdown. Re-exported so index.ts reaches it through the one module it loads
* after a first-boot restore.
*/
export { DatabaseLifecycle };
/**
* The http server buildApp created and bound the ws gateway to.
*
@@ -82,13 +89,14 @@ export async function buildApp(): Promise<INestApplication> {
rawBody: true,
logger: nestLogLevels(readEnv().app.logLevel),
});
// Schema first, before ANY consumer reads it. `database.ts` only opens the
// connection now; migrating and seeding is MikroORM's job and it is async, so
// this is the earliest point it can happen. It has to stay above the
// SettingsService resolution below, which is the boot's first DB read.
// Schema first, before ANY consumer reads it. The connection lifecycle is the
// DatabaseLifecycle provider's: it opens the connection (the ORM's first
// connect usually already has), binds the ORM to later swaps, and runs the
// legacy baseline, the migrations and the seeders. That is async, so this is
// the earliest point it can happen. It has to stay above the SettingsService
// resolution below, which is the boot's first DB read.
const orm = app.get(MikroORM);
attachOrm(orm);
await runSchemaBootstrap(orm);
await app.get(DatabaseLifecycle).open();
const instance = app.getHttpAdapter().getInstance();
// The http server is created HERE, not by the caller after buildApp returns,
// and that ordering is the whole point: Nest binds gateways during app.init(),
+53 -40
View File
@@ -6,49 +6,57 @@ import { readSchemaSnapshot } from './schema-snapshot';
import type Database from 'better-sqlite3';
// In test mode each vitest worker gets an isolated in-memory DB so that
// parallel forks can't race on the same file or share migration state.
const isTest = readEnv().app.isTest;
const dbPath = resolveDbPath();
let _db: Database.Database | null = null;
/**
* Set by `closeDb()`, cleared by `openDb()`. A handle that was never opened is
* opened on first use; one that was closed on purpose (a restore swapping the
* file, the shutdown) stays closed until somebody reopens it, so nothing can
* quietly reconnect to a file that is being replaced.
*/
let closedOnPurpose = false;
/**
* Opens the connection and applies the pragmas. It deliberately does NOT build
* the schema: migrations and seeding are MikroORM's job now and run from
* `db/orm.ts` during `buildApp()`, because the migrator is async and this
* function is called at module load.
* the schema: migrations and seeding are MikroORM's job and run from
* `db/orm.ts` when `DatabaseLifecycle.open()` (`nest/database/`) is called by
* `buildApp()`, because the migrator is async.
*
* The one exception is test mode, where there is no `buildApp()` to hook into
* for the handful of suites that use this singleton directly. Those open a copy
* of a schema snapshot the vitest global setup migrated once for the whole run.
* Nothing calls this at import any more. The first reader opens it: in
* production that is the ORM's first connect inside `NestFactory.create()`,
* which is still after `index.ts` ran a first-boot restore. Opening an open
* connection is a no-op.
*
* In test mode there is no `buildApp()` for the handful of suites that use this
* singleton directly, so those open a copy of a schema snapshot the vitest
* global setup migrated once for the whole run.
*/
function initDb(): void {
if (_db) {
try {
_db.exec('PRAGMA wal_checkpoint(TRUNCATE)');
} catch (e) {}
try {
_db.close();
} catch (e) {}
_db = null;
}
const snapshot = isTest ? readSchemaSnapshot() : null;
_db = openDatabase(snapshot ?? dbPath);
function openDb(): void {
if (_db) return;
// Resolved here rather than at import, so loading the module touches neither
// the environment nor the data directory.
const dbPath = resolveDbPath();
const snapshot = readEnv().app.isTest ? readSchemaSnapshot() : null;
const handle = openDatabase(snapshot ?? dbPath);
// Ahead of the journal switch now: changing journal_mode needs an exclusive
// lock, which a sibling process (reset-admin, the rotation script) may hold.
_db.exec('PRAGMA busy_timeout = 5000');
const durability = applyDurabilityPragmas(_db);
_db.exec('PRAGMA foreign_keys = ON');
// Reported so an operator can see whether their setting took — the test DB is
handle.exec('PRAGMA busy_timeout = 5000');
const durability = applyDurabilityPragmas(handle);
handle.exec('PRAGMA foreign_keys = ON');
_db = handle;
closedOnPurpose = false;
// Reported so an operator can see whether their setting took. The test DB is
// :memory: and has no journal file, so there is nothing to report there.
if (dbPath !== ':memory:') {
console.log(`[DB] journal_mode=${durability.journalMode}, synchronous=${durability.synchronous}`);
}
}
initDb();
/** The open handle, opening it on first use. Throws while it is closed on purpose. */
function current(): Database.Database {
if (!_db && !closedOnPurpose) openDb();
if (!_db) throw new Error('Database connection is not available (restore in progress?)');
return _db;
}
/**
* The live better-sqlite3 handle, for the MikroORM dialect in `orm-driver.ts`.
@@ -58,8 +66,7 @@ initDb();
* The driver calls this on every connect so it always binds the current handle.
*/
function getRawConnection(): Database.Database {
if (!_db) throw new Error('Database connection is not available (restore in progress?)');
return _db;
return current();
}
/**
@@ -80,12 +87,12 @@ function getRawConnection(): Database.Database {
*/
const db = new Proxy({} as Database.Database, {
get(_, prop: string | symbol) {
if (!_db) throw new Error('Database connection is not available (restore in progress?)');
const val = (_db as unknown as Record<string | symbol, unknown>)[prop];
return typeof val === 'function' ? val.bind(_db) : val;
const handle = current();
const val = (handle as unknown as Record<string | symbol, unknown>)[prop];
return typeof val === 'function' ? val.bind(handle) : val;
},
set(_, prop: string | symbol, val: unknown) {
(_db as unknown as Record<string | symbol, unknown>)[prop] = val;
(current() as unknown as Record<string | symbol, unknown>)[prop] = val;
return true;
},
});
@@ -119,8 +126,9 @@ async function runDemoSeed(): Promise<void> {
}
/**
* Lets `db/orm.ts` rebuild the ORM's connection when this module swaps handles,
* without this module importing the ORM (which imports this module back).
* Lets `DatabaseLifecycle` (`nest/database/`) rebuild the ORM's connection when
* this module swaps handles, without this module importing the ORM (which
* imports this module back).
*/
let onReinitialize: (() => Promise<void>) | null = null;
function registerReinitializeHook(hook: () => Promise<void>): void {
@@ -136,6 +144,7 @@ function closeDb(): void {
_db.close();
} catch (e) {}
_db = null;
closedOnPurpose = true;
console.log('[DB] Database connection closed');
}
}
@@ -144,13 +153,17 @@ function closeDb(): void {
* Reopens the connection after a restore has swapped the file underneath us.
*
* Async now, because bringing a restored backup up to date means running the
* migrations, and the hook also has to rebuild the ORM's Kysely client — it
* migrations, and the hook also has to rebuild the ORM's Kysely client: it
* cached the handle this function is about to replace.
*
* Production code reaches this through `DatabaseLifecycle.reopen()`; the export
* stays for the suites that drive the swap directly and for the mock factories
* that stub it.
*/
async function reinitialize(): Promise<void> {
console.log('[DB] Reinitializing database connection after restore...');
if (_db) closeDb();
initDb();
openDb();
if (onReinitialize) await onReinitialize();
console.log('[DB] Database reinitialized successfully');
}
@@ -167,4 +180,4 @@ async function reinitialize(): Promise<void> {
// exported — see its own docstring above for why (~190 test files' static
// typecheck, not a production `src/` need).
export { db, closeDb, reinitialize, getRawConnection, registerReinitializeHook, runDemoSeed };
export { db, openDb, closeDb, reinitialize, getRawConnection, registerReinitializeHook, runDemoSeed };
+4 -24
View File
@@ -1,5 +1,5 @@
import mikroOrmConfig from '../mikro-orm.config';
import { registerReinitializeHook, runDemoSeed } from './database';
import { runDemoSeed } from './database';
import { ensureEmailCaseIndex } from './email-case-index';
import { migrateToHead, NO_SAFETY_NET } from './legacy-baseline';
import { preMigrateSnapshot } from './pre-migrate-snapshot';
@@ -25,7 +25,7 @@ import { MikroORM as SqliteMikroORM } from '@mikro-orm/sqlite';
* class's own `readonly` form while a direct `init()` yields the mutable one;
* nothing here reads that parameter.
*/
type AnyOrm = MikroORM<
export type AnyOrm = MikroORM<
IDatabaseDriver,
EntityManager<IDatabaseDriver>,
readonly (string | EntityClass<AnyEntity> | EntitySchema)[]
@@ -57,8 +57,8 @@ export async function runSchemaBootstrap(orm: AnyOrm, { snapshot = true }: { sna
//
// Plan 3c Task 0b (R9): wrapping the call HERE, the one place
// `runSchemaBootstrap` already has `orm` in scope, covers BOTH of
// `runDemoSeed`'s real call sites at once (`bootstrap.ts`'s initial boot
// and `attachOrm`'s restore hook below, both of which only ever reach
// `runDemoSeed`'s real call sites at once (`DatabaseLifecycle.open()` at
// boot and its restore hook in `reopen()`, both of which only ever reach
// `runDemoSeed` through this function) with no second wrapper to keep in
// sync.
//
@@ -72,26 +72,6 @@ export async function runSchemaBootstrap(orm: AnyOrm, { snapshot = true }: { sna
await withRequestContext(orm, () => runDemoSeed());
}
/**
* Binds the ORM to this process's connection lifecycle.
*
* A restore swaps the SQLite file and reopens the raw handle; Kysely caches
* whatever handle it was given, so without this the ORM would keep talking to a
* closed connection. Closing and reconnecting sends the driver back through
* `createKyselyDialect()`, which picks up the new handle — and the restored file
* may be an older backup, so it gets migrated forward too.
*/
export function attachOrm(orm: AnyOrm): void {
registerReinitializeHook(async () => {
const connection = orm.em.getConnection();
// The handle is owned and closed by `database.ts`; the bound driver's
// `destroy()` is a no-op (see `orm-driver.ts`), so this close never reaches it.
await connection.close(true);
await connection.connect();
await runSchemaBootstrap(orm, { snapshot: false });
});
}
/**
* An ORM instance outside Nest, for the vitest global setup that builds the
* schema snapshot. Production uses the container-owned one from
+4 -4
View File
@@ -6,10 +6,10 @@ import path from 'path';
/**
* A migrated-but-empty database, serialised once per test run.
*
* Migrations are async and `database.ts::initDb()` is not — it runs at module
* load, long before anything can await. Most suites do not care (they mock the
* module or build their own database), but a handful use the singleton directly
* and need a schema to be there the moment they import it.
* Migrations are async and `database.ts::openDb()` is not: it runs on the
* connection's first read, which cannot await. Most suites do not care (they
* mock the module or build their own database), but a handful use the
* singleton directly and need a schema to be there the moment they touch it.
*
* So the vitest global setup migrates one database for the whole run and writes
* it here; every worker opens a copy. `better-sqlite3` deserialises a 1.5 MiB
+16 -6
View File
@@ -2,16 +2,19 @@ import 'reflect-metadata';
import 'dotenv/config';
// Fail-fast env validation — must stay directly after dotenv so a malformed
// variable aborts before any other module runs its import-time side effects
// (config.ts key resolution, db/database.ts initDb, ...).
// (config.ts key resolution, ...).
import './app-config/boot-validate';
import path from 'node:path';
import fs from 'node:fs';
import http from 'node:http';
import type { INestApplication } from '@nestjs/common';
// bootstrap is required inside bootstrap() below, not imported here: importing
// it opens the database, and a first-start restore (#1089) has to put the
// backup's database in place before that happens.
// bootstrap is required inside bootstrap() below, not imported here. Importing
// it no longer opens the database (the ORM's first connect inside buildApp()
// does, through DatabaseLifecycle), but a first-start restore (#1089) has to
// put the backup's database in place before anything near it loads, and
// keeping the require after the restore keeps that true by construction.
import type * as Bootstrap from './bootstrap';
import type { DatabaseLifecycle } from './nest/database/database-lifecycle.service';
// data/tmp is the driver-agnostic global scratch dir (restore-upload spool,
// mirror stream staging) and stays boot-created here. Driver-owned roots — the
@@ -84,6 +87,7 @@ const onListen = () => {
let server: http.Server;
let nestApp: INestApplication;
let database: DatabaseLifecycle | undefined;
// Strangler toggle: prefixes served by Nest (env-overridable, instant rollback).
async function bootstrap(): Promise<void> {
@@ -92,8 +96,9 @@ async function bootstrap(): Promise<void> {
// (/mcp, /.well-known, OAuth SDK, SPA catch-all). buildApp() owns the composition
// order; it is shared with the integration-test harness so they can't drift.
const restore = await restoreBeforeTheDatabaseOpens();
const { buildApp, getHttpServer } = require('./bootstrap') as typeof Bootstrap;
const { buildApp, getHttpServer, DatabaseLifecycle: Lifecycle } = require('./bootstrap') as typeof Bootstrap;
nestApp = await buildApp();
database = nestApp.get(Lifecycle);
if (restore.restored) await finishFirstBootRestore(nestApp, restore);
// The server buildApp created and bound /ws to. Creating a second one here
// would serve the REST API fine and leave the gateway attached to a socket
@@ -185,7 +190,12 @@ function shutdown(signal: string): void {
closeNestApp: async () => { await nestApp?.close(); },
getWsClients: () => getServer()?.clients ?? null,
closeMcpSessions,
closeDb: () => { require('./db/database').closeDb(); },
// Through the lifecycle provider once the app is up; before that (a signal
// that beat bootstrap()) the module function is all there is.
closeDb: () => {
if (database) database.close();
else (require('./db/database') as typeof import('./db/database')).closeDb();
},
logInfo: sLogInfo,
logError: sLogError,
exit: (code: number) => process.exit(code),
+3
View File
@@ -31,6 +31,7 @@ import { TrekExceptionFilter } from './common/trek-exception.filter';
import { ZodValidationPipe } from './common/zod-validation.pipe';
import { ConfigModule } from './config/config.module';
import { OrmModule } from './database/orm.module';
import { DatabaseLifecycleModule } from './database/database-lifecycle.module';
import { DayNotesModule } from './day-notes/day-notes.module';
import { DaysModule } from './days/days.module';
import { DocSyncModule } from './doc-sync/doc-sync.module';
@@ -170,6 +171,8 @@ import { APP_FILTER, APP_GUARD, APP_INTERCEPTOR, APP_PIPE } from '@nestjs/core';
// facade) is gone — Plan 4 Task 4 deleted it once every domain moved onto
// repositories/`UnitOfWork`. `OrmModule` is the module that survives.
OrmModule,
// The connection's open/close/reopen, resolved by buildApp() and index.ts.
DatabaseLifecycleModule,
// MfaPolicyGuard (Plan 3b Task 1) is registered directly below as this
// module's own APP_GUARD, never per-controller — so it is the one place
// that needs `forFeature` for its three repositories; `Users` is not
@@ -0,0 +1,10 @@
import { Module } from '@nestjs/common';
import { DatabaseLifecycle } from './database-lifecycle.service';
/**
* Provides `DatabaseLifecycle`. Not global: the backup port and the app root
* import it by name, so a partial test harness that never touches the
* connection lifecycle does not have to supply the `MikroORM` it injects.
*/
@Module({ providers: [DatabaseLifecycle], exports: [DatabaseLifecycle] })
export class DatabaseLifecycleModule {}
@@ -0,0 +1,76 @@
import { Injectable } from '@nestjs/common';
import { MikroORM } from '@mikro-orm/core';
import { closeDb, getRawConnection, registerReinitializeHook, reinitialize } from '../../db/database';
import { resolveDbPath } from '../../db/db-path';
import { runSchemaBootstrap } from '../../db/orm';
/**
* The one owner of the core database connection's lifecycle: open, close and
* reopen, plus the schema bootstrap that has to follow an open.
*
* The handle itself still lives in `db/database.ts`, because the ORM's bound
* driver (`db/orm-driver.ts`) reads it on every connect and the test suites
* replace that module wholesale with `vi.mock`. What moved here is the
* decision about WHEN it opens and closes: `buildApp()` opens through this and
* the shutdown closes through this. Nothing opens the database at import any
* more; the first connect does, and this provider makes it explicit. The
* restore and the demo reset still call the module's `closeDb()`/`reinitialize()`
* themselves until they move onto `close()`/`reopen()`.
*/
@Injectable()
export class DatabaseLifecycle {
constructor(private readonly orm: MikroORM) {}
/**
* The database file this process runs on, resolved the one way every caller
* shares (`db/db-path.ts`): `TREK_DB_FILE` when set, `data/travel.db`
* otherwise, `:memory:` under test.
*/
get file(): string {
return resolveDbPath();
}
/**
* Opens the connection, binds the ORM to its later swaps and brings the schema
* to head (legacy baseline, migrations, the deferred index, seeders, demo
* seed, in that order: see `db/orm.ts`). Called once by `buildApp()`, before
* anything reads the database.
*/
async open(): Promise<void> {
// Normally a no-op: the ORM's first connect inside NestFactory.create()
// already opened it. Said here so the boot does not depend on that.
getRawConnection();
registerReinitializeHook(() => this.rebindOrm());
await runSchemaBootstrap(this.orm);
}
/** Checkpoints and closes the connection. A closed connection stays closed until `reopen()`. */
close(): void {
closeDb();
}
/**
* Reopens the connection after its file was replaced, rebuilds the ORM's
* cached client around the new handle and migrates the new file forward (a
* restored backup may predate this release).
*/
async reopen(): Promise<void> {
await reinitialize();
}
/**
* Kysely caches whatever handle it was given, so without this the ORM would
* keep talking to the closed one. Closing and reconnecting sends the driver
* back through `createKyselyDialect()`, which picks up the new handle.
*/
private async rebindOrm(): Promise<void> {
const connection = this.orm.em.getConnection();
// The handle is owned and closed by `db/database.ts`; the bound driver's
// `destroy()` is a no-op (see `orm-driver.ts`), so this close never reaches it.
await connection.close(true);
await connection.connect();
// No pre-migrate snapshot: the file was just unpacked from an archive that
// is still there to go back to.
await runSchemaBootstrap(this.orm, { snapshot: false });
}
}
+6 -6
View File
@@ -10,19 +10,19 @@ import { schemaSnapshotPath, writeSchemaSnapshot } from '../src/db/schema-snapsh
import fs from 'node:fs';
async function main(): Promise<void> {
// Delete first, and load database.ts only afterwards — in test mode `initDb()`
// Delete first, and open the database only afterwards: in test mode `openDb()`
// opens a copy of this snapshot when one exists, so a leftover from the last
// run would have this process migrating an already-migrated database and
// seeding it twice. Static imports are evaluated before any of this runs,
// hence the dynamic ones below.
// seeding it twice. The modules are loaded dynamically below so nothing of
// them runs before the delete.
fs.rmSync(schemaSnapshotPath(), { force: true });
const { getRawConnection } = await import('../src/db/database');
const { createStandaloneOrm, runSchemaBootstrap } = await import('../src/db/orm');
// The shared driver binds to the connection database.ts opened on import, so
// this migrates that in-memory database and it can be serialised straight
// after.
// The shared driver binds to the connection database.ts opens on the ORM's
// first connect, so this migrates that in-memory database and it can be
// serialised straight after.
const orm = await createStandaloneOrm();
try {
await runSchemaBootstrap(orm);
+1 -1
View File
@@ -6,7 +6,7 @@ import path from 'node:path';
*
* `src/db/database.ts` used to build the schema itself at module load, so merely
* importing it handed a suite a migrated database. That is MikroORM's job now
* and its migrator is async, which the synchronous `initDb()` cannot await. This
* and its migrator is async, which the synchronous `openDb()` cannot await. This
* is the only hook in the test lifecycle that can: setup files are side-effect
* imports and every other hook is per-file.
*
+2 -1
View File
@@ -12,7 +12,7 @@
* own mock for that exact module gets handed the REAL, unmocked module for
* that one import — Vitest can't give a still-building mock to its own
* re-entrant importer. The real module's test-mode singleton then opens its
* OWN copy of the schema snapshot (`initDb()` → `readSchemaSnapshot()`), so
* OWN copy of the schema snapshot on first use (`openDb()` → `readSchemaSnapshot()`), so
* whichever code path captured `db` this way sees a pristine, never-reset
* database (the snapshot's seeded `admin@trek.local`) instead of the mock
* `buildDbMock` builds — while everything that imports `db/database` AFTER
@@ -135,6 +135,7 @@ export function buildDbMock(testDb: Database.Database) {
return !!testDb.prepare('SELECT id FROM trips WHERE id = ? AND user_id = ?').get(tripId, userId);
},
getRawConnection: () => testDb,
openDb: () => {},
registerReinitializeHook: () => {},
runDemoSeed: () => {},
};
@@ -0,0 +1,55 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import request from 'supertest';
import type { INestApplication } from '@nestjs/common';
import { buildApp } from '../../src/bootstrap';
import { DatabaseLifecycle } from '../../src/nest/database/database-lifecycle.service';
/**
* The core database's lifecycle, owned by the DatabaseLifecycle provider, on a
* real boot.
*
* The connection used to open as a side effect of importing db/database.ts and
* was closed and reopened through module functions the restore and the demo
* reset called directly. buildApp() now opens it through the provider, which
* also binds the ORM to later swaps, and closes and reopens it through the
* same provider. Unmocked on purpose: under NODE_ENV=test the
* module opens a copy of the migrated schema snapshot, and every reopen opens a
* pristine copy again, which is all these cases need.
*/
describe('the database connection lifecycle under buildApp()', () => {
let app: INestApplication;
beforeAll(async () => {
app = await buildApp();
await app.init();
});
afterAll(async () => {
await app?.close();
});
it('LIFECYCLE-001: the boot opened the connection through the provider and the database answers', async () => {
expect(app.get(DatabaseLifecycle).file).toBe(':memory:');
const res = await request(app.getHttpServer()).get('/api/health/ready');
expect(res.status).toBe(200);
expect(res.body).toEqual({ status: 'ready' });
});
it('LIFECYCLE-002: closed, the readiness probe answers 503; reopened, the ORM is bound to the new handle and it answers again', async () => {
const lifecycle = app.get(DatabaseLifecycle);
lifecycle.close();
try {
const down = await request(app.getHttpServer()).get('/api/health/ready');
expect(down.status).toBe(503);
expect(down.body).toEqual({ status: 'unavailable' });
} finally {
await lifecycle.reopen();
}
const up = await request(app.getHttpServer()).get('/api/health/ready');
expect(up.status).toBe(200);
});
});
@@ -6,8 +6,8 @@
* fork (`cron-registrar.service.ts:87-92`, `wrappedTick`). INSIDE that one
* fork, the function reads credentials, calls `closeDb()` then
* `await reinitialize()` (which closes the process's one better-sqlite3
* handle, opens a new one, and — via `orm.ts#attachOrm`'s
* `registerReinitializeHook` — closes and reconnects the ORM's own
* handle, opens a new one, and (through the hook `DatabaseLifecycle.open()`
* registers with `registerReinitializeHook`) closes and reconnects the ORM's own
* Kysely-bound `Connection`, then re-runs `runSchemaBootstrap`), and only
* THEN writes the preserved credentials back through a freshly re-required
* `db` reference.
@@ -28,7 +28,7 @@
* instance's `Configuration`, shared by every fork. `orm-driver.ts`'s
* `BoundSqliteConnection.createKyselyDialect()` reads `getRawConnection()`
* (the CURRENT `db/database.ts` handle) on every `connect()`, and
* `attachOrm`'s hook calls `connection.close(true)` then
* that hook calls `connection.close(true)` then
* `connection.connect()` on that ONE shared `Connection` object — it never
* replaces the object itself, only rebinds its internal Kysely client. So
* any EM fork — pre-swap or fresh — that resolves a query calls
@@ -37,7 +37,7 @@
* this file is what turns that into a measured answer with real evidence
* rather than an assumption.
*
* Test-mode nuance: `NODE_ENV=test` makes `db/database.ts#initDb()` reopen a
* Test-mode nuance: `NODE_ENV=test` makes `db/database.ts#openDb()` reopen a
* PRISTINE copy of the migrated schema snapshot on every call (a fresh
* `better-sqlite3` instance deserialised from a fixed buffer), not the same
* on-disk file demo-reset.ts restores in production — so a row inserted
@@ -8,7 +8,7 @@
* `withRequestContext` then." `runSchemaBootstrap` (Plan 3c Task 0b) now
* wraps the call in `withRequestContext(orm, …)`, the one place it already
* has `orm` in scope — covering both of `runDemoSeed`'s real call sites
* (`bootstrap.ts`'s initial boot and `attachOrm`'s restore hook) with no
* (`DatabaseLifecycle.open()` at boot and its restore hook) with no
* second wrapper to keep in sync.
*
* This is a real `DEMO_MODE` boot proving that wrap doesn't change what the
@@ -0,0 +1,80 @@
/**
* db/database.ts opens the connection on first use, not at import.
*
* It used to call initDb() at module load, so importing anything that reached
* it opened the database: index.ts had to defer loading the app until a
* first-boot restore had put the backup's file in place. The connection is now
* opened by its first reader (the ORM's first connect inside buildApp(), driven
* by DatabaseLifecycle), and a connection closed on purpose for a restore stays
* closed until it is reopened.
*
* The cases share the module's one connection and run in order.
*/
import { describe, expect, it, vi } from 'vitest';
const opened = vi.hoisted(() => ({ count: 0 }));
vi.mock('../../../src/db/connection', async (importOriginal) => {
const actual = await importOriginal<typeof import('../../../src/db/connection')>();
return {
...actual,
openDatabase: (...args: Parameters<typeof actual.openDatabase>) => {
opened.count++;
return actual.openDatabase(...args);
},
};
});
import { closeDb, db, getRawConnection, openDb, registerReinitializeHook, reinitialize } from '../../../src/db/database';
describe('db/database.ts connection lifecycle', () => {
it('DBOPEN-001: importing the module opens nothing', () => {
expect(opened.count).toBe(0);
});
it('DBOPEN-002: the first reader opens it once, with the pragmas applied', () => {
const handle = getRawConnection();
expect(getRawConnection()).toBe(handle);
expect(opened.count).toBe(1);
expect(handle.pragma('foreign_keys', { simple: true })).toBe(1);
expect(handle.pragma('busy_timeout', { simple: true })).toBe(5000);
// The Proxy reads through to the same handle.
expect(db.open).toBe(true);
});
it('DBOPEN-003: a connection closed on purpose stays closed instead of quietly reopening', () => {
const log = vi.spyOn(console, 'log').mockImplementation(() => undefined);
try {
closeDb();
} finally {
log.mockRestore();
}
expect(() => getRawConnection()).toThrow('Database connection is not available (restore in progress?)');
expect(() => db.prepare('SELECT 1')).toThrow('Database connection is not available (restore in progress?)');
expect(opened.count).toBe(1);
});
it('DBOPEN-004: openDb() reopens it, and opening an open connection is a no-op', () => {
openDb();
openDb();
expect(opened.count).toBe(2);
expect(getRawConnection().open).toBe(true);
});
it('DBOPEN-005: reinitialize() swaps in a fresh handle and then runs the registered hook', async () => {
const before = getRawConnection();
const seen: boolean[] = [];
registerReinitializeHook(async () => {
seen.push(getRawConnection() !== before && getRawConnection().open);
});
const log = vi.spyOn(console, 'log').mockImplementation(() => undefined);
try {
await reinitialize();
} finally {
log.mockRestore();
}
expect(before.open).toBe(false);
expect(seen).toEqual([true]);
expect(opened.count).toBe(3);
});
});
@@ -0,0 +1,124 @@
/**
* DatabaseLifecycle: the provider that owns when the core connection opens,
* closes and reopens.
*
* The connection module and the schema bootstrap are mocked: what is pinned
* here is the order the provider drives them in (the boot order the legacy
* baseline depends on) and that the ORM is rebuilt around a reopened handle.
* The real open, migrate and reopen run in tests/integration/database-lifecycle.test.ts.
*/
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import type { MikroORM } from '@mikro-orm/core';
const { dbMock, ormMock, calls } = vi.hoisted(() => {
const calls: string[] = [];
return {
calls,
dbMock: {
getRawConnection: vi.fn(() => {
calls.push('getRawConnection');
return {};
}),
registerReinitializeHook: vi.fn((_hook: () => Promise<void>) => {
calls.push('registerReinitializeHook');
}),
closeDb: vi.fn(() => {
calls.push('closeDb');
}),
reinitialize: vi.fn(async () => {
calls.push('reinitialize');
}),
},
ormMock: {
runSchemaBootstrap: vi.fn(async (_orm: unknown, opts?: { snapshot?: boolean }) => {
calls.push(`runSchemaBootstrap:${JSON.stringify(opts ?? {})}`);
}),
},
};
});
vi.mock('../../../../src/db/database', () => dbMock);
vi.mock('../../../../src/db/orm', () => ormMock);
import { DatabaseLifecycle } from '../../../../src/nest/database/database-lifecycle.service';
function ormStub() {
const connection = {
close: vi.fn(async (force?: boolean) => {
calls.push(`connection.close:${String(force)}`);
}),
connect: vi.fn(async () => {
calls.push('connection.connect');
}),
};
return { orm: { em: { getConnection: () => connection } } as unknown as MikroORM, connection };
}
beforeEach(() => {
vi.clearAllMocks();
calls.length = 0;
});
describe('DatabaseLifecycle', () => {
it('DBLIFE-001: open() opens the connection, binds the ORM to later swaps, then brings the schema to head', async () => {
const { orm } = ormStub();
await new DatabaseLifecycle(orm).open();
expect(calls).toEqual(['getRawConnection', 'registerReinitializeHook', 'runSchemaBootstrap:{}']);
expect(ormMock.runSchemaBootstrap).toHaveBeenCalledWith(orm);
});
it('DBLIFE-002: the hook open() registers rebuilds the ORM client and migrates the new file without a pre-migrate copy', async () => {
const { orm, connection } = ormStub();
await new DatabaseLifecycle(orm).open();
const hook = dbMock.registerReinitializeHook.mock.calls[0][0];
calls.length = 0;
await hook();
expect(connection.close).toHaveBeenCalledWith(true);
expect(calls).toEqual(['connection.close:true', 'connection.connect', 'runSchemaBootstrap:{"snapshot":false}']);
});
it('DBLIFE-003: close() and reopen() go through the connection module', async () => {
const lifecycle = new DatabaseLifecycle(ormStub().orm);
lifecycle.close();
await lifecycle.reopen();
expect(calls).toEqual(['closeDb', 'reinitialize']);
});
it('DBLIFE-004: a reopen that fails is not swallowed', async () => {
dbMock.reinitialize.mockRejectedValueOnce(new Error('database is locked'));
await expect(new DatabaseLifecycle(ormStub().orm).reopen()).rejects.toThrow('database is locked');
});
describe('file', () => {
let dir: string | null = null;
afterEach(() => {
vi.unstubAllEnvs();
if (dir) fs.rmSync(dir, { recursive: true, force: true });
dir = null;
});
it('DBLIFE-005: is the in-memory database under test', () => {
expect(new DatabaseLifecycle(ormStub().orm).file).toBe(':memory:');
});
it('DBLIFE-006: is TREK_DB_FILE when it is set, the one path the connection, the backup and the restore share', () => {
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'trek-lifecycle-'));
const file = path.join(dir, 'nested', 'custom-name.db');
vi.stubEnv('NODE_ENV', 'production');
vi.stubEnv('TREK_DB_FILE', file);
expect(new DatabaseLifecycle(ormStub().orm).file).toBe(file);
});
});
});