fix(demo): save the first baseline after a reopen that seeded too

The demo seed runs in the schema bootstrap, at boot and again after every
reopen. Since the baseline save moved out of the seed into DemoResetJob,
only the boot read the seed's mark. A restore onto a demo instance with no
baseline, whose archive held no admin trips, seeded the example trips
during the reopen and then saved nothing, so the hourly reset kept logging
"No baseline found"; the mark also stayed set for the rest of the process.
Before the move, that path saved the baseline from inside the seed.

DatabaseLifecycle now runs listeners after a reopen's schema bootstrap,
and DemoResetJob registers one in demo mode that does the same check as at
boot: save through the backup port when the seed just ran and no baseline
exists. Each check clears the mark. A failed save is logged, never turned
into a failed reopen. AdminModule imports DatabaseLifecycleModule from the
shared database kernel for it.
This commit is contained in:
Maurice
2026-10-08 15:48:38 +02:00
parent 1dfc5dd8ea
commit 3e1234112d
9 changed files with 149 additions and 18 deletions
+1 -1
View File
@@ -42,7 +42,7 @@ Nest owns everything. Every domain is a DI module under `src/nest/<domain>/` (`c
- **Scheduling**: `src/nest/scheduling/` (`CronRegistrarService`) is the one way to schedule a cron. Jobs are `*.job.ts` providers in their owning domain. The registrar refuses to schedule under `NODE_ENV=test` (an integration test pins it). **Never use a `@Cron` decorator** — it bypasses that gate.
- **Plugins** (`src/nest/plugins/`): the child-process runtime (`supervisor/`, `runtime/plugin-host-entry.ts`, `protocol/envelope.ts`) is **intentionally not Nest** — it is the security boundary. `host/rpc-host.ts` registers an RPC method at spawn only if the plugin holds the unlocking permission (registration = authorization). Install-time gates live in `install/`; the manifest's `trek` version range fails closed, and `TREK_PLUGINS_IGNORE_TREK_RANGE` downgrades that to a warning. The protocol tables in `protocol/envelope.ts` (+ `src/plugin-event-sink.ts`) are **generated** into `plugin-sdk/` and `shared/` — run `gen:plugin-facts` after touching them and commit the output. `runtime/egress-policy.ts` is a hand-kept mirror guarded by a parity test in `plugin-sdk/`; change both sides together.
- **Storage**: `src/nest/storage/` is the upload driver abstraction (local / S3 / mirror behind one interface, with an admin surface and a usage job). Domain services never touch the filesystem or the S3 SDK directly.
- **DB** (`src/db/`): `better-sqlite3`, WAL, FK on. `database.ts` only holds the connection and applies pragmas. It does not build a schema, and importing it opens nothing (the first reader does). The connection's lifecycle belongs to the `DatabaseLifecycle` provider (`nest/database/database-lifecycle.service.ts`): `buildApp()` opens it through `open()` (which binds the ORM to later swaps and runs the schema bootstrap below), the shutdown closes it through `close()`, and a restore or the demo reset swaps the file only through the database backup port, `DATABASE_BACKUP` (`nest/database/database-backup.interface.ts`; the SQLite implementation is `nest/backup/sqlite-database-backup.ts`: `VACUUM INTO` snapshot, close, atomic file swap, `reopen()`). The port works on `DatabaseLifecycle.file`, the one resolved path (`TREK_DB_FILE` or `data/travel.db`), and a snapshot that fails fails the backup rather than archiving the live file. `MaintenanceRepository` is provided only by `MaintenanceModule`'s factory and injected everywhere else; no consumer constructs it. MikroORM's migrations (`db/migrations/`) are the only schema source, **including for tests**: the hand-kept `schema.ts`/`migrations.ts`/`seeds.ts` builders and the parity test that guarded them are gone. Seeding is the MikroORM seeders in `db/seeders/`, applied by `db/orm.ts` from `buildApp()` before anything reads. An install the retired positional runner migrated (a `schema_version` row, no `mikro_orm_migrations` rows) is baselined by `db/legacy-baseline.ts`: the migrations whose `Legacy migration step N` docstring is ≤ its version are recorded as executed (never replayed) in the same transaction as the migrator run, while the baseline migration itself still runs first, as the legacy runner's `createTables()` did on every boot; anything it cannot map refuses the boot. Before a boot applies pending migrations to a file database that holds anything, `db/pre-migrate-snapshot.ts` writes a `VACUUM INTO` copy next to it (`pre-migrate-<schema>-<time>.db`, `<schema>` being `legacy-<schema_version>` or the last recorded migration's timestamp; the newest `TREK_DB_PRE_MIGRATE_SNAPSHOT_KEEP` stay) and the newer-database refusal names the matching copy; a copy that cannot be written refuses the boot unless `TREK_DB_PRE_MIGRATE_SNAPSHOT=false`. It is off under `NODE_ENV=test` unless a suite sets that variable to true, and a restore's re-bootstrap skips it (the archive is the copy). After the migrator, `db/email-case-index.ts` creates the case-insensitive unique index on `users.email` that its migration had to defer on an install holding two accounts whose emails differ only in case (flag `email_case_index_pending`). `db/orm-driver.ts` binds the ORM to that same connection instead of opening a second one — with `:memory:` under test, a second connection is a second, empty database. Every read and write goes through a repository in `db/repositories/` (each extends the shared `TrekRepository` base in `db/repositories/_shared/`, which validates the request context before delegating to MikroORM) and, for multi-statement writes, `UnitOfWork.transactional(...)` (`nest/database/unit-of-work.ts`, provided by the global `OrmModule`) — there is no `DatabaseService`/`DatabaseModule` to inject any more. Entrypoints outside an HTTP request — an MCP tool call, a plugin RPC dispatch, a WebSocket message, a cron job, boot-time seeding, a backup restore — open one explicitly with `withRequestContext()` (`nest/database/request-context.ts`), since only the pathless `mikroOrmRequestContext` middleware `buildApp()` mounts (with `@mikro-orm/nestjs`'s own `registerRequestContext` turned off in `app.module.ts`, UPLOADS-P16) forks an `EntityManager` per HTTP request. `NODE_ENV=test` ⇒ isolated `:memory:` DB per vitest worker, opened from a snapshot `tests/global-setup.ts` migrates once per run; `TREK_DB_FILE` ⇒ explicit file DB (Playwright harness).
- **DB** (`src/db/`): `better-sqlite3`, WAL, FK on. `database.ts` only holds the connection and applies pragmas. It does not build a schema, and importing it opens nothing (the first reader does). The connection's lifecycle belongs to the `DatabaseLifecycle` provider (`nest/database/database-lifecycle.service.ts`): `buildApp()` opens it through `open()` (which binds the ORM to later swaps and runs the schema bootstrap below), the shutdown closes it through `close()`, and a restore or the demo reset swaps the file only through the database backup port, `DATABASE_BACKUP` (`nest/database/database-backup.interface.ts`; the SQLite implementation is `nest/backup/sqlite-database-backup.ts`: `VACUUM INTO` snapshot, close, atomic file swap, `reopen()`). The port works on `DatabaseLifecycle.file`, the one resolved path (`TREK_DB_FILE` or `data/travel.db`), and a snapshot that fails fails the backup rather than archiving the live file. `MaintenanceRepository` is provided only by `MaintenanceModule`'s factory and injected everywhere else; no consumer constructs it. A provider that has to follow every reopen (the demo reset job saves the first demo baseline after a restore's re-bootstrap seeded the example trips) registers with `DatabaseLifecycle.onReopened()`, which runs after the reopen's schema bootstrap; a listener contains its own failures, since one that throws fails the reopen. MikroORM's migrations (`db/migrations/`) are the only schema source, **including for tests**: the hand-kept `schema.ts`/`migrations.ts`/`seeds.ts` builders and the parity test that guarded them are gone. Seeding is the MikroORM seeders in `db/seeders/`, applied by `db/orm.ts` from `buildApp()` before anything reads. An install the retired positional runner migrated (a `schema_version` row, no `mikro_orm_migrations` rows) is baselined by `db/legacy-baseline.ts`: the migrations whose `Legacy migration step N` docstring is ≤ its version are recorded as executed (never replayed) in the same transaction as the migrator run, while the baseline migration itself still runs first, as the legacy runner's `createTables()` did on every boot; anything it cannot map refuses the boot. Before a boot applies pending migrations to a file database that holds anything, `db/pre-migrate-snapshot.ts` writes a `VACUUM INTO` copy next to it (`pre-migrate-<schema>-<time>.db`, `<schema>` being `legacy-<schema_version>` or the last recorded migration's timestamp; the newest `TREK_DB_PRE_MIGRATE_SNAPSHOT_KEEP` stay) and the newer-database refusal names the matching copy; a copy that cannot be written refuses the boot unless `TREK_DB_PRE_MIGRATE_SNAPSHOT=false`. It is off under `NODE_ENV=test` unless a suite sets that variable to true, and a restore's re-bootstrap skips it (the archive is the copy). After the migrator, `db/email-case-index.ts` creates the case-insensitive unique index on `users.email` that its migration had to defer on an install holding two accounts whose emails differ only in case (flag `email_case_index_pending`). `db/orm-driver.ts` binds the ORM to that same connection instead of opening a second one — with `:memory:` under test, a second connection is a second, empty database. Every read and write goes through a repository in `db/repositories/` (each extends the shared `TrekRepository` base in `db/repositories/_shared/`, which validates the request context before delegating to MikroORM) and, for multi-statement writes, `UnitOfWork.transactional(...)` (`nest/database/unit-of-work.ts`, provided by the global `OrmModule`) — there is no `DatabaseService`/`DatabaseModule` to inject any more. Entrypoints outside an HTTP request — an MCP tool call, a plugin RPC dispatch, a WebSocket message, a cron job, boot-time seeding, a backup restore — open one explicitly with `withRequestContext()` (`nest/database/request-context.ts`), since only the pathless `mikroOrmRequestContext` middleware `buildApp()` mounts (with `@mikro-orm/nestjs`'s own `registerRequestContext` turned off in `app.module.ts`, UPLOADS-P16) forks an `EntityManager` per HTTP request. `NODE_ENV=test` ⇒ isolated `:memory:` DB per vitest worker, opened from a snapshot `tests/global-setup.ts` migrates once per run; `TREK_DB_FILE` ⇒ explicit file DB (Playwright harness).
## Rules for new code
+6 -3
View File
@@ -157,13 +157,16 @@ function hasBaseline(): boolean {
}
/**
* Whether the boot-time seed put the example trips in during this process.
* Whether the demo seed put the example trips in since this mark was last read.
*
* The first baseline is saved only after a real seed, as it always was: a demo
* database that already holds data but has no baseline keeps it that way, and
* the hourly reset stays a logged no-op until the admin saves one. The seed
* runs before the container exists, so it leaves this mark and DemoResetJob,
* which has the backup port, saves the baseline once the app is up.
* runs inside the schema bootstrap, at boot before the container exists and
* again after every reopen (a restore, the hourly swap), where it has no backup
* port either. So it leaves this mark, and DemoResetJob, which has the port,
* reads it once the app is up and again after each reopen
* (`DatabaseLifecycle.onReopened`), saving the baseline when it is set.
*/
let seededExampleTrips = false;
+2 -2
View File
@@ -64,8 +64,8 @@ async function seedDemoData(): Promise<{ adminId: number; demoId: number }> {
await seedExampleTrips(demo, adminId, demoId);
// The first baseline follows this seed and only this seed. DemoResetJob saves
// it once the app is up, through the injected backup port this boot-time
// seed has no access to.
// it once the app is up, or right after the reopen this seed ran in, through
// the injected backup port this seed has no access to.
markExampleTripsSeeded();
return { adminId, demoId };
}
+5 -1
View File
@@ -28,6 +28,10 @@ import { AppConfigModule } from '../app-config/app-config.module';
// DatabaseBackupModule (nest/backup/) provides once BackupModule is in the
// graph. It is not imported here on purpose: see that module for the cycle an
// import would close.
// DemoResetJob also listens for reopens on DatabaseLifecycle (the shared
// database kernel, not a domain), to save the first demo baseline after a
// restore's re-bootstrap seeded the example trips.
import { DatabaseLifecycleModule } from '../database/database-lifecycle.module';
import { Users } from '../../db/entities/Users.entity';
import { AuditLog } from '../../db/entities/AuditLog.entity';
import { AppSettings } from '../../db/entities/AppSettings.entity';
@@ -58,7 +62,7 @@ import { PushSubscriptions } from '../../db/entities/PushSubscriptions.entity';
@Module({
imports: [
MikroOrmModule.forFeature([Users, AuditLog, AppSettings, Addons, PhotoProviders, PhotoProviderFields, DocumentProviders, McpTokens, OauthTokens, Trips, Places, TripFiles, PushSubscriptions]),
AppConfigModule, PluginsRuntimeModule, SettingsModule, AuditModule, AddonsModule, AuthModule, NotificationsModule, PackingModule, PermissionsModule, TokensModule, OauthModule, SchedulingModule, KitineraryExtractorModule,
AppConfigModule, DatabaseLifecycleModule, PluginsRuntimeModule, SettingsModule, AuditModule, AddonsModule, AuthModule, NotificationsModule, PackingModule, PermissionsModule, TokensModule, OauthModule, SchedulingModule, KitineraryExtractorModule,
],
controllers: [AdminController],
providers: [AdminService, VersionCheckJob, DemoResetJob],
+22 -9
View File
@@ -4,6 +4,7 @@ import { logInfo, logError } from '../audit/audit-log.logger';
import { RuntimeEnvService } from '../app-config/runtime-env.service';
import { CronRegistrarService } from '../scheduling/cron-registrar.service';
import { DATABASE_BACKUP, type DatabaseBackupStrategy } from '../database/database-backup.interface';
import { DatabaseLifecycle } from '../database/database-lifecycle.service';
import { withRequestContext } from '../database/request-context';
import { hasBaseline, resetDemoUser, saveBaseline, takeExampleTripsSeeded } from '../../demo/demo-reset';
@@ -15,12 +16,16 @@ import { hasBaseline, resetDemoUser, saveBaseline, takeExampleTripsSeeded } from
* side-effect-free, and the close/swap/reopen sequence runs inside
* resetDemoUser at tick time, through the injected backup port.
*
* It also saves the first baseline, on the boot whose seed has just put the
* example trips in and only when no baseline exists yet: the same trigger the
* save had inside the boot-time demo seed, which runs before the container can
* hand the port in. A demo database that already holds data without a baseline
* is left alone, so its hourly reset stays a logged no-op as before. Like the
* old save, it does not depend on the cron registrar being enabled.
* It also saves the first baseline, whenever the demo seed has just put the
* example trips in and no baseline exists yet: the same trigger the save had
* inside the demo seed, which runs before the container can hand the port in.
* That seed runs at boot and again in the schema bootstrap after every reopen
* (an admin restore, the hourly swap), so the job checks once the app is up
* and again after each reopen, through `DatabaseLifecycle.onReopened`. Each
* check clears the seed's mark, so none is left behind. A demo database that
* already holds data without a baseline is left alone, so its hourly reset
* stays a logged no-op as before. Like the old save, it does not depend on the
* cron registrar being enabled.
*/
@Injectable()
export class DemoResetJob implements OnApplicationBootstrap {
@@ -29,6 +34,7 @@ export class DemoResetJob implements OnApplicationBootstrap {
private readonly registrar: CronRegistrarService,
@Inject(DATABASE_BACKUP) private readonly database: DatabaseBackupStrategy,
private readonly em: EntityManager,
private readonly lifecycle: DatabaseLifecycle,
) {}
async onApplicationBootstrap(): Promise<void> {
@@ -39,7 +45,8 @@ export class DemoResetJob implements OnApplicationBootstrap {
this.registrar.register('demo-reset', '0 * * * *', () => void this.tick(), { timezone: 'none' });
logInfo('Demo hourly reset scheduled');
}
if (takeExampleTripsSeeded() && !hasBaseline()) await this.saveFirstBaseline();
this.lifecycle.onReopened(() => this.saveFirstBaselineIfSeeded());
await this.saveFirstBaselineIfSeeded();
}
async tick(): Promise<void> {
@@ -50,8 +57,14 @@ export class DemoResetJob implements OnApplicationBootstrap {
}
}
/** Boot runs outside any request, so the snapshot gets its own context. */
private async saveFirstBaseline(): Promise<void> {
/**
* Saves the first baseline when the demo seed has just seeded and none exists.
* At boot it runs outside any request, so the snapshot gets its own context. Never
* throws: at boot a failure must not stop the app, and after a reopen it must
* not read as the reopen failing.
*/
private async saveFirstBaselineIfSeeded(): Promise<void> {
if (!takeExampleTripsSeeded() || hasBaseline()) return;
try {
await withRequestContext({ em: this.em }, () => saveBaseline(this.database));
} catch (err: unknown) {
@@ -22,6 +22,8 @@ import { runSchemaBootstrap } from '../../db/orm';
*/
@Injectable()
export class DatabaseLifecycle {
private readonly reopenListeners: Array<() => Promise<void>> = [];
constructor(private readonly orm: MikroORM) {}
/**
@@ -61,6 +63,20 @@ export class DatabaseLifecycle {
await reinitialize();
}
/**
* Runs `listener` after every reopen, once the reopened file has been through
* the schema bootstrap (migrations, seeders, demo seed). For work that has to
* follow what that bootstrap did and needs the container, which the bootstrap
* itself has no access to: the demo reset job saves the first demo baseline
* here when a restore's re-bootstrap seeded the example trips.
*
* A listener contains its own failures. One that throws turns into the
* reopen's error, which a restore reports as "restart required".
*/
onReopened(listener: () => Promise<void>): void {
this.reopenListeners.push(listener);
}
/**
* Kysely caches whatever handle it was given, so without this the ORM would
* keep talking to the closed one. Closing and reconnecting sends the driver
@@ -75,5 +91,6 @@ export class DatabaseLifecycle {
// No pre-migrate snapshot: the file was just unpacked from an archive that
// is still there to go back to.
await runSchemaBootstrap(this.orm, { snapshot: false });
for (const listener of this.reopenListeners) await listener();
}
}
+59 -1
View File
@@ -28,6 +28,7 @@ import type { AdminService } from '../../../src/nest/admin/admin.service';
import type { RuntimeEnvService } from '../../../src/nest/app-config/runtime-env.service';
import type { CronRegistrarService } from '../../../src/nest/scheduling/cron-registrar.service';
import type { DatabaseBackupStrategy } from '../../../src/nest/database/database-backup.interface';
import type { DatabaseLifecycle } from '../../../src/nest/database/database-lifecycle.service';
import type { EntityManager } from '@mikro-orm/core';
function registrarStub(enabled = true) {
@@ -80,7 +81,18 @@ describe('DemoResetJob', () => {
function make(demo: boolean, enabled = true) {
const registrar = registrarStub(enabled);
const runtimeEnv = { isDemoMode: () => demo } as RuntimeEnvService;
return { job: new DemoResetJob(runtimeEnv, registrar as unknown as CronRegistrarService, database, em), registrar };
const reopenListeners: Array<() => Promise<void>> = [];
const lifecycle = {
onReopened: vi.fn((listener: () => Promise<void>) => {
reopenListeners.push(listener);
}),
};
const job = new DemoResetJob(runtimeEnv, registrar as unknown as CronRegistrarService, database, em, lifecycle as unknown as DatabaseLifecycle);
// What DatabaseLifecycle does after a reopen's schema bootstrap.
const reopen = async () => {
for (const listener of reopenListeners) await listener();
};
return { job, registrar, lifecycle, reopen };
}
it('AJOB-004 — registers the hourly server-local cron and logs the banner when demo mode is on', async () => {
@@ -143,4 +155,50 @@ describe('DemoResetJob', () => {
await make(true).job.onApplicationBootstrap();
expect(saveBaselineMock).not.toHaveBeenCalled();
});
it('AJOB-011: a reopen whose re-bootstrap seeded the example trips saves the first baseline', async () => {
// A restore onto a demo instance with no baseline: the boot seeded nothing,
// the restored archive had no admin trips, so the reopen's demo seed did.
hasBaselineMock.mockReturnValue(false);
const { job, reopen } = make(true, false);
await job.onApplicationBootstrap();
expect(saveBaselineMock).not.toHaveBeenCalled();
takeExampleTripsSeededMock.mockReturnValueOnce(true);
await reopen();
expect(saveBaselineMock).toHaveBeenCalledTimes(1);
expect(saveBaselineMock).toHaveBeenCalledWith(database);
});
it('AJOB-012: a reopen that seeded nothing, or found a baseline, saves none, and each reopen clears the mark', async () => {
const { job, reopen } = make(true);
await job.onApplicationBootstrap();
takeExampleTripsSeededMock.mockClear();
hasBaselineMock.mockReturnValue(false);
await reopen();
takeExampleTripsSeededMock.mockReturnValueOnce(true);
hasBaselineMock.mockReturnValue(true);
await reopen();
expect(takeExampleTripsSeededMock).toHaveBeenCalledTimes(2);
expect(saveBaselineMock).not.toHaveBeenCalled();
});
it('AJOB-013: a baseline save that fails after a reopen is logged, not turned into a failed reopen', async () => {
hasBaselineMock.mockReturnValue(false);
const { job, reopen } = make(true);
await job.onApplicationBootstrap();
takeExampleTripsSeededMock.mockReturnValueOnce(true);
saveBaselineMock.mockRejectedValueOnce(new Error('database or disk is full'));
await expect(reopen()).resolves.toBeUndefined();
expect(logMock.logError).toHaveBeenCalledWith('Demo baseline: database or disk is full');
});
it('AJOB-014: a non-demo boot does not listen for reopens', async () => {
const { job, lifecycle } = make(false);
await job.onApplicationBootstrap();
expect(lifecycle.onReopened).not.toHaveBeenCalled();
});
});
@@ -84,6 +84,42 @@ describe('DatabaseLifecycle', () => {
expect(calls).toEqual(['connection.close:true', 'connection.connect', 'runSchemaBootstrap:{"snapshot":false}']);
});
it('DBLIFE-007: listeners registered with onReopened run after the schema bootstrap of a reopen, in order', async () => {
const { orm } = ormStub();
const lifecycle = new DatabaseLifecycle(orm);
lifecycle.onReopened(async () => {
calls.push('listener:first');
});
lifecycle.onReopened(async () => {
calls.push('listener:second');
});
await lifecycle.open();
const hook = dbMock.registerReinitializeHook.mock.calls[0][0];
calls.length = 0;
await hook();
expect(calls).toEqual([
'connection.close:true',
'connection.connect',
'runSchemaBootstrap:{"snapshot":false}',
'listener:first',
'listener:second',
]);
});
it('DBLIFE-008: a listener that throws fails the reopen, so a listener must contain its own failures', async () => {
const { orm } = ormStub();
const lifecycle = new DatabaseLifecycle(orm);
lifecycle.onReopened(async () => {
throw new Error('listener broke');
});
await lifecycle.open();
const hook = dbMock.registerReinitializeHook.mock.calls[0][0];
await expect(hook()).rejects.toThrow('listener broke');
});
it('DBLIFE-003: close() and reopen() go through the connection module', async () => {
const lifecycle = new DatabaseLifecycle(ormStub().orm);
+1 -1
View File
@@ -54,7 +54,7 @@ The baseline is the snapshot the hourly reset restores to. The admin can update
In the admin panel, a **Demo Baseline** card sits above the tabs while demo mode is on; its **Save Baseline** button calls this endpoint. The baseline captures the current state of the database, trip data, settings and encrypted API keys included, so demo features (maps, photos, weather) continue to work after each reset. It is written as a consistent snapshot and only then put in place of the previous baseline, so a save that fails leaves the previous one intact.
On first start with demo mode active, TREK seeds three example trips (Tokyo & Kyoto, Barcelona Long Weekend, New York City) owned by the admin and shared with the demo user, then saves the initial baseline automatically once the server is up. Only that seeding start saves a baseline on its own: an instance that already holds data but has no baseline keeps skipping the hourly reset until the admin saves one.
On first start with demo mode active, TREK seeds three example trips (Tokyo & Kyoto, Barcelona Long Weekend, New York City) owned by the admin and shared with the demo user, then saves the initial baseline automatically once the server is up. The same happens when a backup restored onto a demo instance without a baseline holds no admin trips: the example trips are seeded into the restored database and it becomes the baseline. Only a seeding start or restore saves a baseline on its own: an instance that already holds data but has no baseline keeps skipping the hourly reset until the admin saves one.
## Limitations