mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-10-22 15:06:32 +00:00
refactor(server): resolve the data directory in one place
The data and uploads directories were derived independently in about ten files, each counting `..` hops from its own `__dirname`, and the file logger used `process.cwd()` instead. Moving a file or starting the server from another directory quietly moved where secrets, backups or logs went, and the backup code named `data/travel.db` directly, so with TREK_DB_FILE set a backup archived and a restore replaced a file the connection never opened. `resolveDataPaths()` in app-config now anchors data/, uploads/, backups, tmp, logs, the JWT secret and encryption key files and the database file (TREK_DB_FILE when set, else data/travel.db) on the server package root. Code that runs before the container calls it directly; Nest classes inject the new `DataPathsService`. Every previous anchor uses it, and the resolved paths are unchanged for the image and for `npm run dev`.
This commit is contained in:
+1
-1
@@ -49,7 +49,7 @@ Nest owns everything. Every domain is a DI module under `src/nest/<domain>/` (`c
|
||||
- **MCP**: `src/nest-mcp/` is the decorator/registry layer (`@McpController`/`@Tool`/`@Resource`/`@Prompt`; see its `README.md`); tools live in `<domain>.mcp.ts` files beside each module; the HTTP transport is `src/nest/mcp-transport/`; `src/mcp/` holds process-wide state only (scopes, session manager, boot policy). Every tool invocation writes an audit row through the registry's `onInvoke` seam. MCP tools are adapters over the same services and permission checks as REST — a change on one side lands on the other in the same change.
|
||||
- **Scheduling**: `src/nest/scheduling/` (`CronRegistrarService`) is the one way to schedule a cron. Jobs are `*.job.ts` providers in their owning domain. The registrar refuses to schedule under `NODE_ENV=test` (an integration test pins it). **Never use a `@Cron` decorator** — it bypasses that gate.
|
||||
- **Plugins** (`src/nest/plugins/`): the child-process runtime (`supervisor/`, `runtime/plugin-host-entry.ts`, `protocol/envelope.ts`) is **intentionally not Nest** — it is the security boundary. `host/rpc-host.ts` registers an RPC method at spawn only if the plugin holds the unlocking permission (registration = authorization). Install-time gates live in `install/`; the manifest's `trek` version range fails closed, and `TREK_PLUGINS_IGNORE_TREK_RANGE` downgrades that to a warning. The protocol tables in `protocol/envelope.ts` (+ `src/plugin-event-sink.ts`) are **generated** into `plugin-sdk/` and `shared/` — run `gen:plugin-facts` after touching them and commit the output. `runtime/egress-policy.ts` is a hand-kept mirror guarded by a parity test in `plugin-sdk/`; change both sides together.
|
||||
- **Storage**: `src/nest/storage/` is the upload driver abstraction (local / S3 / mirror behind one interface, with an admin surface and a usage job). Domain services never touch the filesystem or the S3 SDK directly.
|
||||
- **Storage**: `src/nest/storage/` is the upload driver abstraction (local / S3 / mirror behind one interface, with an admin surface and a usage job). Domain services never touch the filesystem or the S3 SDK directly. Where the data lives (`data/`, `uploads/`, backups, logs, key files, the DB file) is resolved in one place, `src/app-config/data-paths.ts`; Nest classes inject `DataPathsService`, and nothing anchors a data path on its own `__dirname` or on `process.cwd()`.
|
||||
- **DB** (`src/db/`): `better-sqlite3`, WAL, FK on. `database.ts` only holds the connection and applies pragmas. It does not build a schema, and importing it opens nothing (the first reader does). The connection's lifecycle belongs to the `DatabaseLifecycle` provider (`nest/database/database-lifecycle.service.ts`): `buildApp()` opens it through `open()` (which binds the ORM to later swaps and runs the schema bootstrap below), the shutdown closes it through `close()`, and a restore or the demo reset swaps the file only through the database backup port, `DATABASE_BACKUP` (`nest/database/database-backup.interface.ts`; the SQLite implementation is `nest/backup/sqlite-database-backup.ts`: `VACUUM INTO` snapshot, close, atomic file swap, `reopen()`). The port works on `DatabaseLifecycle.file`, the one resolved path (`TREK_DB_FILE` or `data/travel.db`), and a snapshot that fails fails the backup rather than archiving the live file. `MaintenanceRepository` is provided only by `MaintenanceModule`'s factory and injected everywhere else; no consumer constructs it. A provider that has to follow every reopen (the demo reset job saves the first demo baseline after a restore's re-bootstrap seeded the example trips) registers with `DatabaseLifecycle.onReopened()`, which runs after the reopen's schema bootstrap; a listener contains its own failures, since one that throws fails the reopen. MikroORM's migrations (`db/migrations/`) are the only schema source, **including for tests**: the hand-kept `schema.ts`/`migrations.ts`/`seeds.ts` builders and the parity test that guarded them are gone. Seeding is the MikroORM seeders in `db/seeders/`, applied by `db/orm.ts` from `buildApp()` before anything reads. An install the retired positional runner migrated (a `schema_version` row, no `mikro_orm_migrations` rows) is baselined by `db/legacy-baseline.ts`: the migrations whose `Legacy migration step N` docstring is ≤ its version are recorded as executed (never replayed) in the same transaction as the migrator run, while the baseline migration itself still runs first, as the legacy runner's `createTables()` did on every boot; anything it cannot map refuses the boot. Before a boot applies pending migrations to a file database that holds anything, `db/pre-migrate-snapshot.ts` writes a `VACUUM INTO` copy next to it (`pre-migrate-<schema>-<time>.db`, `<schema>` being `legacy-<schema_version>` or the last recorded migration's timestamp; the newest `TREK_DB_PRE_MIGRATE_SNAPSHOT_KEEP` stay) and the newer-database refusal names the matching copy; a copy that cannot be written refuses the boot unless `TREK_DB_PRE_MIGRATE_SNAPSHOT=false`. It is off under `NODE_ENV=test` unless a suite sets that variable to true, and a restore's re-bootstrap skips it (the archive is the copy). After the migrator, `db/email-case-index.ts` creates the case-insensitive unique index on `users.email` that its migration had to defer on an install holding two accounts whose emails differ only in case (flag `email_case_index_pending`). `db/orm-driver.ts` binds the ORM to that same connection instead of opening a second one — with `:memory:` under test, a second connection is a second, empty database. Kysely statements (`this.kysely<...>()`) type against `DB` from `db/kysely/db.ts`, narrowed with `Pick<DB, 'trips' | 'days'>`; those files are generated from the migrated schema by `scripts/generate-db-types.ts`, so a migration that adds or changes a column is followed by `npm run gen:db-types`, never by a hand-written table interface. Every read and write goes through a repository in `db/repositories/` (each extends the shared `TrekRepository` base in `db/repositories/_shared/`, which validates the request context before delegating to MikroORM) and, for multi-statement writes, `UnitOfWork.transactional(...)` (`nest/database/unit-of-work.ts`, provided by the global `OrmModule`) — there is no `DatabaseService`/`DatabaseModule` to inject any more. Entrypoints outside an HTTP request — an MCP tool call, a plugin RPC dispatch, a WebSocket message, a cron job, boot-time seeding, a backup restore — open one explicitly with `withRequestContext()` (`nest/database/request-context.ts`), since only the pathless `mikroOrmRequestContext` middleware `buildApp()` mounts (with `@mikro-orm/nestjs`'s own `registerRequestContext` turned off in `app.module.ts`, UPLOADS-P16) forks an `EntityManager` per HTTP request. `NODE_ENV=test` ⇒ isolated `:memory:` DB per vitest worker, opened from a snapshot `tests/global-setup.ts` migrates once per run; `TREK_DB_FILE` ⇒ explicit file DB (Playwright harness).
|
||||
|
||||
## Rules for new code
|
||||
|
||||
@@ -10,6 +10,7 @@ env.schema.ts Zod catalog of the whole env surface; fail-fast at boot
|
||||
derive.ts pure (raw env) → typed namespace functions, exact per-site coercions
|
||||
env.ts readEnv() live accessor + validateEnvAtBoot()
|
||||
app-url.ts getAppUrl()/getMcpSafeUrl() instance base-URL resolution
|
||||
data-paths.ts resolveDataPaths(): data/, uploads/, backups, tmp, logs, key files, DB file
|
||||
boot-validate.ts side-effect import used by index.ts (validates before other modules load)
|
||||
parsers.ts shared coercion helpers (boolTrueLoose, numberOr, csvList, …)
|
||||
```
|
||||
@@ -53,6 +54,17 @@ tokens, `RuntimeEnvService`) and consumes the SAME derive functions.
|
||||
rate limits, …) keep freeze-at-import timing, merely sourcing the value from
|
||||
`readEnv()` at module top. Request-time reads stay request-time.
|
||||
|
||||
## data-paths.ts: the one data layout
|
||||
|
||||
`resolveDataPaths()` is the only place the server anchors `data/` and
|
||||
`uploads/` (on `SERVER_ROOT`, the package directory, never on `process.cwd()`)
|
||||
and names what lives in them: backups, the scratch dir, the logs, the JWT
|
||||
secret and encryption key files, and the database file (TREK_DB_FILE when set,
|
||||
else `data/travel.db`). Code that runs before the container exists imports it
|
||||
directly; Nest classes inject `DataPathsService` (`src/nest/app-config/`),
|
||||
which resolves the same layout once per built app. Never derive one of these
|
||||
paths from `__dirname` in a new file: a moved file would quietly move the data.
|
||||
|
||||
## app-url.ts — instance base-URL resolution (moved 2026-07-28)
|
||||
|
||||
`getAppUrl()` (APP_URL → first ALLOWED_ORIGINS entry → `http://localhost:PORT`,
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
import path from 'node:path';
|
||||
import { readEnv, type AppEnv } from './env';
|
||||
|
||||
/**
|
||||
* `<server>/`, the directory holding package.json. `src/app-config` and
|
||||
* `dist/app-config` both sit two levels below it, so the anchor is the same
|
||||
* under vitest, `npm run dev` and the built image.
|
||||
*/
|
||||
export const SERVER_ROOT = path.resolve(__dirname, '..', '..');
|
||||
|
||||
/**
|
||||
* Where TREK keeps what it writes. In the image `data/` and `uploads/` are
|
||||
* symlinks into `/app`, which is why every consumer used to anchor on its own
|
||||
* `__dirname` and why none of them may use `process.cwd()` instead.
|
||||
*/
|
||||
export interface DataPaths {
|
||||
/** `<server>/data`: the database, the key files, backups, logs and plugin trees. */
|
||||
readonly dataDir: string;
|
||||
/** `<server>/uploads`: the local upload categories (files, covers, photos, ...). */
|
||||
readonly uploadsDir: string;
|
||||
/** `data/backups`: archives written by the manual and the automatic backup. */
|
||||
readonly backupsDir: string;
|
||||
/** `data/tmp`: driver-agnostic scratch space (restore spool, mirror staging). */
|
||||
readonly tmpDir: string;
|
||||
/** `data/logs`: the rotating `trek.log`. */
|
||||
readonly logsDir: string;
|
||||
/** `data/.jwt_secret`: written on first start and by the admin's JWT rotation. */
|
||||
readonly jwtSecretFile: string;
|
||||
/** `data/.encryption_key`: the at-rest key when ENCRYPTION_KEY does not provide it. */
|
||||
readonly encryptionKeyFile: string;
|
||||
/**
|
||||
* The SQLite file a non-test process opens: TREK_DB_FILE when set, else
|
||||
* `data/travel.db`. Under NODE_ENV=test the connection is `:memory:` instead
|
||||
* (db/db-path.ts), which is a property of the connection, not of the layout.
|
||||
*/
|
||||
readonly dbFile: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* The data layout, derived from the server root and the live environment.
|
||||
*
|
||||
* The only variable involved is TREK_DB_FILE, read through `readEnv()` like
|
||||
* every other live value, so a test that points it somewhere else is seen on
|
||||
* the next call. Nothing here touches the disk: creating a directory is the
|
||||
* caller's decision.
|
||||
*/
|
||||
export function resolveDataPaths(
|
||||
env: { db: Pick<AppEnv['db'], 'trekDbFile'> } = readEnv(),
|
||||
root: string = SERVER_ROOT,
|
||||
): DataPaths {
|
||||
const dataDir = path.join(root, 'data');
|
||||
return {
|
||||
dataDir,
|
||||
uploadsDir: path.join(root, 'uploads'),
|
||||
backupsDir: path.join(dataDir, 'backups'),
|
||||
tmpDir: path.join(dataDir, 'tmp'),
|
||||
logsDir: path.join(dataDir, 'logs'),
|
||||
jwtSecretFile: path.join(dataDir, '.jwt_secret'),
|
||||
encryptionKeyFile: path.join(dataDir, '.encryption_key'),
|
||||
dbFile: env.db.trekDbFile || path.join(dataDir, 'travel.db'),
|
||||
};
|
||||
}
|
||||
@@ -25,3 +25,4 @@ export {
|
||||
} from './derive';
|
||||
export * from './parsers';
|
||||
export { envSchema } from './env.schema';
|
||||
export { resolveDataPaths, SERVER_ROOT, type DataPaths } from './data-paths';
|
||||
|
||||
@@ -1,10 +1,9 @@
|
||||
import crypto from 'node:crypto';
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { readEnv } from './app-config';
|
||||
import { resolveDataPaths } from './app-config/data-paths';
|
||||
|
||||
const dataDir = path.resolve(__dirname, '../data');
|
||||
const jwtSecretFile = path.join(dataDir, '.jwt_secret');
|
||||
const { dataDir, jwtSecretFile, encryptionKeyFile: encKeyFile } = resolveDataPaths();
|
||||
|
||||
// ENCRYPTION_KEY is used to derive at-rest encryption keys for stored secrets
|
||||
// (API keys, MFA TOTP secrets, SMTP password, OIDC client secret, etc.).
|
||||
@@ -20,7 +19,6 @@ const jwtSecretFile = path.join(dataDir, '.jwt_secret');
|
||||
// data/.encryption_key so JWT rotation can never break decryption later.
|
||||
// 4. Auto-generated — fresh install with none of the above; persisted to
|
||||
// data/.encryption_key.
|
||||
const encKeyFile = path.join(dataDir, '.encryption_key');
|
||||
let _encryptionKey: string = process.env.ENCRYPTION_KEY || '';
|
||||
|
||||
// Under test the suite sets its own fixed ENCRYPTION_KEY. Comparing it with, or
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { readEnv } from '../app-config';
|
||||
import { resolveDataPaths } from '../app-config/data-paths';
|
||||
|
||||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
@@ -19,16 +20,10 @@ export function resolveDbPath(): string {
|
||||
// parallel forks can't race on the same file or share migration state.
|
||||
if (readEnv().app.isTest) return ':memory:';
|
||||
|
||||
const explicit = readEnv().db.trekDbFile;
|
||||
if (explicit) {
|
||||
// Explicit DB file (used by the Playwright E2E harness to run against an
|
||||
// isolated, throwaway database instead of the real data/travel.db).
|
||||
const dir = path.dirname(explicit);
|
||||
if (!fs.existsSync(dir)) fs.mkdirSync(dir, { recursive: true });
|
||||
return explicit;
|
||||
}
|
||||
|
||||
const dataDir = path.join(__dirname, '../../data');
|
||||
if (!fs.existsSync(dataDir)) fs.mkdirSync(dataDir, { recursive: true });
|
||||
return path.join(dataDir, 'travel.db');
|
||||
// TREK_DB_FILE (the Playwright E2E harness runs against a throwaway file this
|
||||
// way) or data/travel.db, both resolved by the one data layout.
|
||||
const file = resolveDataPaths().dbFile;
|
||||
const dir = path.dirname(file);
|
||||
if (!fs.existsSync(dir)) fs.mkdirSync(dir, { recursive: true });
|
||||
return file;
|
||||
}
|
||||
|
||||
@@ -2,11 +2,11 @@ import fs from 'fs';
|
||||
import path from 'path';
|
||||
import { RequestContext, type EntityManager } from '@mikro-orm/core';
|
||||
import { readEnv } from '../app-config';
|
||||
import { resolveDataPaths } from '../app-config/data-paths';
|
||||
import { DemoRepository, type DemoAdminCredentialsRow, type DemoInstanceKeyRow } from '../db/repositories/DemoRepository';
|
||||
import { DatabaseConnectionLostError, type DatabaseBackupStrategy } from '../nest/database/database-backup.interface';
|
||||
|
||||
const dataDir = path.join(__dirname, '../../data');
|
||||
const baselinePath = path.join(dataDir, 'travel-baseline.db');
|
||||
const baselinePath = path.join(resolveDataPaths().dataDir, 'travel-baseline.db');
|
||||
|
||||
/**
|
||||
* The active fork's `EntityManager`, resolved via `RequestContext.getEntityManager()`
|
||||
|
||||
+6
-7
@@ -4,7 +4,6 @@ import 'dotenv/config';
|
||||
// variable aborts before any other module runs its import-time side effects
|
||||
// (config.ts key resolution, ...).
|
||||
import './app-config/boot-validate';
|
||||
import path from 'node:path';
|
||||
import fs from 'node:fs';
|
||||
import http from 'node:http';
|
||||
import type { INestApplication } from '@nestjs/common';
|
||||
@@ -16,6 +15,10 @@ import type { INestApplication } from '@nestjs/common';
|
||||
import type * as Bootstrap from './bootstrap';
|
||||
import type { DatabaseLifecycle } from './nest/database/database-lifecycle.service';
|
||||
|
||||
import { getAppUrl, getMcpSafeUrl, readEnv } from './app-config';
|
||||
import { resolveDataPaths } from './app-config/data-paths';
|
||||
import { resolveDbPath } from './db/db-path';
|
||||
|
||||
// data/tmp is the driver-agnostic global scratch dir (restore-upload spool,
|
||||
// mirror stream staging) and stays boot-created here. Driver-owned roots — the
|
||||
// uploads/ category tree and data/backups — are ensured by LocalDriver.init on
|
||||
@@ -24,12 +27,9 @@ import type { DatabaseLifecycle } from './nest/database/database-lifecycle.servi
|
||||
// fails loudly at startup, inside app.init(), instead of as a stray 500 on
|
||||
// first upload. The Dockerfile `mkdir -p` list is pinned to the registry's
|
||||
// category prefixes by tests/unit/uploads-dirs.test.ts.
|
||||
const tmpDir = path.join(__dirname, '../data/tmp');
|
||||
const { tmpDir } = resolveDataPaths();
|
||||
if (!fs.existsSync(tmpDir)) fs.mkdirSync(tmpDir, { recursive: true });
|
||||
|
||||
import { getAppUrl, getMcpSafeUrl, readEnv } from './app-config';
|
||||
import { resolveDbPath } from './db/db-path';
|
||||
|
||||
const PORT = readEnv().app.port;
|
||||
const HOST = readEnv().app.host;
|
||||
const APP_VERSION: string = readEnv().app.appVersion || (require('../package.json') as { version: string }).version;
|
||||
@@ -130,11 +130,10 @@ async function bootstrap(): Promise<void> {
|
||||
async function restoreBeforeTheDatabaseOpens() {
|
||||
const { restoreOnFirstBoot } = require('./nest/backup/boot-restore') as typeof import('./nest/backup/boot-restore');
|
||||
const env = readEnv();
|
||||
const dataDir = path.join(__dirname, '../data');
|
||||
return restoreOnFirstBoot({
|
||||
archive: env.backup.restoreFromBackup,
|
||||
dbFile: resolveDbPath(),
|
||||
dataDir,
|
||||
dataDir: resolveDataPaths(env).dataDir,
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -3,7 +3,6 @@ import { InjectRepository } from '@mikro-orm/nestjs';
|
||||
import bcrypt from 'bcryptjs';
|
||||
import crypto from 'crypto';
|
||||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
import { ADDON_IDS, MCP_GATED_ADDON_IDS } from '../../addons';
|
||||
import { readEnv } from '../../app-config';
|
||||
import { updateJwtSecret } from '../../config';
|
||||
@@ -23,6 +22,7 @@ import { getPhotoProviderConfig } from '../memories/memories.helpers';
|
||||
import { validatePassword } from '../common/passwordPolicy';
|
||||
import { UserCleanupService } from '../auth/user-cleanup.service';
|
||||
import { UnitOfWork } from '../database/unit-of-work';
|
||||
import { DataPathsService } from '../app-config/data-paths.service';
|
||||
import { AddonsService } from '../addons/addons.service';
|
||||
import { RealtimeService } from '../realtime/realtime.service';
|
||||
import { PasskeyService } from '../auth/passkey.service';
|
||||
@@ -130,6 +130,7 @@ export class AdminService {
|
||||
private readonly realtime: RealtimeService,
|
||||
private readonly uow: UnitOfWork,
|
||||
@Inject(DATABASE_BACKUP) private readonly database: DatabaseBackupStrategy,
|
||||
private readonly dataPaths: DataPathsService,
|
||||
) {}
|
||||
|
||||
// ── User CRUD ──────────────────────────────────────────────────────────────
|
||||
@@ -777,9 +778,7 @@ export class AdminService {
|
||||
|
||||
rotateJwtSecret(): { error?: string; status?: number } {
|
||||
const newSecret = crypto.randomBytes(32).toString('hex');
|
||||
// Re-anchored one directory deeper for nest/admin/ (was '../../data' in services/).
|
||||
const dataDir = path.resolve(__dirname, '../../../data');
|
||||
const secretFile = path.join(dataDir, '.jwt_secret');
|
||||
const { dataDir, jwtSecretFile: secretFile } = this.dataPaths;
|
||||
try {
|
||||
if (!fs.existsSync(dataDir)) fs.mkdirSync(dataDir, { recursive: true });
|
||||
fs.writeFileSync(secretFile, newSecret, { mode: 0o600 });
|
||||
|
||||
@@ -21,6 +21,7 @@ import { Global, Module } from '@nestjs/common';
|
||||
import { ConfigModule as NestConfigModule } from '@nestjs/config';
|
||||
import { BOOT_STABLE_TOKENS } from './tokens';
|
||||
import { RuntimeEnvService } from './runtime-env.service';
|
||||
import { DataPathsService } from './data-paths.service';
|
||||
|
||||
@Global()
|
||||
@Module({
|
||||
@@ -32,7 +33,7 @@ import { RuntimeEnvService } from './runtime-env.service';
|
||||
load: BOOT_STABLE_TOKENS,
|
||||
}),
|
||||
],
|
||||
providers: [RuntimeEnvService],
|
||||
exports: [RuntimeEnvService],
|
||||
providers: [RuntimeEnvService, DataPathsService],
|
||||
exports: [RuntimeEnvService, DataPathsService],
|
||||
})
|
||||
export class AppConfigModule {}
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
import { resolveDataPaths, type DataPaths } from '../../app-config/data-paths';
|
||||
|
||||
/**
|
||||
* The data layout for Nest classes: the data directory, uploads, backups,
|
||||
* scratch space, logs, the key files and the database file.
|
||||
*
|
||||
* Code that runs before the container exists (config.ts key resolution, the
|
||||
* database connection, index.ts, the file logger) calls `resolveDataPaths()`
|
||||
* directly; this provider hands the same answer to everything that can take a
|
||||
* constructor argument, so neither side counts `..` hops from its own file.
|
||||
* Resolved once per built app: none of these move while a process runs.
|
||||
*/
|
||||
@Injectable()
|
||||
export class DataPathsService implements DataPaths {
|
||||
readonly dataDir: string;
|
||||
readonly uploadsDir: string;
|
||||
readonly backupsDir: string;
|
||||
readonly tmpDir: string;
|
||||
readonly logsDir: string;
|
||||
readonly jwtSecretFile: string;
|
||||
readonly encryptionKeyFile: string;
|
||||
readonly dbFile: string;
|
||||
|
||||
constructor() {
|
||||
const paths = resolveDataPaths();
|
||||
this.dataDir = paths.dataDir;
|
||||
this.uploadsDir = paths.uploadsDir;
|
||||
this.backupsDir = paths.backupsDir;
|
||||
this.tmpDir = paths.tmpDir;
|
||||
this.logsDir = paths.logsDir;
|
||||
this.jwtSecretFile = paths.jwtSecretFile;
|
||||
this.encryptionKeyFile = paths.encryptionKeyFile;
|
||||
this.dbFile = paths.dbFile;
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,6 @@
|
||||
export { AppConfigModule } from './app-config.module';
|
||||
export { RuntimeEnvService } from './runtime-env.service';
|
||||
export { DataPathsService } from './data-paths.service';
|
||||
export {
|
||||
httpConfig,
|
||||
sessionConfig,
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { readEnv } from '../../app-config';
|
||||
import { resolveDataPaths } from '../../app-config/data-paths';
|
||||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
|
||||
@@ -30,7 +31,9 @@ const C = {
|
||||
|
||||
// ── File logger with rotation ─────────────────────────────────────────────
|
||||
|
||||
const logsDir = path.join(process.cwd(), 'data/logs');
|
||||
// From the data layout, like every other data path. It used to be cwd-based,
|
||||
// which agreed only because the image and `npm run dev` both start in server/.
|
||||
const logsDir = resolveDataPaths().logsDir;
|
||||
const logFilePath = path.join(logsDir, 'trek.log');
|
||||
let logsDirReady = false;
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@ import path from 'node:path';
|
||||
import fs from 'node:fs';
|
||||
import { logInfo, logError } from '../audit/audit-log.logger';
|
||||
import type { StorageService } from '../storage/storage.service';
|
||||
import { resolveDataPaths } from '../../app-config/data-paths';
|
||||
|
||||
/**
|
||||
* Auto-backup settings and retention — the pure half of the auto-backup cron
|
||||
@@ -13,7 +14,7 @@ import type { StorageService } from '../storage/storage.service';
|
||||
* replicas too.
|
||||
*/
|
||||
|
||||
const dataDir = path.join(__dirname, '../../../data');
|
||||
const { dataDir } = resolveDataPaths();
|
||||
const settingsFile = path.join(dataDir, 'backup-settings.json');
|
||||
|
||||
export const VALID_INTERVALS = ['hourly', 'daily', 'weekly', 'monthly'];
|
||||
|
||||
@@ -2,6 +2,7 @@ import archiver from 'archiver';
|
||||
import path from 'path';
|
||||
import { pipeline } from 'node:stream/promises';
|
||||
import { readEnv } from '../../app-config';
|
||||
import { resolveDataPaths } from '../../app-config/data-paths';
|
||||
import fs from 'fs';
|
||||
import { logError, logWarn } from '../audit/audit-log.logger';
|
||||
import type { DatabaseBackupStrategy } from '../database/database-backup.interface';
|
||||
@@ -33,7 +34,7 @@ const describeError = (err: unknown): string => (err instanceof Error ? err.mess
|
||||
|
||||
// The scratch and key directory. The database itself may live elsewhere
|
||||
// (TREK_DB_FILE); the database port knows where.
|
||||
const dataDir = path.join(__dirname, '../../../data');
|
||||
const { dataDir, encryptionKeyFile } = resolveDataPaths();
|
||||
const PRECOMPRESSED = /\.(jpe?g|png|webp|gif|heic|heif|avif|mp4|mov|m4v|webm|pdf|zip|gz)$/i;
|
||||
|
||||
// Compressed upload cap for restore archives. Defaults to 500 MB, raisable via
|
||||
@@ -281,7 +282,7 @@ export async function createBackup(
|
||||
// them. NOTE: this makes the backup file as sensitive as the key itself —
|
||||
// store/transfer it securely. Skipped when ENCRYPTION_KEY is provided via
|
||||
// env, since in that case the file is not the source of truth.
|
||||
const encKeyPath = path.join(dataDir, '.encryption_key');
|
||||
const encKeyPath = encryptionKeyFile;
|
||||
if (!readEnv().backup.encryptionKeyFromEnv && fs.existsSync(encKeyPath)) {
|
||||
archive.file(encKeyPath, { name: '.encryption_key' });
|
||||
}
|
||||
@@ -438,7 +439,7 @@ export async function restoreFromZip({ storage, database }: BackupDeps, zipPath:
|
||||
// still overrides the file).
|
||||
const extractedEncKey = path.join(extractDir, '.encryption_key');
|
||||
if (fs.existsSync(extractedEncKey)) {
|
||||
fs.copyFileSync(extractedEncKey, path.join(dataDir, '.encryption_key'));
|
||||
fs.copyFileSync(extractedEncKey, encryptionKeyFile);
|
||||
}
|
||||
} finally {
|
||||
// The restored DB has different permission-override rows from
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import path from 'path';
|
||||
import { readEnv } from '../../app-config';
|
||||
import { resolveDataPaths } from '../../app-config/data-paths';
|
||||
|
||||
/**
|
||||
* File-domain constants and pure helpers, split out of FilesService because
|
||||
@@ -43,10 +44,8 @@ export const BLOCKED_EXTENSIONS = [
|
||||
// Executables
|
||||
'.exe', '.bat', '.sh', '.cmd', '.msi', '.dll', '.com', '.vbs', '.ps1', '.app',
|
||||
];
|
||||
// One directory level deeper than the legacy src/services/fileService.ts, so
|
||||
// the extra '..' keeps the same absolute <server>/uploads/files under both the
|
||||
// src (vitest) and dist (runtime) layouts.
|
||||
export const filesDir = path.join(__dirname, '../../../uploads/files');
|
||||
/** `<server>/uploads/files`, from the one data layout. */
|
||||
export const filesDir = path.join(resolveDataPaths().uploadsDir, 'files');
|
||||
|
||||
/**
|
||||
* Whether a trip-file upload named `originalname` passes the extension rules:
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { readEnv } from '../../app-config';
|
||||
import { resolveDataPaths } from '../../app-config/data-paths';
|
||||
|
||||
/**
|
||||
* Filesystem layout for the plugin system (#plugins). Code and data are two
|
||||
@@ -12,7 +13,7 @@ import { readEnv } from '../../app-config';
|
||||
* so a Docker deployment can point them at dedicated volumes.
|
||||
*/
|
||||
|
||||
const DATA_ROOT = path.resolve(__dirname, '../../../data');
|
||||
const DATA_ROOT = resolveDataPaths().dataDir;
|
||||
|
||||
// Read lazily so a deployment (or a test) can point these at dedicated volumes
|
||||
// via env without import-order surprises.
|
||||
|
||||
@@ -1,23 +1,20 @@
|
||||
import path from 'node:path';
|
||||
import { resolveDataPaths } from '../../app-config/data-paths';
|
||||
|
||||
/**
|
||||
* The storage domain's one `__dirname` anchor (the uploads-root.ts idiom, and
|
||||
* eventually its replacement: the registry's defaults make this the single
|
||||
* source of truth the other 16 copies collapse onto in later slices).
|
||||
*
|
||||
* `__dirname` moves with the file — `server/{src,dist}/nest/storage` sits
|
||||
* three levels under `server/`, so three `..` hops resolve to `<server>/`
|
||||
* under both the src (vitest) and dist (runtime) layouts. The depth is stated
|
||||
* rather than counted by hand and pinned by storage-keys.test.ts, exactly
|
||||
* like uploads-root.test.ts pins UPLOADS_ROOT. In Docker both anchors are
|
||||
* symlinks (`/app/server/uploads → /app/uploads`, `/app/server/data →
|
||||
* /app/data`), which is why LocalDriver realpaths its root at init.
|
||||
* The storage domain's view of the data layout. Both roots come from
|
||||
* `resolveDataPaths()` (src/app-config/data-paths.ts), the one place the server
|
||||
* anchors `data/` and `uploads/`, so this file no longer counts `..` hops of
|
||||
* its own. In Docker both are symlinks (`/app/server/uploads -> /app/uploads`,
|
||||
* `/app/server/data -> /app/data`), which is why LocalDriver realpaths its root
|
||||
* at init. storage-keys.test.ts pins the resolved values.
|
||||
*/
|
||||
export const DEFAULT_UPLOADS_ROOT = path.resolve(__dirname, '..', '..', '..', 'uploads');
|
||||
export const DATA_ROOT = path.resolve(__dirname, '..', '..', '..', 'data');
|
||||
export const DEFAULT_BACKUPS_ROOT = path.join(DATA_ROOT, 'backups');
|
||||
const LAYOUT = resolveDataPaths();
|
||||
export const DEFAULT_UPLOADS_ROOT = LAYOUT.uploadsDir;
|
||||
export const DATA_ROOT = LAYOUT.dataDir;
|
||||
export const DEFAULT_BACKUPS_ROOT = LAYOUT.backupsDir;
|
||||
/** Driver-agnostic global scratch space (`data/tmp`) — see StorageService.tempDir(). */
|
||||
export const GLOBAL_TEMP_DIR = path.join(DATA_ROOT, 'tmp');
|
||||
export const GLOBAL_TEMP_DIR = LAYOUT.tmpDir;
|
||||
/** Seed-once boot provisioning file — imported only when no storage.* row exists. */
|
||||
const DEFAULT_SEED_CONFIG_PATH = path.join(DATA_ROOT, 'storage-config.json');
|
||||
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
/**
|
||||
* The data layout: one anchor for data/, uploads/ and the files inside them,
|
||||
* shared by the pre-container code (resolveDataPaths) and the Nest classes
|
||||
* (DataPathsService). These cases pin where each path lands, because a moved
|
||||
* anchor silently moves the database, the key files or the backups.
|
||||
*/
|
||||
import { describe, it, expect, afterEach, vi } from 'vitest';
|
||||
import path from 'node:path';
|
||||
import { resolveDataPaths, SERVER_ROOT } from '../../../src/app-config/data-paths';
|
||||
import { DataPathsService } from '../../../src/nest/app-config/data-paths.service';
|
||||
import { DEFAULT_BACKUPS_ROOT, DEFAULT_UPLOADS_ROOT, DATA_ROOT, GLOBAL_TEMP_DIR } from '../../../src/nest/storage/storage-paths';
|
||||
import { filesDir } from '../../../src/nest/files/files.constants';
|
||||
|
||||
const SERVER_DIR = path.resolve(__dirname, '..', '..', '..');
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllEnvs();
|
||||
});
|
||||
|
||||
describe('resolveDataPaths', () => {
|
||||
it('DATAPATHS-001: anchors on the server package directory, not on the working directory', () => {
|
||||
expect(SERVER_ROOT).toBe(SERVER_DIR);
|
||||
const paths = resolveDataPaths({ db: { trekDbFile: undefined } });
|
||||
expect(paths.dataDir).toBe(path.join(SERVER_DIR, 'data'));
|
||||
expect(paths.uploadsDir).toBe(path.join(SERVER_DIR, 'uploads'));
|
||||
});
|
||||
|
||||
it('DATAPATHS-002: places backups, scratch space, logs and the key files inside data/', () => {
|
||||
const paths = resolveDataPaths({ db: { trekDbFile: undefined } }, '/srv/trek');
|
||||
expect(paths).toEqual({
|
||||
dataDir: path.join('/srv/trek', 'data'),
|
||||
uploadsDir: path.join('/srv/trek', 'uploads'),
|
||||
backupsDir: path.join('/srv/trek', 'data', 'backups'),
|
||||
tmpDir: path.join('/srv/trek', 'data', 'tmp'),
|
||||
logsDir: path.join('/srv/trek', 'data', 'logs'),
|
||||
jwtSecretFile: path.join('/srv/trek', 'data', '.jwt_secret'),
|
||||
encryptionKeyFile: path.join('/srv/trek', 'data', '.encryption_key'),
|
||||
dbFile: path.join('/srv/trek', 'data', 'travel.db'),
|
||||
});
|
||||
});
|
||||
|
||||
it('DATAPATHS-003: TREK_DB_FILE moves only the database file', () => {
|
||||
const paths = resolveDataPaths({ db: { trekDbFile: '/elsewhere/e2e.db' } }, '/srv/trek');
|
||||
expect(paths.dbFile).toBe('/elsewhere/e2e.db');
|
||||
expect(paths.dataDir).toBe(path.join('/srv/trek', 'data'));
|
||||
});
|
||||
|
||||
it('DATAPATHS-004: reads TREK_DB_FILE from the live environment by default', () => {
|
||||
vi.stubEnv('TREK_DB_FILE', '/tmp/live.db');
|
||||
expect(resolveDataPaths().dbFile).toBe('/tmp/live.db');
|
||||
vi.stubEnv('TREK_DB_FILE', '');
|
||||
expect(resolveDataPaths().dbFile).toBe(path.join(SERVER_DIR, 'data', 'travel.db'));
|
||||
});
|
||||
});
|
||||
|
||||
describe('every consumer resolves the same layout', () => {
|
||||
it('DATAPATHS-005: the storage defaults and the trip-files directory come from it', () => {
|
||||
const paths = resolveDataPaths();
|
||||
expect(DATA_ROOT).toBe(paths.dataDir);
|
||||
expect(DEFAULT_UPLOADS_ROOT).toBe(paths.uploadsDir);
|
||||
expect(DEFAULT_BACKUPS_ROOT).toBe(paths.backupsDir);
|
||||
expect(GLOBAL_TEMP_DIR).toBe(paths.tmpDir);
|
||||
expect(filesDir).toBe(path.join(paths.uploadsDir, 'files'));
|
||||
});
|
||||
|
||||
it('DATAPATHS-006: DataPathsService hands Nest classes the same answer', () => {
|
||||
vi.stubEnv('TREK_DB_FILE', '/tmp/service.db');
|
||||
const service = new DataPathsService();
|
||||
expect({ ...service }).toEqual(resolveDataPaths());
|
||||
expect(service.dbFile).toBe('/tmp/service.db');
|
||||
});
|
||||
});
|
||||
@@ -72,6 +72,7 @@ import { BudgetService } from '../../../src/nest/budget/budget.service';
|
||||
import { ExchangeRatesService } from '../../../src/nest/budget/exchange-rates.service';
|
||||
import { NotificationsService } from '../../../src/nest/notifications/notifications.service';
|
||||
import { AdminService } from '../../../src/nest/admin/admin.service';
|
||||
import { DataPathsService } from '../../../src/nest/app-config/data-paths.service';
|
||||
import { VersionCheckJob } from '../../../src/nest/admin/version-check.job';
|
||||
import type { CronRegistrarService } from '../../../src/nest/scheduling/cron-registrar.service';
|
||||
import type { RuntimeEnvService } from '../../../src/nest/app-config/runtime-env.service';
|
||||
@@ -174,6 +175,7 @@ beforeAll(async () => {
|
||||
realtime,
|
||||
await createTestUnitOfWork(testDb),
|
||||
databaseBackupStub,
|
||||
new DataPathsService(),
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -54,6 +54,7 @@ import { BudgetService } from '../../../src/nest/budget/budget.service';
|
||||
import { ExchangeRatesService } from '../../../src/nest/budget/exchange-rates.service';
|
||||
import { NotificationsService } from '../../../src/nest/notifications/notifications.service';
|
||||
import { AdminService } from '../../../src/nest/admin/admin.service';
|
||||
import { DataPathsService } from '../../../src/nest/app-config/data-paths.service';
|
||||
import { makeNotificationsService, makeNotificationPreferencesService } from '../../helpers/notifications';
|
||||
import { EphemeralTokenService } from '../../../src/nest/auth/ephemeral-token.service';
|
||||
import { AllowedFileTypesService } from '../../../src/nest/files/allowed-file-types.service';
|
||||
@@ -139,6 +140,7 @@ beforeAll(async () => {
|
||||
realtime,
|
||||
await createTestUnitOfWork(testDb),
|
||||
databaseBackupStub,
|
||||
new DataPathsService(),
|
||||
);
|
||||
});
|
||||
const checkAndNotifyVersion = () => svc.checkAndNotifyVersion();
|
||||
|
||||
Reference in New Issue
Block a user