mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-10-19 13:36:48 +00:00
3a0604b3f5
* feat(config): put the operator-owned surfaces behind TREK_MANAGED The switch landed last release with nothing behind it. This is what it turns off, plus the four fixes that came out of walking every one of these surfaces. Nothing here changes anything for an install that does not set the flag. That is not a hope, it is what managed-inert.test.ts asserts: the mode stays off for any unparseable value, no other variable can turn it on, every settings key stays writable, and splitManagedKeys hands back the same object rather than a copy so no write path can quietly diverge on self-hosted installs only. WHAT THE MODE WITHHOLDS Fourteen routes carry @ManagedForbidden: key validation, static MCP tokens, JWT rotation, the OIDC admin pair, restore and auto-backup, plugin sideload and reload, the Ollama model list and pull, and the SMTP test send. Each names the reason it is not the instance admin's, and the reason is mandatory. Eighteen settings keys are filtered instead of refused. SMTP, the OIDC toggles and the WebAuthn pair share one handler with keys the admin does own, and the llm_* family is reachable through per-user settings, the instance defaults and the addon config alike, so a route-level refusal would be both too coarse and incomplete. A locked key is skipped and reported as managed_keys; the rest of the save goes through. Refusing the request would take the registration toggles down with the SMTP block. GET /api/auth/me/settings stops returning the three API keys in the clear. It is the only endpoint that decrypts a stored credential back to the caller, which is fine when the admin pasted it in and wrong when the operator supplied it. Two multipart routes check in the handler rather than the guard: a guard throws before the parser drains the body and the client gets an ECONNRESET instead of the 403 (PROFILE-015). They still carry the marker so the boot gate sees them. WHAT KEEPS IT FROM DRIFTING validate-managed-routes.ts refuses to boot on a marking that is not on the reviewed list, a list entry whose route lost its marking, or a marking with no reason. The third case has no equivalent in the guard gate next door and is the one that matters most: a blank reason destroys the only record of why a control was taken away, and nothing recovers that later. It found two wrong entries in my own list while I was writing this. managed-keys.test.ts asserts that every key in ADMIN_SETTINGS_KEYS and DEFAULTABLE_USER_SETTING_KEYS belongs to exactly one of two sets. A key added to either list belongs to neither until somebody decides, and that is the failure this prevents: the lists grow, the mode does not. PLACES_API_BASE AND PLACES_API_KEY The nine Places endpoints spell out googleapis.com, so an install that wants those calls to leave through something of its own has no way to say so. One variable, substituted where every call already funnels through. A second one supplies the key, and when it is set the users table is not consulted at all — a per-user key would route around whatever the endpoint counts, and those columns are readable by any admin of the instance. FOUR FIXES THAT ARE NOT ABOUT THE MODE An admin setting somebody's password now bumps password_version and clears their MCP tokens. An admin sets a password for one reason, and until now verifyJwtAndLoadUser kept accepting every cookie the intruder already had: the one action taken to lock them out was the one action that did not. The four OIDC provider calls go through the SSRF guard. Three are worse than they look — the token, userinfo and JWKS URLs come out of the discovery response, so they are chosen by the other end, not by an admin. safeFetchLlm is renamed safeFetchAdminConfigured and keeps its old name as an alias, because it is the variant that allows loopback and LAN: a self-hoster with Authentik on localhost:9000 must keep working. The token exchange follows no redirect at all, since one would hand client_secret to a second host. DELETE /api/admin/users/:id/mfa clears another account's TOTP. The passkey half has existed since passkeys landed; the TOTP half never did, which left "somebody on the trip lost their phone" with no answer short of an operator in the database. Never for the caller themselves. TREK_PLUGIN_PERMISSIONS=off refuses to boot when the install is centrally administered. A warning is right for a machine its owner chose to trust; where the person who set the flag and the people whose data is in the database are not the same person, fail closed. TEXTS Four strings lose the phrase "self-hosted" because they describe somebody else's software — an Ollama endpoint, an AirTrail instance, a service a plugin talks to — and read better without it in both modes. The one string that describes TREK itself keeps it, and the mode picks a second key instead, so self-hosters lose nothing. A grep gate in shared/ keeps the canonical locale honest, because the parity check compares the file set and the top-level keys and never looks at a value: re-wording a string is invisible to CI today, and so is putting it back. * feat(maps): supply the map credential with the instance when centrally administered MAPBOX_ACCESS_TOKEN joins PLACES_API_KEY as an operator-supplied value. SettingsService injects it when the settings are read, and picks GL as the provider if the reader is still on Leaflet, since a token is always present. The key moves from the customer set to the locked set. It is the one locked key whose value is public by design — a pk.* token has to reach the browser to be useful — so this is not about secrecy: a per-user save landing on top of the instance token would only break the map for that user. Both settings fields for it go away with it. * fix(auth): count the operator key when reporting whether Google features work has_maps_key only looked at the users table, but getMapsKey prefers the PLACES_API_KEY env over anything stored there. On an install that supplies the key that way and has an empty column, every lookup worked while the client was told there was no key and hid the search, the autocomplete and the place photos. Found by walking the running instance rather than the code: the map came up, the search box did not. * feat(admin): hide the settings a centrally administered install cannot act on A control that answers 403 is worse than no control: the AI section rendered the refusal as "Request failed with status code 403", which tells an admin they broke something rather than that the decision is not theirs. So the surfaces go rather than being disabled in place: - AI parsing and AirTrail leave the addon list, and the AI addon route now refuses outright. The endpoint, the model and what a document costs come with the instance, so there is no instance decision left — not even whether it runs. - Immich and Synology Photos leave it too. Both connect to a server the reader runs at home, and there is no route from here to one. - The SMTP block, the admin webhook and ntfy targets, and the passkey RP ID and origins follow the same rule: they configure something the operator owns, and a wrong RP ID on a shared parent domain reaches past this instance entirely. - The About tab goes. It is about the project — what TREK is, where to file a bug, where to support it — and is aimed at the person running the software. The version and the link to the source move to the sidebar footer, on both the desktop page and the mobile twin, because AGPL 13 wants that offer wherever people reach the software over a network. - The update check stops asking GitHub and reports nothing available. Separately, the version footers lose the "self-hosted" suffix everywhere, not just here. It read as a label on the software rather than on the install, and the version alone says what it needs to. * feat(config): refuse to boot centrally administered without an at-rest key Measured on a running instance rather than assumed: with ENCRYPTION_KEY unset, the at-rest key falls back to a file in the data volume, and backupService bundles that file into every archive it builds. The archive is downloadable by any instance admin, so the encryption ships with its own key in the same zip. The skip in backupService already handles this — it only omits the file when the key came from the env — but nothing made sure the env actually carried one. Provisioning sets it; forgetting it was silent. validateEnvAtBoot now carries a second kind of check next to the schema: cross- field rules that only apply with the flag set, for combinations where every value is individually valid and the result is wrong. Empty without the flag, so an install that never asked to be centrally administered cannot meet it. * feat(client): add an attachment point for install-specific screens Two empty exports on the client, an empty module on the server, and the four places that read them. Nothing here adds a route, a nav entry or an endpoint — the arrays are empty and the module contributes nothing, which managed-ext.test pins so it stays that way. Read unconditionally, never behind a flag. On the server a conditional import would tie the module graph to an environment variable, and the graph is built once per app while the flag is read live; on the client an empty array costs nothing to spread, so every build walks the same path instead of one build walking a path nobody tests. Anything attached here follows the same rules as the rest of src/nest and src/components: a real module with a controller and a provider, bodies through the Zod pipe, routes under the global guards, tests. * test(shared): move the wording gate out of src so shared typechecks again CI installs shared with its own dependencies only (npm ci --workspace shared), so there is no @types/node and the `node:fs` import inside src/ failed to compile. The tests passed the whole time — vitest supplies the module — which is why this only showed up in the typecheck step. The walk moves to scripts/, next to i18n-parity.mjs, and the spec imports it the same way that one does. It also runs on its own now (`node scripts/i18n-selfhost-wording.mjs`), which is what you want when CI names a line you have to go find. * test(server): repair the call sites typecheck:tests caught tests/ sits outside the build tsconfig, so none of this showed up in a green test run — which is exactly the gap `npm run typecheck:tests` exists to close. Three kinds, all mine: - Seven `import type` paths for RuntimeEnvService carried one `../` too many. Type-only imports are erased, so vitest never resolved them and nothing failed. - VersionCheckJob and PluginsController grew a constructor parameter and eleven hand-wired `new` calls kept passing the old count. The argument arrived as undefined and no assertion happened to read it. - ConditionContext grew `managed`, and the shared context object in the conditions suite did not. The condition also had no test of its own, so it has one now: both directions, since the registry uses `is: false` to keep a notice away and a notice that only belongs on a centrally administered install would use `is: true`.
140 lines
8.2 KiB
TypeScript
140 lines
8.2 KiB
TypeScript
import nodeHttp from 'node:http';
|
|
import express from 'express';
|
|
import type { Request, Response, NextFunction, RequestHandler } from 'express';
|
|
import { NestFactory } from '@nestjs/core';
|
|
import { ExpressAdapter } from '@nestjs/platform-express';
|
|
import type { INestApplication } from '@nestjs/common';
|
|
import type { ConfigType } from '@nestjs/config';
|
|
import { AppModule } from './nest/app.module';
|
|
import { httpConfig } from './nest/app-config';
|
|
import { applyGlobalMiddleware } from './middleware/globalMiddleware';
|
|
import { applyPlatformUploads, applyPlatformStatic } from './nest/platform/platform.routes';
|
|
import { apiDocsEnabled } from './nest/common/api-docs.kill-switch';
|
|
import { setupApiDocs } from './nest/platform/api-docs';
|
|
import { MCP_METADATA_MIDDLEWARE } from './nest/platform/mcp-metadata.middleware';
|
|
import { validateBodyContracts } from './nest/common/validate-body-contracts';
|
|
import { validateRouteGuards } from './nest/common/validate-route-guards';
|
|
import { validateManagedRoutes } from './nest/common/validate-managed-routes';
|
|
import { TrekWsAdapter } from './nest/realtime/trek-ws.adapter';
|
|
|
|
/**
|
|
* Builds the unified TREK NestJS application that serves the ENTIRE surface — the
|
|
* former Express app is gone. One builder is shared by the production bootstrap
|
|
* (index.ts) and the integration-test harness so the two can never drift.
|
|
*
|
|
* Composition order is load-bearing. The few remaining pre-`app.init()` pieces
|
|
* are registered on the underlying Express instance first, because Nest's
|
|
* router terminates an unmatched request by throwing NotFoundException — it
|
|
* does NOT fall through to a route registered after init, so a post-init
|
|
* Express route is unreachable. Each pre-init piece matches its own requests
|
|
* and `next()`s everything else through to the Nest controllers:
|
|
*
|
|
* 1. applyGlobalMiddleware — helmet/CSP, CORS, HSTS, forced-HTTPS, request
|
|
* logging + cookie-parser.
|
|
* 2. applyPlatformUploads — the static + guarded /uploads/* routes.
|
|
* 3. MCP_METADATA_MIDDLEWARE — the SDK's OAuth discovery router + addon
|
|
* gate, container-built (DI) but mounted pathless here (see below).
|
|
* 4. applyPlatformStatic — the production built-client static assets (so a
|
|
* real asset request returns the file before the Nest router 404s it).
|
|
* 4b. setupApiDocs — Swagger UI/spec at /api/docs* when TREK_API_DOCS_ENABLED.
|
|
* 4c. the named body-parser wrappers — the global 100kb JSON/urlencoded
|
|
* parsers, exempting /mcp so the MCP SDK reads the raw stream.
|
|
* 5. app.init() — every domain controller, including the transport surface
|
|
* that used to live pre-init: /api/health (FeaturesController), OAuth
|
|
* discovery documents (DiscoveryController), the /oauth/consent COOP
|
|
* override (ConsentCoopMiddleware), /oauth/authorize + /oauth/register
|
|
* (SDK routers via OauthModule.configure) and /mcp (McpTransportModule).
|
|
*
|
|
* The SPA index.html fallback (unmatched GET → index.html in production) is the
|
|
* SpaFallbackFilter (APP_FILTER in AppModule); the global error envelope is the
|
|
* TrekExceptionFilter (also APP_FILTER).
|
|
*/
|
|
let boundHttpServer: nodeHttp.Server | null = null;
|
|
|
|
/**
|
|
* The http server buildApp created and bound the ws gateway to.
|
|
*
|
|
* index.ts listens on it; the websocket suites connect to it. Anyone creating a
|
|
* second server around the same express instance would get a working REST API
|
|
* and a /ws that no client can reach.
|
|
*/
|
|
export function getHttpServer(): nodeHttp.Server {
|
|
if (!boundHttpServer) throw new Error('buildApp() has not run yet');
|
|
return boundHttpServer;
|
|
}
|
|
|
|
export async function buildApp(): Promise<INestApplication> {
|
|
// rawBody keeps the unparsed request bytes on req.rawBody so a plugin webhook
|
|
// route can verify a provider's HMAC signature over the exact payload (the
|
|
// parsed JSON alone can't be re-serialised byte-for-byte).
|
|
const app = await NestFactory.create(AppModule, new ExpressAdapter(), { rawBody: true });
|
|
const instance = app.getHttpAdapter().getInstance();
|
|
// The http server is created HERE, not by the caller after buildApp returns,
|
|
// and that ordering is the whole point: Nest binds gateways during app.init(),
|
|
// so the ws adapter has to exist before it and has to already know which
|
|
// server to attach to. Handing it the app instead would bind the ws server to
|
|
// Nest's own internal http server, which this process never listens on: the
|
|
// boot succeeds, the gateway logs as registered, every test passes, and no
|
|
// browser can connect. Callers take the server from getHttpServer() below.
|
|
boundHttpServer = nodeHttp.createServer(instance);
|
|
app.useWebSocketAdapter(new TrekWsAdapter(boundHttpServer));
|
|
// ConfigModule.forRoot's load factories already ran inside NestFactory.create,
|
|
// so the boot-stable snapshot is resolvable here, BEFORE app.init() — this is
|
|
// the one bridge that lets the pre-init Express layer consume the validated
|
|
// config instead of reading process.env itself.
|
|
const http = app.get<ConfigType<typeof httpConfig>>(httpConfig.KEY);
|
|
applyGlobalMiddleware(instance, { http });
|
|
applyPlatformUploads(instance);
|
|
// The SDK discovery router (+ its addon gate). Container-built so its deps
|
|
// are injected (same pre-init consumption bridge as httpConfig above), but
|
|
// applied here as a PATHLESS app.use: the SDK router matches absolute
|
|
// /.well-known/* paths against req.url, and any Nest wildcard forRoutes()
|
|
// mount is an Express pattern mount that strips the matched prefix from
|
|
// req.url before the middleware runs.
|
|
instance.use(app.get<RequestHandler>(MCP_METADATA_MIDDLEWARE));
|
|
applyPlatformStatic(instance);
|
|
// Pin the request-body ceiling explicitly. The Express shell used to set
|
|
// '100kb' and stopped doing it when the Nest instance took over parsing, which
|
|
// left the limit implicit — the same number, but nowhere anybody would find
|
|
// it, and one NestFactory default away from silently becoming something else.
|
|
// The verify hook keeps req.rawBody, which the plugin webhook routes need to
|
|
// check a provider's HMAC over the exact payload (same hook Nest's own
|
|
// useBodyParser would install).
|
|
//
|
|
// /mcp is exempted and stays RAW: the MCP SDK's StreamableHTTPServerTransport
|
|
// reads the request stream itself (its own size handling and JSON-RPC error
|
|
// bodies), and the transport controller passes the still-undefined req.body
|
|
// through as handleRequest's parsedBody argument. No Nest route may ever
|
|
// @Body() on /mcp. The wrapper NAMES are load-bearing:
|
|
// ExpressAdapter.registerParserMiddleware skips its own init-time parsers
|
|
// only when layers named jsonParser/urlencodedParser already sit in the stack.
|
|
const rawBodyKeeper = (req: Request, _res: Response, buffer: Buffer) => {
|
|
if (Buffer.isBuffer(buffer)) (req as Request & { rawBody?: Buffer }).rawBody = buffer;
|
|
};
|
|
const json = express.json({ limit: '100kb', verify: rawBodyKeeper });
|
|
const urlencoded = express.urlencoded({ limit: '100kb', extended: true, verify: rawBodyKeeper });
|
|
const isMcp = (req: Request) => req.path === '/mcp' || req.path === '/mcp/';
|
|
instance.use(function jsonParser(req: Request, res: Response, next: NextFunction) {
|
|
return isMcp(req) ? next() : json(req, res, next);
|
|
});
|
|
instance.use(function urlencodedParser(req: Request, res: Response, next: NextFunction) {
|
|
return isMcp(req) ? next() : urlencoded(req, res, next);
|
|
});
|
|
if (apiDocsEnabled()) setupApiDocs(app);
|
|
await app.init();
|
|
// Fail closed on unvalidated mutation bodies: every POST/PUT/PATCH @Body()
|
|
// must carry a createZodDto class (validated by the global ZodValidationPipe)
|
|
// or sit on the ratchet-only legacy allow-list — otherwise refuse to boot.
|
|
validateBodyContracts(app);
|
|
// Fail closed on the anonymous surface too: a route that answers without a
|
|
// session must carry @Public() with a reason AND be on the reviewed list.
|
|
// Default-deny protects what exists; this is what keeps the exemptions honest.
|
|
validateRouteGuards(app);
|
|
// And on the other direction of the same question: a route that a centrally
|
|
// administered install withholds from its own admin must say why, and be on a
|
|
// list somebody reviewed. Runs in every e2e harness because they share this
|
|
// builder, so a forgotten marker fails in CI rather than at a customer.
|
|
validateManagedRoutes(app);
|
|
return app;
|
|
}
|