mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-10-18 21:16:47 +00:00
Maurice
88d196bae8
Most features have a desktop view and a phone view, and each carried its own copy of the feature's logic, so SonarCloud's duplication condition was the gate a pull request touching both failed most often, and only after a scan of twenty minutes. lint:dup finds the same copies before the push. It runs jscpd's detector (@jscpd/core with @jscpd/tokenizer) over the source files under src/ at the thresholds the SonarCloud gate applies to TypeScript, 100 tokens over at least 10 lines, and counts per file the lines inside a block that repeats elsewhere or in the same file. Tests and declaration files are left out as Sonar leaves them out, and .ts and .tsx are read as one language, so a hook that repeats a view's code is a copy too. The files are read sorted and with readText, so every machine counts the same. scripts/dup-baseline.json holds the files that still hold copies with their count; a file may hold no more than its entry, a file without an entry none, and an entry above its file or for a file that is gone fails until --update lowers it. --list prints every block with its other side. The engine packages are used instead of the jscpd command, which walks the tree in file system order, skips files past its size limits without a word and drops every result once it meets a file it has no format for.
1013 lines
39 KiB
YAML
1013 lines
39 KiB
YAML
name: Tests & SonarQube
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
# The repo's test/quality CI and SonarQube Cloud analysis in one workflow, so
|
|
# each test suite runs once per event instead of once for the old test.yml and
|
|
# once more for Sonar's coverage. Pushes to dev and main produce branch
|
|
# analysis; pull requests get PR analysis with decoration on the PR itself
|
|
# (the scanner picks the context up from the Actions environment).
|
|
#
|
|
# Job layout — three tiers:
|
|
# - Preflight: plugin-facts, i18n-parity and the quality jobs
|
|
# (server-quality, client-quality, plugin-sdk-quality) hold the cheap
|
|
# generated-code/typecheck/lint/smoke gates. Every test job `needs:` all
|
|
# of them, so a gate failure stops the run before the expensive coverage
|
|
# fan-out spends ~15 runner-minutes on a result the gates already sank.
|
|
# - Tests: the *-coverage jobs run the suites and upload lcov; s3-contract
|
|
# runs the MinIO contract suite. Coverage jobs run tests only — with the
|
|
# gates cleared in preflight, they can only fail because tests failed.
|
|
# client-e2e drives the real app in Chromium and WebKit, client-bundle
|
|
# builds the client and checks each map engine keeps a chunk of its own,
|
|
# and postgres-probe runs the dialect helpers and the repository
|
|
# statements it reaches against a Postgres service. None of them feeds the scan.
|
|
# - Scan `needs:` exactly the four coverage jobs. A coverage job failing
|
|
# must stop the scan — a missing lcov file would make every line of that
|
|
# package report as uncovered, and the dashboard would show a collapse
|
|
# that did not happen. (The preflight gates stop it transitively.)
|
|
# - Fork PRs and runs Dependabot triggers don't receive the repo's Actions
|
|
# secrets, so the two jobs that need them skip themselves: s3-contract
|
|
# (MINIO_LICENSE_KEY) and scan (SONAR_TOKEN). ci-ok counts a skip as
|
|
# passed. Everything else runs on those PRs so outside contributors and
|
|
# dependency bumps still get the full test signal.
|
|
# - changes runs first and decides which of the jobs below a change needs
|
|
# (scripts/ci/changed-areas.mjs holds the rules). A docs-only push or PR
|
|
# runs nothing else. The filter lives in a job, not in `on.paths`, because
|
|
# a workflow skipped by its path filter never reports a check, and a
|
|
# required check that never reports blocks the PR for good.
|
|
# - docker-image builds the Dockerfile and boots the result, and
|
|
# deploy-manifests lints and renders the Helm chart and checks the
|
|
# compose files. Both run beside the preflight rather than after it.
|
|
# - ci-ok `needs:` every gate, test, image and chart job, and is the one
|
|
# check the branch ruleset requires. It fails when any of them failed or
|
|
# was cancelled and passes when they passed or were skipped (by changes,
|
|
# or on a fork PR without secrets). The scan is not in it: the quality
|
|
# gate reports on the PR by itself, and a SonarCloud outage should not
|
|
# block a merge.
|
|
#
|
|
# Automatic Analysis must stay disabled on sonarcloud.io — it is mutually
|
|
# exclusive with CI-based analysis, and it cannot import the lcov coverage
|
|
# this workflow produces.
|
|
#
|
|
# Every job runs on ubuntu-24.04 rather than ubuntu-latest, so the move of
|
|
# that label to a new Ubuntu release is a change made here, in a PR, and not
|
|
# one that lands on its own.
|
|
on:
|
|
push:
|
|
branches: [main, dev]
|
|
pull_request:
|
|
branches: [main, dev]
|
|
workflow_dispatch:
|
|
|
|
# A newer push to the same branch/PR supersedes the running one — cancel it
|
|
# rather than spending the coverage fan-out on a stale commit. Safe here
|
|
# because the Sonar scan job runs last and only the latest commit matters.
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
changes:
|
|
name: Changed Areas
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 10
|
|
outputs:
|
|
code: ${{ steps.areas.outputs.code }}
|
|
image: ${{ steps.areas.outputs.image }}
|
|
deploy: ${{ steps.areas.outputs.deploy }}
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
# On a pull request HEAD is the merge commit GitHub builds of the PR
|
|
# into its base, and its first parent is the base, so two commits
|
|
# are enough to see exactly what the PR changes.
|
|
fetch-depth: 2
|
|
persist-credentials: false
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version-file: .nvmrc
|
|
|
|
- name: Test the classifier
|
|
run: npm run test:ci-scripts
|
|
|
|
- name: List the changed files
|
|
id: list
|
|
env:
|
|
EVENT: ${{ github.event_name }}
|
|
BEFORE: ${{ github.event.before }}
|
|
AFTER: ${{ github.sha }}
|
|
run: |
|
|
set -euo pipefail
|
|
list="$RUNNER_TEMP/changed-files.txt"
|
|
# --no-renames lists a moved file under its old path too, so a move
|
|
# out of an area still runs that area.
|
|
changed_between() { git -c core.quotePath=false diff --no-renames --name-only "$1" "$2" > "$list"; }
|
|
case "$EVENT" in
|
|
pull_request)
|
|
changed_between HEAD^1 HEAD
|
|
;;
|
|
push)
|
|
# A new branch has no previous commit, and a force push can leave
|
|
# it unreachable. Both run everything.
|
|
if [ -n "$BEFORE" ] && [ "$BEFORE" != "0000000000000000000000000000000000000000" ] && git fetch --no-tags --depth=1 origin "$BEFORE"; then
|
|
changed_between "$BEFORE" "$AFTER"
|
|
else
|
|
echo "No previous commit to compare with, so everything runs."
|
|
list=""
|
|
fi
|
|
;;
|
|
*)
|
|
list=""
|
|
;;
|
|
esac
|
|
echo "list=$list" >> "$GITHUB_OUTPUT"
|
|
[ -z "$list" ] || cat "$list"
|
|
|
|
- name: Decide what runs
|
|
id: areas
|
|
env:
|
|
LIST: ${{ steps.list.outputs.list }}
|
|
run: |
|
|
if [ -n "$LIST" ]; then
|
|
node scripts/ci/changed-areas.mjs --files "$LIST"
|
|
else
|
|
node scripts/ci/changed-areas.mjs --all
|
|
fi
|
|
|
|
plugin-facts:
|
|
name: Plugin Permission Facts
|
|
needs: changes
|
|
if: needs.changes.outputs.code == 'true'
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version-file: .nvmrc
|
|
cache: npm
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install dependencies
|
|
run: npm ci --workspace server
|
|
|
|
# The plugin permission facts live once, in
|
|
# server/src/nest/plugins/protocol/envelope.ts. Every other copy — the
|
|
# SDK's generated/host-facts.ts and shared's plugin-permissions.ts — is
|
|
# generated from it and checked in, because publish-plugin-sdk builds
|
|
# from a standalone checkout with no server present. This fails the run
|
|
# if a copy drifts, if a new permission has no admin.plugins.perm.*
|
|
# translation, or if the CLI has no hint for it.
|
|
- name: Check the generated plugin facts are in sync
|
|
run: cd server && npm run check:plugin-facts
|
|
|
|
i18n-parity:
|
|
name: i18n Key Parity
|
|
needs: changes
|
|
if: needs.changes.outputs.code == 'true'
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version-file: .nvmrc
|
|
|
|
- name: Check i18n key parity
|
|
run: node shared/scripts/i18n-parity.mjs --strict
|
|
|
|
server-quality:
|
|
name: Server Types & Lint
|
|
needs: changes
|
|
if: needs.changes.outputs.code == 'true'
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version-file: .nvmrc
|
|
cache: npm
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install dependencies
|
|
run: npm ci
|
|
|
|
- name: Build shared
|
|
run: npm run build --workspace=shared
|
|
|
|
- name: Typecheck shared
|
|
# Rides along here rather than in shared-coverage: this job already
|
|
# pays the root install and shared build, and a coverage job may only
|
|
# fail because tests failed (it gates the scan).
|
|
run: cd shared && npm run typecheck
|
|
|
|
- name: Build server (tsc -> dist)
|
|
run: cd server && npm run build
|
|
|
|
- name: Smoke production require chain
|
|
# Vitest aliases the MCP SDK deep imports, so only this exercises what
|
|
# production runs: the built dist resolving the MCP SDK's subpath
|
|
# exports through the tsconfig-paths/register runtime hook.
|
|
run: cd server && node --require tsconfig-paths/register -e "require('./dist/nest-mcp')"
|
|
|
|
- name: Typecheck
|
|
run: cd server && npm run typecheck
|
|
|
|
- name: Typecheck tests
|
|
# tests/ is outside the build tsconfig, so a service that grows a
|
|
# constructor parameter does not break its hand-wired call sites at
|
|
# compile time: the argument arrives as undefined and throws at
|
|
# runtime, if anything reads it at all. This is the step that
|
|
# catches it.
|
|
run: cd server && npm run typecheck:tests
|
|
|
|
- name: Typecheck entity generator
|
|
# tsconfig.scripts.json covers generate-entities.ts and migrate-encryption.ts;
|
|
# gen-plugin-facts.ts uses import.meta and does not compile under this
|
|
# config (needs a newer --module target), so the gate is named for what
|
|
# it actually covers rather than the whole directory.
|
|
run: cd server && npm run typecheck:scripts
|
|
|
|
- name: Entity files match the generator
|
|
run: cd server && npm run check:entities
|
|
|
|
- name: Kysely table types match the migrated schema
|
|
# src/db/kysely/ is generated from the migrations by gen:db-types; a
|
|
# migration that changes a column regenerates it in the same change.
|
|
run: cd server && npm run check:db-types
|
|
|
|
- name: Lint
|
|
# Type-aware rules build the src + tests program, which outgrows
|
|
# Node's default ~4 GB heap on the runner.
|
|
env:
|
|
NODE_OPTIONS: --max-old-space-size=8192
|
|
run: cd server && npm run lint:check
|
|
|
|
- name: DB call-graph gate (no sync DB-touching methods, no raw transactions)
|
|
run: cd server && npm run db:call-graph -- --sync --tx
|
|
|
|
- name: File size check
|
|
# A source file may hold no more lines than scripts/size-baseline.json
|
|
# allows (1000 for a file without an entry, long lines counted
|
|
# wrapped), so a big service is split by concern instead of growing.
|
|
run: cd server && npm run lint:size
|
|
|
|
- name: Import boundaries and cycles
|
|
# No new load-time cycle, no new reach into another domain's
|
|
# internals, src/db stays below src/nest. Today's violations sit in
|
|
# scripts/import-boundaries-baseline.json and may only shrink.
|
|
run: cd server && npm run lint:boundaries
|
|
|
|
- name: SQLite-only SQL spellings may only shrink
|
|
# INSERT OR IGNORE, datetime(), json_extract, insertId and friends
|
|
# outside src/db/dialect/ are held to scripts/dialect-baseline.json,
|
|
# so the code stays portable to a second engine.
|
|
run: cd server && npm run lint:dialect
|
|
|
|
- name: Writes outside a transaction
|
|
# A method that writes twice does it inside one uow.transactional.
|
|
# Today's exceptions sit in scripts/tx-writes-baseline.json and may
|
|
# only shrink; @txStandalone/@txIndependent mark the deliberate ones.
|
|
run: cd server && npm run lint:tx
|
|
|
|
- name: Raw SQL in tests may only shrink
|
|
# Counts the better-sqlite3 .prepare() calls under tests/ per file
|
|
# against scripts/test-sql-baseline.json. New fixtures seed and read
|
|
# through tests/helpers/factories (MikroORM) instead.
|
|
run: cd server && npm run lint:test-sql
|
|
|
|
- name: Open request schemas may only shrink
|
|
# Needs the shared build above. Counts open shapes (records of
|
|
# unknown, passthrough, any) in shared's request schemas.
|
|
run: cd shared && npm run contracts:open
|
|
|
|
client-quality:
|
|
name: Client Types & Lint
|
|
needs: changes
|
|
if: needs.changes.outputs.code == 'true'
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version-file: .nvmrc
|
|
cache: npm
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install dependencies
|
|
run: npm ci --workspace shared && npm ci --workspace client
|
|
|
|
- name: Build shared
|
|
run: npm run build --workspace=shared
|
|
|
|
- name: Typecheck
|
|
run: cd client && npm run typecheck
|
|
|
|
- name: Lint
|
|
# Fails on any ESLint error, like lint:check did, and on any rule
|
|
# whose warning count rises above scripts/eslint-baseline.json.
|
|
run: cd client && npm run lint:warnings
|
|
|
|
- name: Page pattern check
|
|
run: cd client && npm run lint:pages
|
|
|
|
- name: Reading direction check
|
|
# Arabic runs right to left. A file may hold no more physical
|
|
# left/right classes and styles than scripts/rtl-baseline.json allows,
|
|
# so new code uses the logical start/end forms (scripts/rtl-lint.mjs).
|
|
run: cd client && npm run lint:rtl
|
|
|
|
- name: File size check
|
|
# A source file or stylesheet may hold no more lines than
|
|
# scripts/size-baseline.json allows (1000 without an entry, 2000 for a
|
|
# test, long lines counted wrapped), so a big file is split by concern
|
|
# instead of growing (scripts/size-lint.mjs).
|
|
run: cd client && npm run lint:size
|
|
|
|
- name: Formatting check
|
|
# Every file outside scripts/format-baseline.json must be Prettier
|
|
# formatted; the list only shrinks.
|
|
run: cd client && npm run lint:format
|
|
|
|
- name: Layering check
|
|
# Imports go downwards: components never import pages/ or mobile/.
|
|
run: cd client && npm run lint:layers
|
|
|
|
- name: Offline data path check
|
|
# No view file calls src/api/ directly beyond
|
|
# scripts/offline-baseline.json; writes go through repo/ and the
|
|
# mutation queue.
|
|
run: cd client && npm run lint:offline
|
|
|
|
- name: Duplication check
|
|
# A file may hold no more lines inside a block of code that repeats
|
|
# elsewhere than scripts/dup-baseline.json allows, counted at the
|
|
# thresholds of the SonarCloud gate, so the logic a desktop view and
|
|
# its phone twin share stays in one hook (scripts/dup-lint.mjs).
|
|
run: cd client && npm run lint:dup
|
|
|
|
- name: Skipped and focused tests check
|
|
run: cd client && npm run lint:skips
|
|
|
|
- name: i18n keys check
|
|
# Every translation key src/ names must exist in shared en.
|
|
run: cd client && npm run lint:i18n-keys
|
|
|
|
plugin-sdk-quality:
|
|
name: Plugin SDK Types
|
|
needs: changes
|
|
if: needs.changes.outputs.code == 'true'
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
defaults:
|
|
run:
|
|
working-directory: plugin-sdk
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version-file: .nvmrc
|
|
|
|
# plugin-sdk is standalone (own lockfile, not a root workspace), so it
|
|
# installs from its own directory.
|
|
- name: Install dependencies
|
|
run: npm ci
|
|
|
|
- name: Typecheck
|
|
run: npm run typecheck
|
|
|
|
client-bundle:
|
|
name: Client Bundle
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
needs: [plugin-facts, i18n-parity, server-quality, client-quality, plugin-sdk-quality]
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version-file: .nvmrc
|
|
cache: npm
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install dependencies
|
|
run: npm ci --workspace shared && npm ci --workspace client
|
|
|
|
- name: Build shared
|
|
run: npm run build --workspace=shared
|
|
|
|
- name: Build client
|
|
run: cd client && npm run build
|
|
|
|
- name: Each map engine in a chunk of its own
|
|
# One static import of mapbox-gl or maplibre-gl in the map tree folds
|
|
# both engines into one chunk again, and nothing else notices: the build
|
|
# stays green and the app still works, only every map user downloads
|
|
# both. scripts/check-gl-split.mjs explains the markers it looks for.
|
|
run: cd client && npm run check:gl-split
|
|
|
|
client-e2e:
|
|
name: Client End-to-End
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
needs: [plugin-facts, i18n-parity, server-quality, client-quality, plugin-sdk-quality]
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version-file: .nvmrc
|
|
cache: npm
|
|
cache-dependency-path: package-lock.json
|
|
|
|
# The run boots the real server (e2e/server-launch.mjs builds it and
|
|
# starts it against a throwaway SQLite file) and the Vite dev server in
|
|
# front of it, so it needs every workspace.
|
|
- name: Install dependencies
|
|
run: npm ci
|
|
|
|
- name: Build shared
|
|
run: npm run build --workspace=shared
|
|
|
|
# Keyed on the lockfile, which pins @playwright/test and with it the
|
|
# browser builds. The system libraries the browsers need are not cached
|
|
# and are installed on every run.
|
|
- name: Cache browsers
|
|
id: browsers
|
|
uses: actions/cache@v6
|
|
with:
|
|
path: ~/.cache/ms-playwright
|
|
key: playwright-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
|
|
|
|
# Chromium for the desktop and phone flows, WebKit for the iPad and
|
|
# iPhone ones (every browser on iOS and iPadOS is WebKit underneath).
|
|
- name: Install browsers
|
|
if: steps.browsers.outputs.cache-hit != 'true'
|
|
run: cd client && npx playwright install --with-deps chromium webkit
|
|
|
|
- name: Install browser dependencies
|
|
if: steps.browsers.outputs.cache-hit == 'true'
|
|
run: cd client && npx playwright install-deps chromium webkit
|
|
|
|
# The public and app projects are the regression flows. The screenshot
|
|
# and help-media projects make pictures for the wiki and the help center
|
|
# and assert nothing, so they stay out of CI.
|
|
- name: Run the end-to-end flows
|
|
run: cd client && npx playwright test --project=public --project=app
|
|
|
|
- name: Upload traces
|
|
if: failure()
|
|
uses: actions/upload-artifact@v6
|
|
with:
|
|
name: e2e-results
|
|
path: client/test-results/
|
|
retention-days: 7
|
|
|
|
s3-contract:
|
|
name: S3 Driver Contract (MinIO)
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
needs: [plugin-facts, i18n-parity, server-quality, client-quality, plugin-sdk-quality]
|
|
# Fork PRs don't get repo secrets, so MINIO_LICENSE_KEY would be empty and
|
|
# the AIStor container would fail to boot. A run Dependabot triggers gets
|
|
# only Dependabot secrets, so its PRs (same repo, not a fork) hit the same
|
|
# wall. Pushes and manual dispatches by anyone else do have secrets, so
|
|
# only those two are filtered out. (The implicit success() on this
|
|
# expression also keeps the job skipped when a preflight fails.)
|
|
if: >-
|
|
github.actor != 'dependabot[bot]' &&
|
|
(github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository)
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version-file: .nvmrc
|
|
cache: npm
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install dependencies
|
|
run: npm ci
|
|
|
|
- name: Ensure @swc/core's Linux binary for unplugin-swc
|
|
# The lockfile was generated on Windows and omits @swc/core's Linux
|
|
# optional native binary, so npm ci/install skips it on the runner.
|
|
# Install the matching version explicitly so the server's SWC transform
|
|
# (server/vitest.config.ts) can load.
|
|
run: |
|
|
SWC_VERSION=$(node -p "require('@swc/core/package.json').version")
|
|
npm install --no-save --legacy-peer-deps "@swc/core-linux-x64-gnu@$SWC_VERSION"
|
|
|
|
- name: Build shared
|
|
run: npm run build --workspace=shared
|
|
|
|
- name: Write MinIO license
|
|
run: printf '%s' "${{ secrets.MINIO_LICENSE_KEY }}" > minio-licence.key
|
|
|
|
- name: Start MinIO
|
|
# `services:` can't pass the `server /data --license ...` command to
|
|
# the AIStor image, so it starts via docker run. 127.0.0.1 endpoint is
|
|
# load-bearing: aws-lite picks path-style only for IP/localhost hosts,
|
|
# which is all stock MinIO/AIStor speaks.
|
|
run: |
|
|
docker run -d --name minio -p 127.0.0.1:9000:9000 \
|
|
-v "$PWD/minio-licence.key:/minio.license:ro" \
|
|
-e MINIO_ROOT_USER=trekci -e MINIO_ROOT_PASSWORD=trekci-secret \
|
|
quay.io/minio/aistor/minio:RELEASE.2026-08-07T18-34-35Z server /data --license /minio.license
|
|
|
|
- name: Wait for MinIO
|
|
# curl was removed from the AIStor image — poll from the runner instead.
|
|
# 60 x 2s gives a wider margin on cold runners than the 30 x 1s the
|
|
# pre-6fbff835 job used.
|
|
run: |
|
|
for _ in $(seq 1 60); do
|
|
curl -sf http://127.0.0.1:9000/minio/health/live && exit 0
|
|
sleep 2
|
|
done
|
|
echo "MinIO never became healthy" >&2
|
|
docker logs minio >&2
|
|
exit 1
|
|
|
|
- name: Run the S3 driver contract suite
|
|
# No --coverage: the src/nest/storage/** 94/84/97/94 pin must keep
|
|
# holding from hermetic tests alone. The suite's beforeAll creates
|
|
# the trek-test bucket idempotently — no separate bucket-init step.
|
|
env:
|
|
TREK_TEST_S3_ENDPOINT: http://127.0.0.1:9000
|
|
TREK_TEST_S3_BUCKET: trek-test
|
|
TREK_TEST_S3_ACCESS_KEY_ID: trekci
|
|
TREK_TEST_S3_SECRET_ACCESS_KEY: trekci-secret
|
|
run: cd server && npx vitest run tests/unit/nest/storage/s3.driver.contract.test.ts
|
|
|
|
# What of the server's SQL runs on Postgres today (server/scripts/pg-probe.ts).
|
|
# TREK runs on SQLite; the dialect layer in server/src/db/dialect/ is written
|
|
# so a second engine is a branch there and not a sweep through the
|
|
# repositories, and this is the job that holds it to that. Against an empty
|
|
# Postgres it derives a schema from the entities, runs every dialect helper
|
|
# on SQLite and Postgres (every case must pass), and calls every public
|
|
# repository method once, each statement in its own rolled-back transaction.
|
|
# The statements Postgres refuses as SQL are held per method to
|
|
# server/scripts/pg-probe-baseline.json, and so is the list of methods it
|
|
# could not measure (uncovered); both may only shrink. An unmeasured
|
|
# baseline (failing or uncovered null) or a run that called no method fails. The measured
|
|
# baseline and the full report are uploaded either way, so the first run's
|
|
# artifact seeds the file and a change that fixes a statement can lower it
|
|
# without a local Postgres.
|
|
postgres-probe:
|
|
name: Postgres Probe
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 10
|
|
needs: [changes, plugin-facts, i18n-parity, server-quality, client-quality, plugin-sdk-quality]
|
|
if: needs.changes.outputs.code == 'true'
|
|
services:
|
|
postgres:
|
|
image: postgres:17
|
|
env:
|
|
POSTGRES_PASSWORD: probe
|
|
POSTGRES_DB: trek_probe
|
|
ports:
|
|
- 5432:5432
|
|
options: >-
|
|
--health-cmd "pg_isready -U postgres -d trek_probe"
|
|
--health-interval 5s
|
|
--health-timeout 5s
|
|
--health-retries 20
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
persist-credentials: false
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version-file: .nvmrc
|
|
cache: npm
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install dependencies
|
|
run: npm ci
|
|
|
|
# The probe reads the repositories' parameter types, some of which come
|
|
# from @trek/shared's built declarations.
|
|
- name: Build shared
|
|
run: npm run build --workspace=shared
|
|
|
|
- name: Run the Postgres probe
|
|
env:
|
|
TREK_PG_PROBE_URL: postgres://postgres:probe@127.0.0.1:5432/trek_probe
|
|
run: >-
|
|
cd server && npm run probe:pg --
|
|
--next-baseline="$RUNNER_TEMP/pg-probe-baseline.json"
|
|
--report="$RUNNER_TEMP/pg-probe-report.json"
|
|
|
|
- name: Upload the measured baseline and the report
|
|
if: always()
|
|
uses: actions/upload-artifact@v6
|
|
with:
|
|
name: pg-probe-baseline
|
|
path: |
|
|
${{ runner.temp }}/pg-probe-baseline.json
|
|
${{ runner.temp }}/pg-probe-report.json
|
|
if-no-files-found: ignore
|
|
retention-days: 14
|
|
|
|
server-coverage:
|
|
name: Server Coverage
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
needs: [plugin-facts, i18n-parity, server-quality, client-quality, plugin-sdk-quality]
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version-file: .nvmrc
|
|
cache: npm
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install dependencies
|
|
run: npm ci
|
|
|
|
- name: Ensure @swc/core's Linux binary for unplugin-swc
|
|
# The lockfile was generated on Windows and omits @swc/core's Linux
|
|
# optional native binary, so npm ci/install skips it on the runner.
|
|
# Install the matching version explicitly so the server's SWC transform
|
|
# (server/vitest.config.ts) can load.
|
|
run: |
|
|
SWC_VERSION=$(node -p "require('@swc/core/package.json').version")
|
|
npm install --no-save --legacy-peer-deps "@swc/core-linux-x64-gnu@$SWC_VERSION"
|
|
|
|
- name: Build shared
|
|
run: npm run build --workspace=shared
|
|
|
|
- name: Run tests with coverage
|
|
run: cd server && npm run test:coverage
|
|
|
|
- name: Upload coverage
|
|
uses: actions/upload-artifact@v6
|
|
with:
|
|
name: sonar-server-coverage
|
|
path: server/coverage/
|
|
retention-days: 1
|
|
|
|
client-coverage:
|
|
name: Client Coverage
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
needs: [plugin-facts, i18n-parity, server-quality, client-quality, plugin-sdk-quality]
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version-file: .nvmrc
|
|
cache: npm
|
|
cache-dependency-path: package-lock.json
|
|
|
|
# Only the shared and client workspaces — a full root npm ci is not
|
|
# needed, and the scoped install keeps @testing-library/dom (a peer of
|
|
# @testing-library/react) in the tree without the re-resolution dance
|
|
# the folded coverage job used to need.
|
|
- name: Install dependencies
|
|
run: npm ci --workspace shared && npm ci --workspace client
|
|
|
|
- name: Build shared
|
|
run: npm run build --workspace=shared
|
|
|
|
- name: Run tests with coverage
|
|
run: cd client && npm run test:coverage
|
|
|
|
- name: Upload coverage
|
|
uses: actions/upload-artifact@v6
|
|
with:
|
|
name: sonar-client-coverage
|
|
path: client/coverage/
|
|
retention-days: 1
|
|
|
|
shared-coverage:
|
|
name: Shared Coverage
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
needs: [plugin-facts, i18n-parity, server-quality, client-quality, plugin-sdk-quality]
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version-file: .nvmrc
|
|
cache: npm
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install dependencies
|
|
run: npm ci --workspace shared
|
|
|
|
- name: Run tests with coverage
|
|
run: cd shared && npm run test:coverage
|
|
|
|
- name: Upload coverage
|
|
uses: actions/upload-artifact@v6
|
|
with:
|
|
name: sonar-shared-coverage
|
|
path: shared/coverage/
|
|
retention-days: 1
|
|
|
|
plugin-sdk-coverage:
|
|
name: Plugin SDK Coverage
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
needs: [plugin-facts, i18n-parity, server-quality, client-quality, plugin-sdk-quality]
|
|
defaults:
|
|
run:
|
|
working-directory: plugin-sdk
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
# The image's Node runs the plugins, and it has the node:sqlite the
|
|
# dev-server tests exercise.
|
|
node-version-file: .nvmrc
|
|
|
|
# plugin-sdk is standalone (own lockfile, not a root workspace), so it
|
|
# installs from its own directory.
|
|
- name: Install dependencies
|
|
run: npm ci
|
|
|
|
- name: Run tests with coverage
|
|
run: npm run test:coverage
|
|
|
|
- name: Upload coverage
|
|
uses: actions/upload-artifact@v6
|
|
with:
|
|
name: sonar-plugin-sdk-coverage
|
|
path: plugin-sdk/coverage/
|
|
retention-days: 1
|
|
|
|
scan:
|
|
name: Scan
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
# Fork PRs and Dependabot runs don't receive SONAR_TOKEN, so skip the scan
|
|
# rather than fail it.
|
|
if: >-
|
|
github.actor != 'dependabot[bot]' &&
|
|
(github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository)
|
|
# Coverage jobs failing must stop the scan: a missing lcov file would make
|
|
# every line of that package report as uncovered, and the dashboard would
|
|
# show a collapse that did not happen. The preflight gates already stop it
|
|
# transitively through these four, so they aren't repeated here.
|
|
needs: [server-coverage, client-coverage, shared-coverage, plugin-sdk-coverage]
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
# Sonar reads git blame to date each issue and to decide what counts
|
|
# as new code. A shallow clone leaves it blind to both.
|
|
fetch-depth: 0
|
|
|
|
- name: Download server coverage
|
|
uses: actions/download-artifact@v7
|
|
with:
|
|
name: sonar-server-coverage
|
|
path: server/coverage/
|
|
|
|
- name: Download client coverage
|
|
uses: actions/download-artifact@v7
|
|
with:
|
|
name: sonar-client-coverage
|
|
path: client/coverage/
|
|
|
|
- name: Download shared coverage
|
|
uses: actions/download-artifact@v7
|
|
with:
|
|
name: sonar-shared-coverage
|
|
path: shared/coverage/
|
|
|
|
- name: Download plugin-sdk coverage
|
|
uses: actions/download-artifact@v7
|
|
with:
|
|
name: sonar-plugin-sdk-coverage
|
|
path: plugin-sdk/coverage/
|
|
|
|
- name: Scan
|
|
uses: SonarSource/sonarqube-scan-action@d209202bc7d53ff1cc128f7f907dac145c9d6ae9 # v8.3.0
|
|
env:
|
|
# SonarQube Cloud is the scan action's default host, so no
|
|
# SONAR_HOST_URL — the token is a sonarcloud.io token for the
|
|
# liketrek org.
|
|
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
|
|
with:
|
|
# New Code is configured as "previous version", so this string is what
|
|
# draws that line — changing it re-bases every "new code" metric.
|
|
# Pinned rather than read from package.json on purpose: package.json is
|
|
# still on 3.4.1 while the project has been analysed as 4.0.0 all along,
|
|
# and taking the lower number would silently move the baseline. Bump it
|
|
# here as part of the release, together with the version bump.
|
|
#
|
|
# Everything else — sources, tests, exclusions, the documented
|
|
# false-positive list — lives in sonar-project.properties so it stays
|
|
# reviewable in a PR rather than buried in workflow YAML.
|
|
args: -Dsonar.projectVersion=4.0.0
|
|
|
|
# The image every release ships, built the way docker.yml builds it and
|
|
# started the way docker-compose.yml starts it (read-only root, every
|
|
# capability dropped but the three the entrypoint needs, bind-mounted data
|
|
# and uploads). Nothing else in CI builds the Dockerfile: the unit tests only
|
|
# read it as text, which is how a Dockerfile that could not be parsed
|
|
# reached dev unseen. It is separate from the vulnerability scan on purpose,
|
|
# so a newly published CVE cannot hide a broken build or the other way round.
|
|
docker-image:
|
|
name: Docker Image
|
|
needs: changes
|
|
if: needs.changes.outputs.image == 'true'
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
persist-credentials: false
|
|
|
|
- uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
|
|
|
|
# Only pushes write the layer cache. Pull requests read what the last
|
|
# push to their base left, so a stream of PRs cannot evict it and the
|
|
# cache stays within the repository's quota.
|
|
- name: Build the image
|
|
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
|
|
with:
|
|
context: .
|
|
platforms: linux/amd64
|
|
push: false
|
|
load: true
|
|
tags: trek:ci
|
|
build-args: |
|
|
APP_VERSION=0.0.0-ci
|
|
cache-from: type=gha,scope=trek-image
|
|
cache-to: ${{ github.event_name == 'push' && 'type=gha,mode=max,scope=trek-image' || '' }}
|
|
|
|
- name: Start the container
|
|
run: |
|
|
mkdir -p "$RUNNER_TEMP/trek/data" "$RUNNER_TEMP/trek/uploads"
|
|
docker run -d --name trek-ci -p 127.0.0.1:3000:3000 \
|
|
--read-only --tmpfs /tmp:noexec,nosuid,size=128m \
|
|
--security-opt no-new-privileges:true \
|
|
--cap-drop ALL --cap-add CHOWN --cap-add SETUID --cap-add SETGID \
|
|
-v "$RUNNER_TEMP/trek/data:/app/data" \
|
|
-v "$RUNNER_TEMP/trek/uploads:/app/uploads" \
|
|
-e ENCRYPTION_KEY="$(openssl rand -hex 32)" \
|
|
trek:ci
|
|
|
|
# Ready means migrated, seeded and answering from the database. A first
|
|
# boot migrates an empty database, which takes seconds; three minutes
|
|
# is the bound, not the expectation.
|
|
- name: Wait until the server is ready
|
|
run: |
|
|
for _ in $(seq 1 90); do
|
|
if [ "$(docker inspect -f '{{.State.Running}}' trek-ci)" != "true" ]; then
|
|
echo "::error::The container exited before it became ready."
|
|
exit 1
|
|
fi
|
|
if curl -sf http://127.0.0.1:3000/api/health/ready; then
|
|
echo
|
|
exit 0
|
|
fi
|
|
sleep 2
|
|
done
|
|
echo "::error::/api/health/ready did not answer within 180 seconds."
|
|
exit 1
|
|
|
|
- name: Check what the image serves
|
|
run: |
|
|
set -euo pipefail
|
|
version="$(docker exec trek-ci cat /app/server/VERSION)"
|
|
[ "$version" = "0.0.0-ci" ] || { echo "::error::VERSION holds '$version'"; exit 1; }
|
|
curl -sf http://127.0.0.1:3000/ | grep -q '<div id="root">' \
|
|
|| { echo "::error::/ does not serve the client"; exit 1; }
|
|
|
|
- name: Container log
|
|
if: always()
|
|
run: docker logs trek-ci 2>&1 || true
|
|
|
|
# The Helm chart and the compose files are what self-hosters deploy from,
|
|
# and before this job only a release ever touched them. The chart is linted,
|
|
# rendered with the defaults and with each secret and persistence mode, and
|
|
# the output checked against the Kubernetes API schemas. Both tools come
|
|
# from their release pages, pinned by version and checksum.
|
|
deploy-manifests:
|
|
name: Helm Chart & Compose
|
|
needs: changes
|
|
if: needs.changes.outputs.deploy == 'true'
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 15
|
|
env:
|
|
HELM_VERSION: v3.22.0
|
|
HELM_SHA256: 1e4ab49e429626cf6c6958d914248b78c9730803c2751b87627e171dc800e7bb
|
|
KUBECONFORM_VERSION: v0.8.0
|
|
KUBECONFORM_SHA256: 9bc2bffbf71f261128533edaf912153948b7ff238f9a531ae6d34466ec287883
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
persist-credentials: false
|
|
|
|
- name: Install helm and kubeconform
|
|
run: |
|
|
set -euo pipefail
|
|
bin="$RUNNER_TEMP/bin"
|
|
mkdir -p "$bin"
|
|
cd "$RUNNER_TEMP"
|
|
curl -sSfLo helm.tgz "https://get.helm.sh/helm-$HELM_VERSION-linux-amd64.tar.gz"
|
|
echo "$HELM_SHA256 helm.tgz" | sha256sum -c -
|
|
tar -xzf helm.tgz --strip-components=1 -C "$bin" linux-amd64/helm
|
|
curl -sSfLo kubeconform.tgz "https://github.com/yannh/kubeconform/releases/download/$KUBECONFORM_VERSION/kubeconform-linux-amd64.tar.gz"
|
|
echo "$KUBECONFORM_SHA256 kubeconform.tgz" | sha256sum -c -
|
|
tar -xzf kubeconform.tgz -C "$bin" kubeconform
|
|
echo "$bin" >> "$GITHUB_PATH"
|
|
|
|
- name: Lint the chart
|
|
run: helm lint --strict charts/trek
|
|
|
|
- name: Render the chart
|
|
run: |
|
|
set -euo pipefail
|
|
out="$RUNNER_TEMP/rendered"
|
|
mkdir -p "$out"
|
|
helm template trek charts/trek > "$out/defaults.yaml"
|
|
helm template trek charts/trek --set existingSecret=trek-secrets > "$out/existing-secret.yaml"
|
|
helm template trek charts/trek --set generateEncryptionKey=true > "$out/generated-key.yaml"
|
|
helm template trek charts/trek \
|
|
--set persistence.enabled=false \
|
|
--set ingress.enabled=true \
|
|
--set 'ingress.tls[0].secretName=trek-tls' \
|
|
--set 'ingress.tls[0].hosts[0]=chart-example.local' \
|
|
--set 'extraEnv[0].name=SESSION_DURATION' \
|
|
--set 'extraEnv[0].value=7d' \
|
|
--set 'extraEnvFrom[0].secretRef.name=trek-smtp' \
|
|
--set probes.liveness=null \
|
|
> "$out/variants.yaml"
|
|
|
|
- name: Check the manifests against the Kubernetes schemas
|
|
run: kubeconform -strict -summary "$RUNNER_TEMP"/rendered/*.yaml
|
|
|
|
- name: Check every compose file
|
|
run: |
|
|
set -euo pipefail
|
|
shopt -s nullglob
|
|
files=(docker-compose*.yml docker-compose*.yaml)
|
|
[ "${#files[@]}" -gt 0 ] || { echo "::error::No compose file found."; exit 1; }
|
|
for f in "${files[@]}"; do
|
|
echo "$f"
|
|
docker compose -f "$f" config --quiet
|
|
done
|
|
|
|
# The single check the branch ruleset requires. It needs every job that
|
|
# gates a merge, so listing a job here is what makes it required. The scan
|
|
# stays out (see the layout notes at the top).
|
|
ci-ok:
|
|
needs:
|
|
- changes
|
|
- plugin-facts
|
|
- i18n-parity
|
|
- server-quality
|
|
- client-quality
|
|
- plugin-sdk-quality
|
|
- client-bundle
|
|
- client-e2e
|
|
- s3-contract
|
|
- postgres-probe
|
|
- server-coverage
|
|
- client-coverage
|
|
- shared-coverage
|
|
- plugin-sdk-coverage
|
|
- docker-image
|
|
- deploy-manifests
|
|
# always(): without it a failed dependency would skip this job, and the
|
|
# ruleset reads a skipped required check as passed.
|
|
if: always()
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 5
|
|
steps:
|
|
- name: Every needed job passed or was skipped
|
|
env:
|
|
NEEDS: ${{ toJSON(needs) }}
|
|
run: |
|
|
set -euo pipefail
|
|
echo "$NEEDS" | jq -r 'to_entries[] | "\(.value.result)\t\(.key)"' | sort
|
|
failed="$(echo "$NEEDS" | jq -r '[to_entries[] | select(.value.result != "success" and .value.result != "skipped") | .key] | join(", ")')"
|
|
if [ -n "$failed" ]; then
|
|
echo "::error::Not passed: $failed"
|
|
exit 1
|
|
fi
|