diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index e875e6f1b..2651ed744 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -349,6 +349,13 @@ jobs: # mutation queue. run: cd client && npm run lint:offline + - name: Duplication check + # A file may hold no more lines inside a block of code that repeats + # elsewhere than scripts/dup-baseline.json allows, counted at the + # thresholds of the SonarCloud gate, so the logic a desktop view and + # its phone twin share stays in one hook (scripts/dup-lint.mjs). + run: cd client && npm run lint:dup + - name: Skipped and focused tests check run: cd client && npm run lint:skips diff --git a/CLAUDE.md b/CLAUDE.md index 2260e4b68..8c1eb83f7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -35,6 +35,7 @@ cd client && npm run test # client vitest run cd client && npm run e2e # Playwright (CI runs the public and app projects; the screenshot and help-media ones are local) cd client && npm run lint:pages # enforce the Page pattern (CI gate) cd client && npm run lint:rtl # physical left/right sides may only shrink per file (CI gate) +cd client && npm run lint:dup # lines in copied code may only shrink per file, at SonarCloud's thresholds (CI gate) cd client && npm run lint:size # no source file past 1000 lines, longer ones only shrink (CI gate; server has the same) cd client && npm run lint:warnings # eslint errors fail, warnings per rule may only shrink (CI gate) cd client && npm run lint:format # Prettier outside the baseline list (CI gate) @@ -121,7 +122,7 @@ These principles come out of a full-repo audit and shape all new code: ## Quality gates in CI -`.github/workflows/test.yml` runs, per package: typecheck (server also `typecheck:tests`), lint (client through `lint:warnings`, which also holds the warning count per rule), `lint:pages`, `lint:rtl`, `lint:size` (client and server), the server's `lint:boundaries`, `lint:tx`, `lint:dialect` and `lint:test-sql`, the client's `lint:format`, `lint:layers`, `lint:offline`, `lint:skips` and `lint:i18n-keys`, shared's `contracts:open`, `check:plugin-facts`, `i18n:parity:strict` (with the untranslated-strings ratchet), the S3 contract suite, the Postgres probe (`postgres-probe`: every dialect helper on SQLite and Postgres, and the statements every repository method reaches with sample arguments against a Postgres service; the SQL errors per method and the list of methods it cannot measure are held to a baseline that only shrinks and fails while it is still unmeasured; see `server/CLAUDE.md`), the browser end-to-end flows (Chromium and WebKit), a client build with `check:gl-split`, coverage for all four packages, then a SonarCloud scan. It also builds the Docker image and boots it until `/api/health/ready` answers (`docker-image`), and lints and renders the Helm chart against the Kubernetes schemas and runs `docker compose config` on every compose file (`deploy-manifests`). `lint-prettier.yml` additionally runs `lint` + `format:check` on `shared/`. `plugin-sdk` has no lint script and is outside the eslint gates. **Not in CI**: `theme:lint` and the screenshot and help-media Playwright projects; run them locally. There are no git hooks; nothing runs before a commit except you. Other workflows build Docker images, scan them, and publish `trek-plugin-sdk` on `plugin-sdk-v*` tags. +`.github/workflows/test.yml` runs, per package: typecheck (server also `typecheck:tests`), lint (client through `lint:warnings`, which also holds the warning count per rule), `lint:pages`, `lint:rtl`, `lint:size` (client and server), the server's `lint:boundaries`, `lint:tx`, `lint:dialect` and `lint:test-sql`, the client's `lint:format`, `lint:layers`, `lint:offline`, `lint:dup`, `lint:skips` and `lint:i18n-keys`, shared's `contracts:open`, `check:plugin-facts`, `i18n:parity:strict` (with the untranslated-strings ratchet), the S3 contract suite, the Postgres probe (`postgres-probe`: every dialect helper on SQLite and Postgres, and the statements every repository method reaches with sample arguments against a Postgres service; the SQL errors per method and the list of methods it cannot measure are held to a baseline that only shrinks and fails while it is still unmeasured; see `server/CLAUDE.md`), the browser end-to-end flows (Chromium and WebKit), a client build with `check:gl-split`, coverage for all four packages, then a SonarCloud scan. It also builds the Docker image and boots it until `/api/health/ready` answers (`docker-image`), and lints and renders the Helm chart against the Kubernetes schemas and runs `docker compose config` on every compose file (`deploy-manifests`). `lint-prettier.yml` additionally runs `lint` + `format:check` on `shared/`. `plugin-sdk` has no lint script and is outside the eslint gates. **Not in CI**: `theme:lint` and the screenshot and help-media Playwright projects; run them locally. There are no git hooks; nothing runs before a commit except you. Other workflows build Docker images, scan them, and publish `trek-plugin-sdk` on `plugin-sdk-v*` tags. - **`ci-ok` is the required check.** The branch ruleset requires `ci-ok` (plus `lint` and `check-target` from their own workflows), and `ci-ok` needs every job above except the scan. It fails if any of them failed or was cancelled and passes if they passed or were skipped. A new gate job counts only once it is listed in `ci-ok`'s `needs:`. - **What runs is decided by `changes`, not by `on.paths`**: `scripts/ci/changed-areas.mjs` maps the changed files to the `code`, `image` and `deploy` areas (tests: `npm run test:ci-scripts`). A path filter on the workflow would leave `ci-ok` unreported on a docs-only PR and block it. Add a new top-level file the build reads to the right area there. @@ -138,7 +139,7 @@ These principles come out of a full-repo audit and shape all new code: | Security / reliability / maintainability rating | A | | Security hotspots reviewed | 100% | -- **Duplication is the one that bites.** Desktop and phone shells (`client/src/components/` ↔ `client/src/mobile/screens/.../M`) are deliberate mirrors that are already heavily duplicated. Every line added identically to both counts against the 3% budget, so a feature touching both shells fails on its own. Put the logic in ONE hook/module (precedents: `useInstanceSettings`, `useRangeBypass` under `client/src/components/Admin/`) and leave only markup in each shell. Locale files are excluded from duplication, so i18n additions are free. +- **Duplication is the one that bites.** Desktop and phone shells (`client/src/components/` ↔ `client/src/mobile/screens/.../M`) are deliberate mirrors that are already heavily duplicated. Every line added identically to both counts against the 3% budget, so a feature touching both shells fails on its own. Put the logic in ONE hook/module (precedents: `useInstanceSettings`, `useRangeBypass` under `client/src/components/Admin/`) and leave only markup in each shell. `lint:dup` finds the same copies in `client/src` before the push and holds every file to `client/scripts/dup-baseline.json`. Locale files are excluded from duplication, so i18n additions are free. - **Coverage is per new line across all four packages.** A new file without tests drags the PR under 80%. Files in `sonar.coverage.exclusions` don't count either way. - **Ratings are about new issues.** One new bug drops reliability; one new vulnerability drops security. Don't introduce hotspot patterns (`Math.random` in anything security-adjacent, `child_process` through PATH, backtracking regexes, secret-looking strings). Path-wide suppressions live only in `sonar-project.properties` with a written justification; one-off findings are resolved on SonarCloud, never with an inline `// NOSONAR`. - **Check it yourself.** The `sonarqube` MCP is connected: `list_pull_requests` → `get_project_quality_gate_status` shows each condition's value; `search_duplicated_files` + `get_duplications` show what tripped duplication; `search_sonar_issues_in_projects` lists new issues. Test files are excluded from analysis — when duplication fires, the source is the problem. diff --git a/client/CLAUDE.md b/client/CLAUDE.md index 3d0ccf4f9..5cb9e7d2d 100644 --- a/client/CLAUDE.md +++ b/client/CLAUDE.md @@ -16,6 +16,7 @@ npm run lint:size # no source file or stylesheet past 1000 lines, no tes npm run lint:format # Prettier: every .ts/.tsx/.mjs/.css file under src/, tests/, e2e/ and scripts/ outside scripts/format-baseline.json must be formatted, the list only shrinks (CI gate; npx prettier --write , then --update) npm run lint:layers # imports go downwards: components never import pages/ or mobile/, nothing under the views imports a view; per-file counts may only shrink (CI gate; --list, --update) npm run lint:offline # no view file (components/mobile/pages/hooks) imports src/api/ beyond scripts/offline-baseline.json, which only shrinks (CI gate; --list, --update) +npm run lint:dup # copied code: the lines of a file inside a block that repeats elsewhere (100 tokens over 10 lines, SonarCloud's thresholds; .ts and .tsx read as one language) may only shrink against scripts/dup-baseline.json (CI gate; --list shows each block with its other side, --update lowers it) npm run lint:skips # no .only anywhere; skipped/todo/fixme tests per file may only shrink against scripts/skip-baseline.json (CI gate; skipIf/runIf and Playwright's test.skip(condition, 'why') are fine, but skipIf(true) and runIf(false) count as skips) npm run lint:i18n-keys # every translation key src/ names exists in shared en (CI gate; --unused lists en keys nothing reaches) npm run test # vitest run (tests/** + co-located src/**/*.test.{ts,tsx}); also test:unit / test:integration / test:coverage @@ -73,7 +74,7 @@ The offline core is flagship work surrounded by a periphery that ignores it. New ## Desktop ↔ mobile shells and the SonarQube duplication gate -Every desktop component with a phone twin is a deliberate copy that is already heavily duplicated. CI's SonarCloud gate allows **≤ 3% duplicated lines on the PR's new code**, so adding the same lines to both shells fails the PR by itself. Put logic in one shared hook/module (`useInstanceSettings`, `useRangeBypass` under `components/Admin/` are the pattern) and keep only markup in each shell. Full gate rules in the root `CLAUDE.md`. +Every desktop component with a phone twin is a deliberate copy that is already heavily duplicated. CI's SonarCloud gate allows **≤ 3% duplicated lines on the PR's new code**, so adding the same lines to both shells fails the PR by itself. Put logic in one shared hook/module (`useInstanceSettings`, `useRangeBypass` under `components/Admin/` are the pattern) and keep only markup in each shell. `npm run lint:dup` finds such copies before the push and holds every file to `scripts/dup-baseline.json`. Full gate rules in the root `CLAUDE.md`. ## Tests diff --git a/client/package.json b/client/package.json index 944e97450..49bb80c4d 100644 --- a/client/package.json +++ b/client/package.json @@ -27,6 +27,7 @@ "lint:format": "node scripts/format-lint.mjs", "lint:layers": "node scripts/layer-lint.mjs", "lint:offline": "node scripts/offline-lint.mjs", + "lint:dup": "node scripts/dup-lint.mjs", "lint:skips": "node scripts/skip-lint.mjs", "lint:i18n-keys": "node scripts/i18n-keys.mjs", "check:gl-split": "node scripts/check-gl-split.mjs", @@ -82,6 +83,8 @@ }, "devDependencies": { "@eslint/js": "^10.0.1", + "@jscpd/core": "^4.2.5", + "@jscpd/tokenizer": "^4.2.6", "@playwright/test": "^1.63.0", "@testing-library/jest-dom": "^6.10.0", "@testing-library/react": "^16.3.3", diff --git a/client/scripts/dup-baseline.json b/client/scripts/dup-baseline.json new file mode 100644 index 000000000..652cd2c7a --- /dev/null +++ b/client/scripts/dup-baseline.json @@ -0,0 +1,37 @@ +{ + "src/components/Admin/AddonManager.tsx": 23, + "src/components/Files/FileManagerMarkdownPreviewModal.tsx": 19, + "src/components/Files/FileManagerPdfPreviewModal.tsx": 19, + "src/components/Map/MapView.tsx": 56, + "src/components/Map/MapViewGL.tsx": 51, + "src/components/Map/ReservationOverlay.tsx": 75, + "src/components/Map/RouteCalculator.ts": 27, + "src/components/Map/reservationsMapbox.ts": 72, + "src/components/Packing/ApplyTemplateButton.tsx": 19, + "src/components/Packing/PackingListPanelHeader.tsx": 16, + "src/components/Planner/AddressInput.tsx": 38, + "src/components/Planner/AirportSelect.tsx": 44, + "src/components/Planner/DayPlanSidebar.tsx": 326, + "src/components/Planner/LocationSelect.tsx": 56, + "src/components/Planner/ReservationModal.tsx": 26, + "src/components/Planner/TransportModal.tsx": 26, + "src/components/Studio/StudioInspector.tsx": 19, + "src/components/Studio/StudioTravelInspector.tsx": 19, + "src/components/Studio/TravelElements.tsx": 10, + "src/components/Studio/roadRoute.ts": 17, + "src/components/shared/CustomTimePicker.tsx": 24, + "src/mobile/screens/admin/MAdminAddonManager.tsx": 23, + "src/mobile/screens/admin/MAdminStoragePanel.tsx": 38, + "src/mobile/screens/trip/sheets/MReservationSheet.tsx": 79, + "src/mobile/screens/trip/sheets/MTransportFormSheet.tsx": 139, + "src/mobile/screens/trip/tabs/MBookingsTab.tsx": 88, + "src/mobile/screens/trip/tabs/MTransportsTab.tsx": 90, + "src/mobile/screens/vacay/MVacayInviteSheet.tsx": 21, + "src/mobile/screens/vacay/MVacayShareSheet.tsx": 21, + "src/pages/TripPlannerPage.tsx": 82, + "src/pages/admin/GoogleOptions.tsx": 13, + "src/pages/admin/TrekApiCard.tsx": 14, + "src/pages/atlas/AtlasCountrySearch.tsx": 38, + "src/store/slices/budgetSlice.ts": 16, + "src/store/slices/remoteEventHandler.ts": 17 +} diff --git a/client/scripts/dup-lint.mjs b/client/scripts/dup-lint.mjs new file mode 100644 index 000000000..5e57dc669 --- /dev/null +++ b/client/scripts/dup-lint.mjs @@ -0,0 +1,29 @@ +#!/usr/bin/env node +/* + * lint:dup: keeps copied code from coming back. + * + * Most features have a desktop view and a phone view, and each used to carry + * its own copy of the feature's logic, so a fix reached one of them and the + * other kept the bug. The logic now lives in one shared hook or module per + * feature, and this check holds the copies that remain to what they are. It + * counts, per file under src/, the lines inside a block of code that repeats + * elsewhere (or in the same file), at the thresholds SonarCloud uses on a + * pull request. The files that still hold such lines are listed in + * scripts/dup-baseline.json with their count, and a file may hold no more + * than its entry (a file without an entry holds none). An entry above what + * its file holds now, or for a file that is gone, fails as well until + * --update lowers it. + * + * npm run lint:dup check against the baseline (CI) + * npm run lint:dup -- --list print every copied block with its other side + * npm run lint:dup -- --update lower the baseline to what the files hold now; + * it never raises an entry + * + * The measuring lives in scripts/lib/dup.mjs. + */ +import { fileURLToPath } from 'node:url'; +import { check } from './lib/dup.mjs'; +import { runCli } from './lib/ratchet.mjs'; + +const root = fileURLToPath(new URL('..', import.meta.url)); +await runCli('lint:dup', (args) => check({ root, update: args.includes('--update'), list: args.includes('--list') })); diff --git a/client/scripts/lib/dup.mjs b/client/scripts/lib/dup.mjs new file mode 100644 index 000000000..cad8d62fa --- /dev/null +++ b/client/scripts/lib/dup.mjs @@ -0,0 +1,139 @@ +/* + * The measuring and the comparison behind lint:dup (scripts/dup-lint.mjs). + * + * Copies are found by jscpd's detector at the thresholds SonarCloud applies to + * TypeScript: a block counts once MIN_TOKENS tokens repeat over at least + * MIN_LINES lines. Both .ts and .tsx are read as TypeScript, because Sonar + * treats them as one language and a hook in a .ts file that repeats a view's + * .tsx code is a copy all the same. A file's count is the number of its lines + * inside a copied block, the duplicated lines Sonar reports. Which of three + * copies a block is matched against depends on the order the files are read + * in, so they are read sorted by path, and with readText, so every machine + * counts the same. + */ +import { Detector, getModeHandler, MemoryStore } from '@jscpd/core'; +import { Tokenizer } from '@jscpd/tokenizer'; +import { join } from 'node:path'; +import { + countMap, + listFiles, + lowerCounts, + readBaseline, + readText, + reportStale, + staleCounts, + TEST_FILE, + toKey, + writeBaseline, +} from './ratchet.mjs'; + +export const MIN_TOKENS = 100; +export const MIN_LINES = 10; + +export const DIRS = ['src']; + +/** The files SonarCloud analyses as source: tests and declaration files are left out. */ +export function accepts(key) { + return /\.tsx?$/.test(key) && !key.endsWith('.d.ts') && !TEST_FILE.test(key); +} + +/** + * Every copied block under root as a pair of line ranges, one per side. Both + * sides can be the same file. + * + * @returns {Promise<{ key: string, start: number, end: number }[][]>} + */ +export async function findCopies(root) { + const keys = listFiles(root, DIRS, accepts) + .map((path) => toKey(root, path)) + .sort(); + const detector = new Detector(new Tokenizer(), new MemoryStore(), [], { + minTokens: MIN_TOKENS, + minLines: MIN_LINES, + mode: getModeHandler('mild'), + }); + const side = ({ sourceId, start, end }) => ({ key: sourceId, start: start.line, end: end.line }); + const copies = []; + for (const key of keys) { + const clones = await detector.detect(key, readText(join(root, key)), 'typescript'); + for (const clone of clones) copies.push([side(clone.duplicationA), side(clone.duplicationB)]); + } + return copies; +} + +/** Per file in path order, how many of its lines sit inside a copied block. */ +export function countLines(copies) { + const lines = {}; + for (const pair of copies) { + for (const { key, start, end } of pair) { + const set = (lines[key] ??= new Set()); + for (let line = start; line <= end; line++) set.add(line); + } + } + return Object.fromEntries( + Object.entries(lines) + .filter(([, set]) => set.size > 0) + .sort(([a], [b]) => (a < b ? -1 : 1)) + .map(([key, set]) => [key, set.size]) + ); +} + +/** Per file, its copied blocks as "start-end = other:start-end", in line order. */ +export function describeCopies(copies) { + const found = {}; + const range = ({ start, end }) => `${start}-${end}`; + for (const [a, b] of copies) { + (found[a.key] ??= []).push({ at: a.start, text: `${range(a)} = ${b.key}:${range(b)}` }); + (found[b.key] ??= []).push({ at: b.start, text: `${range(b)} = ${a.key}:${range(a)}` }); + } + return Object.fromEntries( + Object.entries(found).map(([key, list]) => [key, list.sort((x, y) => x.at - y.at).map(({ text }) => text)]) + ); +} + +/** Runs the check against the baseline at baselinePath and returns the exit code. */ +export async function check({ + root, + baselinePath = join(root, 'scripts/dup-baseline.json'), + update = false, + list = false, + log = console.log, + error = console.error, +}) { + const copies = await findCopies(root); + const counts = countLines(copies); + const blocks = describeCopies(copies); + let baseline = readBaseline(baselinePath, countMap); + + if (list) { + for (const [key, n] of Object.entries(counts)) { + log(`${key} (${n}, baseline ${baseline[key] ?? 0})`); + for (const line of blocks[key]) log(` ${line}`); + } + } + + if (update) { + baseline = lowerCounts(baseline, counts); + writeBaseline(baselinePath, baseline); + } + + const grown = Object.entries(counts).filter(([key, n]) => n > (baseline[key] ?? 0)); + const stale = staleCounts(baseline, counts); + + for (const [key, n] of grown) { + error(`FAIL ${key}: ${n} line(s) in copied blocks, baseline ${baseline[key] ?? 0}:`); + for (const line of blocks[key]) error(` ${line}`); + } + if (grown.length) { + error( + 'Put the code both places need in one shared hook or module and import it from both instead of copying it ' + + '(useInstanceSettings and useRangeBypass under src/components/Admin/ are the pattern).' + ); + } + reportStale(stale, { file: 'dup-baseline.json', command: 'lint:dup', root, error }); + const sum = (map) => Object.values(map).reduce((a, b) => a + b, 0); + log( + `dup: ${sum(counts)} line(s) in copied blocks in ${Object.keys(counts).length} file(s), baseline allows ${sum(baseline)}` + ); + return grown.length || stale.length ? 1 : 0; +} diff --git a/client/tests/unit/scripts/dup-lint.test.ts b/client/tests/unit/scripts/dup-lint.test.ts new file mode 100644 index 000000000..1542438c5 --- /dev/null +++ b/client/tests/unit/scripts/dup-lint.test.ts @@ -0,0 +1,145 @@ +import { readFileSync } from 'node:fs'; +import { afterEach, describe, expect, it } from 'vitest'; +import { check, MIN_LINES } from '../../../scripts/lib/dup.mjs'; +import { RatchetError } from '../../../scripts/lib/ratchet.mjs'; +import { ratchetTree, type RatchetTree } from '../../helpers/ratchetFixture'; + +let tree: RatchetTree; +afterEach(() => tree?.remove()); + +const BASELINE = 'scripts/dup-baseline.json'; + +/** + * n lines of about twenty tokens each, every line different, so a block of + * MIN_LINES + 4 of them is long enough to count as a copy and repeats nothing + * inside itself. Two blocks with the same name are the same code. + */ +function block(name = 'shared', n = MIN_LINES + 4): string { + return Array.from({ length: n }, (_, i) => `export const ${name}${i} = compute(${i}, 'x', [1, 2, 3]);\n`).join(''); +} + +function run( + baseline: Record | string, + files: Record, + options: { update?: boolean; list?: boolean } = {} +) { + tree = ratchetTree({ + [BASELINE]: typeof baseline === 'string' ? baseline : JSON.stringify(baseline), + ...files, + }); + return check({ root: tree.root, baselinePath: tree.path(BASELINE), ...options, ...tree.out }); +} + +describe('lint:dup', () => { + it('DUP-001: fails both sides of a block two files share, naming the other side', async () => { + expect(await run({}, { 'src/a/View.tsx': block(), 'src/b/MView.tsx': block() })).toBe(1); + const out = tree.error.join('\n'); + expect(out).toMatch( + /FAIL {2}src\/a\/View\.tsx: 14 line\(s\) in copied blocks, baseline 0:\n {8}1-14 = src\/b\/MView\.tsx:1-14/ + ); + expect(out).toMatch( + /FAIL {2}src\/b\/MView\.tsx: 14 line\(s\) in copied blocks, baseline 0:\n {8}1-14 = src\/a\/View\.tsx:1-14/ + ); + expect(out).toMatch(/one shared hook or module/); + expect(tree.log.join('\n')).toMatch(/dup: 28 line\(s\) in copied blocks in 2 file\(s\), baseline allows 0/); + }); + + it('DUP-002: passes different code, and a repeat too short in lines or tokens to count', async () => { + const shortLines = block('k', MIN_LINES - 1); + const fewTokens = Array.from({ length: MIN_LINES + 4 }, (_, i) => `s${i}()\n`).join(''); + const files = { + 'src/a.ts': block('a') + fewTokens, + 'src/b.ts': block('b') + fewTokens, + 'src/c.ts': shortLines, + 'src/d.ts': shortLines, + }; + expect(await run({}, files)).toBe(0); + expect(tree.error).toEqual([]); + }); + + it('DUP-003: reads .ts and .tsx as one language, so a hook and a view that repeat it are a copy', async () => { + expect(await run({}, { 'src/components/X/useThing.ts': block(), 'src/mobile/MThing.tsx': block() })).toBe(1); + const out = tree.error.join('\n'); + expect(out).toMatch(/FAIL {2}src\/components\/X\/useThing\.ts: 14 line/); + expect(out).toMatch(/FAIL {2}src\/mobile\/MThing\.tsx: 14 line/); + }); + + it('DUP-004: leaves tests and declaration files out, as SonarCloud does', async () => { + const files = { + 'src/a.test.tsx': block(), + 'src/b.test.ts': block(), + 'src/c.d.ts': block(), + 'src/real.ts': block('real'), + }; + expect(await run({}, files)).toBe(0); + expect(tree.error).toEqual([]); + }); + + it('DUP-005: counts a block one file repeats inside itself', async () => { + expect(await run({}, { 'src/a.ts': block() + block('mid') + block() })).toBe(1); + expect(tree.error.join('\n')).toMatch(/FAIL {2}src\/a\.ts: 28 line\(s\)[^\n]*\n {8}1-14 = src\/a\.ts:29-42/); + }); + + it('DUP-006: holds a baselined file at its entry and fails a new copy or a grown one', async () => { + const held = { 'src/a.ts': 14, 'src/b.ts': 14 }; + expect(await run(held, { 'src/a.ts': block(), 'src/b.ts': block() })).toBe(0); + tree.remove(); + // A third copy of code that is already copied is still a new copy. + expect(await run(held, { 'src/a.ts': block(), 'src/b.ts': block(), 'src/c.ts': block() })).toBe(1); + expect(tree.error.join('\n')).toMatch(/FAIL {2}src\/c\.ts: 14 line\(s\) in copied blocks, baseline 0/); + expect(tree.error.join('\n')).not.toMatch(/FAIL {2}src\/[ab]\.ts/); + tree.remove(); + const grown = block('x') + block('y'); + expect(await run(held, { 'src/a.ts': grown, 'src/b.ts': grown })).toBe(1); + expect(tree.error.join('\n')).toMatch(/FAIL {2}src\/a\.ts: 28 line\(s\) in copied blocks, baseline 14/); + }); + + it('DUP-007: --update lowers an entry, drops one without copies and never raises', async () => { + const baseline = { 'src/a.ts': 30, 'src/b.ts': 10, 'src/gone.ts': 5, 'src/clean.ts': 10 }; + const files = { 'src/a.ts': block(), 'src/b.ts': block(), 'src/clean.ts': block('clean') }; + expect(await run(baseline, files, { update: true })).toBe(1); + expect(JSON.parse(readFileSync(tree.path(BASELINE), 'utf8'))).toEqual({ 'src/a.ts': 14, 'src/b.ts': 10 }); + expect(tree.error.join('\n')).toMatch(/FAIL {2}src\/b\.ts: 14 line\(s\) in copied blocks, baseline 10/); + }); + + it('DUP-008: fails an entry above what its file holds, or for a file that is gone, until --update', async () => { + expect(await run({ 'src/a.ts': 20, 'src/b.ts': 14 }, { 'src/a.ts': block(), 'src/b.ts': block() })).toBe(1); + expect(tree.error.join('\n')).toMatch( + /src\/a\.ts is held at 20 in scripts\/dup-baseline\.json, but there are 14 now/ + ); + expect(tree.error.join('\n')).toMatch(/npm run lint:dup -- --update/); + tree.remove(); + expect(await run({ 'src/gone.ts': 14 }, { 'src/a.ts': block('a') })).toBe(1); + expect(tree.error.join('\n')).toMatch(/src\/gone\.ts is held at 14 .*but the file is gone/); + tree.remove(); + // A file whose copies are all gone keeps no allowance either. + expect(await run({ 'src/a.ts': 14 }, { 'src/a.ts': block('a') })).toBe(1); + expect(tree.error.join('\n')).toMatch(/src\/a\.ts is held at 14 .*but there are 0 now/); + tree.remove(); + expect(await run({ 'src/a.ts': 14, 'src/gone.ts': 14 }, { 'src/a.ts': block('a') }, { update: true })).toBe(0); + }); + + it('DUP-009: counts a file with CRLF line ends the same as one with LF', async () => { + expect(await run({}, { 'src/a.ts': block().replace(/\n/g, '\r\n'), 'src/b.ts': block() })).toBe(1); + expect(tree.error.join('\n')).toMatch(/FAIL {2}src\/a\.ts: 14 line\(s\)[^\n]*\n {8}1-14 = src\/b\.ts:1-14/); + }); + + it('DUP-010: --list prints every copied block with its other side', async () => { + expect( + await run({ 'src/a.ts': 14, 'src/b.ts': 14 }, { 'src/a.ts': block(), 'src/b.ts': block() }, { list: true }) + ).toBe(0); + expect(tree.log).toEqual([ + 'src/a.ts (14, baseline 14)', + ' 1-14 = src/b.ts:1-14', + 'src/b.ts (14, baseline 14)', + ' 1-14 = src/a.ts:1-14', + 'dup: 28 line(s) in copied blocks in 2 file(s), baseline allows 28', + ]); + }); + + it('DUP-011: a broken baseline or a missing src/ stops the check', async () => { + await expect(run('{', { 'src/a.ts': block('a') })).rejects.toThrow(RatchetError); + tree.remove(); + await expect(run({}, { 'tests/a.ts': block('a') })).rejects.toThrow(/src\/ does not exist/); + }); +}); diff --git a/package-lock.json b/package-lock.json index 4dc6b1358..7db011aa4 100644 --- a/package-lock.json +++ b/package-lock.json @@ -69,6 +69,8 @@ }, "devDependencies": { "@eslint/js": "^10.0.1", + "@jscpd/core": "^4.2.5", + "@jscpd/tokenizer": "^4.2.6", "@playwright/test": "^1.63.0", "@testing-library/jest-dom": "^6.10.0", "@testing-library/react": "^16.3.3", @@ -4626,6 +4628,27 @@ "@jridgewell/sourcemap-codec": "^1.4.14" } }, + "node_modules/@jscpd/core": { + "version": "4.2.5", + "resolved": "https://registry.npmjs.org/@jscpd/core/-/core-4.2.5.tgz", + "integrity": "sha512-Esf2deHxaoNEjePwf2jqP6Urzj+BAOsJVPFLbnnSsV+q7rLNMcn0UEEoKBXIOOt4qMkrkhl9DfwpMyPPOr6GkQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "eventemitter3": "^5.0.1" + } + }, + "node_modules/@jscpd/tokenizer": { + "version": "4.2.6", + "resolved": "https://registry.npmjs.org/@jscpd/tokenizer/-/tokenizer-4.2.6.tgz", + "integrity": "sha512-/eyFjINWLs2mrBTU4H681bs855r5oRyl1O3mZxcd7TpL1JIG85a7pps0RkRPAe9c/2KcjVCc82HbhOz86M0n5g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jscpd/core": "4.2.5", + "spark-md5": "^3.0.2" + } + }, "node_modules/@keyv/serialize": { "version": "1.1.1", "resolved": "https://registry.npmjs.org/@keyv/serialize/-/serialize-1.1.1.tgz", @@ -11793,6 +11816,13 @@ "node": ">= 0.6" } }, + "node_modules/eventemitter3": { + "version": "5.0.4", + "resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.4.tgz", + "integrity": "sha512-mlsTRyGaPBjPedk6Bvw+aqbsXDtoAyAzm5MO7JgU+yVRyMQ5O8bD4Kcci7BS85f93veegeCPkL8R4GLClnjLFw==", + "dev": true, + "license": "MIT" + }, "node_modules/events-universal": { "version": "1.0.1", "resolved": "https://registry.npmjs.org/events-universal/-/events-universal-1.0.1.tgz", @@ -19216,6 +19246,13 @@ "url": "https://github.com/sponsors/wooorm" } }, + "node_modules/spark-md5": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/spark-md5/-/spark-md5-3.0.2.tgz", + "integrity": "sha512-wcFzz9cDfbuqe0FZzfi2or1sgyIrsDwmPwfZC4hiNidPdPINjeUwNfv5kldczoEAcjl9Y1L3SM7Uz2PUEQzxQw==", + "dev": true, + "license": "(WTFPL OR MIT)" + }, "node_modules/split2": { "version": "4.2.0", "resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz", diff --git a/wiki/Development-environment.md b/wiki/Development-environment.md index f6cf02e37..07ed57a5e 100644 --- a/wiki/Development-environment.md +++ b/wiki/Development-environment.md @@ -205,6 +205,7 @@ The `@trek/shared` package is the single source of truth for code shared between | `npm run lint:format` | Every file outside the shrinking baseline is Prettier-formatted (CI gate) | | `npm run lint:layers` | Imports only go downwards through the layers (CI gate) | | `npm run lint:offline` | Views reach `src/api/` only through the offline core; the baseline only shrinks (CI gate) | +| `npm run lint:dup` | Lines in copied code blocks may only shrink per file, at SonarCloud's thresholds (CI gate) | | `npm run lint:skips` | No focused test, and skipped tests may only go away (CI gate) | | `npm run lint:i18n-keys` | Every translation key the client names exists in `en` (CI gate) | | `npm run theme:lint` | Flag styling that bypasses the appearance tokens (not run in CI) | @@ -242,6 +243,6 @@ Then open a Pull Request from your fork to `liketrek/TREK` targeting the `dev` b - Run tests before pushing: `npm test` at the repo root runs all workspaces. That alone is not the full CI gate. With `shared` built, these are the checks CI runs before any test (`.github/workflows/test.yml` is the source of truth): - in `shared/`: `npm run typecheck && npm run contracts:open` - in `server/`: `npm run build && npm run typecheck && npm run typecheck:tests && npm run typecheck:scripts && npm run check:entities && npm run check:db-types && npm run lint:check && npm run db:call-graph -- --sync --tx && npm run lint:size && npm run lint:boundaries && npm run lint:tx && npm run lint:test-sql && npm run lint:dialect && npm run check:plugin-facts` - - in `client/`: `npm run typecheck && npm run lint:warnings && npm run lint:pages && npm run lint:rtl && npm run lint:size && npm run lint:format && npm run lint:layers && npm run lint:offline && npm run lint:skips && npm run lint:i18n-keys` + - in `client/`: `npm run typecheck && npm run lint:warnings && npm run lint:pages && npm run lint:rtl && npm run lint:size && npm run lint:format && npm run lint:layers && npm run lint:offline && npm run lint:dup && npm run lint:skips && npm run lint:i18n-keys` - at the root, if you touched translations: `npm run i18n:parity:strict --workspace=shared` - Follow the commit message conventions described in the [[Contributing]] guidelines.