mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-10-19 05:26:47 +00:00
ci(client): lint:dup holds copied code to a baseline that only shrinks
Most features have a desktop view and a phone view, and each carried its own copy of the feature's logic, so SonarCloud's duplication condition was the gate a pull request touching both failed most often, and only after a scan of twenty minutes. lint:dup finds the same copies before the push. It runs jscpd's detector (@jscpd/core with @jscpd/tokenizer) over the source files under src/ at the thresholds the SonarCloud gate applies to TypeScript, 100 tokens over at least 10 lines, and counts per file the lines inside a block that repeats elsewhere or in the same file. Tests and declaration files are left out as Sonar leaves them out, and .ts and .tsx are read as one language, so a hook that repeats a view's code is a copy too. The files are read sorted and with readText, so every machine counts the same. scripts/dup-baseline.json holds the files that still hold copies with their count; a file may hold no more than its entry, a file without an entry none, and an entry above its file or for a file that is gone fails until --update lowers it. --list prints every block with its other side. The engine packages are used instead of the jscpd command, which walks the tree in file system order, skips files past its size limits without a word and drops every result once it meets a file it has no format for.
This commit is contained in:
Maurice
@@ -349,6 +349,13 @@ jobs:
|
||||
# mutation queue.
|
||||
run: cd client && npm run lint:offline
|
||||
|
||||
- name: Duplication check
|
||||
# A file may hold no more lines inside a block of code that repeats
|
||||
# elsewhere than scripts/dup-baseline.json allows, counted at the
|
||||
# thresholds of the SonarCloud gate, so the logic a desktop view and
|
||||
# its phone twin share stays in one hook (scripts/dup-lint.mjs).
|
||||
run: cd client && npm run lint:dup
|
||||
|
||||
- name: Skipped and focused tests check
|
||||
run: cd client && npm run lint:skips
|
||||
|
||||
|
||||
@@ -35,6 +35,7 @@ cd client && npm run test # client vitest run
|
||||
cd client && npm run e2e # Playwright (CI runs the public and app projects; the screenshot and help-media ones are local)
|
||||
cd client && npm run lint:pages # enforce the Page pattern (CI gate)
|
||||
cd client && npm run lint:rtl # physical left/right sides may only shrink per file (CI gate)
|
||||
cd client && npm run lint:dup # lines in copied code may only shrink per file, at SonarCloud's thresholds (CI gate)
|
||||
cd client && npm run lint:size # no source file past 1000 lines, longer ones only shrink (CI gate; server has the same)
|
||||
cd client && npm run lint:warnings # eslint errors fail, warnings per rule may only shrink (CI gate)
|
||||
cd client && npm run lint:format # Prettier outside the baseline list (CI gate)
|
||||
@@ -121,7 +122,7 @@ These principles come out of a full-repo audit and shape all new code:
|
||||
|
||||
## Quality gates in CI
|
||||
|
||||
`.github/workflows/test.yml` runs, per package: typecheck (server also `typecheck:tests`), lint (client through `lint:warnings`, which also holds the warning count per rule), `lint:pages`, `lint:rtl`, `lint:size` (client and server), the server's `lint:boundaries`, `lint:tx`, `lint:dialect` and `lint:test-sql`, the client's `lint:format`, `lint:layers`, `lint:offline`, `lint:skips` and `lint:i18n-keys`, shared's `contracts:open`, `check:plugin-facts`, `i18n:parity:strict` (with the untranslated-strings ratchet), the S3 contract suite, the Postgres probe (`postgres-probe`: every dialect helper on SQLite and Postgres, and the statements every repository method reaches with sample arguments against a Postgres service; the SQL errors per method and the list of methods it cannot measure are held to a baseline that only shrinks and fails while it is still unmeasured; see `server/CLAUDE.md`), the browser end-to-end flows (Chromium and WebKit), a client build with `check:gl-split`, coverage for all four packages, then a SonarCloud scan. It also builds the Docker image and boots it until `/api/health/ready` answers (`docker-image`), and lints and renders the Helm chart against the Kubernetes schemas and runs `docker compose config` on every compose file (`deploy-manifests`). `lint-prettier.yml` additionally runs `lint` + `format:check` on `shared/`. `plugin-sdk` has no lint script and is outside the eslint gates. **Not in CI**: `theme:lint` and the screenshot and help-media Playwright projects; run them locally. There are no git hooks; nothing runs before a commit except you. Other workflows build Docker images, scan them, and publish `trek-plugin-sdk` on `plugin-sdk-v*` tags.
|
||||
`.github/workflows/test.yml` runs, per package: typecheck (server also `typecheck:tests`), lint (client through `lint:warnings`, which also holds the warning count per rule), `lint:pages`, `lint:rtl`, `lint:size` (client and server), the server's `lint:boundaries`, `lint:tx`, `lint:dialect` and `lint:test-sql`, the client's `lint:format`, `lint:layers`, `lint:offline`, `lint:dup`, `lint:skips` and `lint:i18n-keys`, shared's `contracts:open`, `check:plugin-facts`, `i18n:parity:strict` (with the untranslated-strings ratchet), the S3 contract suite, the Postgres probe (`postgres-probe`: every dialect helper on SQLite and Postgres, and the statements every repository method reaches with sample arguments against a Postgres service; the SQL errors per method and the list of methods it cannot measure are held to a baseline that only shrinks and fails while it is still unmeasured; see `server/CLAUDE.md`), the browser end-to-end flows (Chromium and WebKit), a client build with `check:gl-split`, coverage for all four packages, then a SonarCloud scan. It also builds the Docker image and boots it until `/api/health/ready` answers (`docker-image`), and lints and renders the Helm chart against the Kubernetes schemas and runs `docker compose config` on every compose file (`deploy-manifests`). `lint-prettier.yml` additionally runs `lint` + `format:check` on `shared/`. `plugin-sdk` has no lint script and is outside the eslint gates. **Not in CI**: `theme:lint` and the screenshot and help-media Playwright projects; run them locally. There are no git hooks; nothing runs before a commit except you. Other workflows build Docker images, scan them, and publish `trek-plugin-sdk` on `plugin-sdk-v*` tags.
|
||||
|
||||
- **`ci-ok` is the required check.** The branch ruleset requires `ci-ok` (plus `lint` and `check-target` from their own workflows), and `ci-ok` needs every job above except the scan. It fails if any of them failed or was cancelled and passes if they passed or were skipped. A new gate job counts only once it is listed in `ci-ok`'s `needs:`.
|
||||
- **What runs is decided by `changes`, not by `on.paths`**: `scripts/ci/changed-areas.mjs` maps the changed files to the `code`, `image` and `deploy` areas (tests: `npm run test:ci-scripts`). A path filter on the workflow would leave `ci-ok` unreported on a docs-only PR and block it. Add a new top-level file the build reads to the right area there.
|
||||
@@ -138,7 +139,7 @@ These principles come out of a full-repo audit and shape all new code:
|
||||
| Security / reliability / maintainability rating | A |
|
||||
| Security hotspots reviewed | 100% |
|
||||
|
||||
- **Duplication is the one that bites.** Desktop and phone shells (`client/src/components/<X>` ↔ `client/src/mobile/screens/.../M<X>`) are deliberate mirrors that are already heavily duplicated. Every line added identically to both counts against the 3% budget, so a feature touching both shells fails on its own. Put the logic in ONE hook/module (precedents: `useInstanceSettings`, `useRangeBypass` under `client/src/components/Admin/`) and leave only markup in each shell. Locale files are excluded from duplication, so i18n additions are free.
|
||||
- **Duplication is the one that bites.** Desktop and phone shells (`client/src/components/<X>` ↔ `client/src/mobile/screens/.../M<X>`) are deliberate mirrors that are already heavily duplicated. Every line added identically to both counts against the 3% budget, so a feature touching both shells fails on its own. Put the logic in ONE hook/module (precedents: `useInstanceSettings`, `useRangeBypass` under `client/src/components/Admin/`) and leave only markup in each shell. `lint:dup` finds the same copies in `client/src` before the push and holds every file to `client/scripts/dup-baseline.json`. Locale files are excluded from duplication, so i18n additions are free.
|
||||
- **Coverage is per new line across all four packages.** A new file without tests drags the PR under 80%. Files in `sonar.coverage.exclusions` don't count either way.
|
||||
- **Ratings are about new issues.** One new bug drops reliability; one new vulnerability drops security. Don't introduce hotspot patterns (`Math.random` in anything security-adjacent, `child_process` through PATH, backtracking regexes, secret-looking strings). Path-wide suppressions live only in `sonar-project.properties` with a written justification; one-off findings are resolved on SonarCloud, never with an inline `// NOSONAR`.
|
||||
- **Check it yourself.** The `sonarqube` MCP is connected: `list_pull_requests` → `get_project_quality_gate_status` shows each condition's value; `search_duplicated_files` + `get_duplications` show what tripped duplication; `search_sonar_issues_in_projects` lists new issues. Test files are excluded from analysis — when duplication fires, the source is the problem.
|
||||
|
||||
+2
-1
@@ -16,6 +16,7 @@ npm run lint:size # no source file or stylesheet past 1000 lines, no tes
|
||||
npm run lint:format # Prettier: every .ts/.tsx/.mjs/.css file under src/, tests/, e2e/ and scripts/ outside scripts/format-baseline.json must be formatted, the list only shrinks (CI gate; npx prettier --write <file>, then --update)
|
||||
npm run lint:layers # imports go downwards: components never import pages/ or mobile/, nothing under the views imports a view; per-file counts may only shrink (CI gate; --list, --update)
|
||||
npm run lint:offline # no view file (components/mobile/pages/hooks) imports src/api/ beyond scripts/offline-baseline.json, which only shrinks (CI gate; --list, --update)
|
||||
npm run lint:dup # copied code: the lines of a file inside a block that repeats elsewhere (100 tokens over 10 lines, SonarCloud's thresholds; .ts and .tsx read as one language) may only shrink against scripts/dup-baseline.json (CI gate; --list shows each block with its other side, --update lowers it)
|
||||
npm run lint:skips # no .only anywhere; skipped/todo/fixme tests per file may only shrink against scripts/skip-baseline.json (CI gate; skipIf/runIf and Playwright's test.skip(condition, 'why') are fine, but skipIf(true) and runIf(false) count as skips)
|
||||
npm run lint:i18n-keys # every translation key src/ names exists in shared en (CI gate; --unused lists en keys nothing reaches)
|
||||
npm run test # vitest run (tests/** + co-located src/**/*.test.{ts,tsx}); also test:unit / test:integration / test:coverage
|
||||
@@ -73,7 +74,7 @@ The offline core is flagship work surrounded by a periphery that ignores it. New
|
||||
|
||||
## Desktop ↔ mobile shells and the SonarQube duplication gate
|
||||
|
||||
Every desktop component with a phone twin is a deliberate copy that is already heavily duplicated. CI's SonarCloud gate allows **≤ 3% duplicated lines on the PR's new code**, so adding the same lines to both shells fails the PR by itself. Put logic in one shared hook/module (`useInstanceSettings`, `useRangeBypass` under `components/Admin/` are the pattern) and keep only markup in each shell. Full gate rules in the root `CLAUDE.md`.
|
||||
Every desktop component with a phone twin is a deliberate copy that is already heavily duplicated. CI's SonarCloud gate allows **≤ 3% duplicated lines on the PR's new code**, so adding the same lines to both shells fails the PR by itself. Put logic in one shared hook/module (`useInstanceSettings`, `useRangeBypass` under `components/Admin/` are the pattern) and keep only markup in each shell. `npm run lint:dup` finds such copies before the push and holds every file to `scripts/dup-baseline.json`. Full gate rules in the root `CLAUDE.md`.
|
||||
|
||||
## Tests
|
||||
|
||||
|
||||
@@ -27,6 +27,7 @@
|
||||
"lint:format": "node scripts/format-lint.mjs",
|
||||
"lint:layers": "node scripts/layer-lint.mjs",
|
||||
"lint:offline": "node scripts/offline-lint.mjs",
|
||||
"lint:dup": "node scripts/dup-lint.mjs",
|
||||
"lint:skips": "node scripts/skip-lint.mjs",
|
||||
"lint:i18n-keys": "node scripts/i18n-keys.mjs",
|
||||
"check:gl-split": "node scripts/check-gl-split.mjs",
|
||||
@@ -82,6 +83,8 @@
|
||||
},
|
||||
"devDependencies": {
|
||||
"@eslint/js": "^10.0.1",
|
||||
"@jscpd/core": "^4.2.5",
|
||||
"@jscpd/tokenizer": "^4.2.6",
|
||||
"@playwright/test": "^1.63.0",
|
||||
"@testing-library/jest-dom": "^6.10.0",
|
||||
"@testing-library/react": "^16.3.3",
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"src/components/Admin/AddonManager.tsx": 23,
|
||||
"src/components/Files/FileManagerMarkdownPreviewModal.tsx": 19,
|
||||
"src/components/Files/FileManagerPdfPreviewModal.tsx": 19,
|
||||
"src/components/Map/MapView.tsx": 56,
|
||||
"src/components/Map/MapViewGL.tsx": 51,
|
||||
"src/components/Map/ReservationOverlay.tsx": 75,
|
||||
"src/components/Map/RouteCalculator.ts": 27,
|
||||
"src/components/Map/reservationsMapbox.ts": 72,
|
||||
"src/components/Packing/ApplyTemplateButton.tsx": 19,
|
||||
"src/components/Packing/PackingListPanelHeader.tsx": 16,
|
||||
"src/components/Planner/AddressInput.tsx": 38,
|
||||
"src/components/Planner/AirportSelect.tsx": 44,
|
||||
"src/components/Planner/DayPlanSidebar.tsx": 326,
|
||||
"src/components/Planner/LocationSelect.tsx": 56,
|
||||
"src/components/Planner/ReservationModal.tsx": 26,
|
||||
"src/components/Planner/TransportModal.tsx": 26,
|
||||
"src/components/Studio/StudioInspector.tsx": 19,
|
||||
"src/components/Studio/StudioTravelInspector.tsx": 19,
|
||||
"src/components/Studio/TravelElements.tsx": 10,
|
||||
"src/components/Studio/roadRoute.ts": 17,
|
||||
"src/components/shared/CustomTimePicker.tsx": 24,
|
||||
"src/mobile/screens/admin/MAdminAddonManager.tsx": 23,
|
||||
"src/mobile/screens/admin/MAdminStoragePanel.tsx": 38,
|
||||
"src/mobile/screens/trip/sheets/MReservationSheet.tsx": 79,
|
||||
"src/mobile/screens/trip/sheets/MTransportFormSheet.tsx": 139,
|
||||
"src/mobile/screens/trip/tabs/MBookingsTab.tsx": 88,
|
||||
"src/mobile/screens/trip/tabs/MTransportsTab.tsx": 90,
|
||||
"src/mobile/screens/vacay/MVacayInviteSheet.tsx": 21,
|
||||
"src/mobile/screens/vacay/MVacayShareSheet.tsx": 21,
|
||||
"src/pages/TripPlannerPage.tsx": 82,
|
||||
"src/pages/admin/GoogleOptions.tsx": 13,
|
||||
"src/pages/admin/TrekApiCard.tsx": 14,
|
||||
"src/pages/atlas/AtlasCountrySearch.tsx": 38,
|
||||
"src/store/slices/budgetSlice.ts": 16,
|
||||
"src/store/slices/remoteEventHandler.ts": 17
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
#!/usr/bin/env node
|
||||
/*
|
||||
* lint:dup: keeps copied code from coming back.
|
||||
*
|
||||
* Most features have a desktop view and a phone view, and each used to carry
|
||||
* its own copy of the feature's logic, so a fix reached one of them and the
|
||||
* other kept the bug. The logic now lives in one shared hook or module per
|
||||
* feature, and this check holds the copies that remain to what they are. It
|
||||
* counts, per file under src/, the lines inside a block of code that repeats
|
||||
* elsewhere (or in the same file), at the thresholds SonarCloud uses on a
|
||||
* pull request. The files that still hold such lines are listed in
|
||||
* scripts/dup-baseline.json with their count, and a file may hold no more
|
||||
* than its entry (a file without an entry holds none). An entry above what
|
||||
* its file holds now, or for a file that is gone, fails as well until
|
||||
* --update lowers it.
|
||||
*
|
||||
* npm run lint:dup check against the baseline (CI)
|
||||
* npm run lint:dup -- --list print every copied block with its other side
|
||||
* npm run lint:dup -- --update lower the baseline to what the files hold now;
|
||||
* it never raises an entry
|
||||
*
|
||||
* The measuring lives in scripts/lib/dup.mjs.
|
||||
*/
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { check } from './lib/dup.mjs';
|
||||
import { runCli } from './lib/ratchet.mjs';
|
||||
|
||||
const root = fileURLToPath(new URL('..', import.meta.url));
|
||||
await runCli('lint:dup', (args) => check({ root, update: args.includes('--update'), list: args.includes('--list') }));
|
||||
@@ -0,0 +1,139 @@
|
||||
/*
|
||||
* The measuring and the comparison behind lint:dup (scripts/dup-lint.mjs).
|
||||
*
|
||||
* Copies are found by jscpd's detector at the thresholds SonarCloud applies to
|
||||
* TypeScript: a block counts once MIN_TOKENS tokens repeat over at least
|
||||
* MIN_LINES lines. Both .ts and .tsx are read as TypeScript, because Sonar
|
||||
* treats them as one language and a hook in a .ts file that repeats a view's
|
||||
* .tsx code is a copy all the same. A file's count is the number of its lines
|
||||
* inside a copied block, the duplicated lines Sonar reports. Which of three
|
||||
* copies a block is matched against depends on the order the files are read
|
||||
* in, so they are read sorted by path, and with readText, so every machine
|
||||
* counts the same.
|
||||
*/
|
||||
import { Detector, getModeHandler, MemoryStore } from '@jscpd/core';
|
||||
import { Tokenizer } from '@jscpd/tokenizer';
|
||||
import { join } from 'node:path';
|
||||
import {
|
||||
countMap,
|
||||
listFiles,
|
||||
lowerCounts,
|
||||
readBaseline,
|
||||
readText,
|
||||
reportStale,
|
||||
staleCounts,
|
||||
TEST_FILE,
|
||||
toKey,
|
||||
writeBaseline,
|
||||
} from './ratchet.mjs';
|
||||
|
||||
export const MIN_TOKENS = 100;
|
||||
export const MIN_LINES = 10;
|
||||
|
||||
export const DIRS = ['src'];
|
||||
|
||||
/** The files SonarCloud analyses as source: tests and declaration files are left out. */
|
||||
export function accepts(key) {
|
||||
return /\.tsx?$/.test(key) && !key.endsWith('.d.ts') && !TEST_FILE.test(key);
|
||||
}
|
||||
|
||||
/**
|
||||
* Every copied block under root as a pair of line ranges, one per side. Both
|
||||
* sides can be the same file.
|
||||
*
|
||||
* @returns {Promise<{ key: string, start: number, end: number }[][]>}
|
||||
*/
|
||||
export async function findCopies(root) {
|
||||
const keys = listFiles(root, DIRS, accepts)
|
||||
.map((path) => toKey(root, path))
|
||||
.sort();
|
||||
const detector = new Detector(new Tokenizer(), new MemoryStore(), [], {
|
||||
minTokens: MIN_TOKENS,
|
||||
minLines: MIN_LINES,
|
||||
mode: getModeHandler('mild'),
|
||||
});
|
||||
const side = ({ sourceId, start, end }) => ({ key: sourceId, start: start.line, end: end.line });
|
||||
const copies = [];
|
||||
for (const key of keys) {
|
||||
const clones = await detector.detect(key, readText(join(root, key)), 'typescript');
|
||||
for (const clone of clones) copies.push([side(clone.duplicationA), side(clone.duplicationB)]);
|
||||
}
|
||||
return copies;
|
||||
}
|
||||
|
||||
/** Per file in path order, how many of its lines sit inside a copied block. */
|
||||
export function countLines(copies) {
|
||||
const lines = {};
|
||||
for (const pair of copies) {
|
||||
for (const { key, start, end } of pair) {
|
||||
const set = (lines[key] ??= new Set());
|
||||
for (let line = start; line <= end; line++) set.add(line);
|
||||
}
|
||||
}
|
||||
return Object.fromEntries(
|
||||
Object.entries(lines)
|
||||
.filter(([, set]) => set.size > 0)
|
||||
.sort(([a], [b]) => (a < b ? -1 : 1))
|
||||
.map(([key, set]) => [key, set.size])
|
||||
);
|
||||
}
|
||||
|
||||
/** Per file, its copied blocks as "start-end = other:start-end", in line order. */
|
||||
export function describeCopies(copies) {
|
||||
const found = {};
|
||||
const range = ({ start, end }) => `${start}-${end}`;
|
||||
for (const [a, b] of copies) {
|
||||
(found[a.key] ??= []).push({ at: a.start, text: `${range(a)} = ${b.key}:${range(b)}` });
|
||||
(found[b.key] ??= []).push({ at: b.start, text: `${range(b)} = ${a.key}:${range(a)}` });
|
||||
}
|
||||
return Object.fromEntries(
|
||||
Object.entries(found).map(([key, list]) => [key, list.sort((x, y) => x.at - y.at).map(({ text }) => text)])
|
||||
);
|
||||
}
|
||||
|
||||
/** Runs the check against the baseline at baselinePath and returns the exit code. */
|
||||
export async function check({
|
||||
root,
|
||||
baselinePath = join(root, 'scripts/dup-baseline.json'),
|
||||
update = false,
|
||||
list = false,
|
||||
log = console.log,
|
||||
error = console.error,
|
||||
}) {
|
||||
const copies = await findCopies(root);
|
||||
const counts = countLines(copies);
|
||||
const blocks = describeCopies(copies);
|
||||
let baseline = readBaseline(baselinePath, countMap);
|
||||
|
||||
if (list) {
|
||||
for (const [key, n] of Object.entries(counts)) {
|
||||
log(`${key} (${n}, baseline ${baseline[key] ?? 0})`);
|
||||
for (const line of blocks[key]) log(` ${line}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (update) {
|
||||
baseline = lowerCounts(baseline, counts);
|
||||
writeBaseline(baselinePath, baseline);
|
||||
}
|
||||
|
||||
const grown = Object.entries(counts).filter(([key, n]) => n > (baseline[key] ?? 0));
|
||||
const stale = staleCounts(baseline, counts);
|
||||
|
||||
for (const [key, n] of grown) {
|
||||
error(`FAIL ${key}: ${n} line(s) in copied blocks, baseline ${baseline[key] ?? 0}:`);
|
||||
for (const line of blocks[key]) error(` ${line}`);
|
||||
}
|
||||
if (grown.length) {
|
||||
error(
|
||||
'Put the code both places need in one shared hook or module and import it from both instead of copying it ' +
|
||||
'(useInstanceSettings and useRangeBypass under src/components/Admin/ are the pattern).'
|
||||
);
|
||||
}
|
||||
reportStale(stale, { file: 'dup-baseline.json', command: 'lint:dup', root, error });
|
||||
const sum = (map) => Object.values(map).reduce((a, b) => a + b, 0);
|
||||
log(
|
||||
`dup: ${sum(counts)} line(s) in copied blocks in ${Object.keys(counts).length} file(s), baseline allows ${sum(baseline)}`
|
||||
);
|
||||
return grown.length || stale.length ? 1 : 0;
|
||||
}
|
||||
@@ -0,0 +1,145 @@
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { afterEach, describe, expect, it } from 'vitest';
|
||||
import { check, MIN_LINES } from '../../../scripts/lib/dup.mjs';
|
||||
import { RatchetError } from '../../../scripts/lib/ratchet.mjs';
|
||||
import { ratchetTree, type RatchetTree } from '../../helpers/ratchetFixture';
|
||||
|
||||
let tree: RatchetTree;
|
||||
afterEach(() => tree?.remove());
|
||||
|
||||
const BASELINE = 'scripts/dup-baseline.json';
|
||||
|
||||
/**
|
||||
* n lines of about twenty tokens each, every line different, so a block of
|
||||
* MIN_LINES + 4 of them is long enough to count as a copy and repeats nothing
|
||||
* inside itself. Two blocks with the same name are the same code.
|
||||
*/
|
||||
function block(name = 'shared', n = MIN_LINES + 4): string {
|
||||
return Array.from({ length: n }, (_, i) => `export const ${name}${i} = compute(${i}, 'x', [1, 2, 3]);\n`).join('');
|
||||
}
|
||||
|
||||
function run(
|
||||
baseline: Record<string, number> | string,
|
||||
files: Record<string, string>,
|
||||
options: { update?: boolean; list?: boolean } = {}
|
||||
) {
|
||||
tree = ratchetTree({
|
||||
[BASELINE]: typeof baseline === 'string' ? baseline : JSON.stringify(baseline),
|
||||
...files,
|
||||
});
|
||||
return check({ root: tree.root, baselinePath: tree.path(BASELINE), ...options, ...tree.out });
|
||||
}
|
||||
|
||||
describe('lint:dup', () => {
|
||||
it('DUP-001: fails both sides of a block two files share, naming the other side', async () => {
|
||||
expect(await run({}, { 'src/a/View.tsx': block(), 'src/b/MView.tsx': block() })).toBe(1);
|
||||
const out = tree.error.join('\n');
|
||||
expect(out).toMatch(
|
||||
/FAIL {2}src\/a\/View\.tsx: 14 line\(s\) in copied blocks, baseline 0:\n {8}1-14 = src\/b\/MView\.tsx:1-14/
|
||||
);
|
||||
expect(out).toMatch(
|
||||
/FAIL {2}src\/b\/MView\.tsx: 14 line\(s\) in copied blocks, baseline 0:\n {8}1-14 = src\/a\/View\.tsx:1-14/
|
||||
);
|
||||
expect(out).toMatch(/one shared hook or module/);
|
||||
expect(tree.log.join('\n')).toMatch(/dup: 28 line\(s\) in copied blocks in 2 file\(s\), baseline allows 0/);
|
||||
});
|
||||
|
||||
it('DUP-002: passes different code, and a repeat too short in lines or tokens to count', async () => {
|
||||
const shortLines = block('k', MIN_LINES - 1);
|
||||
const fewTokens = Array.from({ length: MIN_LINES + 4 }, (_, i) => `s${i}()\n`).join('');
|
||||
const files = {
|
||||
'src/a.ts': block('a') + fewTokens,
|
||||
'src/b.ts': block('b') + fewTokens,
|
||||
'src/c.ts': shortLines,
|
||||
'src/d.ts': shortLines,
|
||||
};
|
||||
expect(await run({}, files)).toBe(0);
|
||||
expect(tree.error).toEqual([]);
|
||||
});
|
||||
|
||||
it('DUP-003: reads .ts and .tsx as one language, so a hook and a view that repeat it are a copy', async () => {
|
||||
expect(await run({}, { 'src/components/X/useThing.ts': block(), 'src/mobile/MThing.tsx': block() })).toBe(1);
|
||||
const out = tree.error.join('\n');
|
||||
expect(out).toMatch(/FAIL {2}src\/components\/X\/useThing\.ts: 14 line/);
|
||||
expect(out).toMatch(/FAIL {2}src\/mobile\/MThing\.tsx: 14 line/);
|
||||
});
|
||||
|
||||
it('DUP-004: leaves tests and declaration files out, as SonarCloud does', async () => {
|
||||
const files = {
|
||||
'src/a.test.tsx': block(),
|
||||
'src/b.test.ts': block(),
|
||||
'src/c.d.ts': block(),
|
||||
'src/real.ts': block('real'),
|
||||
};
|
||||
expect(await run({}, files)).toBe(0);
|
||||
expect(tree.error).toEqual([]);
|
||||
});
|
||||
|
||||
it('DUP-005: counts a block one file repeats inside itself', async () => {
|
||||
expect(await run({}, { 'src/a.ts': block() + block('mid') + block() })).toBe(1);
|
||||
expect(tree.error.join('\n')).toMatch(/FAIL {2}src\/a\.ts: 28 line\(s\)[^\n]*\n {8}1-14 = src\/a\.ts:29-42/);
|
||||
});
|
||||
|
||||
it('DUP-006: holds a baselined file at its entry and fails a new copy or a grown one', async () => {
|
||||
const held = { 'src/a.ts': 14, 'src/b.ts': 14 };
|
||||
expect(await run(held, { 'src/a.ts': block(), 'src/b.ts': block() })).toBe(0);
|
||||
tree.remove();
|
||||
// A third copy of code that is already copied is still a new copy.
|
||||
expect(await run(held, { 'src/a.ts': block(), 'src/b.ts': block(), 'src/c.ts': block() })).toBe(1);
|
||||
expect(tree.error.join('\n')).toMatch(/FAIL {2}src\/c\.ts: 14 line\(s\) in copied blocks, baseline 0/);
|
||||
expect(tree.error.join('\n')).not.toMatch(/FAIL {2}src\/[ab]\.ts/);
|
||||
tree.remove();
|
||||
const grown = block('x') + block('y');
|
||||
expect(await run(held, { 'src/a.ts': grown, 'src/b.ts': grown })).toBe(1);
|
||||
expect(tree.error.join('\n')).toMatch(/FAIL {2}src\/a\.ts: 28 line\(s\) in copied blocks, baseline 14/);
|
||||
});
|
||||
|
||||
it('DUP-007: --update lowers an entry, drops one without copies and never raises', async () => {
|
||||
const baseline = { 'src/a.ts': 30, 'src/b.ts': 10, 'src/gone.ts': 5, 'src/clean.ts': 10 };
|
||||
const files = { 'src/a.ts': block(), 'src/b.ts': block(), 'src/clean.ts': block('clean') };
|
||||
expect(await run(baseline, files, { update: true })).toBe(1);
|
||||
expect(JSON.parse(readFileSync(tree.path(BASELINE), 'utf8'))).toEqual({ 'src/a.ts': 14, 'src/b.ts': 10 });
|
||||
expect(tree.error.join('\n')).toMatch(/FAIL {2}src\/b\.ts: 14 line\(s\) in copied blocks, baseline 10/);
|
||||
});
|
||||
|
||||
it('DUP-008: fails an entry above what its file holds, or for a file that is gone, until --update', async () => {
|
||||
expect(await run({ 'src/a.ts': 20, 'src/b.ts': 14 }, { 'src/a.ts': block(), 'src/b.ts': block() })).toBe(1);
|
||||
expect(tree.error.join('\n')).toMatch(
|
||||
/src\/a\.ts is held at 20 in scripts\/dup-baseline\.json, but there are 14 now/
|
||||
);
|
||||
expect(tree.error.join('\n')).toMatch(/npm run lint:dup -- --update/);
|
||||
tree.remove();
|
||||
expect(await run({ 'src/gone.ts': 14 }, { 'src/a.ts': block('a') })).toBe(1);
|
||||
expect(tree.error.join('\n')).toMatch(/src\/gone\.ts is held at 14 .*but the file is gone/);
|
||||
tree.remove();
|
||||
// A file whose copies are all gone keeps no allowance either.
|
||||
expect(await run({ 'src/a.ts': 14 }, { 'src/a.ts': block('a') })).toBe(1);
|
||||
expect(tree.error.join('\n')).toMatch(/src\/a\.ts is held at 14 .*but there are 0 now/);
|
||||
tree.remove();
|
||||
expect(await run({ 'src/a.ts': 14, 'src/gone.ts': 14 }, { 'src/a.ts': block('a') }, { update: true })).toBe(0);
|
||||
});
|
||||
|
||||
it('DUP-009: counts a file with CRLF line ends the same as one with LF', async () => {
|
||||
expect(await run({}, { 'src/a.ts': block().replace(/\n/g, '\r\n'), 'src/b.ts': block() })).toBe(1);
|
||||
expect(tree.error.join('\n')).toMatch(/FAIL {2}src\/a\.ts: 14 line\(s\)[^\n]*\n {8}1-14 = src\/b\.ts:1-14/);
|
||||
});
|
||||
|
||||
it('DUP-010: --list prints every copied block with its other side', async () => {
|
||||
expect(
|
||||
await run({ 'src/a.ts': 14, 'src/b.ts': 14 }, { 'src/a.ts': block(), 'src/b.ts': block() }, { list: true })
|
||||
).toBe(0);
|
||||
expect(tree.log).toEqual([
|
||||
'src/a.ts (14, baseline 14)',
|
||||
' 1-14 = src/b.ts:1-14',
|
||||
'src/b.ts (14, baseline 14)',
|
||||
' 1-14 = src/a.ts:1-14',
|
||||
'dup: 28 line(s) in copied blocks in 2 file(s), baseline allows 28',
|
||||
]);
|
||||
});
|
||||
|
||||
it('DUP-011: a broken baseline or a missing src/ stops the check', async () => {
|
||||
await expect(run('{', { 'src/a.ts': block('a') })).rejects.toThrow(RatchetError);
|
||||
tree.remove();
|
||||
await expect(run({}, { 'tests/a.ts': block('a') })).rejects.toThrow(/src\/ does not exist/);
|
||||
});
|
||||
});
|
||||
Generated
+37
@@ -69,6 +69,8 @@
|
||||
},
|
||||
"devDependencies": {
|
||||
"@eslint/js": "^10.0.1",
|
||||
"@jscpd/core": "^4.2.5",
|
||||
"@jscpd/tokenizer": "^4.2.6",
|
||||
"@playwright/test": "^1.63.0",
|
||||
"@testing-library/jest-dom": "^6.10.0",
|
||||
"@testing-library/react": "^16.3.3",
|
||||
@@ -4626,6 +4628,27 @@
|
||||
"@jridgewell/sourcemap-codec": "^1.4.14"
|
||||
}
|
||||
},
|
||||
"node_modules/@jscpd/core": {
|
||||
"version": "4.2.5",
|
||||
"resolved": "https://registry.npmjs.org/@jscpd/core/-/core-4.2.5.tgz",
|
||||
"integrity": "sha512-Esf2deHxaoNEjePwf2jqP6Urzj+BAOsJVPFLbnnSsV+q7rLNMcn0UEEoKBXIOOt4qMkrkhl9DfwpMyPPOr6GkQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"eventemitter3": "^5.0.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@jscpd/tokenizer": {
|
||||
"version": "4.2.6",
|
||||
"resolved": "https://registry.npmjs.org/@jscpd/tokenizer/-/tokenizer-4.2.6.tgz",
|
||||
"integrity": "sha512-/eyFjINWLs2mrBTU4H681bs855r5oRyl1O3mZxcd7TpL1JIG85a7pps0RkRPAe9c/2KcjVCc82HbhOz86M0n5g==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@jscpd/core": "4.2.5",
|
||||
"spark-md5": "^3.0.2"
|
||||
}
|
||||
},
|
||||
"node_modules/@keyv/serialize": {
|
||||
"version": "1.1.1",
|
||||
"resolved": "https://registry.npmjs.org/@keyv/serialize/-/serialize-1.1.1.tgz",
|
||||
@@ -11793,6 +11816,13 @@
|
||||
"node": ">= 0.6"
|
||||
}
|
||||
},
|
||||
"node_modules/eventemitter3": {
|
||||
"version": "5.0.4",
|
||||
"resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.4.tgz",
|
||||
"integrity": "sha512-mlsTRyGaPBjPedk6Bvw+aqbsXDtoAyAzm5MO7JgU+yVRyMQ5O8bD4Kcci7BS85f93veegeCPkL8R4GLClnjLFw==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/events-universal": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/events-universal/-/events-universal-1.0.1.tgz",
|
||||
@@ -19216,6 +19246,13 @@
|
||||
"url": "https://github.com/sponsors/wooorm"
|
||||
}
|
||||
},
|
||||
"node_modules/spark-md5": {
|
||||
"version": "3.0.2",
|
||||
"resolved": "https://registry.npmjs.org/spark-md5/-/spark-md5-3.0.2.tgz",
|
||||
"integrity": "sha512-wcFzz9cDfbuqe0FZzfi2or1sgyIrsDwmPwfZC4hiNidPdPINjeUwNfv5kldczoEAcjl9Y1L3SM7Uz2PUEQzxQw==",
|
||||
"dev": true,
|
||||
"license": "(WTFPL OR MIT)"
|
||||
},
|
||||
"node_modules/split2": {
|
||||
"version": "4.2.0",
|
||||
"resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
|
||||
|
||||
@@ -205,6 +205,7 @@ The `@trek/shared` package is the single source of truth for code shared between
|
||||
| `npm run lint:format` | Every file outside the shrinking baseline is Prettier-formatted (CI gate) |
|
||||
| `npm run lint:layers` | Imports only go downwards through the layers (CI gate) |
|
||||
| `npm run lint:offline` | Views reach `src/api/` only through the offline core; the baseline only shrinks (CI gate) |
|
||||
| `npm run lint:dup` | Lines in copied code blocks may only shrink per file, at SonarCloud's thresholds (CI gate) |
|
||||
| `npm run lint:skips` | No focused test, and skipped tests may only go away (CI gate) |
|
||||
| `npm run lint:i18n-keys` | Every translation key the client names exists in `en` (CI gate) |
|
||||
| `npm run theme:lint` | Flag styling that bypasses the appearance tokens (not run in CI) |
|
||||
@@ -242,6 +243,6 @@ Then open a Pull Request from your fork to `liketrek/TREK` targeting the `dev` b
|
||||
- Run tests before pushing: `npm test` at the repo root runs all workspaces. That alone is not the full CI gate. With `shared` built, these are the checks CI runs before any test (`.github/workflows/test.yml` is the source of truth):
|
||||
- in `shared/`: `npm run typecheck && npm run contracts:open`
|
||||
- in `server/`: `npm run build && npm run typecheck && npm run typecheck:tests && npm run typecheck:scripts && npm run check:entities && npm run check:db-types && npm run lint:check && npm run db:call-graph -- --sync --tx && npm run lint:size && npm run lint:boundaries && npm run lint:tx && npm run lint:test-sql && npm run lint:dialect && npm run check:plugin-facts`
|
||||
- in `client/`: `npm run typecheck && npm run lint:warnings && npm run lint:pages && npm run lint:rtl && npm run lint:size && npm run lint:format && npm run lint:layers && npm run lint:offline && npm run lint:skips && npm run lint:i18n-keys`
|
||||
- in `client/`: `npm run typecheck && npm run lint:warnings && npm run lint:pages && npm run lint:rtl && npm run lint:size && npm run lint:format && npm run lint:layers && npm run lint:offline && npm run lint:dup && npm run lint:skips && npm run lint:i18n-keys`
|
||||
- at the root, if you touched translations: `npm run i18n:parity:strict --workspace=shared`
|
||||
- Follow the commit message conventions described in the [[Contributing]] guidelines.
|
||||
|
||||
Reference in New Issue
Block a user