k1nq/TREK
1
0
Fork 0
mirror of https://github.com/mauriceboe/TREK.git synced 2026-10-18 21:16:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(docsync): follow a rename on Synology, and settle restores and purges in one run

Browse Source 
Where the id is the path, a rename or a move upstream lists the same
copy under a new id. The upstream-only loop re-paired it with a touch,
which went wrong twice. The pairing had already been counted as
vanished, so the same run flagged it missing as well and counted it
towards the mass-delete guard. And touch wrote the provider's new name
into remote_name while the trip file kept the old one, so the next run
read that as a rename made in TREK and renamed the provider's copy
back. The heuristic also matched on a content hash only, which Synology
never sends: on the one provider without stable ids it did not fire at
all, and every rename there ended as a missing flag plus a second
download of the same bytes.

The planner now moves such a pairing first, with a relocate action that
updates remote_id and the version and leaves the agreed name alone. The
pair loop then treats it like any listed document, so the new name
comes into TREK through rename_local exactly as it does where ids are
stable, and nothing counts as vanished. A copy is recognised by its
content hash where the listing has one, otherwise by size and
modification time, which a rename or a move leaves untouched. Only a
one-to-one match is followed, and a copy TREK binned itself never ties
with a live twin of the same bytes. The rows now load remote_size and
remote_modified_at, which were written on pull and never read; touch
keeps them current and a push records them. rename_local and the
provider branch of resolveConflict take the provider name in its
cleaned form, since a raw name with a colon or a slash read as a rename
made in TREK on the next run.

Restores and purges:
- A file taken back out of TREK's trash whose copy somebody else had
  deleted meanwhile resumed as synced and was then counted as vanished,
  so a few such restores tripped the mass-delete guard on every run
  until the circuit opened. It now lands as remote_missing directly,
  outside the guard, and the service asks for a full listing while such
  a restore is pending.
- A row in error that once held a file now reads as a deletion once the
  file is purged, told apart from a first download that never landed by
  the agreed hash. "Sync now" used to download the deleted document
  again.
- Moves are followed before a restore is judged, so a binned copy
  restored at the provider under a new path resumes its pairing instead
  of being duplicated on both sides.
- relocate and both renames lift a remote_missing flag the way touch
  does.
- A double rename under trek_wins or provider_wins is settled with a
  rename instead of a transfer.
This commit is contained in:
Maurice
2026-09-19 13:35:55 +02:00
parent 98e2938033
commit 71400b7e85
4 changed files with 1136 additions and 155 deletions
Download Patch File Download Diff File Expand all files Collapse all files
server/tests/unit/nest/doc-sync/doc-sync.helpers.test.ts
+473 -32
View File
@@ -3,6 +3,7 @@ import {
backoffSeconds,
isAllowedByOperator,
isBlockedName,
needsFullListing,
planReconcile,
sanitizeIncomingName,
type LocalDocument,
@@ -13,8 +14,8 @@ import type { RemoteDocument } from '../../../../src/nest/doc-sync/document-prov
/**
* The planner decides everything that matters about two-way sync, and it does
* it without a database, a container or a network. That is the point of having
* it: the cases that are painful to reproduce against a live provider — an
* unmounted share, a rename with no stable id, TREK's own write echoing back —
* it: the cases that are painful to reproduce against a live provider (an
* unmounted share, a rename with no stable id, TREK's own write echoing back)
* are cheap to state here as data.
*/
@@ -47,6 +48,8 @@ const item = (over: Partial<SyncItemState> = {}): SyncItemState => ({
remoteId: 'r1',
remoteVersion: 'v1',
remoteName: 'boarding.pdf',
remoteSize: null,
remoteModifiedAt: null,
contentSha256: 'aaa',
pushedSha256: null,
state: 'synced',
@@ -156,22 +159,6 @@ describe('planReconcile', () => {
expect(p.actions.filter((a) => a.kind === 'mark_remote_missing')).toHaveLength(0);
});
/**
* Synology FileStation has no stable file id, so a rename upstream looks like
* a deletion plus a new file. Re-pairing on content avoids downloading the
* same bytes again and leaving a phantom behind.
*/
it('re-pairs a renamed document when the provider has no stable ids', () => {
const p = plan({
items: [item({ remoteId: '/trek/old.pdf', contentSha256: 'hash-x' })],
remote: [remote({ remoteId: '/trek/new.pdf', contentHash: 'hash-x' })],
local: [local()],
stableRemoteIds: false,
});
expect(p.actions.some((a) => a.kind === 'pull')).toBe(false);
expect(p.actions.some((a) => a.kind === 'touch')).toBe(true);
});
/**
* Which side renamed is decided against the name both sides last agreed on.
* Reading a `both` binding as "TREK always wins" renamed the provider's copy
@@ -346,8 +333,8 @@ describe('backoffSeconds', () => {
*
* Found in a live database at 46 attempts against a limit of 6: the counter and
* the backoff were written on every failure and never consulted when planning
* the next run, so a document a provider refuses — a file it reads as corrupt,
* a type it will not take, a full quota — was re-uploaded on every cron tick
* the next run, so a document a provider refuses (a file it reads as corrupt,
* a type it will not take, a full quota) was re-uploaded on every cron tick
* for as long as it existed.
*/
describe('planReconcile > shelving a row that keeps failing', () => {
@@ -416,7 +403,7 @@ describe('planReconcile > shelving a row that keeps failing', () => {
it('leaves a shelved row alone even when its provider copy is gone too', () => {
// "Vanished" is only said about a row that was synced last run, and a row is
// only shelved while it is in error, so these two never describe the same
// row — nothing is planned for it either way.
// row. Nothing is planned for it either way.
const p = plan({
items: [item({ id: 30, attempts: 99, state: 'error', remoteId: 'r-gone' })],
remote: [],
@@ -434,7 +421,7 @@ describe('planReconcile > shelving a row that keeps failing', () => {
* they then return an empty document list. The core ignored the flag and read
* that emptiness as a folder somebody had emptied: from the third run of every
* idle Nextcloud or OpenCloud binding onward, every run ended in the mass-delete
* guard — `partial`, `missing: 6`, forever, until the failure counter opened the
* guard: `partial`, `missing: 6`, forever, until the failure counter opened the
* circuit and the scheduler dropped the binding entirely. Reproduced against a
* live Nextcloud before this was written.
*/
@@ -600,8 +587,8 @@ describe('planReconcile > a pairing whose download never landed', () => {
/**
* Names are compared through the same sanitiser the local one went through.
*
* A provider name TREK had to clean up — a slash, a control character, a
* leading dot — otherwise reads as "TREK renamed this document" on the very
* A provider name TREK had to clean up (a slash, a control character, a
* leading dot) otherwise reads as "TREK renamed this document" on the very
* next run, and the provider's copy is renamed to the cleaned version without
* anybody asking for it.
*/
@@ -623,6 +610,325 @@ describe('planReconcile > a provider name TREK had to clean up', () => {
});
expect(p.actions.map(a => a.kind)).toEqual(['rename_local']);
});
it('takes a new name into TREK the way a download would have written it, so it stays', () => {
// Taken over raw, the next run compared the cleaned provider name with the
// raw one in TREK, read the difference as a rename made in TREK, and sent
// the name back upstream: a Paperless title with a colon, renamed to itself.
const renamed = remote({ remoteId: 'r1', name: 'Hotel: Kyoto.pdf', remoteVersion: 'v1' });
const first = plan({
items: [item({ remoteName: 'b.pdf' })],
remote: [renamed],
local: [local({ fileId: 1, name: 'b.pdf' })],
});
expect(first.actions).toEqual([{ kind: 'rename_local', itemId: 10, fileId: 1, name: 'Hotel_ Kyoto.pdf' }]);
const second = plan({
items: [item({ remoteName: 'Hotel_ Kyoto.pdf' })],
remote: [renamed],
local: [local({ fileId: 1, name: 'Hotel_ Kyoto.pdf' })],
});
expect(second.actions).toEqual([{ kind: 'touch', itemId: 10, remote: renamed }]);
});
});
/**
* A rename upstream where the id is the path, which is Synology FileStation.
*
* It bounced. The upstream-only loop re-paired the renamed copy with a `touch`,
* and `touch` wrote the provider's new name into the rename arbiter while the
* trip file kept the old one: the next run read that as a rename made in TREK
* and renamed the provider's copy back. The same run had flagged the pairing
* missing too, because the vanished list was drawn up before the re-pairing.
* Observed: run 1 [touch b.pdf, mark_remote_missing], run 2 [rename_remote a.pdf].
*/
describe('planReconcile > a rename upstream where the id is the path', () => {
const at = '2026-09-18T10:00:00Z';
const paired = (over: Partial<SyncItemState> = {}) => item({
remoteId: '/trek/a.pdf', remoteVersion: 'va', remoteName: 'a.pdf', contentSha256: 'hash-a',
remoteSize: 1024, remoteModifiedAt: at, ...over,
});
const listed = (over: Partial<RemoteDocument> = {}) => remote({
remoteId: '/trek/b.pdf', name: 'b.pdf', remoteVersion: 'vb', contentHash: 'hash-a',
size: 1024, remoteModifiedAt: at, ...over,
});
// No local hash, as in production: TREK has no way to change a file's bytes,
// so the service never reports one (see loadLocalDocuments).
const file = (over: Partial<LocalDocument> = {}) => local({ name: 'a.pdf', sha256: null, ...over });
const pathIds = (over: Partial<Parameters<typeof planReconcile>[0]> = {}) => plan({ stableRemoteIds: false, ...over });
const kinds = (p: ReturnType<typeof plan>) => p.actions.map(a => a.kind);
it('takes the new name into TREK and leaves it there on the next run', () => {
const first = pathIds({ items: [paired()], remote: [listed()], local: [file()] });
expect(first.actions).toEqual([
{ kind: 'relocate', itemId: 10, remote: listed() },
{ kind: 'rename_local', itemId: 10, fileId: 1, name: 'b.pdf' },
]);
// What the executor leaves behind: the pairing on the new path, the agreed
// name moved on by rename_local, the trip file renamed.
const second = pathIds({
items: [paired({ remoteId: '/trek/b.pdf', remoteVersion: 'vb', remoteName: 'b.pdf' })],
remote: [listed()],
local: [file({ name: 'b.pdf' })],
});
expect(second.actions).toEqual([{ kind: 'touch', itemId: 10, remote: listed() }]);
});
it('ends the way it ends where ids are stable, apart from moving the pairing', () => {
const pathKeyed = pathIds({ items: [paired()], remote: [listed()], local: [file()] });
const stable = plan({
items: [paired({ remoteId: '/trek/b.pdf', remoteVersion: 'vb' })],
remote: [listed()],
local: [file()],
});
expect(pathKeyed.actions.filter(a => a.kind !== 'relocate')).toEqual(stable.actions);
});
it('does not flag the renamed copy as missing', () => {
const p = pathIds({ items: [paired()], remote: [listed()], local: [file()] });
expect(p.missingCount).toBe(0);
expect(kinds(p)).not.toContain('mark_remote_missing');
});
it('does not read a whole folder moved in one go as a mass deletion', () => {
const rows = Array.from({ length: 10 }, (_, i) => paired({
id: 100 + i, fileId: 100 + i, remoteId: `/trek/${i}.pdf`, remoteName: `${i}.pdf`, contentSha256: `hash-${i}`,
}));
const p = pathIds({
items: rows,
remote: rows.map((_, i) => listed({ remoteId: `/trek/2026/${i}.pdf`, name: `${i}.pdf`, contentHash: `hash-${i}` })),
local: rows.map((r, i) => file({ fileId: r.fileId as number, name: `${i}.pdf` })),
});
expect(p.massDeleteGuardTripped).toBe(false);
expect(p.missingCount).toBe(0);
expect(kinds(p).filter(k => k === 'relocate')).toHaveLength(10);
expect(kinds(p)).not.toContain('pull');
});
it('follows a copy moved to another folder and leaves its name alone', () => {
const moved = listed({ remoteId: '/trek/receipts/a.pdf', name: 'a.pdf' });
const p = pathIds({ items: [paired()], remote: [moved], local: [file()] });
expect(p.actions).toEqual([
{ kind: 'relocate', itemId: 10, remote: moved },
{ kind: 'touch', itemId: 10, remote: moved },
]);
});
it('recognises the copy by size and time where the listing carries no hash', () => {
// Synology offers no sha256 at all, so this is the case that actually
// happens there; the hash-only version of this never fired.
const p = pathIds({ items: [paired()], remote: [listed({ contentHash: null })], local: [file()] });
expect(kinds(p)).toEqual(['relocate', 'rename_local']);
});
it('does not follow into a copy with other bytes, whatever its size and time', () => {
const p = pathIds({ items: [paired()], remote: [listed({ contentHash: 'hash-other' })], local: [file()] });
expect(kinds(p)).toEqual(['pull', 'mark_remote_missing']);
});
it('does not follow into a copy that was edited as well', () => {
const edited = listed({ contentHash: null, remoteModifiedAt: '2026-09-19T08:00:00Z' });
const p = pathIds({ items: [paired()], remote: [edited], local: [file()] });
expect(kinds(p)).toEqual(['pull', 'mark_remote_missing']);
});
it('does not guess from size and time it never recorded', () => {
const p = pathIds({
items: [paired({ remoteSize: null, remoteModifiedAt: null })],
remote: [listed({ contentHash: null })],
local: [file()],
});
expect(kinds(p)).toEqual(['pull', 'mark_remote_missing']);
});
it('pairs nothing when two copies of the same bytes were renamed at once', () => {
// Either way round fits the content, and the crosswise one puts each name
// on the other trip file. A missing flag and a second download can be seen
// and undone; a swapped pairing cannot be seen at all.
const p = pathIds({
items: [
paired({ id: 1, fileId: 1, remoteId: '/trek/a1.pdf', remoteName: 'a1.pdf' }),
paired({ id: 2, fileId: 2, remoteId: '/trek/a2.pdf', remoteName: 'a2.pdf' }),
],
remote: [listed({ remoteId: '/trek/b1.pdf', name: 'b1.pdf' }), listed({ remoteId: '/trek/b2.pdf', name: 'b2.pdf' })],
local: [file({ fileId: 1, name: 'a1.pdf' }), file({ fileId: 2, name: 'a2.pdf' })],
});
expect(kinds(p).sort()).toEqual(['mark_remote_missing', 'mark_remote_missing', 'pull', 'pull']);
});
it('tells two copies of the same bytes apart when they were moved and kept their names', () => {
const one = listed({ remoteId: '/trek/2026/a1.pdf', name: 'a1.pdf' });
const two = listed({ remoteId: '/trek/2026/a2.pdf', name: 'a2.pdf' });
const p = pathIds({
items: [
paired({ id: 1, fileId: 1, remoteId: '/trek/a1.pdf', remoteName: 'a1.pdf' }),
paired({ id: 2, fileId: 2, remoteId: '/trek/a2.pdf', remoteName: 'a2.pdf' }),
],
remote: [two, one],
local: [file({ fileId: 1, name: 'a1.pdf' }), file({ fileId: 2, name: 'a2.pdf' })],
});
expect(p.actions.filter(a => a.kind === 'relocate')).toEqual([
{ kind: 'relocate', itemId: 1, remote: one },
{ kind: 'relocate', itemId: 2, remote: two },
]);
expect(kinds(p).filter(k => k !== 'relocate')).toEqual(['touch', 'touch']);
});
it('follows the one of two identical copies that was renamed and leaves the other', () => {
const kept = listed({ remoteId: '/trek/a2.pdf', name: 'a2.pdf', remoteVersion: 'va' });
const renamed = listed({ remoteId: '/trek/b1.pdf', name: 'b1.pdf' });
const p = pathIds({
items: [
paired({ id: 1, fileId: 1, remoteId: '/trek/a1.pdf', remoteName: 'a1.pdf' }),
paired({ id: 2, fileId: 2, remoteId: '/trek/a2.pdf', remoteName: 'a2.pdf' }),
],
remote: [kept, renamed],
local: [file({ fileId: 1, name: 'a1.pdf' }), file({ fileId: 2, name: 'a2.pdf' })],
});
expect(p.actions).toEqual([
{ kind: 'relocate', itemId: 1, remote: renamed },
{ kind: 'rename_local', itemId: 1, fileId: 1, name: 'b1.pdf' },
{ kind: 'touch', itemId: 2, remote: kept },
]);
});
it('finds a copy again after TREK renamed it upstream', () => {
// The adapter cannot say what path a rename produced, so the row keeps the
// old one (and the name rename_remote agreed) until the next listing.
const p = pathIds({
items: [paired({ remoteName: 'b.pdf', remoteVersion: 'vb' })],
remote: [listed({ contentHash: null })],
local: [file({ name: 'b.pdf' })],
});
expect(kinds(p)).toEqual(['relocate', 'touch']);
});
it('keeps a document deleted in TREK deleted when its copy is renamed upstream', () => {
// Where ids are stable a pairing already acted on stays put. Here the
// renamed copy would otherwise come back as a new document.
const p = pathIds({
items: [paired({ state: 'local_deleted' })],
remote: [listed()],
local: [file({ deletedAt: '2026-09-18 08:00:00' })],
});
expect(p.actions).toEqual([{ kind: 'relocate', itemId: 10, remote: listed() }]);
});
describe('next to a twin TREK binned', () => {
// Deleting one of two identical files in TREK is the usual way two rows
// come to share content. The binned one used to count as a candidate, so a
// rename of the live one was a tie and came back as missing plus new.
const binned = (over: Partial<SyncItemState> = {}) => paired({
id: 1, fileId: 1, remoteId: '/trek/a1.pdf', remoteName: 'a1.pdf',
state: 'local_deleted', remoteTrashedAt: '2026-09-18 08:05:00', ...over,
});
const live = paired({ id: 2, fileId: 2, remoteId: '/trek/a2.pdf', remoteName: 'a2.pdf' });
const renamed = listed({ remoteId: '/trek/b.pdf', name: 'b.pdf', contentHash: null });
it('follows the rename of the live one', () => {
const p = pathIds({
items: [binned(), live],
remote: [renamed],
local: [file({ fileId: 1, name: 'a1.pdf', deletedAt: '2026-09-18 08:00:00' }), file({ fileId: 2, name: 'a2.pdf' })],
});
expect(p.actions).toEqual([
{ kind: 'relocate', itemId: 2, remote: renamed },
{ kind: 'rename_local', itemId: 2, fileId: 2, name: 'b.pdf' },
]);
expect(p.missingCount).toBe(0);
});
it('does so once the binned one is purged as well', () => {
const p = pathIds({ items: [binned({ fileId: null }), live], remote: [renamed], local: [file({ fileId: 2, name: 'a2.pdf' })] });
expect(kinds(p)).toEqual(['relocate', 'rename_local']);
expect(p.actions[0]).toMatchObject({ itemId: 2 });
});
it('still recognises the binned copy when it comes back elsewhere on its own', () => {
const back = listed({ remoteId: '/trek/restored/a1.pdf', name: 'a1.pdf', contentHash: null });
const p = pathIds({
items: [binned()],
remote: [back],
local: [file({ fileId: 1, name: 'a1.pdf', deletedAt: '2026-09-18 08:00:00' })],
});
expect(p.actions).toEqual([
{ kind: 'relocate', itemId: 1, remote: back },
{ kind: 'remote_restored', itemId: 1 },
]);
});
});
it('leaves a shelved row where it is: that waits for a person', () => {
const p = pathIds({
items: [paired({ state: 'error', attempts: 6, fileId: null, contentSha256: null })],
remote: [listed({ contentHash: null })],
local: [],
});
expect(kinds(p)).not.toContain('relocate');
});
it('follows nothing where ids are stable: a new id there is a new document', () => {
const p = plan({ items: [paired()], remote: [listed()], local: [file()] });
expect(kinds(p)).toEqual(['pull', 'mark_remote_missing']);
});
it('follows nothing under an unchanged upstream, which lists nothing to follow into', () => {
const p = pathIds({ items: [paired()], remote: [listed()], local: [file()], remoteUnchanged: true });
expect(p.actions).toEqual([]);
});
});
/**
* Both sides renamed a document whose bytes agree: only the name is in dispute.
*
* `trek_wins` used to settle it like changed bytes, with a push_update. That
* re-sent identical bytes, and where the provider replaces in place (WebDAV,
* and Synology once renames there were followed) the upload kept the
* provider's name: the next run read it as a rename made upstream and renamed
* TREK's file to it. Observed: run 1 [relocate, push_update], run 2 [rename_local].
*/
describe('planReconcile > both sides renamed', () => {
const at = '2026-09-18T10:00:00Z';
const paired = item({ remoteId: '/trek/a.pdf', remoteName: 'a.pdf', remoteSize: 1024, remoteModifiedAt: at });
const theirs = remote({ remoteId: '/trek/c.pdf', name: 'c.pdf', remoteModifiedAt: at });
const mine = local({ name: 'b.pdf', sha256: null });
const pathIds = (over: Partial<Parameters<typeof planReconcile>[0]> = {}) =>
plan({ stableRemoteIds: false, items: [paired], remote: [theirs], local: [mine], ...over });
it('carries TREK\'s name upstream when TREK wins, and sends no bytes', () => {
const p = pathIds({ conflictPolicy: 'trek_wins' });
expect(p.actions).toEqual([
{ kind: 'relocate', itemId: 10, remote: theirs },
{ kind: 'rename_remote', itemId: 10, remoteId: '/trek/c.pdf', name: 'b.pdf' },
]);
});
it('takes the provider\'s name into TREK when the provider wins, and fetches nothing', () => {
const p = pathIds({ conflictPolicy: 'provider_wins' });
expect(p.actions).toEqual([
{ kind: 'relocate', itemId: 10, remote: theirs },
{ kind: 'rename_local', itemId: 10, fileId: 1, name: 'c.pdf' },
]);
});
it('does the same where ids are stable', () => {
const stable = (conflictPolicy: 'trek_wins' | 'provider_wins') => plan({
conflictPolicy,
items: [item({ remoteName: 'old.pdf' })],
remote: [remote({ name: 'theirs.pdf' })],
local: [local({ name: 'mine.pdf' })],
}).actions;
expect(stable('trek_wins')).toEqual([{ kind: 'rename_remote', itemId: 10, remoteId: 'r1', name: 'mine.pdf' }]);
expect(stable('provider_wins')).toEqual([{ kind: 'rename_local', itemId: 10, fileId: 1, name: 'theirs.pdf' }]);
});
it('parks it when the direction forbids what the winner would need', () => {
expect(pathIds({ conflictPolicy: 'trek_wins', direction: 'pull' }).actions.map(a => a.kind))
.toEqual(['relocate', 'conflict']);
expect(pathIds({ conflictPolicy: 'provider_wins', direction: 'push' }).actions.map(a => a.kind))
.toEqual(['relocate', 'conflict']);
});
});
/**
@@ -685,6 +991,47 @@ describe('planReconcile > a deletion TREK already acted on', () => {
});
expect(p.actions).toEqual([{ kind: 'remote_restored', itemId: 50 }]);
});
/**
* A failed update keeps the file it had and sets `error`. Deleted and purged
* while the row was held back, it read as a first download that never
* landed, and the next attempt ("Sync now" clears the counter) brought the
* deleted document straight back.
*/
describe('on a row that failed an update first', () => {
const failedUpdate = (over: Partial<SyncItemState> = {}) =>
item({ fileId: null, state: 'error', attempts: 2, contentSha256: 'aaa', ...over });
it('reads the purge as a deletion', () => {
const p = plan({ items: [failedUpdate()], remote: [remote()], local: [] });
expect(p.actions).toEqual([{ kind: 'local_deleted', itemId: 10, remoteId: 'r1' }]);
});
it('does the same under an unchanged upstream', () => {
const p = plan({ items: [failedUpdate()], remote: [], local: [], remoteUnchanged: true });
expect(p.actions).toEqual([{ kind: 'local_deleted', itemId: 10, remoteId: 'r1' }]);
});
it('waits out the backoff like any other action for the row', () => {
const p = plan({
items: [failedUpdate({ nextAttemptAt: '2030-01-01 00:00:00' })],
remote: [remote()],
local: [],
now: '2026-09-19 08:00:00',
});
expect(p.actions).toEqual([]);
});
it('treats a row waiting on a conflict answer the same way', () => {
const p = plan({ items: [failedUpdate({ state: 'pending', attempts: 0 })], remote: [remote()], local: [] });
expect(p.actions).toEqual([{ kind: 'local_deleted', itemId: 10, remoteId: 'r1' }]);
});
it('still retries a first download, which never agreed on a hash', () => {
const p = plan({ items: [failedUpdate({ contentSha256: null })], remote: [remote()], local: [] });
expect(p.actions.map(a => a.kind)).toEqual(['pull_update']);
});
});
});
/**
@@ -713,15 +1060,25 @@ describe('planReconcile > a file taken back out of TREK trash', () => {
it('flags a copy somebody else deleted meanwhile instead of uploading it', () => {
const p = plan({ items: [settled()], remote: [], local: [back] });
expect(p.actions).toEqual([
{ kind: 'local_restored', itemId: 50 },
{ kind: 'mark_remote_missing', itemId: 50 },
]);
expect(p.actions).toEqual([{ kind: 'local_restored', itemId: 50, missing: true }]);
expect(p.missingCount).toBe(1);
});
it('resumes under an unchanged upstream and leaves the listing to a later run', () => {
it('waits under an unchanged upstream, which shows nothing of the copy', () => {
// Resumed as synced here, a copy that was long gone counted as one that
// vanished on the next real listing, and could trip the guard with it.
const p = plan({ items: [settled()], remote: [], local: [back], remoteUnchanged: true });
expect(p.actions).toEqual([{ kind: 'local_restored', itemId: 50 }]);
expect(p.actions).toEqual([]);
});
it('waits on a truncated listing that does not show the copy', () => {
const p = plan({ items: [settled()], remote: [], local: [back], remoteTruncated: true });
expect(p.actions).toEqual([]);
});
it('resumes from a truncated listing that does show the copy', () => {
const p = plan({ items: [settled()], remote: [remote({ remoteId: 'r5' })], local: [back], remoteTruncated: true });
expect(p.actions.map(a => a.kind)).toEqual(['local_restored', 'touch']);
});
it('uploads it again when TREK was the one that binned the copy', () => {
@@ -766,10 +1123,94 @@ describe('planReconcile > a file taken back out of TREK trash', () => {
expect(p.actions.filter(a => a.kind === 'push')).toHaveLength(10);
});
it('counts many restored rows whose copies vanished like any synced rows, guard included', () => {
it('does not read many restores whose copies are gone as a mass deletion', () => {
// Nothing was recorded about those copies while TREK's files sat in the
// bin, so they may have been gone for weeks: the listing did not shrink.
// Counted as vanished, they tripped the guard, the run was dropped before
// the restores were written, and every run after it planned the same.
const rows = Array.from({ length: 10 }, (_, i) =>
settled({ id: 100 + i, fileId: 100 + i, remoteId: `r${100 + i}` }));
const p = plan({ items: rows, remote: [], local: rows.map((r) => local({ fileId: r.fileId as number })) });
expect(p.massDeleteGuardTripped).toBe(true);
expect(p.massDeleteGuardTripped).toBe(false);
expect(p.actions).toEqual(rows.map((r) => ({ kind: 'local_restored', itemId: r.id, missing: true })));
expect(p.missingCount).toBe(10);
});
describe('beside a binding that is otherwise in step', () => {
const synced = Array.from({ length: 6 }, (_, i) => item({ id: 200 + i, fileId: 200 + i, remoteId: `s${i}` }));
const restored = Array.from({ length: 5 }, (_, i) => settled({ id: 300 + i, fileId: 300 + i, remoteId: `g${i}` }));
const files = [...synced, ...restored].map((r) => local({ fileId: r.fileId as number }));
const newUpstream = remote({ remoteId: 'fresh', name: 'visa.pdf' });
it('leaves the guard alone and syncs the rest as usual', () => {
const p = plan({
items: [...synced, ...restored],
remote: [...synced.map((s) => remote({ remoteId: s.remoteId as string })), newUpstream],
local: files,
});
expect(p.massDeleteGuardTripped).toBe(false);
expect(p.actions.filter(a => a.kind === 'local_restored'))
.toEqual(restored.map((r) => ({ kind: 'local_restored', itemId: r.id, missing: true })));
expect(p.actions).toContainEqual({ kind: 'pull', remote: newUpstream, itemId: null });
expect(p.actions.map(a => a.kind)).not.toContain('mark_remote_missing');
});
it('still trips the guard when the synced rows themselves vanish', () => {
const p = plan({ items: synced, remote: [remote({ remoteId: 's0' })], local: files });
expect(p.massDeleteGuardTripped).toBe(true);
});
});
it('resumes when the binned copy was restored at the provider under a new path', () => {
// Detached first, the row lost its id before the move could be followed:
// the restored copy came down as a new document and TREK's file went up as
// another, two of each.
const p = plan({
stableRemoteIds: false,
items: [binnedByTrek({ remoteId: '/trek/a.pdf', remoteName: 'a.pdf', remoteSize: 1024, remoteModifiedAt: '2026-09-18T10:00:00Z' })],
remote: [remote({ remoteId: '/trek/a restored.pdf', name: 'a restored.pdf' })],
local: [local({ fileId: 5, name: 'a.pdf', sha256: null })],
});
expect(p.actions.map(a => a.kind)).toEqual(['relocate', 'local_restored', 'rename_local']);
});
it('follows a copy moved while the file was in the bin, when TREK did not bin it', () => {
const moved = remote({ remoteId: '/trek/2026/a.pdf', name: 'a.pdf' });
const p = plan({
stableRemoteIds: false,
items: [settled({ remoteId: '/trek/a.pdf', remoteName: 'a.pdf', remoteSize: 1024, remoteModifiedAt: '2026-09-18T10:00:00Z' })],
remote: [moved],
local: [local({ fileId: 5, name: 'a.pdf', sha256: null })],
});
expect(p.actions).toEqual([
{ kind: 'relocate', itemId: 50, remote: moved },
{ kind: 'local_restored', itemId: 50 },
{ kind: 'touch', itemId: 50, remote: moved },
]);
});
});
describe('needsFullListing', () => {
const settled = (over: Partial<SyncItemState> = {}) =>
item({ id: 50, fileId: 5, remoteId: 'r5', state: 'local_deleted', ...over });
it('asks for one once a file whose copy TREK left alone is back', () => {
expect(needsFullListing([settled()], [local({ fileId: 5 })])).toBe(true);
});
it('does not while the file is still in the trash', () => {
expect(needsFullListing([settled()], [local({ fileId: 5, deletedAt: '2026-09-18 08:00:00' })])).toBe(false);
});
it('does not for a copy TREK binned: that gap is known, and the file goes up again', () => {
expect(needsFullListing([settled({ remoteTrashedAt: '2026-09-18 08:05:00' })], [local({ fileId: 5 })])).toBe(false);
});
it('does not once the file is purged', () => {
expect(needsFullListing([settled({ fileId: null })], [])).toBe(false);
});
it('does not for pairings that are simply in step', () => {
expect(needsFullListing([item()], [local()])).toBe(false);
});
});
server/tests/unit/nest/doc-sync/doc-sync.service.test.ts
+335 -19
View File
@@ -8,7 +8,7 @@
* read back with plain SQL rather than through the service that wrote it.
*
* The provider is a fake because a unit test must not open a socket. Storage
* and the files domain are fakes because the bytes are not what is under test —
* and the files domain are fakes because the bytes are not what is under test,
* but the fake files service writes real `trip_files` rows, so "nothing was
* downloaded" is asserted against the table the file manager reads, not against
* a spy alone.
@@ -129,7 +129,7 @@ const provider = {
listScopes: vi.fn(async (): Promise<DocResult<DocumentScopeOption[]>> => ok([SCOPE])),
createScope: vi.fn(async (): Promise<DocResult<DocumentScopeOption>> => ok(SCOPE)),
resolveScope: vi.fn(async (): Promise<DocResult<DocumentScopeOption>> => ok(SCOPE)),
list: vi.fn(async (): Promise<DocResult<Listing>> => ok(listing())),
list: vi.fn(async (_conn: DocumentConnectionRef, _scope: DocumentScopeRef): Promise<DocResult<Listing>> => ok(listing())),
fetch: vi.fn(async (): Promise<DocResult<FetchResult>> =>
ok({ body: Readable.from([FILE_BYTES]), size: FILE_BYTES.length, mimeType: 'application/pdf', remoteVersion: 'v9' })),
push: vi.fn(async (
@@ -188,6 +188,7 @@ let scopeSeq = 0;
function makeLink(over: {
direction?: string;
deletePolicy?: string;
conflictPolicy?: string;
syncEnabled?: number;
failureCount?: number;
nextAttemptAt?: string | null;
@@ -199,11 +200,11 @@ function makeLink(over: {
`INSERT INTO trip_document_links
(trip_id, connection_id, provider_id, remote_scope_key, remote_root_id, remote_root_path, remote_label,
direction, delete_policy, conflict_policy, sync_enabled, failure_count, next_attempt_at, created_by)
VALUES (?, ?, 'paperless', ?, '1', '/TREK/japan', 'Japan 2026', ?, ?, 'manual', ?, ?, ?, ?)`,
VALUES (?, ?, 'paperless', ?, '1', '/TREK/japan', 'Japan 2026', ?, ?, ?, ?, ?, ?, ?)`,
)
.run(
tripId, connectionId, `tag:${scopeSeq}`,
over.direction ?? 'both', over.deletePolicy ?? 'unlink',
over.direction ?? 'both', over.deletePolicy ?? 'unlink', over.conflictPolicy ?? 'manual',
over.syncEnabled ?? 1, over.failureCount ?? 0, over.nextAttemptAt ?? null, ownerId,
);
if (over.lastSyncAt !== undefined) {
@@ -222,19 +223,24 @@ function seedItem(link: LinkRow, over: {
state?: string;
contentSha256?: string;
pushedSha256?: string;
remoteSize?: number;
remoteModifiedAt?: string;
remoteMissingAt?: string;
} = {}): number {
uidSeq += 1;
const info = testDb
.prepare(
`INSERT INTO document_sync_items
(link_id, trip_id, file_id, trek_doc_uid, remote_id, remote_name, remote_version,
content_sha256, pushed_sha256, state)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
remote_size, remote_modified_at, content_sha256, pushed_sha256, state, remote_missing_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
)
.run(
link.id, link.trip_id, over.fileId ?? null, `uid-${uidSeq}`,
over.remoteId ?? null, over.remoteName ?? null, over.remoteVersion ?? null,
over.remoteSize ?? null, over.remoteModifiedAt ?? null,
over.contentSha256 ?? null, over.pushedSha256 ?? null, over.state ?? 'synced',
over.remoteMissingAt ?? null,
);
return Number(info.lastInsertRowid);
}
@@ -549,7 +555,7 @@ describe('DocSyncService', () => {
expect(afterPush.content_sha256).toBe(FILE_SHA);
expect(afterPush.remote_id).toBe('r-pushed');
// Same bytes back, under a new version marker — which is what a provider
// Same bytes back, under a new version marker, which is what a provider
// reports for TREK's own write.
provider.list.mockResolvedValueOnce(ok(listing([
remoteDoc({ remoteId: 'r-pushed', name: 'boarding.pdf', remoteVersion: 'v2', contentHash: FILE_SHA }),
@@ -567,7 +573,7 @@ describe('DocSyncService', () => {
it('refuses to move a pairing onto a document another file already owns', async () => {
// Papra answers a push of bytes it already holds with the document that
// has them. Two trip files with the same content therefore push to ONE
// document — and the unique index on (link_id, remote_id) turned that
// document, and the unique index on (link_id, remote_id) turned that
// into a constraint error that aborted this run and every run after it.
const link = makeLink();
makeFile({ name: 'boarding.pdf' });
@@ -576,8 +582,8 @@ describe('DocSyncService', () => {
const run = await service.syncLink(link);
// Counted as a conflict, which is what it is — a document two files want
// — and reported the same way as any other conflict rather than as a
// Counted as a conflict, which is what it is (a document two files
// want), and reported the same way as any other conflict rather than as a
// failed run.
expect(run.pushed).toBe(1);
expect(run.conflicts).toBe(1);
@@ -639,7 +645,7 @@ describe('DocSyncService', () => {
/**
* A chat screenshot and a note attachment live in `trip_files` like every
* other document, but they belong to a conversation rather than to the
* trip's paperwork — pushing them would put someone's chat into a shared
* trip's paperwork: pushing them would put someone's chat into a shared
* Paperless.
*/
it('leaves chat and note attachments out of the upload entirely', async () => {
@@ -749,6 +755,64 @@ describe('DocSyncService', () => {
expect(service.issues(tripId).map((r) => r.id)).toContain(itemId);
});
it('settles a double rename TREK wins with a rename, not a second upload', async () => {
const link = makeLink({ conflictPolicy: 'trek_wins' });
const fileId = makeFile({ name: 'mine.pdf' });
const itemId = seedItem(link, {
remoteId: 'r1', remoteName: 'old.pdf', remoteVersion: 'v1', fileId, state: 'synced',
});
provider.list.mockResolvedValueOnce(ok(listing([remoteDoc({ remoteId: 'r1', name: 'theirs.pdf' })])));
const res = await service.syncLink(link);
expect(res.conflicts).toBe(0);
expect(provider.push).not.toHaveBeenCalled();
expect(provider.rename).toHaveBeenCalledWith(expect.anything(), expect.anything(), 'r1', 'mine.pdf');
expect(itemRow(itemId)).toMatchObject({ state: 'synced', remote_name: 'mine.pdf' });
provider.list.mockResolvedValueOnce(ok(listing([remoteDoc({ remoteId: 'r1', name: 'mine.pdf', remoteVersion: 'v2' })])));
await service.syncLink(config.getLink(link.id));
expect(provider.rename).toHaveBeenCalledTimes(1);
expect(fileRows().map((f) => f.original_name)).toEqual(['mine.pdf']);
});
/**
* Only `touch` used to lift the missing flag, and a copy that came back
* under another name gets a rename instead, so it stayed on the issues list
* for another run.
*/
it('takes a copy on record as missing off the list when it is back under a new name', async () => {
const link = makeLink();
const fileId = makeFile({ name: 'old.pdf' });
const itemId = seedItem(link, {
remoteId: 'r1', remoteName: 'old.pdf', remoteVersion: 'v1', fileId, state: 'remote_missing',
remoteMissingAt: '2026-09-18 09:00:00',
});
provider.list.mockResolvedValueOnce(ok(listing([remoteDoc({ remoteId: 'r1', name: 'new-upstream.pdf' })])));
await service.syncLink(link);
expect(fileRows()[0].original_name).toBe('new-upstream.pdf');
expect(itemRow(itemId)).toMatchObject({ state: 'synced', remote_missing_at: null, remote_name: 'new-upstream.pdf' });
expect(service.issues(tripId)).toEqual([]);
});
it('does the same when the name TREK gave it meanwhile is carried upstream', async () => {
const link = makeLink();
const fileId = makeFile({ name: 'new.pdf' });
const itemId = seedItem(link, {
remoteId: 'r1', remoteName: 'old.pdf', remoteVersion: 'v1', fileId, state: 'remote_missing',
remoteMissingAt: '2026-09-18 09:00:00',
});
provider.list.mockResolvedValueOnce(ok(listing([remoteDoc({ remoteId: 'r1', name: 'old.pdf' })])));
await service.syncLink(link);
expect(provider.rename).toHaveBeenCalledWith(expect.anything(), expect.anything(), 'r1', 'new.pdf');
expect(itemRow(itemId)).toMatchObject({ state: 'synced', remote_missing_at: null, remote_name: 'new.pdf' });
});
/**
* The size a listing claims is not a promise. The transfer is cut off at
* the cap while it is still being written, so an oversized body can never
@@ -792,7 +856,7 @@ describe('DocSyncService', () => {
/**
* Pins what happens today, and it is wrong. The failed pull already wrote
* the provider's version marker into the pairing, so the next run sees no
* change and emits `touch` — and `touch` promotes `error` to `synced`. The
* change and emits `touch`, and `touch` promotes `error` to `synced`. The
* document is then booked as synced with no `file_id`, out of the issues
* list and never retried. Reported rather than fixed here; this is the test
* that turns red once it is.
@@ -958,7 +1022,7 @@ describe('DocSyncService', () => {
* pin what the code does TODAY, and it is the wrong way round: 'trek'
* clears `content_sha256`, which is the only record that TREK's copy ever
* moved, so the run pulls the provider's version instead of sending TREK's.
* Reported rather than fixed here — these are the tests that turn red when
* Reported rather than fixed here; these are the tests that turn red when
* the two branches are put back.
*/
it('does not pull the provider copy when the conflict was resolved in TREK favour', async () => {
@@ -994,8 +1058,8 @@ describe('DocSyncService', () => {
/**
* The double rename is the only conflict a person can actually produce, and
* it is arbitrated by `remote_name` alone. Clearing a version marker left
* all three names untouched, so the next run raised the same conflict again
* — whatever the owner chose, the issue came back within minutes, forever.
* all three names untouched, so the next run raised the same conflict again:
* whatever the owner chose, the issue came back within minutes, forever.
*/
describe('a double rename', () => {
function seedRenameConflict(link: LinkRow) {
@@ -1039,6 +1103,27 @@ describe('DocSyncService', () => {
expect(file.original_name).toBe('Rechnung.pdf');
});
it('takes the provider name back the way a download would have written it', async () => {
// Written raw, the colon made the next run read TREK's name as a second
// rename of its own, and the conflict the owner had just settled came
// straight back.
const link = makeLink();
const fileId = makeFile({ name: 'Invoice.pdf' });
const itemId = seedItem(link, {
remoteId: 'r1', remoteName: 'Hotel: Kyoto.pdf', remoteVersion: 'v1', fileId, state: 'conflict', contentSha256: 'agreed-hash',
});
provider.list.mockResolvedValue(ok(listing([
remoteDoc({ remoteId: 'r1', name: 'Hotel: Kyoto 2026.pdf', remoteVersion: 'v1' }),
])));
await service.resolveConflict(itemId, 'provider');
expect(itemRow(itemId).state).not.toBe('conflict');
expect(provider.rename).not.toHaveBeenCalled();
const file = testDb.prepare('SELECT original_name FROM trip_files WHERE id = ?').get(fileId) as { original_name: string };
expect(file.original_name).toBe('Hotel_ Kyoto 2026.pdf');
});
it('does not leave the row in conflict either way', async () => {
for (const keep of ['trek', 'provider'] as const) {
const link = makeLink();
@@ -1090,7 +1175,7 @@ describe('DocSyncService', () => {
* An upstream edit replaces TREK's copy instead of joining it.
*
* `pull_update` routes into the same `pull()` as a first download, and
* `FilesService.createFile` only inserts — so the edit arrived as a SECOND
* `FilesService.createFile` only inserts, so the edit arrived as a SECOND
* document while the first stayed in the file manager holding the old bytes.
* Worse, the pairing moves to the new row, which leaves the old one with no
* sync item: the next run reads it as a document TREK gained and pushes the
@@ -1317,6 +1402,15 @@ describe('DocSyncService', () => {
const restore = (fileId: number) =>
testDb.prepare('UPDATE trip_files SET deleted_at = NULL WHERE id = ?').run(fileId);
/**
* A provider answering the way the WebDAV adapters do: asked with the
* cursor it handed out, "nothing changed" and no documents at all; asked
* without one, the whole folder.
*/
const quietUnlessAsked = (documents: RemoteDocument[]) =>
async (_conn: DocumentConnectionRef, scope: DocumentScopeRef) =>
ok(scope.cursor === null ? listing(documents) : listing([], { cursorUnchanged: true }));
/** A synced pairing whose file was deleted in TREK and whose deletion one run has seen. */
async function deletedAndSeen(link: LinkRow): Promise<{ itemId: number; fileId: number }> {
const fileId = makeFile({ name: 'boarding.pdf', deletedAt: '2026-09-18 08:00:00' });
@@ -1447,8 +1541,8 @@ describe('DocSyncService', () => {
it('does not upload a copy TREK binned that was restored at the provider meanwhile', async () => {
// Seen while TREK's file was still in the trash, so the note that TREK
// binned it is dropped. Without that, a later restore under an unchanged
// upstream (no listing to look at) would upload a second copy.
// binned it is dropped. Without that, a later restore on a quiet binding
// would upload a second copy.
const link = makeLink({ deletePolicy: 'trash' });
const { itemId, fileId } = await deletedAndSeen(link);
@@ -1458,7 +1552,7 @@ describe('DocSyncService', () => {
expect(provider.fetch).not.toHaveBeenCalled();
restore(fileId);
provider.list.mockResolvedValueOnce(ok(listing([], { cursorUnchanged: true })));
provider.list.mockImplementationOnce(quietUnlessAsked([remoteDoc({ remoteId: 'r1' })]));
await service.syncLink(config.getLink(link.id));
expect(provider.push).not.toHaveBeenCalled();
@@ -1466,6 +1560,60 @@ describe('DocSyncService', () => {
expect(itemRow(itemId).remote_id).toBe('r1');
});
it('asks a quiet binding for the whole listing and flags a copy that is gone', async () => {
// An unchanged cursor shows nothing of the copy, and on a quiet binding
// it stays unchanged until somebody touches the folder. Resumed blind,
// the row was booked as synced and counted as vanished later on.
const link = makeLink({ deletePolicy: 'unlink' });
const { itemId, fileId } = await deletedAndSeen(link);
const visa = remoteDoc({ remoteId: 'r-visa', name: 'visa.pdf' });
restore(fileId);
provider.list.mockImplementation(quietUnlessAsked([visa]));
const run = await service.syncLink(config.getLink(link.id));
expect(provider.list.mock.calls[1][1].cursor).toBeNull();
expect(run).toMatchObject({ state: 'ok', pulled: 1, pushed: 0, missing: 1 });
expect(itemRow(itemId)).toMatchObject({ state: 'remote_missing', remote_trashed_at: null });
expect(itemRow(itemId).remote_missing_at).not.toBeNull();
// Once the restore is settled the binding goes back to its cursor.
await service.syncLink(config.getLink(link.id));
expect(provider.list.mock.calls[2][1].cursor).toBe('cursor-1');
});
it('keeps the binding running when several restored files lost their copies meanwhile', async () => {
// Counted as vanished, five of them next to six in step tripped the
// mass-delete guard. The run was dropped before the restores were
// written, the next run planned the same, and the binding synced nothing
// at all, new documents included, until its circuit opened.
const link = makeLink({ deletePolicy: 'unlink' });
const inStep = Array.from({ length: 6 }, (_, i) => {
const fileId = makeFile({ name: `doc-${i}.pdf`, storageKey: `key-doc-${i}` });
seedItem(link, { remoteId: `s${i}`, remoteName: `doc-${i}.pdf`, remoteVersion: 'v1', fileId, contentSha256: FILE_SHA });
return remoteDoc({ remoteId: `s${i}`, name: `doc-${i}.pdf` });
});
const restored = Array.from({ length: 5 }, (_, i) => seedItem(link, {
remoteId: `g${i}`, remoteName: `gone-${i}.pdf`, remoteVersion: 'v1', state: 'local_deleted', contentSha256: FILE_SHA,
fileId: makeFile({ name: `gone-${i}.pdf`, storageKey: `key-gone-${i}` }),
}));
makeFile({ name: 'new-here.pdf', storageKey: 'key-new-here' });
const newThere = remoteDoc({ remoteId: 'r-new', name: 'new-there.pdf' });
provider.list.mockResolvedValueOnce(ok(listing([...inStep, newThere])));
const first = await service.syncLink(link);
provider.list.mockResolvedValueOnce(ok(listing([
...inStep, newThere, remoteDoc({ remoteId: 'r-pushed', name: 'new-here.pdf' }),
])));
const second = await service.syncLink(config.getLink(link.id));
expect(first).toMatchObject({ state: 'ok', pulled: 1, pushed: 1, missing: 5 });
expect(first.errorCode).toBeUndefined();
expect(second).toMatchObject({ state: 'ok', pulled: 0, pushed: 0, missing: 0 });
expect(linkRow(link.id)).toMatchObject({ failure_count: 0, last_sync_state: 'ok' });
for (const id of restored) expect(itemRow(id).state).toBe('remote_missing');
});
it('flags the copy as missing when it vanished upstream by somebody else', async () => {
const link = makeLink({ deletePolicy: 'unlink' });
const { itemId, fileId } = await deletedAndSeen(link);
@@ -1520,6 +1668,174 @@ describe('DocSyncService', () => {
expect(fileRows()).toHaveLength(0);
expect(itemRow(itemId).state).toBe('local_deleted');
});
it('does not bring back a document purged while its failed update was held back', async () => {
// A failed update keeps the file and leaves the row in `error`, where the
// backoff drops the deletion. Purged in that window, the row read as a
// first download that never landed, and "Sync now" fetched it again.
const link = makeLink({ deletePolicy: 'trash' });
const fileId = makeFile({ name: 'boarding.pdf' });
const itemId = seedItem(link, {
remoteId: 'r1', remoteName: 'boarding.pdf', remoteVersion: 'v1', fileId, state: 'synced', contentSha256: FILE_SHA,
});
provider.list.mockResolvedValue(ok(listing([remoteDoc({ remoteId: 'r1', remoteVersion: 'v2' })])));
provider.fetch.mockResolvedValueOnce(fail('timeout'));
await service.syncLink(link);
expect(itemRow(itemId)).toMatchObject({ state: 'error', file_id: fileId });
purge(fileId);
service.retryShelvedItems(link.id);
await service.syncLink(config.getLink(link.id));
expect(provider.fetch).toHaveBeenCalledTimes(1);
expect(fileRows()).toHaveLength(0);
expect(provider.trash).toHaveBeenCalledTimes(1);
expect(itemRow(itemId).state).toBe('local_deleted');
});
});
/**
* A rename where the id is the path, walked through real runs.
*
* On Synology a rename upstream lists the same copy under a new id. The
* pairing used to be re-attached with a `touch` that wrote the new name into
* the arbiter while the trip file kept the old one, and was flagged missing
* in the same run; the run after that renamed the provider's copy back.
*/
describe('a rename where the id is the path', () => {
const AT = '2026-09-18T10:00:00Z';
beforeEach(() => {
capabilities = { ...BASE_CAPABILITIES, stableId: false, contentHashInListing: false };
});
it('takes a rename made at the provider into TREK and does not rename it back', async () => {
const link = makeLink();
const fileId = makeFile({ name: 'a.pdf' });
const itemId = seedItem(link, {
remoteId: '/trek/a.pdf', remoteName: 'a.pdf', remoteVersion: 'va', fileId, state: 'synced',
contentSha256: FILE_SHA, remoteSize: 1024, remoteModifiedAt: AT,
});
provider.list.mockResolvedValue(ok(listing([
remoteDoc({ remoteId: '/trek/b.pdf', name: 'b.pdf', remoteVersion: 'vb', contentHash: FILE_SHA }),
])));
const first = await service.syncLink(link);
expect(first).toMatchObject({ state: 'ok', missing: 0, pulled: 0 });
expect(fileRows().map((f) => [f.id, f.original_name, f.deleted_at])).toEqual([[fileId, 'b.pdf', null]]);
expect(itemRow(itemId)).toMatchObject({
remote_id: '/trek/b.pdf', remote_name: 'b.pdf', remote_version: 'vb', state: 'synced', remote_missing_at: null,
});
const second = await service.syncLink(config.getLink(link.id));
expect(second).toMatchObject({ state: 'ok', missing: 0 });
expect(provider.rename).not.toHaveBeenCalled();
expect(provider.fetch).not.toHaveBeenCalled();
expect(provider.push).not.toHaveBeenCalled();
expect(fileRows().map((f) => f.original_name)).toEqual(['b.pdf']);
expect(itemRows()).toHaveLength(1);
});
it('takes a copy on record as missing off the list in the run that finds it renamed', async () => {
const link = makeLink();
const fileId = makeFile({ name: 'a.pdf' });
const itemId = seedItem(link, {
remoteId: '/trek/a.pdf', remoteName: 'a.pdf', remoteVersion: 'va', fileId, state: 'remote_missing',
contentSha256: FILE_SHA, remoteSize: 1024, remoteModifiedAt: AT, remoteMissingAt: '2026-09-18 09:00:00',
});
provider.list.mockResolvedValue(ok(listing([
remoteDoc({ remoteId: '/trek/b.pdf', name: 'b.pdf', remoteVersion: 'vb' }),
])));
await service.syncLink(link);
expect(itemRow(itemId)).toMatchObject({ remote_id: '/trek/b.pdf', state: 'synced', remote_missing_at: null });
expect(fileRows().map((f) => f.original_name)).toEqual(['b.pdf']);
expect(service.issues(tripId)).toEqual([]);
await service.syncLink(config.getLink(link.id));
expect(itemRow(itemId).state).toBe('synced');
expect(provider.rename).not.toHaveBeenCalled();
});
it('recognises the copy by size and time where the listing has no hash', async () => {
// What Synology actually sends. The pull records size and time, which is
// all there is to go on once the path has changed.
const link = makeLink();
provider.list.mockResolvedValueOnce(ok(listing([
remoteDoc({ remoteId: '/trek/a.pdf', name: 'a.pdf', remoteVersion: 'va' }),
])));
await service.syncLink(link);
expect(provider.fetch).toHaveBeenCalledTimes(1);
const fileId = Number(fileRows()[0].id);
provider.list.mockResolvedValue(ok(listing([
remoteDoc({ remoteId: '/trek/2026/b.pdf', name: 'b.pdf', remoteVersion: 'vb' }),
])));
const renamed = await service.syncLink(config.getLink(link.id));
await service.syncLink(config.getLink(link.id));
expect(renamed.missing).toBe(0);
expect(provider.fetch).toHaveBeenCalledTimes(1);
expect(provider.rename).not.toHaveBeenCalled();
expect(fileRows().map((f) => [f.id, f.original_name, f.deleted_at])).toEqual([[fileId, 'b.pdf', null]]);
expect(itemRows().map((r) => [r.remote_id, r.remote_name, r.state])).toEqual([['/trek/2026/b.pdf', 'b.pdf', 'synced']]);
});
it('recognises a copy it uploaded itself when it is renamed before the next run', async () => {
const link = makeLink();
makeFile({ name: 'boarding.pdf' });
provider.list.mockResolvedValueOnce(ok(listing([])));
provider.push.mockResolvedValueOnce(ok({
remoteId: '/trek/boarding.pdf', remoteVersion: 'v1', remoteModifiedAt: AT, deduplicated: false,
}));
await service.syncLink(link);
provider.list.mockResolvedValue(ok(listing([
remoteDoc({ remoteId: '/trek/flight.pdf', name: 'flight.pdf', remoteVersion: 'v2', size: FILE_BYTES.length }),
])));
await service.syncLink(config.getLink(link.id));
await service.syncLink(config.getLink(link.id));
expect(provider.push).toHaveBeenCalledTimes(1);
expect(provider.fetch).not.toHaveBeenCalled();
expect(provider.rename).not.toHaveBeenCalled();
expect(fileRows().map((f) => f.original_name)).toEqual(['flight.pdf']);
expect(itemRows().map((r) => [r.remote_id, r.state])).toEqual([['/trek/flight.pdf', 'synced']]);
});
it('finds the copy again after sending a rename made in TREK upstream', async () => {
// The adapter cannot report the path its rename produced, so the row
// keeps the old one until the next listing shows where the copy went.
const link = makeLink();
const fileId = makeFile({ name: 'b.pdf' });
const itemId = seedItem(link, {
remoteId: '/trek/a.pdf', remoteName: 'a.pdf', remoteVersion: 'va', fileId, state: 'synced',
contentSha256: FILE_SHA, remoteSize: 1024, remoteModifiedAt: AT,
});
provider.list.mockResolvedValueOnce(ok(listing([
remoteDoc({ remoteId: '/trek/a.pdf', name: 'a.pdf', remoteVersion: 'va' }),
])));
provider.rename.mockResolvedValueOnce(ok({ remoteVersion: 'vb' }));
await service.syncLink(link);
expect(provider.rename).toHaveBeenCalledWith(expect.anything(), expect.anything(), '/trek/a.pdf', 'b.pdf');
provider.list.mockResolvedValue(ok(listing([
remoteDoc({ remoteId: '/trek/b.pdf', name: 'b.pdf', remoteVersion: 'vb' }),
])));
const next = await service.syncLink(config.getLink(link.id));
await service.syncLink(config.getLink(link.id));
expect(next.missing).toBe(0);
expect(provider.rename).toHaveBeenCalledTimes(1);
expect(provider.fetch).not.toHaveBeenCalled();
expect(provider.push).not.toHaveBeenCalled();
expect(fileRows()).toHaveLength(1);
expect(itemRow(itemId)).toMatchObject({ remote_id: '/trek/b.pdf', remote_name: 'b.pdf', state: 'synced' });
});
});
describe('resolveConflict across trips', () => {