feat(offline): stamp queued writes with their format and skip newer ones on replay

A queued write outlives the bundle that wrote it. After a deploy a tab can
still run the previous bundle, and an install can be rolled back, so a row
written by a newer build may be replayed by an older one that does not know
its shape.

Each row enqueue() writes now carries schemaVersion (MUTATION_SCHEMA_VERSION)
and the UI build that queued it. Both fields are optional, so rows queued
before this read as format 1 and replay as before. A flush leaves a row from a
newer format in the queue without sending, marking or dropping it, and holds
back the later writes to the same entity as it does behind a parked write, so
they keep their order until a build that knows the format replays them.
This commit is contained in:
Maurice
2026-10-08 15:56:31 +02:00
parent 3e1234112d
commit 5bb45bc188
5 changed files with 192 additions and 4 deletions
+1 -1
View File
@@ -43,7 +43,7 @@ The layering is **component → feature hook → store/slice → `repo/` → `ap
- **`src/repo/`** — per-entity repositories mediating between the network and the offline cache: online, call REST and upsert into Dexie; offline, read Dexie. Always go through a repo for trip data.
- **`src/api/client.ts`** — the single Axios instance, typed from `@trek/shared`. New API surface goes in a per-domain `api/<domain>.ts`, not the god module.
- **`src/db/offlineDb.ts`** — the Dexie schema. Database and table names are the on-device contract — renaming one orphans every user's cache.
- **`src/sync/`**: in `mutationQueue.ts`, offline writes do an optimistic Dexie write (temporary **negative id** for creates), then `flush()` replays REST on reconnect with the mutation's UUID as the **`X-Idempotency-Key`** header (matching the server interceptor) and reconciles the row. Writes to one entity keep their order: a write parked as `failed` or `conflict` holds back the later writes to the same `resource` + entity id until the user retries, resolves or discards it, and only that action deletes a parked row (trip eviction never does). Repos whose writes carry no version token (visits, day clears, tours, driving settings) ask `mutationQueue.mustQueue()` before an online write and queue it while an older write to the entity is still queued, so a retried write cannot land over a newer one made here online; places and packing items send `X-Base-Updated-At`, so the server refuses the stale replay with a 409 instead. `flush()` takes the Web Lock `trek-mutation-flush:<db name>` so only one tab replays at a time; a tab that finds it taken waits for it, and a `flush()` called while this tab's flush runs or waits for the lock gets that same flush back (with one more pass asked for), so `flush()` never resolves while a tab is still replaying and a re-seed chained onto it cannot lay older server rows over those writes (the in-tab flush alone where the API is missing). The rest of the directory drives reconnection sync, the auth gate that stops background loops before logout swaps the DB, offline preferences, and map pre-download (raster and vector prefetchers).
- **`src/sync/`**: in `mutationQueue.ts`, offline writes do an optimistic Dexie write (temporary **negative id** for creates), then `flush()` replays REST on reconnect with the mutation's UUID as the **`X-Idempotency-Key`** header (matching the server interceptor) and reconciles the row. Writes to one entity keep their order: a write parked as `failed` or `conflict` holds back the later writes to the same `resource` + entity id until the user retries, resolves or discards it, and only that action deletes a parked row (trip eviction never does). Every row `enqueue()` writes carries `schemaVersion` (`MUTATION_SCHEMA_VERSION` in `offlineDb.ts`) and `buildVersion`; bump the constant when a row's fields or the way the replay reads them change incompatibly. A flush replays its own format and older ones (an unstamped row is format 1) and skips a row from a newer format without sending, marking or dropping it, holding back the later writes to its entity, so a tab still on an older bundle never replays a shape it does not know. Repos whose writes carry no version token (visits, day clears, tours, driving settings) ask `mutationQueue.mustQueue()` before an online write and queue it while an older write to the entity is still queued, so a retried write cannot land over a newer one made here online; places and packing items send `X-Base-Updated-At`, so the server refuses the stale replay with a 409 instead. `flush()` takes the Web Lock `trek-mutation-flush:<db name>` so only one tab replays at a time; a tab that finds it taken waits for it, and a `flush()` called while this tab's flush runs or waits for the lock gets that same flush back (with one more pass asked for), so `flush()` never resolves while a tab is still replaying and a re-seed chained onto it cannot lay older server rows over those writes (the in-tab flush alone where the API is missing). The rest of the directory drives reconnection sync, the auth gate that stops background loops before logout swaps the DB, offline preferences, and map pre-download (raster and vector prefetchers).
## Rules for new code
+20
View File
@@ -24,6 +24,19 @@ export interface CachedTour extends TourListItem {
// to the user for a keep-mine / keep-theirs decision rather than dropped.
export type MutationStatus = 'pending' | 'syncing' | 'failed' | 'conflict';
/**
* The format of a queued write as this build writes and replays it. A queued
* row outlives the bundle that wrote it: it can be replayed by the next build
* after a deploy, or by an older one when a tab still runs the previous bundle
* or the install is rolled back. Bump this when a row's fields or the way the
* replay reads them change so that an older build would send it wrongly. A
* build replays rows stamped with its own format or an older one, and leaves a
* row stamped with a newer one in the queue untouched for a build that knows
* it (see mutationQueue's flush). Rows queued before the stamp existed carry
* none and read as format 1.
*/
export const MUTATION_SCHEMA_VERSION = 1;
export interface QueuedMutation {
/** UUID — also used as X-Idempotency-Key sent to the server */
id: string;
@@ -75,6 +88,13 @@ export interface QueuedMutation {
* on every trigger.
*/
retryAfter?: number;
/**
* MUTATION_SCHEMA_VERSION of the build that queued the row. Optional because
* rows queued before it existed carry none; those read as format 1.
*/
schemaVersion?: number;
/** The UI build that queued the row. For diagnosis only, never compared. */
buildVersion?: string;
}
export interface SyncMeta {
+28 -3
View File
@@ -5,7 +5,7 @@
* offline create/update/delete → enqueue() → optimistic Dexie write (in repo)
* online trigger → flush() → replay REST with X-Idempotency-Key header → update Dexie
*/
import { offlineDb } from '../db/offlineDb'
import { offlineDb, MUTATION_SCHEMA_VERSION } from '../db/offlineDb'
import { apiClient } from '../api/client'
import { isAuthed } from './authGate'
import { isEffectivelyOffline } from './networkMode'
@@ -146,6 +146,20 @@ function holdEntity(held: Map<string, number>, m: QueuedMutation): void {
if (since === undefined || m.createdAt < since) held.set(key, m.createdAt)
}
// The UI build stamped onto each queued write, for diagnosis. Empty where the
// bundler defined none.
const BUILD_VERSION: string = typeof __TREK_UI_VERSION__ === 'string' ? __TREK_UI_VERSION__ : ''
/**
* Was this write queued by a build whose row format this one does not know? A
* tab still on the previous bundle, or an install rolled back, would otherwise
* replay a shape it cannot read. Such a row stays in the queue as it is, for
* the build that wrote it; a row without a stamp predates it and is format 1.
*/
function isFromNewerFormat(m: QueuedMutation): boolean {
return (m.schemaVersion ?? 1) > MUTATION_SCHEMA_VERSION
}
/** Pull the server's current entity out of a 409 response body ({ server: {...} }). */
function extractConflictServer(err: unknown): unknown {
const data = (err as { response?: { data?: unknown } })?.response?.data
@@ -222,7 +236,7 @@ export const mutationQueue = {
* Returns the UUID (= idempotency key).
*/
async enqueue(
mutation: Omit<QueuedMutation, 'status' | 'attempts' | 'createdAt' | 'lastError'>,
mutation: Omit<QueuedMutation, 'status' | 'attempts' | 'createdAt' | 'lastError' | 'schemaVersion' | 'buildVersion'>,
): Promise<string> {
const now = Date.now()
_lastTs = now > _lastTs ? now : _lastTs + 1
@@ -232,6 +246,8 @@ export const mutationQueue = {
attempts: 0,
createdAt: _lastTs,
lastError: null,
schemaVersion: MUTATION_SCHEMA_VERSION,
buildVersion: BUILD_VERSION,
}
await offlineDb.mutationQueue.put(item)
return item.id
@@ -277,7 +293,9 @@ export const mutationQueue = {
* holds back only its own trip, retried with growing gaps, and is parked as
* failed once it has failed MAX_SERVER_ERROR_ATTEMPTS times. A write parked as
* failed or as a conflict holds back the later writes to the same entity, so
* they keep their order whatever the user decides about it.
* they keep their order whatever the user decides about it. A write queued
* by a newer build (a higher schemaVersion) is skipped and left pending, and
* holds back the later writes to its entity the same way.
*
* A call that arrives while a flush is running, or still waiting for another
* tab's lock, asks it for one more round (the running pass read the pending
@@ -362,6 +380,13 @@ export const mutationQueue = {
const key = entityKey(mutation)
const heldSince = key === undefined ? undefined : heldEntities.get(key)
if (heldSince !== undefined && heldSince < mutation.createdAt) continue
// Queued by a newer build: not sent, not marked, not dropped. It keeps
// its place for that build, and the later writes to the same entity
// wait behind it as they do behind a parked one.
if (isFromNewerFormat(mutation)) {
holdEntity(heldEntities, mutation)
continue
}
if (mutation.retryAfter !== undefined && mutation.retryAfter > Date.now()) {
blockedTrips.add(mutation.tripId)
continue
@@ -0,0 +1,141 @@
/**
* The format stamp on queued writes.
*
* A queued write can be replayed by a different build than the one that queued
* it. Rows are stamped with MUTATION_SCHEMA_VERSION; a build replays its own
* format and older ones (an unstamped row is format 1), and leaves a row from a
* newer format in the queue untouched, holding back the later writes to the
* same entity.
*/
import 'fake-indexeddb/auto';
import { http, HttpResponse } from 'msw';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { clearAll, MUTATION_SCHEMA_VERSION, offlineDb, type QueuedMutation } from '../../../src/db/offlineDb';
import { setAuthed } from '../../../src/sync/authGate';
import { generateUUID, mutationQueue } from '../../../src/sync/mutationQueue';
import { buildPlace } from '../../helpers/factories';
import { server } from '../../helpers/msw/server';
beforeEach(async () => {
await clearAll();
mutationQueue._resetFlushing();
setAuthed(true);
Object.defineProperty(navigator, 'onLine', { value: true, writable: true, configurable: true });
});
afterEach(() => {
vi.restoreAllMocks();
setAuthed(false);
});
/** A queued PUT of place `placeId`, written straight into the queue as a build would. */
function queuedPut(placeId: number, createdAt: number, extra: Partial<QueuedMutation> = {}): QueuedMutation {
return {
id: generateUUID(),
tripId: 1,
method: 'PUT',
url: `/trips/1/places/${placeId}`,
body: { name: `Place ${placeId}` },
createdAt,
status: 'pending',
attempts: 0,
lastError: null,
resource: 'places',
entityId: placeId,
...extra,
};
}
/** Answer every place PUT with the place, and record which ids were sent. */
function answerPlacePuts(): number[] {
const sent: number[] = [];
server.use(
http.put('/api/trips/1/places/:id', ({ params }) => {
const id = Number(params.id);
sent.push(id);
return HttpResponse.json({ place: buildPlace({ id, trip_id: 1 }) });
})
);
return sent;
}
describe('mutationQueue format stamp', () => {
it('stamps a queued write with the format and the build that queued it', async () => {
const id = await mutationQueue.enqueue({
id: generateUUID(),
tripId: 1,
method: 'POST',
url: '/trips/1/places',
body: { name: 'Belem' },
resource: 'places',
});
const stored = await offlineDb.mutationQueue.get(id);
expect(stored?.schemaVersion).toBe(MUTATION_SCHEMA_VERSION);
expect(stored?.buildVersion).toBe(__TREK_UI_VERSION__);
});
it('replays a write queued before the stamp existed', async () => {
const legacy = queuedPut(21, 1);
expect(legacy.schemaVersion).toBeUndefined();
await offlineDb.mutationQueue.put(legacy);
const sent = answerPlacePuts();
await mutationQueue.flush();
expect(sent).toEqual([21]);
expect(await offlineDb.mutationQueue.get(legacy.id)).toBeUndefined();
});
it('replays a write stamped with its own format', async () => {
const own = queuedPut(21, 1, { schemaVersion: MUTATION_SCHEMA_VERSION });
await offlineDb.mutationQueue.put(own);
const sent = answerPlacePuts();
await mutationQueue.flush();
expect(sent).toEqual([21]);
expect(await offlineDb.mutationQueue.get(own.id)).toBeUndefined();
});
it('leaves a write from a newer format in the queue, unsent and unchanged', async () => {
const newer = queuedPut(21, 1, { schemaVersion: MUTATION_SCHEMA_VERSION + 1, buildVersion: '99.0.0' });
await offlineDb.mutationQueue.put(newer);
const sent = answerPlacePuts();
await mutationQueue.flush();
expect(sent).toEqual([]);
expect(await offlineDb.mutationQueue.get(newer.id)).toEqual(newer);
expect(await mutationQueue.pendingCount()).toBe(1);
expect(await mutationQueue.failedCount()).toBe(0);
});
it('holds back later writes to the same entity, and only those', async () => {
const newer = queuedPut(21, 1, { schemaVersion: MUTATION_SCHEMA_VERSION + 1 });
const sameEntity = queuedPut(21, 2, { schemaVersion: MUTATION_SCHEMA_VERSION });
const otherEntity = queuedPut(22, 3, { schemaVersion: MUTATION_SCHEMA_VERSION });
await offlineDb.mutationQueue.bulkPut([newer, sameEntity, otherEntity]);
const sent = answerPlacePuts();
await mutationQueue.flush();
expect(sent).toEqual([22]);
expect(await offlineDb.mutationQueue.get(otherEntity.id)).toBeUndefined();
expect((await offlineDb.mutationQueue.get(newer.id))?.status).toBe('pending');
expect((await offlineDb.mutationQueue.get(sameEntity.id))?.status).toBe('pending');
});
it('keeps a write from a newer format pending when Try again puts parked writes back', async () => {
const newer = queuedPut(21, 1, { schemaVersion: MUTATION_SCHEMA_VERSION + 1, status: 'failed', attempts: 3 });
await offlineDb.mutationQueue.put(newer);
const sent = answerPlacePuts();
await mutationQueue.retryFailed();
expect(sent).toEqual([]);
const stored = await offlineDb.mutationQueue.get(newer.id);
expect(stored).toMatchObject({ status: 'pending', schemaVersion: MUTATION_SCHEMA_VERSION + 1 });
});
});
+2
View File
@@ -97,6 +97,8 @@ If a change you made offline collides with a newer change on the server, it is s
The stats panel shows cached trips, pending changes, conflicts and failed changes. A change the server keeps refusing with a server error is retried with growing gaps (from 30 seconds up to about two hours) and only holds back the other changes of its own trip; after eight attempts it counts as failed. Later changes you make on this device to the same item (the same place, packing item, visit, tour or driving settings) wait behind a failed change or an open conflict instead of overtaking it, so they reach the server in the order you made them; for visits, tours and driving settings that includes changes you make while online. Places and packing items remember which version you edited, so trying a failed change again never overwrites a newer version saved in the meantime, on this device or by another member of the trip: it turns into a conflict for you to decide. Visits, tours and driving settings carry no such version, so trying again overwrites what another member changed on the same item in the meantime, the same as any change sent after being offline. Failed changes come with **Try again** and **Discard**: trying again sends them first and then the changes that waited behind them; discarding keeps the server's version, sends the changes that waited, and the next sync puts the server's version back on the device. **Clear cache** removes all offline data from IndexedDB after you confirm it in TREK's dialog (you can re-sync any time while online). Each cached trip entry shows its date range, place/file count and last successful sync.
Changes waiting to sync remember which version of TREK queued them. After an update, a browser tab that still runs the previous version leaves changes queued by the newer version alone instead of sending them in a form it does not know: they stay under pending changes, together with the later changes to the same item, until a tab on the current version (reload the page) sends them. Nothing is lost, and the same applies after an install is rolled back to an older release.
## Limitations
- Offline **editing** is supported for places and packing items (with conflict detection), plus a visit's start and end time, its **End the day here** flag and the road trip **Driving settings** (queued, without conflict detection). A tour can be saved or edited offline as well (queued, without conflict detection), but drawing a new route needs a connection, since routing and elevation come from the routing server, and the GPX import is an upload. Other entities — budget, to-dos, reservations, days — require connectivity to edit; while forced offline those edits still go to the live server when a connection is actually present.