mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-10-18 21:16:47 +00:00
7ccc8dc03d
A document the model read correctly could still come back as "No reservations could be extracted". Every type mapper can answer null when the node carries no reservationFor, or one with no name, and both call sites pushed the item only if there was one — with no else. The unknown-@type branch next to them does warn, so a node TREK did not recognise was reported while a node it recognised but could not map vanished: no item, no warning, nothing in the log. Because the file did produce a node, the "no reservations found" fallback was skipped too, and the preview arrived empty and silent. Every recognised type could disappear this way, on both the kitinerary and the AI path. Each of them now warns, naming the file, the index, the type and the reason. Three smaller silences went with it. The debug line labelled "Raw LLM Response" printed the already-parsed array, so its inspect formatting — single quotes, bare keys — read as a provider emitting broken JSON and sent this report at the JSON parser, which has handled that shape since 4.0.0. A single reservation answered on its own, with no array and no wrapper, was dropped by the list unwrapper although the schema downstream would have taken it. And a response nobody could read returned an empty list, which is the same answer as a voucher with no booking in it; it now raises, on the Anthropic client too, where a reply cut off at the token limit produced exactly this report's symptom. The plugin AI surface keeps its old empty answer, since a value is what a plugin contracts for. The retry ladder gained a rung that sends no response_format at all, for providers that support neither json_schema nor json_object. Closes #2375
120 lines
4.4 KiB
TypeScript
120 lines
4.4 KiB
TypeScript
import { describe, it, expect } from 'vitest';
|
|
|
|
import { parseLenientJson, toReservationList } from '../../../../src/nest/llm-parse/lenient-json';
|
|
|
|
describe('parseLenientJson', () => {
|
|
it('parses strict JSON', () => {
|
|
expect(parseLenientJson('{"a":1,"b":"x"}')).toEqual({ a: 1, b: 'x' });
|
|
});
|
|
|
|
it('strips a ```json code fence', () => {
|
|
expect(parseLenientJson('```json\n[{"@type":"FlightReservation"}]\n```')).toEqual([
|
|
{ '@type': 'FlightReservation' },
|
|
]);
|
|
});
|
|
|
|
it('parses single-quoted, unquoted-key, trailing-comma output (Gemini, #1638)', () => {
|
|
const gemini = `[
|
|
{
|
|
'@type': 'LodgingReservation',
|
|
checkinTime: '2026-08-28T00:00:00',
|
|
checkoutTime: '2026-08-30T11:00:00',
|
|
price: 146.25,
|
|
priceCurrency: 'EUR',
|
|
}
|
|
]`;
|
|
expect(parseLenientJson(gemini)).toEqual([
|
|
{
|
|
'@type': 'LodgingReservation',
|
|
checkinTime: '2026-08-28T00:00:00',
|
|
checkoutTime: '2026-08-30T11:00:00',
|
|
price: 146.25,
|
|
priceCurrency: 'EUR',
|
|
},
|
|
]);
|
|
});
|
|
|
|
it('parses a code-fenced non-strict object', () => {
|
|
expect(parseLenientJson("```\n{ reservations: [{ '@type': 'TrainReservation' }] }\n```")).toEqual({
|
|
reservations: [{ '@type': 'TrainReservation' }],
|
|
});
|
|
});
|
|
|
|
it('returns null on empty or truly unparseable input', () => {
|
|
expect(parseLenientJson('')).toBeNull();
|
|
expect(parseLenientJson(null)).toBeNull();
|
|
expect(parseLenientJson(undefined)).toBeNull();
|
|
expect(parseLenientJson('this is prose, not json')).toBeNull();
|
|
});
|
|
});
|
|
|
|
/**
|
|
* The shapes a forced tool call actually comes back in (#1968).
|
|
*
|
|
* The tool declares `reservations` as an array and the call forces the tool, so
|
|
* the answer should be an array and usually is. Anthropic sometimes serialises
|
|
* its tool input as a JSON-encoded string instead, and the check that only
|
|
* accepted arrays dropped those extractions silently: no error, no log, and an
|
|
* import that reported "no reservations found" — indistinguishable from a
|
|
* document that really had none. Because it depends on how the model serialises
|
|
* that particular call, the same file imported fine one minute and empty the
|
|
* next.
|
|
*/
|
|
describe('toReservationList', () => {
|
|
const node = { '@type': 'LodgingReservation', reservationNumber: 'ABC123' };
|
|
|
|
it('passes an array straight through, which is the ordinary case', () => {
|
|
expect(toReservationList([node])).toEqual([node]);
|
|
});
|
|
|
|
it('unwraps the object shape the OpenAI-compatible path produces', () => {
|
|
expect(toReservationList({ reservations: [node] })).toEqual([node]);
|
|
});
|
|
|
|
it('parses a stringified array', () => {
|
|
expect(toReservationList(JSON.stringify([node]))).toEqual([node]);
|
|
});
|
|
|
|
it('parses a stringified object that wraps the list again', () => {
|
|
expect(toReservationList(JSON.stringify({ reservations: [node] }))).toEqual([node]);
|
|
});
|
|
|
|
/*
|
|
* A single booking answered on its own, with no array and no wrapper. The
|
|
* check only knew the two list shapes, so a perfectly good extraction was
|
|
* dropped without a word — the same silent empty preview as #1968, from a
|
|
* different direction (#2375).
|
|
*/
|
|
it('takes a single reservation answered on its own', () => {
|
|
expect(toReservationList(node)).toEqual([node]);
|
|
expect(toReservationList(JSON.stringify(node))).toEqual([node]);
|
|
});
|
|
|
|
it('takes the relaxed JSON a model might emit inside that string', () => {
|
|
expect(toReservationList("[{'@type': 'FlightReservation', flightNumber: 'LH400',}]"))
|
|
.toEqual([{ '@type': 'FlightReservation', flightNumber: 'LH400' }]);
|
|
});
|
|
|
|
/*
|
|
* The other half of the fix: it must not reach so far that prose starts
|
|
* counting as a booking. Anything that is not a list after one unwrap is not
|
|
* a list.
|
|
*/
|
|
it('answers empty for something that is genuinely not a list', () => {
|
|
expect(toReservationList(undefined)).toEqual([]);
|
|
expect(toReservationList(null)).toEqual([]);
|
|
expect(toReservationList('there are no reservations in this document')).toEqual([]);
|
|
expect(toReservationList({ hotel: 'Grand' })).toEqual([]);
|
|
expect(toReservationList(42)).toEqual([]);
|
|
});
|
|
|
|
it('does not unwrap twice, so a doubly-encoded string stays refused', () => {
|
|
expect(toReservationList(JSON.stringify(JSON.stringify([node])))).toEqual([]);
|
|
});
|
|
|
|
it('keeps an empty list an empty list', () => {
|
|
expect(toReservationList([])).toEqual([]);
|
|
expect(toReservationList('{"reservations":[]}')).toEqual([]);
|
|
});
|
|
});
|