mirror of
https://github.com/mauriceboe/TREK.git
synced 2026-10-19 21:46:49 +00:00
RemEx
b1c05bca83
Whether the AI Parsing model reads images is an admin setting now, and it lands with the two features that send a photo, so it has something to control. The setting is `vision: 'auto' | 'on' | 'off'` on the addon config. On Automatic a local Ollama is asked what the model can do (`/api/show`), and a cloud provider means no; Yes and No are applied as set. The old `multimodal` flag is not read: the admin screens wrote it false on every save, so a stored false meant nothing. The per-user toggle keeps its on/off meaning and is read by the extraction now. The lookup is server-side only: capped read, validated body, never throws, cached per server and model. GET /api/llm/capabilities answers it for the signed-in caller, and both photo pickers below ask it. The booking import takes JPG, PNG and WEBP when the model reads images. The photo goes as an image to every provider (image_url, an Anthropic image block, Ollama's native chat with `images`), capped at 1600px, and skips KDE Itinerary. A scanned PDF, which used to reach the model as empty text, has its first two pages drawn by pdf-parse and sent the same way. The rest is the import as it was: background job, review, the file attached to the booking, the linked cost. On a real qwen3.5:4b a train ticket came back as bare clock times until the router gave the model every type's field meanings, and its price as an object inside a string, which read as 4900; both are handled. Costs gets Scan receipt: a dialog laid out like the booking import's takes one photo, read on the import job queue, then reviewed in the existing expense editor pre-filled with merchant, total, currency, day and lines, the photo waiting in Receipts & Invoices to be uploaded on save. No table, no migration, no category guess, no document-type detection.
21 lines
1.1 KiB
TypeScript
21 lines
1.1 KiB
TypeScript
import { describe, it, expect, vi } from 'vitest';
|
|
import { GUARDS_METADATA } from '@nestjs/common/constants';
|
|
import { LlmCapabilitiesController } from '../../../../src/nest/llm-parse/llm-capabilities.controller';
|
|
import { JwtAuthGuard } from '../../../../src/nest/auth/jwt-auth.guard';
|
|
import type { LlmParseService } from '../../../../src/nest/llm-parse/llm-parse.service';
|
|
import type { User } from '../../../../src/types';
|
|
|
|
describe('LlmCapabilitiesController', () => {
|
|
it("answers whether the caller's model reads images", async () => {
|
|
const readsImages = vi.fn(async (id: number) => id === 1);
|
|
const c = new LlmCapabilitiesController({ readsImages } as unknown as LlmParseService);
|
|
await expect(c.capabilities({ id: 1 } as User)).resolves.toEqual({ images: true });
|
|
await expect(c.capabilities({ id: 2 } as User)).resolves.toEqual({ images: false });
|
|
expect(readsImages).toHaveBeenCalledWith(2);
|
|
});
|
|
|
|
it('needs a session', () => {
|
|
expect(Reflect.getMetadata(GUARDS_METADATA, LlmCapabilitiesController)).toContain(JwtAuthGuard);
|
|
});
|
|
});
|