k1nq/TREK
1
0
Fork 0
mirror of https://github.com/mauriceboe/TREK.git synced 2026-10-18 21:16:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(llm-parse): LLM_MAX_OUTPUT_TOKENS sets the answer cap, a cut-off answer says so

Browse Source 
The OpenAI-compatible client always asked for 4096 tokens, and a document
whose bookings needed more came back as "no reservations" with nothing that
pointed at the cap. LLM_MAX_OUTPUT_TOKENS now sets the cap for every
provider; unset keeps 4096 and 8192 so a small local model's context still
fits. A finish_reason of "length" is reported like Anthropic's max_tokens
stop, naming the limit and the variable that raises it.

Closes #2677
This commit is contained in:
Maurice
2026-10-18 11:52:32 +02:00
parent 8f64431ff1
commit f901982903
12 changed files with 87 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
server/.env.example
+1
View File
@@ -84,6 +84,7 @@ DEMO_MODE=false # Demo mode: resets data hourly
# KITINERARY_EXTRACTOR_PATH= # Full path to the kitinerary-extractor binary. Unset, TREK searches /usr/lib/*/libexec/kf6/ and then PATH; the official image pins it. Set it to run a newer extractor than your distribution packages — the version in use is in the startup log and in GET /api/admin/system-info. (default: auto-detected)
# LLM_TIMEOUT_MS=900000 # Ceiling (ms) for one AI-parsing call before it is abandoned. One value for every provider, and the underlying HTTP client is held to it too. Generous by default so heavier parsing work fits without a code change; lower it on a cloud provider to fail fast. (default: 900000)
# LLM_MAX_OUTPUT_TOKENS=8192 # Longest answer (tokens) one AI-parsing call may ask for. Raise it when a document with many bookings comes back cut off; keep prompt plus cap inside your model's context. (default: 4096 for OpenAI-compatible, 8192 for Anthropic)
# VAPID_PUBLIC_KEY= # Web Push: the public half of the key pair browsers subscribe with (base64url, 65 bytes). Leave both halves unset and TREK generates a pair on first start and keeps it in the database, so it travels with every backup. Set both to use a pair of your own, e.g. printed by `npx web-push generate-vapid-keys`. Changing the pair later means every device has to turn push on again.
# VAPID_PRIVATE_KEY= # Web Push: the private half (base64url, 32 bytes). Only used together with VAPID_PUBLIC_KEY; a pair whose halves do not belong together, or only one half, turns push off with an error in the log until the pair is fixed. A pair stored in the database is not used in its place, and every device stays registered.