Reproducable stray artifacts
Hi,
I'm getting some weird behaviour I can't wrap my head around. Maybe you have an idea on what is going on?
Every other file write a stray artifact dangles at eof. They always look somthing like this:</content><tool_call>1a2b3c4d <tool_call>1a2b3c4d</tool_call>
also I sometimes still recieve messages instead of tool_call execution
<acp": "### TODO-003: HTTP-Engine (Parallelrequests, Basic-Auth, Body-Handling JSON/HTML)\n- Status: [x]\n- Datum:
2026-10-14\n- Beschreibung: code/httpengine/engine.go (Send + formValues), Tests via httptest.\n- Commit-hash:
(offen)\n- Verifizierung: go test ./httpengine/... OK, go build+go vet clean.", "newText": "### TODO-003: HTTP-Engine
(Parallelrequests, Basic-Auth, Body-Handling JSON/HTML)\n- Status: [x]\n- Datum: 2026-10-14\n- Beschreibung:
code/httpengine/engine.go (Send + formValues), Tests via httptest.\n- Commit-hash: (offen)\n- Verifizierung: go test
./httpengine/... OK, go build+go vet clean.", "oldText": "### TODO-003: HTTP-Engine (Parallelrequests, Basic-Auth,
Body-Handling JSON/HTML)\n- Status: [ ]\n- Commit-hash: (offen)"}
</tool_call>
furthermore llama.cpp frequently logs:
241.13.494.524 I slot release: id 0 | task 100254 | stop processing: n_tokens = 77551, truncated = 0
WARNING: JSON schema conversion was incomplete: pattern ^[^-] is not supported (not anchored with '^' and '$'), accepting any string; pattern ^[^-] is not supported (not anchored with '^' and '$'), accepting any string; pattern ^[^-] is not supported (not anchored with '^' and '$'), accepting any string; pattern ^[^-] is not supported (not anchored with '^' and '$'), accepting any string; pattern ^[^-] is not supported (not anchored with '^' and '$'), accepting any string; pattern ^[^-] is not supported (not anchored with '^' and '$'), accepting any string
llama-server start parameters on 14700K / 32GB / RTX4070:
llama-server -m Cyber-Tiel-Coder-35B-A3B.APEX-I-MiniPlus-V2.1.gguf -ngl 999 -ctk q8_0 -ctv q8_0 -b 8192 -ub 2048 -n 256 -np 1 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.05 --presence-penalty 0 --repeat-penalty 1.0 --frequency-penalty 0 --jinja --chat-template-file chat_template.jinja --cache-reuse 256 -fa on --cont-batching --n-cpu-moe 22 -c 163840 -t 8 -tb 14 --load-mode dio --mmproj mmproj-Q8_0.gguf --image-min-tokens 2048
Can you tell what is going on?
Best regards,
derFusel
The stray closing artifacts (</content><tool_call>...) and raw text output stem from a combination of two factors in your setup: first, llama-server is being run with -n 256 (--n-predict 256), which abruptly truncates long file-writing calls midway through generation, leaving malformed chunks and orphan closing tags that the agent framework attempts to re-emit or repair in subsequent turns; second, the log warning indicates that your agent schema passes an unanchored regex pattern (^[^-]), which causes llama.cpp to abort strict GBNF grammar compilation and accept unconstrained strings. To resolve this, increase your server prediction budget by setting -n 4096 (or -n -1 to delegate length management to the client) and ensure your tool schema uses anchored patterns so that llama.cpp can enforce valid JSON tool-call syntax without truncation.
The positive: I can confirm that '-n -1' reduces tool_call artifacts. Negative side effect is that it's significantly slower than 256 ( 74 tg/s vs 54 tg/s at the same context size ) for my machine.
The neutral: The very most of the encountered stalls and artifacts went away by getting rid of 'pi-gitnexus' and recent updates to 'billion-context'. My agent identified a busy bili proxy as the remaining source of the stated behavior.
The negative: Ditching the chat-template had a significant impact on reducing stalls and artifacts, while keeping the jinja flag active. Changing the model from Tiel-Coder to Ornith-1.5 reduced faulty behavior to 0.
Hi @derFusel ,
Thanks for the detailed feedback and testing! That last observation about Ornith-1.5 actually pinpoints the root cause:
1. The Chat Template discrepancy
In Tiel-Coder MiniPlus, the uploaded chat_template.jinja file was an older revision that contained unescaped physical multiline string literals in the tool-calling definitions. While standard Python Jinja accepts that, minja (llama.cpp's internal C++ Jinja parser) chokes on raw line breaks, producing corrupted formatting and leaked XML tokens.
Ornith-1.5 had already been updated with the normalized template, which is why your errors dropped to zero there. I have just synchronized and pushed the clean, normalized chat_template.jinja to this repository as well.
2. Throughput vs. Length Budget (-n 256 vs -n -1)
Setting -n -1 forces llama-server to allocate context buffers for the worst-case ceiling, which adds overhead and dropped your throughput from 74 tg/s down to 54 tg/s. On the other hand, -n 256 is too tight for code writes, cutting JSON tool payloads in half and leaving orphan closing tags.
The sweet spot to recover your ~74 tg/s without truncate errors is setting a sensible fixed limit like -n 4096 (or passing max_tokens: 4096 client-side from your agent payload).
Pull or re-download the updated chat_template.jinja and you should be good to go!