Jinja issue?
I'm running UD-Q6_K_XL on llama-server (b5538) and it crashes quite often when I use the --jinja flag. Soon as I remove it, the crashes stop.
Oh weird I utilized whatever DeepSeek provided - it's possible the tool calling part might be broken?
So far, I've just been asking it some basic tech questions to get a gauge on it's quality of responses. Was getting it to write some bash code blocks and Ansible playbooks for updating containers. Might just be coincidence, but a large number of the crashes were within the thinking portion, inside of a code block, and following the @ symbol.
OK, for the sake of comparison, I downloaded the bartowski GGUF to compare and it's the same behaviour. Crashes with --jinja and works without. Must be an issue upstream with the Deepseek template. Hope it's something you can still spot-fix.
I've dug in a bit more and I think it's a problem in llama-server. I used coredumpctl to examine the dump file and ran bt to get the last function calls before the crash. Looks like it occurs when it's trying to compute diffs between messages, so maybe when attempting to determine if context has changed or can be appended to?? Starting to get out of my depth here.
Looks like it occurs when it's trying to compute diffs between messages
I can't comment on the exact underlying cause, but I am also having crashes with the same trace, and I assume it's a llama.cpp issue.
This seems to be llama-server problem as recently there was a merge of PR introducing streaming support with tool execution
here is the issue PR