100% cpu spinning on tool call, generastion cannot be aborted

[schmorp]

When koboldcpp received this (possibly malformed) tool call from gemma-4, it goes into a 100% cpu spinning loop.

Output: <|tool_call>call:vim_replace_lines{buffer:1,expected_changedtick:2,line:1,new_lines:["use strict;","use warnings;","use Time::HiRes qw(sleep);","my $w = 40;","my $h = 20;","my $p = 20;","my $s = 0;","my @world = map { ' ' x $w } 1..$h;","sub draw {"," system('clear');"," print "Score: $s\n";"," print '=' x ($w+2)."\n";"," for (@world) { print "|$_\n"; }"," print '=' x ($w+2)."\n";", "}", "sub update {"," pop @world;"," my $l = ' ' x $w;"," if (rand() < 0.2) { substr($l, int(rand($w)), 1) = 'Y'; }"," unshift @world, $l;"," substr($world[-1], $p, 1) = 'S';", " return 0 if substr($world[-1], $p, 1) eq 'Y';"," $s++; return 1;","}", "system('stty cbreak raw -echo');","eval {"," while (1) {"," draw();"," if (select(STDIN, undef, undef, 0.1)) {"," my $k = ;"," if ($k eq 'a' && $p > 0) { $p--; }"," elsif ($k eq 'd' && $p < $w-1) { $p++; }"," elsif ($k eq 'q') { last; }"," }", " last unless update();", " }", "};","system('stty sane');","print "\nGame Over! Score: $s\n";"],old_lines:[" "],target:<|"|>oben<|"|>}<tool_call|>

gdb backtrace is not very helpful, but shows it spinning in python, the only thread active:

#0 frame_dealloc (f=0x7f1ea0211590) at /usr/local/src/conda/python-3.8.20/Objects/frameobject.c:430
#1 0x00007f2779113747 in _Py_DECREF (op=0x7f1ea0211590, lineno=4314, filename=0x7f2779202190 "/home/conda/feedstock_root/build_artifacts/python-split_1727717877616/work/Python/ceval.c") at /usr/local/src/conda/python-3.8.20/Include/object.h:478
#2 _PyEval_EvalCodeWithName (_co=, globals=, locals=locals@entry=0x0, args=args@entry=0x7f1e9c06a900, argcount=argcount@entry=0, kwnames=, kwargs=0x7f1e9c06a900, kwcount=, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x7f2762b86c10, name=0x7f2777ab5a30, qualname=0x7f2777ab80f0) at /usr/local/src/conda/python-3.8.20/Python/ceval.c:4314
#3 0x00007f277912161d in _PyFunction_Vectorcall (func=, stack=0x7f1e9c06a900, nargsf=, kwnames=) at /usr/local/src/conda/python-3.8.20/Objects/call.c:436
#4 0x00007f2779114936 in _PyObject_Vectorcall (kwnames=0x0, nargsf=, args=0x7f1e9c06a900, callable=0x7f2762b693a0) at /usr/local/src/conda/python-3.8.20/Include/cpython/abstract.h:127
#5 call_function (kwnames=0x0, oparg=, pp_stack=, tstate=0x5592bdc5d1c0) at /usr/local/src/conda/python-3.8.20/Python/ceval.c:4963
#6 _PyEval_EvalFrameDefault (f=, throwflag=) at /usr/local/src/conda/python-3.8.20/Python/ceval.c:3500
#7 0x00007f2779113711 in _PyEval_EvalCodeWithName (_co=, globals=, locals=locals@entry=0x0, args=args@entry=0x7f275c1819f8, argcount=argcount@entry=0, kwnames=, kwargs=0x7f275c1819f8, kwcount=, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x7f275c18e520, name=0x7f2777ab59b0, qualname=0x7f2777ab82d0) at /usr/local/src/conda/python-3.8.20/Python/ceval.c:4298
#8 0x00007f277912161d in _PyFunction_Vectorcall (func=, stack=0x7f275c1819f8, nargsf=, kwnames=) at /usr/local/src/conda/python-3.8.20/Objects/call.c:436
#9 0x00007f2779114936 in _PyObject_Vectorcall (kwnames=0x0, nargsf=, args=0x7f275c1819f8, callable=0x7f2762b250d0) at /usr/local/src/conda/python-3.8.20/Include/cpython/abstract.h:127
#10 call_function (kwnames=0x0, oparg=, pp_stack=, tstate=0x5592bdc5d1c0) at /usr/local/src/conda/python-3.8.20/Python/ceval.c:4963
#11 _PyEval_EvalFrameDefault (f=, throwflag=) at /usr/local/src/conda/python-3.8.20/Python/ceval.c:3500
#12 0x00007f2779113711 in _PyEval_EvalCodeWithName (_co=, globals=, locals=locals@entry=0x0, args=args@entry=0x7f1ea0246750, argcount=argcount@entry=0, kwnames=, kwargs=0x7f1ea0246750, kwcount=, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x7f2762b86c10, name=0x7f2777ab5a30, qualname=0x7f2777ab80f0) at /usr/local/src/conda/python-3.8.20/Python/ceval.c:4298

pressing abort causes koboldcpp to print:

"Generation Aborted"

but it's not true, it's not aborted, it just keep spinning in a 100% cpu loop.

this is specific to this tool call. a lot of other tool calls worked fine.

"--jinja --jinjatools" was in effect to work around koboldcpp starting new kv contexts for every tool call otherwise.

[schmorp]

pre-built koboldcp 1.121 btw.

[schmorp]

happens on 1.222 the same

[LostRuins]

Should be fixed in the rolling build here: https://github.com/LostRuins/koboldcpp/releases/tag/rolling

[firedesiresa-bit]

Independent confirmation on Linux, from an OpenAI-compatible client that sends tools.

Symptom. The server accepts a request and never finishes it — no response, no error — while pinning a
core. Measured with ps on the server process: 1:15:15 of CPU inside 1:06:38 of wall (sampled
01:13:11 → 01:13:31 over 20 s, i.e. a steady 100% of one core), while nvidia-smi read 0 % GPU with
the model still resident (4.8 GB). GET /v1/models kept answering in 36 ms throughout, so the HTTP
side was alive and only generation was stuck. Restarting the server cleared it immediately.

Environment. koboldcpp 1.121, koboldcpp-linux-x64, --jinja --jinja_tools --quantkv f16 --contextsize 65536 --batchsize 256 --threads 8; Nemotron-3-Nano-4B Q4_K_M GGUF; request to
/v1/chat/completions with a 5-tool tools array.

Trigger here. A normal-sized request mid-run — 11,363 characters ≈ 2,840 estimated tokens, 5 tools,
nowhere near the 65,536 context. It was the third request of a three-request run; the run's journal shows
it started and never ended. Nothing else about it looked unusual.

Client-side cost, in case it helps prioritise. Clients that read a run's exit rather than the
provider's liveness will wait out their whole timeout against the dead server and then retry into the same
wall: we measured runs of 610.9 s, 615.9 s, 780.9 s and 1530.9 s before we stopped it — over an hour of a
benchmark pass spent measuring a server that was never going to answer, with the failed runs
indistinguishable from ordinary test failures in the record.

Thanks for the work on this project.