LFM2.5 tool calling is broken in streaming mode on both vLLM and llama.cpp (root cause, self-contained repros, workaround)

README.md

LFM2.5 tool calling is broken in streaming mode on both vLLM and llama.cpp

Tested 2026-08-08 with LiquidAI/LFM2.5-2.6B.

Both engines translate the model's tool call into JSON incrementally, while the model is still generating. That translation goes wrong whenever the arguments contain characters needing escapes. The two engines fail on complementary inputs, and in opposite ways.

argument payload	vLLM v0.26.0 (--tool-call-parser lfm2)	llama.cpp b10156 (--jinja)
plain text	OK	OK
braces only ({alpha: 1})	OK	OK
escaped double quotes only (say \"hi\")	OK	OK
braces and escaped quotes ({\"a\": 1})	silently corrupted	OK
realistic nested JSON document	silently corrupted	OK, byte-exact
echo 'hello world'	OK	HTTP 500
awk '{print $1}' data.csv	OK	HTTP 500
python code mixing ' and " with newlines	OK	HTTP 500
two tool calls in one turn, single quotes	OK	HTTP 500

Non-streaming is correct on both engines in every case tested, and on llama.cpp the content argument round-trips byte for byte.

This matters for agentic use. An agent that writes JSON files hits the vLLM bug, and one that runs shell commands hits the llama.cpp bug, so an agent doing both is broken on either engine. The failure is easy to miss because ordinary chat works, and so do simple tool calls like get_weather(city="Paris").

Pythonic tool-call format

Most models emit tool calls as JSON. LFM2.5 emits Python between special tokens:

<|tool_call_start|>[write(path="a.json", content="{\"a\": 1}")]<|tool_call_end|>

Every engine therefore has to translate Python syntax into JSON, and doing that translation on a half-finished string is where both implementations break.

Silent corruption in vLLM

The root cause is make_valid_python in vllm/tool_parsers/utils.py. It scans the partial call text to append closing characters before ast.parse, tracking brackets and quotes on one stack, and it pushes bracket characters even when they occur inside a string literal:

if char in {"[", "(", "{"}:
    bracket_stack.append(char)          # pushed even inside a string literal
...
elif char in {"'", '"'}:
    if bracket_stack and bracket_stack[-1] == char:   # closing quote
        if index > 0 and text[index - 1] == "\\":
            pass                        # escape handled ONLY on this branch
        else:
            bracket_stack.pop()
    elif bracket_stack and bracket_stack[-1] in {"'", '"'}:
        pass
    else:
        bracket_stack.append(char)      # treated as an OPENING quote

Walking c="{\"a\": 1}" shows how the stack desynchronises. The opening " pushes a quote. The { sits inside that string but is pushed anyway, so the top of the stack becomes { and the scanner stops believing it is inside a string. The escaped \" then misses both quote branches and reaches the final else, where it is pushed as an opening quote. When the JSON's closing } arrives it pops that quote and raises UnexpectedAstError: Mismatched curly braces. The escape check never runs, because it exists only on the branch where the top of the stack already equals the quote character.

From that point every longer prefix raises, including the complete text:

len=27  '[w(p="a.json", c="{\"a\": 1'    -> None (gives up)
len=28  '[w(p="a.json", c="{\"a\": 1}'   -> RAISES UnexpectedAstError
len=31  '[w(p="a.json", c="{\"a\": 1}")]'-> RAISES UnexpectedAstError

Lfm2ToolParser.extract_tool_calls_streaming catches the exception and falls back to content-only, so the arguments stay frozen at whatever prefix was already emitted. The client receives truncated, unparseable JSON and no error is raised. Non-streaming is unaffected, because extract_tool_calls runs ast.parse on the complete text and never calls make_valid_python.

Observed over HTTP against a live endpoint at temperature=0, same prompt:

stream=False   parseable 4/4
stream=True    parseable 0/4

expected: {"alpha": 1, "beta": "two", "gamma": "three four five"}
streamed: {"\lpha": 1, "\eta": "\wo", "\amma": "\hree\four\five"}

make_valid_python and compute_tool_delta are shared with llama4_pythonic_tool_parser, which is likely affected too, though I have not verified that.

Run repro_vllm.py to see it. The script needs no GPU and no running engine, and its only network access is fetching two source files from GitHub.

HTTP 500 in llama.cpp

string_diff in common/chat.cpp requires each new serialization to start with the previous one:

if (!string_starts_with(current, last)) {
    ...
    throw std::runtime_error("Invalid diff: '" + last + "' not found at start of '" + current + "'");
}

Re-serialized partial JSON is not monotonic under escaping, so the request dies with HTTP 500. The measured trigger is any single quote in an argument value.

This is ggml-org/llama.cpp#26658, still open. Issues #23838, #20245 and #20814 cover related LFM2 tool-call failures that have already been fixed.

Run repro_llamacpp.py against a local llama-server --jinja.

Workaround

Since non-streaming is correct on both engines, the workaround is to stop translating mid-generation. nostream_shim.py is a dependency-free proxy that accepts the client's streaming request, forwards it upstream with "stream": false, and re-emits the finished reply as a short SSE stream. Heartbeat comments keep long generations from tripping idle timeouts.

llama-server -m LFM2.5-2.6B-BF16.gguf --jinja --port 8099 &
python3 nostream_shim.py --upstream http://127.0.0.1:8099 --port 8100
# point your agent at http://127.0.0.1:8100/v1

Measured against llama.cpp b10156 with LFM2.5-2.6B-BF16.gguf:

case	direct	through shim
echo 'hello world'	500	OK
echo '{"done": true}'	500	OK
awk '{print $1}' data.csv	500	OK
python code, mixed quotes	500	byte-exact
JSON deliverable	OK	byte-exact
parallel tool calls	500	OK

An agent cannot act on a half-received tool call anyway, so the only thing lost is token-by-token display of the arguments.

Upstream fixes

For vLLM, the bracket scanner needs to become string-aware, so that [, ( and { are not pushed while inside a string literal, and escape handling runs before the opening-quote branch. The same patch covers llama4_pythonic.

For llama.cpp, string_diff could tolerate a non-prefix by re-emitting the full current value instead of throwing, or the server could buffer tool-call arguments until the call is complete.

Environment

Models were LiquidAI/LFM2.5-2.6B, plus LiquidAI/LFM2.5-2.6B-GGUF at BF16 for llama.cpp. The vLLM side ran v0.26.0, the latest release at the time of writing, started with the flags from the official LFM2.5 cookbook:

vllm serve LiquidAI/LFM2.5-2.6B \
  --enable-auto-tool-choice \
  --tool-call-parser lfm2 \
  --reasoning-parser qwen3

The llama.cpp side ran build b10156, the official prebuilt CUDA server, with --jinja, on an NVIDIA GB10. Its requests used the sampling parameters from the model card, temperature 0.1, top_k 50 and repetition_penalty 1.1.

Caveat

repro_vllm.py runs vLLM's parser sources unmodified, but stubs the surrounding pydantic protocol types with dataclasses. It is a faithful test of the parser algorithm. Confirm any fix against the shipped container before trusting it.

nostream_shim.py

#!/usr/bin/env python3
"""Buffering shim for OpenAI-compatible servers with broken streamed tool calls.

Both vLLM's `lfm2` parser and llama.cpp translate a model's tool call into JSON
incrementally, while the model is still generating. That translation is wrong
whenever the arguments contain characters that need escaping:

  * vLLM v0.26.0 silently corrupts arguments that contain braces AND escaped
    quotes -- i.e. a JSON document, which is what every shellbench-structured
    task asks the agent to write.
  * llama.cpp b10156 returns HTTP 500 ("Invalid diff") when an argument
    contains a single quote, which covers most real shell commands.

Both engines are correct when the request is not streamed. This shim exploits
that: it accepts a streaming request from the agent, forwards it upstream with
"stream": false, and re-emits the finished reply as a short SSE stream. Agents
need the complete tool call before they can act on it anyway, so nothing is
lost but token-by-token display.

A heartbeat comment is emitted while waiting so that clients and proxies with
idle timeouts do not drop the connection during a long generation.

Usage:
    nostream_shim.py --upstream http://127.0.0.1:8099 --port 8100
"""

from __future__ import annotations

import argparse
import json
import threading
import time
import urllib.error
import urllib.request
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

HEARTBEAT_SECONDS = 10


class Handler(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"
    upstream = "http://127.0.0.1:8099"

    def log_message(self, fmt, *args):  # quieter logs
        pass

    def do_POST(self) -> None:
        body = self.rfile.read(int(self.headers.get("Content-Length") or 0))
        try:
            payload = json.loads(body or b"{}")
        except json.JSONDecodeError:
            self._send_json(400, {"error": {"message": "invalid JSON body"}})
            return

        wants_stream = bool(payload.get("stream"))
        if not wants_stream or not self.path.endswith("/chat/completions"):
            self._passthrough(body)
            return

        # Ask upstream for the complete answer; never let it stream.
        upstream_payload = dict(payload)
        upstream_payload["stream"] = False
        upstream_payload.pop("stream_options", None)

        result: dict = {}
        done = threading.Event()

        def fetch() -> None:
            try:
                result["data"] = self._upstream_json(upstream_payload)
            except urllib.error.HTTPError as exc:
                result["error"] = (exc.code, exc.read().decode("utf-8", "replace"))
            except Exception as exc:  # noqa: BLE001 - surfaced to the client
                result["error"] = (502, str(exc))
            finally:
                done.set()

        worker = threading.Thread(target=fetch, daemon=True)
        worker.start()

        self.send_response(200)
        self.send_header("Content-Type", "text/event-stream")
        self.send_header("Cache-Control", "no-cache")
        self.send_header("Connection", "close")
        self.end_headers()

        # Hold the connection open with SSE comments until upstream answers.
        while not done.wait(HEARTBEAT_SECONDS):
            try:
                self.wfile.write(b": keepalive\n\n")
                self.wfile.flush()
            except (BrokenPipeError, ConnectionResetError):
                return

        if "error" in result:
            code, detail = result["error"]
            self._write_event({"error": {"code": code, "message": detail[:2000]}})
            self._write_raw(b"data: [DONE]\n\n")
            return

        include_usage = bool(
            (payload.get("stream_options") or {}).get("include_usage")
        )
        for event in self._to_stream_events(result["data"], include_usage):
            self._write_event(event)
        self._write_raw(b"data: [DONE]\n\n")

    # GET is used for /health, /props and similar probes.
    def do_GET(self) -> None:
        try:
            with urllib.request.urlopen(
                urllib.request.Request(self.upstream + self.path), timeout=30
            ) as resp:
                data = resp.read()
                status = resp.status
                ctype = resp.headers.get("Content-Type", "application/json")
        except urllib.error.HTTPError as exc:
            data, status, ctype = exc.read(), exc.code, "application/json"
        except Exception as exc:  # noqa: BLE001
            data, status, ctype = str(exc).encode(), 502, "text/plain"
        self.send_response(status)
        self.send_header("Content-Type", ctype)
        self.send_header("Content-Length", str(len(data)))
        self.end_headers()
        self.wfile.write(data)

    def _upstream_json(self, payload: dict) -> dict:
        req = urllib.request.Request(
            self.upstream + self.path,
            data=json.dumps(payload).encode(),
            headers={"Content-Type": "application/json",
                     "Authorization": self.headers.get("Authorization", "")},
        )
        with urllib.request.urlopen(req, timeout=3600) as resp:
            return json.loads(resp.read().decode())

    def _passthrough(self, body: bytes) -> None:
        try:
            req = urllib.request.Request(
                self.upstream + self.path, data=body,
                headers={"Content-Type": "application/json",
                         "Authorization": self.headers.get("Authorization", "")},
            )
            with urllib.request.urlopen(req, timeout=3600) as resp:
                data, status = resp.read(), resp.status
        except urllib.error.HTTPError as exc:
            data, status = exc.read(), exc.code
        except Exception as exc:  # noqa: BLE001
            data, status = json.dumps({"error": {"message": str(exc)}}).encode(), 502
        self.send_response(status)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(data)))
        self.end_headers()
        self.wfile.write(data)

    @staticmethod
    def _to_stream_events(completion: dict, include_usage: bool = False) -> list[dict]:
        """Render a finished completion as the SSE chunks a client expects."""
        base = {
            "id": completion.get("id", "chatcmpl-shim"),
            "object": "chat.completion.chunk",
            "created": completion.get("created", int(time.time())),
            "model": completion.get("model", ""),
        }
        events: list[dict] = []
        for choice in completion.get("choices", []):
            index = choice.get("index", 0)
            message = choice.get("message") or {}

            delta: dict = {"role": message.get("role", "assistant")}
            if message.get("reasoning_content"):
                delta["reasoning_content"] = message["reasoning_content"]
            if message.get("content"):
                delta["content"] = message["content"]
            if message.get("tool_calls"):
                delta["tool_calls"] = [
                    {
                        "index": i,
                        "id": call.get("id", f"call_{i}"),
                        "type": call.get("type", "function"),
                        "function": {
                            "name": (call.get("function") or {}).get("name", ""),
                            # complete, already-validated argument string
                            "arguments": (call.get("function") or {}).get("arguments", ""),
                        },
                    }
                    for i, call in enumerate(message["tool_calls"])
                ]
            events.append({**base, "choices": [
                {"index": index, "delta": delta, "finish_reason": None}]})
            events.append({**base, "choices": [
                {"index": index, "delta": {},
                 "finish_reason": choice.get("finish_reason", "stop")}]})

        # A trailing chunk with an empty `choices` list is only expected when the
        # caller asked for usage; emitting it unconditionally breaks clients that
        # index choices[0] on every chunk.
        if include_usage and completion.get("usage"):
            events.append({**base, "choices": [], "usage": completion["usage"]})
        return events

    def _write_event(self, obj: dict) -> None:
        self._write_raw(b"data: " + json.dumps(obj).encode() + b"\n\n")

    def _write_raw(self, data: bytes) -> None:
        try:
            self.wfile.write(data)
            self.wfile.flush()
        except (BrokenPipeError, ConnectionResetError):
            pass

    def _send_json(self, status: int, obj: dict) -> None:
        data = json.dumps(obj).encode()
        self.send_response(status)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(data)))
        self.end_headers()
        self.wfile.write(data)


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--upstream", default="http://127.0.0.1:8099")
    parser.add_argument("--host", default="0.0.0.0")
    parser.add_argument("--port", type=int, default=8100)
    args = parser.parse_args()

    Handler.upstream = args.upstream.rstrip("/")
    server = ThreadingHTTPServer((args.host, args.port), Handler)
    print(f"nostream shim: {args.host}:{args.port} -> {Handler.upstream}", flush=True)
    server.serve_forever()


if __name__ == "__main__":
    main()

repro_llamacpp.py

#!/usr/bin/env python3
"""Reproduce the llama.cpp LFM2.5 streamed tool-call failure (ggml-org/llama.cpp#26658).

Start a server first:

    llama-server -m LFM2.5-2.6B-BF16.gguf --jinja --host 127.0.0.1 --port 8099

then:

    python3 repro_llamacpp.py                       # direct: single quotes -> HTTP 500
    python3 repro_llamacpp.py --url http://127.0.0.1:8100  # through nostream_shim.py

Only stdlib is required.

Trigger: a single quote anywhere in a tool-call argument value. Non-streaming is
correct in every case; streaming returns
`{"error":{"code":500,"message":"Invalid diff: ... not found at start of ..."}}`.
"""

from __future__ import annotations

import argparse
import json
import urllib.error
import urllib.request

BASH = {
    "type": "function",
    "function": {
        "name": "bash",
        "description": "Run a shell command.",
        "parameters": {
            "type": "object",
            "properties": {"command": {"type": "string"}},
            "required": ["command"],
        },
    },
}
WRITE = {
    "type": "function",
    "function": {
        "name": "write",
        "description": "Write a file to disk.",
        "parameters": {
            "type": "object",
            "properties": {"path": {"type": "string"},
                           "content": {"type": "string"}},
            "required": ["path", "content"],
        },
    },
}

PY_CODE = (
    "import matplotlib.pyplot as plt\n"
    "import numpy as np\n\n"
    "x = np.linspace(0, 2*np.pi, 100)\n"
    "plt.plot(x, np.sin(x), 'red', label=\"wave\")\n"
    "plt.legend(loc='best')\n"
)

CASES = [
    ("bash, no quotes", [BASH],
     "Run the bash tool with command: ls -la /app/data", None),
    ("bash, double quotes", [BASH],
     'Run the bash tool with command: echo "hello world"', None),
    ("bash, SINGLE quotes", [BASH],
     "Run the bash tool with command: echo 'hello world'", None),
    ("bash, single-quoted JSON", [BASH],
     'Run the bash tool with command: echo \'{"done": true}\'', None),
    ("bash, awk single quotes", [BASH],
     "Run the bash tool with command: awk '{print $1}' data.csv", None),
    ("write, python w/ mixed quotes", [WRITE],
     f"Use the write tool to write /tmp/plot.py with content exactly:\n{PY_CODE}",
     PY_CODE),
    ("parallel calls, single quotes", [WRITE, BASH],
     'Call the write tool for /tmp/g.json with content {"k": "v"} and also '
     "call the bash tool with command: echo 'done'", None),
]


def call(url: str, prompt: str, tools: list, stream: bool):
    body = {
        "messages": [{"role": "user", "content": prompt}],
        "tools": tools, "tool_choice": "auto",
        # model card's recommended sampling
        "temperature": 0.1, "top_k": 50, "repetition_penalty": 1.1,
        "max_tokens": 1024, "stream": stream,
    }
    req = urllib.request.Request(
        url + "/v1/chat/completions", data=json.dumps(body).encode(),
        headers={"Content-Type": "application/json"})
    try:
        raw = urllib.request.urlopen(req, timeout=600).read().decode()
    except urllib.error.HTTPError as exc:
        return None, f"HTTP {exc.code}: {exc.read().decode()[:160]}"

    if not stream:
        msg = json.loads(raw)["choices"][0]["message"]
        return [(t["function"]["name"], t["function"]["arguments"])
                for t in (msg.get("tool_calls") or [])], None

    acc, err = {}, None
    for line in raw.splitlines():
        if not line.startswith("data: "):
            continue
        payload = line[6:].strip()
        if payload == "[DONE]":
            break
        chunk = json.loads(payload)
        if "error" in chunk:
            err = json.dumps(chunk["error"])[:160]
            continue
        if not chunk.get("choices"):
            continue
        for t in ((chunk["choices"][0].get("delta") or {}).get("tool_calls") or []):
            slot = acc.setdefault(t.get("index", 0), {"n": "", "a": ""})
            fn = t.get("function") or {}
            if fn.get("name"):
                slot["n"] = fn["name"]
            slot["a"] += fn.get("arguments") or ""
    return [(v["n"], v["a"]) for v in acc.values()], err


def verdict(rows, err, expect_content):
    if err:
        return "HTTP 500" if "500" in err else f"ERROR {err[:40]}"
    if not rows:
        return "NO TOOL CALL"
    try:
        args = [json.loads(a) for _, a in rows]
    except Exception:
        return "CORRUPT"
    if expect_content is not None:
        got = args[0].get("content", "")
        return "EXACT" if got == expect_content else "MISMATCH"
    return "OK"


def main() -> int:
    ap = argparse.ArgumentParser()
    ap.add_argument("--url", default="http://127.0.0.1:8099")
    args = ap.parse_args()

    print(f"target: {args.url}\n")
    print(f"{'case':<32} {'non-streaming':<16} {'streaming':<16}")
    failures = 0
    for label, tools, prompt, expect in CASES:
        ns, ns_err = call(args.url, prompt, tools, False)
        st, st_err = call(args.url, prompt, tools, True)
        ns_v = verdict(ns, ns_err, expect)
        st_v = verdict(st, st_err, expect)
        if st_v not in ("OK", "EXACT"):
            failures += 1
        print(f"{label:<32} {ns_v:<16} {st_v:<16}")

    print(f"\n{failures} streaming failure(s)")
    return 1 if failures else 0


if __name__ == "__main__":
    raise SystemExit(main())

repro_vllm.py

#!/usr/bin/env python3
"""Self-contained reproduction of the vLLM lfm2 streaming tool-parser defect.

No GPU, no engine, no model. Fetches vLLM's own parser sources at a pinned tag,
stubs the handful of protocol types they import, then drives the real
`extract_tool_calls_streaming` with a simulated token stream and compares it
against the real `extract_tool_calls` on the same complete text.

    pip install partial-json-parser openai regex pydantic
    python3 repro_vllm.py

Expected: only the case whose argument value contains BOTH braces and escaped
quotes disagrees with non-streaming, at every delta size.
"""

from __future__ import annotations

import json
import pathlib
import sys
import tempfile
import urllib.request

VLLM_TAG = "v0.26.0"
RAW = f"https://raw.githubusercontent.com/vllm-project/vllm/{VLLM_TAG}/vllm"

STUBS = {
    "vllm/__init__.py": "",
    "vllm/envs.py": "VLLM_TOOL_PARSE_REGEX_TIMEOUT_SECONDS = 1\n",
    "vllm/logger.py": (
        "import logging\n"
        "def init_logger(name):\n"
        "    return logging.getLogger(name)\n"
    ),
    "vllm/tokenizers/__init__.py": "class TokenizerLike:\n    pass\n",
    "vllm/entrypoints/__init__.py": "",
    "vllm/entrypoints/openai/__init__.py": "",
    "vllm/entrypoints/openai/chat_completion/__init__.py": "",
    "vllm/entrypoints/openai/chat_completion/protocol.py": (
        "from dataclasses import dataclass\n"
        "from typing import Any\n\n"
        "@dataclass\n"
        "class ChatCompletionRequest:\n"
        "    tools: Any = None\n"
        "    tool_choice: Any = None\n"
        "    skip_special_tokens: bool = True\n\n"
        "class ChatCompletionNamedToolChoiceParam: pass\n"
        "class ChatCompletionToolsParam: pass\n"
    ),
    "vllm/entrypoints/openai/responses/__init__.py": "",
    "vllm/entrypoints/openai/responses/protocol.py": "class ResponsesRequest: pass\n",
    "vllm/entrypoints/openai/engine/__init__.py": "",
    "vllm/entrypoints/openai/engine/protocol.py": (
        "from dataclasses import dataclass, field\n"
        "from typing import Any\n\n"
        "@dataclass\n"
        "class FunctionCall:\n"
        "    name: str | None = None\n"
        "    arguments: str = ''\n\n"
        "@dataclass\n"
        "class ToolCall:\n"
        "    function: FunctionCall\n"
        "    id: str | None = None\n"
        "    type: str = 'function'\n\n"
        "@dataclass\n"
        "class DeltaFunctionCall:\n"
        "    name: str | None = None\n"
        "    arguments: str | None = None\n\n"
        "@dataclass\n"
        "class DeltaToolCall:\n"
        "    index: int = 0\n"
        "    id: str | None = None\n"
        "    type: str | None = None\n"
        "    function: DeltaFunctionCall | None = None\n\n"
        "@dataclass\n"
        "class DeltaMessage:\n"
        "    content: str | None = None\n"
        "    tool_calls: list = field(default_factory=list)\n\n"
        "@dataclass\n"
        "class ExtractedToolCallInformation:\n"
        "    tools_called: bool = False\n"
        "    tool_calls: list = field(default_factory=list)\n"
        "    content: Any = None\n"
    ),
    "vllm/tool_parsers/__init__.py": "",
    "vllm/tool_parsers/abstract_tool_parser.py": (
        "class Tool: pass\n\n"
        "class ToolParser:\n"
        "    def __init__(self, tokenizer, tools=None):\n"
        "        self.model_tokenizer = tokenizer\n"
        "        self.vocab = tokenizer.get_vocab()\n"
        "        self.prev_tool_call_arr = []\n"
        "        self.current_tool_id = -1\n"
        "        self.current_tool_name_sent = False\n"
        "        self.streamed_args_for_tool = []\n\n"
        "    def adjust_request(self, request):\n"
        "        return request\n"
    ),
}

START, END = "<|tool_call_start|>", "<|tool_call_end|>"

CASES = {
    "A no braces, no escaped quotes": f'{START}[w(p="a.txt", c="hello world")]{END}',
    "B braces only":                   f'{START}[w(p="a.txt", c="{{alpha: 1}}")]{END}',
    "C escaped quotes only":           f'{START}[w(p="a.txt", c="say \\"hi\\" now")]{END}',
    "D braces AND escaped quotes":     f'{START}[w(p="a.json", c="{{\\"a\\": 1}}")]{END}',
}


def build_tree(root: pathlib.Path) -> None:
    for rel, body in STUBS.items():
        path = root / rel
        path.parent.mkdir(parents=True, exist_ok=True)
        path.write_text(body)
    for name in ("utils.py", "lfm2_tool_parser.py"):
        url = f"{RAW}/tool_parsers/{name}"
        with urllib.request.urlopen(url, timeout=60) as resp:
            (root / "vllm" / "tool_parsers" / name).write_bytes(resp.read())


def main() -> int:
    tmp = pathlib.Path(tempfile.mkdtemp(prefix="vllm-lfm2-repro-"))
    print(f"fetching vLLM {VLLM_TAG} parser sources into {tmp}")
    build_tree(tmp)
    sys.path.insert(0, str(tmp))

    from vllm.entrypoints.openai.chat_completion.protocol import ChatCompletionRequest
    from vllm.tool_parsers.lfm2_tool_parser import Lfm2ToolParser

    class FakeTokenizer:
        def get_vocab(self):
            return {START: 100_000, END: 100_001}

    def non_streaming(text: str):
        parser = Lfm2ToolParser(FakeTokenizer())
        info = parser.extract_tool_calls(text, ChatCompletionRequest())
        return [(c.function.name, c.function.arguments) for c in info.tool_calls]

    def streaming(text: str, chunk: int):
        parser = Lfm2ToolParser(FakeTokenizer())
        req = ChatCompletionRequest()
        acc: dict[int, dict] = {}
        prev = ""
        for i in range(0, len(text), chunk):
            cur = text[: i + chunk]
            msg = parser.extract_tool_calls_streaming(
                prev, cur, cur[len(prev):], [], [], [], req)
            if msg is not None:
                for call in (msg.tool_calls or []):
                    slot = acc.setdefault(call.index, {"name": None, "args": ""})
                    if call.function is not None:
                        if call.function.name:
                            slot["name"] = call.function.name
                        if call.function.arguments:
                            slot["args"] += call.function.arguments
            prev = cur
        return [(v["name"], v["args"]) for v in acc.values()]

    def parses(s: str) -> bool:
        try:
            json.loads(s)
            return True
        except Exception:
            return False

    print(f"\n{'case':<32} {'non-streaming':<14} streaming (delta=1/3/7)")
    failures = 0
    for label, text in CASES.items():
        expect = non_streaming(text)
        ns = "OK" if expect and all(parses(a) for _, a in expect) else "BAD"
        verdicts = []
        for chunk in (1, 3, 7):
            got = streaming(text, chunk)
            good = got == expect and all(parses(a) for _, a in got)
            verdicts.append("OK" if good else "CORRUPT")
            failures += 0 if good else 1
        print(f"{label:<32} {ns:<14} {'/'.join(verdicts)}")
        if "CORRUPT" in verdicts:
            print(f"{'':<32} streamed -> {streaming(text, 3)}")
            print(f"{'':<32} expected -> {expect}")

    print("\nDEFECT REPRODUCED" if failures else "\nno defect observed")
    return 1 if failures else 0


if __name__ == "__main__":
    raise SystemExit(main())