The Problem
Streaming tool calls present a reliability challenge that the standard OpenAI API does not address:- Fragmented delivery — tool call
argumentsarrive as a sequence of small JSON fragments across multiple SSE chunks. Each individual chunk is not valid JSON on its own. - Mid-argument truncation — if the model reaches
max_tokenswhile writing tool arguments, the stream ends withfinish_reason: "length". The final argument string is truncated mid-JSON — for example, a closing}or"may be missing entirely. - Client-side errors — without intervention, your application receives the raw fragments, attempts to parse them as JSON, and throws an exception. For write-heavy tools (such as
write_filewith large file content), this is a common failure mode that silently drops work.
What Piramyd Does Automatically
Piramyd applies three layers of protection to all streaming tool calls, transparently and without any configuration:1. Buffered Validation
Rather than forwarding each rawfunction.arguments delta to your client as it arrives, Piramyd accumulates all argument fragments internally. The complete, assembled argument string is emitted to your client as a single validated chunk once the tool call is fully formed. You never need to concatenate argument fragments in your application.
2. Auto-Repair
When the assembled argument string is detected as truncated JSON — for example, an unclosed string literal, a missing array bracket, or a dangling object key — Piramyd deterministically repairs it by closing all open structures. The repaired JSON is what your client receives.3. Transparent Continuation
When a model stops mid-tool-call withfinish_reason: "length" (indicating it hit max_tokens before finishing), Piramyd transparently issues a continuation request to the upstream model and merges the result. From your client’s perspective, the stream completes normally with a full finish_reason: "tool_calls".
No client-side changes, retry logic, or error handling is needed for any of these cases.
Result
Your client always receives complete, valid tool call JSON — regardless of how many tokens the model generated, how the arguments arrived in fragments, or whether the model was cut off mid-response. This means you can safely parsetool_calls[n].function.arguments directly with JSON.parse() or json.loads() without defensive error handling for truncated output.
For a full guide on structuring tool definitions, handling tool results, and building multi-step agent loops, see the Tool Calling guide.