mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-05-30 03:25:42 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix(openai): stop streaming tool-call double-emission when autoparser is active (#10055)

Browse Source 
Streaming /v1/chat/completions could emit the same logical tool call at
multiple `index` values. In processStreamWithTools the Go-side iterative
parser (ParseXMLIterative / ParseJSONIterative) runs on every token and
emits tool-call deltas, while the C++ chat-template autoparser delivers
its own tool calls via ChatDeltas that are flushed at end-of-stream by
ToolCallsFromChatDeltas -> buildDeferredToolCallChunks. With both paths
active the same call is emitted twice at different indices, so OpenAI
clients that accumulate tool calls by `index` dispatch the tool N times.

Skip the Go-side iterative parser once the autoparser is producing tool
calls (hasChatDeltaToolCalls). The deferred flush stays guarded by
lastEmittedCount, so the race where the Go parser emitted before the flag
flipped also remains single-emission. Backends without an autoparser
(e.g. vLLM) keep hasChatDeltaToolCalls=false and are unaffected.

Refs #9722

Signed-off-by: bozhouDev <259759010+bozhouDev@users.noreply.github.com>
Co-authored-by: bozhouDev <259759010+bozhouDev@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
泊舟
2026-05-29 17:39:09 +08:00
committed by GitHub
parent 73cfedc023
commit e1a782b70f
1 changed files with 13 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
core/http/endpoints/openai/chat_stream_workers.go
View File

@@ -341,6 +341,19 @@ func processStreamWithTools(
}
}
// Issue #9722: when the C++ autoparser is already producing tool
// calls (it delivers them via ChatDeltas, which are flushed at
// end-of-stream by ToolCallsFromChatDeltas -> buildDeferredToolCallChunks),
// skip the Go-side iterative parser below. Running both parsers makes
// the same logical tool call surface at multiple `index` values.
// The deferred flush is guarded by lastEmittedCount, so the race where
// the Go parser already emitted before this flag flipped also stays
// single-emission. Backends without an autoparser (e.g. vLLM) keep
// hasChatDeltaToolCalls=false and are unaffected.
if hasChatDeltaToolCalls {
return true
}
// Try incremental XML parsing for streaming support using iterative parser
// This allows emitting partial tool calls as they're being generated
cleanedResult := functions.CleanupLLMResult(result, cfg.FunctionsConfig)