Ettore Di Giacinto 6786f05c64 feat(vllm): wire native tool/reasoning parsers + chat deltas + logprobs ...

- Use vLLM's ToolParserManager/ReasoningParserManager to extract structured
  output (tool calls, reasoning content) instead of reimplementing parsing
- Convert proto Messages to dicts and pass tools to apply_chat_template
- Emit ChatDelta with content/reasoning_content/tool_calls in Reply
- Extract prompt_tokens, completion_tokens, and logprobs from output
- Replace boolean GuidedDecoding with proper GuidedDecodingParams from Grammar
- Add TokenizeString and Free RPC methods
- Fix missing `time` import used by load_video()

2026-04-12 14:48:28 +00:00

8.7 KiB

Raw Blame History

View Raw