mirror/LocalAI

mirror of https://github.com/mudler/LocalAI.git synced 2026-04-16 21:08:16 -04:00

Files

History

Ettore Di Giacinto 6786f05c64 feat(vllm): wire native tool/reasoning parsers + chat deltas + logprobs

- Use vLLM's ToolParserManager/ReasoningParserManager to extract structured
  output (tool calls, reasoning content) instead of reimplementing parsing
- Convert proto Messages to dicts and pass tools to apply_chat_template
- Emit ChatDelta with content/reasoning_content/tool_calls in Reply
- Extract prompt_tokens, completion_tokens, and logprobs from output
- Replace boolean GuidedDecoding with proper GuidedDecodingParams from Grammar
- Add TokenizeString and Free RPC methods
- Fix missing `time` import used by load_video()

2026-04-12 14:48:28 +00:00

..

backend.py

feat(vllm): wire native tool/reasoning parsers + chat deltas + logprobs

2026-04-12 14:48:28 +00:00

install.sh

chore: Update to Ubuntu24.04 (cont #7423 ) (#7769 )

2026-01-06 15:26:42 +01:00

Makefile

feat(mlx): add mlx backend (#6049 )

2025-08-22 08:42:29 +02:00

README.md

refactor: move backends into the backends directory (#1279 )

2023-11-13 22:40:16 +01:00

requirements-after.txt

fix(python): move vllm to after deps, drop diffusers main deps

2024-08-07 23:34:37 +02:00

requirements-cpu.txt

Revert "chore(deps): bump torch from 2.7.0 to 2.7.1+xpu in /backend/python/vllm in the pip group across 1 directory" (#8367 )

2026-02-03 08:34:54 +01:00

requirements-cublas12-after.txt

fix(vllm): Update flash-attn to specific wheel URL

2025-11-21 18:06:46 +01:00

requirements-cublas12.txt

Revert "chore(deps): bump torch from 2.7.0 to 2.7.1+xpu in /backend/python/vllm in the pip group across 1 directory" (#8367 )

2026-02-03 08:34:54 +01:00

requirements-hipblas.txt

feat(rocm): bump to 7.x (#9323 )

2026-04-12 08:51:30 +02:00

requirements-install.txt

feat: migrate python backends from conda to uv (#2215 )

2024-05-10 15:08:08 +02:00

requirements-intel.txt

feat(qwen-tts): add Qwen-tts backend (#8163 )

2026-01-23 15:18:41 +01:00

requirements.txt

chore(deps): bump grpcio from 1.78.1 to 1.80.0 in /backend/python/vllm (#9177 )

2026-03-31 10:10:17 +02:00

run.sh

feat: Add backend gallery (#5607 )

2025-06-15 14:56:52 +02:00

test.py

feat(vllm): wire native tool/reasoning parsers + chat deltas + logprobs

2026-04-12 14:48:28 +00:00

test.sh

feat: Add backend gallery (#5607 )

2025-06-15 14:56:52 +02:00

README.md

Creating a separate environment for the vllm project

make vllm