mirror/LocalAI

mirror of https://github.com/mudler/LocalAI.git synced 2026-06-25 09:09:07 -04:00

Files

History

Ettore Di Giacinto 5e3774dfe3 fix(vllm): fail Score cleanly when the engine returns no prompt_logprobs

Audit of the Score path against vllm-metal (MLX on macOS): the engine accepts
SamplingParams(prompt_logprobs=1) but returns an all-None prompt_logprobs list
rather than computing it, so scoring is not supported there. The old guard
treated the truthy [None] list as valid and silently scored every candidate as
0. Detect the all-None case and return UNIMPLEMENTED instead. No-op on
Linux/CUDA, which populate real entries.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:opus-4.8 [Claude Code]

2026-06-24 21:31:41 +00:00

..

backend.py

fix(vllm): fail Score cleanly when the engine returns no prompt_logprobs

2026-06-24 21:31:41 +00:00

install.sh

fix(vllm): fetch the vllm-metal wheel without the GitHub API

2026-06-24 20:54:30 +00:00

Makefile

feat(mlx): add mlx backend (#6049 )

2025-08-22 08:42:29 +02:00

package.sh

feat(vllm, distributed): tensor parallel distributed workers (#9612 )

2026-05-06 00:22:50 +02:00

README.md

refactor: move backends into the backends directory (#1279 )

2023-11-13 22:40:16 +01:00

requirements-after.txt

feat(vllm): parity with llama.cpp backend (#9328 )

2026-04-13 11:00:29 +02:00

requirements-cpu-after.txt

feat(vllm): parity with llama.cpp backend (#9328 )

2026-04-13 11:00:29 +02:00

requirements-cpu.txt

feat(vllm): parity with llama.cpp backend (#9328 )

2026-04-13 11:00:29 +02:00

requirements-cublas12-after.txt

fix(vllm): drop flash-attn wheel to avoid torch 2.10 ABI mismatch (#9557 )

2026-04-25 15:38:13 +00:00

requirements-cublas12.txt

fix(vllm): drop flash-attn wheel to avoid torch 2.10 ABI mismatch (#9557 )

2026-04-25 15:38:13 +00:00

requirements-cublas13-after.txt

feat(vllm): macOS/Metal support via vllm-metal (MLX)

2026-06-24 17:17:50 +00:00

requirements-cublas13.txt

feat(backends): add CUDA 13 + L4T arm64 CUDA 13 variants for vllm/vllm-omni/sglang (#9553 )

2026-04-25 12:26:29 +02:00

requirements-hipblas-after.txt

feat(vllm): parity with llama.cpp backend (#9328 )

2026-04-13 11:00:29 +02:00

requirements-hipblas.txt

feat(rocm): bump to 7.x (#9323 )

2026-04-12 08:51:30 +02:00

requirements-install.txt

fix(vllm): seed pybind11 for fastsafetensors build under --no-build-isolation

2026-04-28 20:08:26 +00:00

requirements-intel-after.txt

feat(vllm, distributed): tensor parallel distributed workers (#9612 )

2026-05-06 00:22:50 +02:00

requirements-intel.txt

feat(vllm, distributed): tensor parallel distributed workers (#9612 )

2026-05-06 00:22:50 +02:00

requirements-l4t13-after.txt

fix(L4T13 backends): switch vllm/sglang/vllm-omni to PyPI aarch64+cu130 wheels (#9950 )

2026-05-22 23:01:22 +02:00

requirements-l4t13.txt

fix(L4T13 backends): switch vllm/sglang/vllm-omni to PyPI aarch64+cu130 wheels (#9950 )

2026-05-22 23:01:22 +02:00

requirements.txt

chore(deps): bump grpcio from 1.81.0 to 1.81.1 in /backend/python/vllm (#10347 )

2026-06-15 22:57:38 +02:00

run.sh

fix(python-backend): make JIT subprocesses work on hosts of any size (#9679 )

2026-05-06 00:28:01 +02:00

test.py

feat(vllm): progressive streaming via parser.extract_tool_calls_streaming (follow-up to #10346 ) (#10351 )

2026-06-21 17:07:15 +02:00

test.sh

feat: Add backend gallery (#5607 )

2025-06-15 14:56:52 +02:00

README.md

Creating a separate environment for the vllm project

make vllm