mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-07-08 07:18:31 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix(backends): pin grpcio-tools to the installed grpcio in runProtogen

Browse Source 
runProtogen installed grpcio-tools unpinned, so the protoc it bundles
stamped backend_pb2.py with the newest Protobuf gencode (7.35.0). When a
backend caps the protobuf runtime lower -- vLLM pins protobuf to 6.33.6 --
the import-time guarantee runtime >= gencode fails:

  google.protobuf.runtime_version.VersionError: Detected incompatible
  Protobuf Gencode/Runtime versions ... gencode 7.35.0 runtime 6.33.6

The backend crashes on `import backend_pb2` before it can serve, which
surfaces to the user as "grpc service not ready". It was mis-reported as a
ROCm/gfx1201 failure in #10718 but is not GPU-specific and affects every
vLLM variant (and any backend that caps protobuf below the latest gencode).

Pin grpcio-tools to the grpcio version the backend already installed --
they release in lockstep -- so the generated gencode stays in step with
the protobuf runtime. Falls back to unpinned when grpcio isn't present.

Closes #10718

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
This commit is contained in:
Ettore Di Giacinto
2026-07-07 21:08:05 +00:00
parent 2f33cc7bc4
commit ebd3dcf3d8
1 changed files with 21 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
backend/python/common/libbackend.sh
View File

@@ -397,10 +397,29 @@ function ensureVenv() {
function runProtogen() {
ensureVenv
# Match grpcio-tools to the grpcio already installed by the backend's
# requirements. grpcio and grpcio-tools are released in lockstep, and the
# protoc that grpcio-tools bundles stamps a Protobuf "gencode" version into
# backend_pb2.py. Left unpinned, `uv pip install grpcio-tools` pulls the
# newest release, whose newer gencode (e.g. 7.35.0) trips Protobuf's
# runtime >= gencode guarantee at import time when a backend caps the
# protobuf runtime lower (vLLM pins it to 6.33.6), crashing the backend with
# "grpc service not ready" before it ever loads a model. Pinning
# grpcio-tools to the installed grpcio version keeps the gencode in step with
# the runtime. Falls back to unpinned when grpcio isn't installed yet.
# See mudler/LocalAI#10718.
local grpcio_tools_spec="grpcio-tools"
local grpcio_version
grpcio_version="$(python -c 'import importlib.metadata as m; print(m.version("grpcio"))' 2>/dev/null || true)"
if [ -n "${grpcio_version}" ]; then
grpcio_tools_spec="grpcio-tools==${grpcio_version}"
fi
if [ "x${USE_PIP}" == "xtrue" ]; then
pip install grpcio-tools
pip install "${grpcio_tools_spec}"
else
uv pip install grpcio-tools
uv pip install "${grpcio_tools_spec}"
fi
pushd "${EDIR}" >/dev/null
# use the venv python (ensures correct interpreter & sys.path)