mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-21 13:44:55 -04:00
mudler's LocalAI [bot]andEttore Di Giacinto
102d91414e
The Go bindings mirror vllm.h by hand and refuse a library whose vllm_abi_version differs from what they were written against. Two automated pin bumps (#11174, #11352) moved VLLM_CPP_VERSION onto engines declaring ABI v10 while govllmcpp.go still mirrored v5, so every vllm-cpp image built since then panics at startup on every platform: panic: vllm-cpp: ABI mismatch: library reports v10, backend built against v5 Grow both PODs to the v10 layout: vllm_model_params gains speculative_config, enable_prefix_caching, max_num_batched_tokens, scheduling_policy, kv_transfer_config and enable_jump_forward (88 bytes), vllm_sampling_params gains the v8 logits-processor pair (136 bytes). The offsets in the specs come from offsetof() against the pinned header. All of the new fields are inert when zeroed, so the engine behaves exactly as it did under v5; the backend sets none of them. Nothing cross-checked the two files, which is why a blind pin bump could ship a backend that cannot load. The library build now runs abi-check first: it compares VLLM_ABI_VERSION in the fetched header against abiVersion in govllmcpp.go and fails the build naming both, instead of leaving the mismatch for a user's runtime. Fixes #11379 Assisted-by: Claude:claude-fable-5 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>