mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 09:57:57 -04:00
df5e2a6d27
vllm.cpp landed ABI v10 on main while this branch was open. The backend's runtime handshake refuses any library whose reported ABI differs from the mirrors', so the pin and the Go PODs move together or not at all. v10 appends one int32, `enable_jump_forward`, AFTER the v9 fields. Nothing else in the config surface changed: the SGLang reconciliation that carried it explicitly dropped its own duplicate scheduler_policy int in favour of the v9 `scheduling_policy` string this branch already wires, and a diff of EngineParams and the server flags across the window turns up jump forward and nothing else. So the exposure is one new knob, `engine_args.enable_jump_forward` - SGLang's grammar-speed subset, which emits grammar-forced tokens without spending a model step and therefore only affects constrained requests. It is the SECOND tri-state on this struct, and it repeats the trap the first one had: 0 is not "off", it is "defer" (to the environment here, to the model capability for prefix caching), so an explicit `false` has to reach the engine as 2. The bool->tri-state helper and the log renderer are now shared rather than duplicated per field, and named for the encoding instead of for prefix caching, since the next tri-state will want them too. The docs say this outright, because "omitting the key" and "setting it false" being different is not guessable. The Go mirror grows the field plus an EXPLICIT trailing pad: the struct is 8-aligned and now ends on a lone int32, so it is 88 bytes rather than 84. The offset assertions cover it, and the real-library handshake spec (VLLM_CPP_LIBRARY against a CPU libvllm.so built at the new pin) confirms the version agrees: 43 specs pass, ABI reported 10. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5 [ClaudeCode]