Files
LocalAI/backend/go
LocalAI [bot] ec933b837d chore(moss-transcribe-cpp): bump pin to CUDA K-quant embed fix (#10862) (#10878)
chore(moss-transcribe-cpp): bump pin to CUDA K-quant embed fix

Bumps the moss-transcribe.cpp pin to 190a569c, which merges the
host-side embed-lookup fallback for K-quant token_embd tensors
(localai-org/moss-transcribe.cpp#2).

Before this, running a q5_K/q4_K/q6_K moss-transcribe GGUF on CUDA
(or any non-CPU backend) aborted in getrows.cu with
"unsupported src0 type: q5_K" because ggml's GET_ROWS op has no
K-quant implementation on GPU, killing the backend process on the
first request. The engine now dequantizes the needed rows on the
host when the backend cannot run GET_ROWS for the tensor type,
producing bit-identical results.

Fixes #10862


Assisted-by: Claude:claude-opus-4-8 [Bash] [Edit]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-07-17 12:47:13 +02:00
..