mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-05-19 14:17:21 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix(buun-llama-cpp): backport logit_bias_eog field to grpc-server copy

Browse Source 
LocalAI's shared grpc-server.cpp reaches
ctx_server.get_meta().logit_bias_eog twice (the twin params_from_json_cmpl
callsites). That accessor was added to server_context_meta upstream after
buun's 2026-04-05 fork-point, so compiling against buun errors with
  'struct server_context_meta' has no member named 'logit_bias_eog'.

Rewrite the call sites — only in the buun grpc-server.cpp copy — to source
the vector from params_base.sampling.logit_bias_eog instead. That vector is
the underlying data the upstream meta accessor eventually returns (buun
still carries common_params_sampling::logit_bias_eog at common.h:280), so
the substitution yields identical behavior on both trees.

The sed is guarded by a grep for the call site, so this patch is
self-disabling once buun rebases past the upstream refactor.

Assisted-by: Claude:Opus-4.7 [Read] [Edit] [Bash] [WebFetch]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto
2026-04-24 11:12:51 +00:00
parent 45756b19dc
commit b27d38a53d
1 changed files with 15 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
backend/cpp/buun-llama-cpp/patch-grpc-server.sh
View File

@@ -125,6 +125,21 @@ else
echo "==> DFlash option-handler patch OK"
fi
if grep -q 'ctx_server\.get_meta()\.logit_bias_eog' "$SRC"; then
echo "==> patching $SRC to source logit_bias_eog from params_base.sampling (buun predates server_context_meta::logit_bias_eog accessor)"
# Upstream llama.cpp exposes logit_bias_eog through server_context_meta
# after buun's 2026-04-05 fork-point. Buun still carries the underlying
# data on common_params_sampling::logit_bias_eog (the struct field the
# meta accessor eventually returns). Rewriting the call site to read
# params_base.sampling.logit_bias_eog works against both trees — upstream
# still populates that same vector the newer accessor returns.
sed 's/ctx_server\.get_meta()\.logit_bias_eog/params_base.sampling.logit_bias_eog/g' "$SRC" > "$SRC.tmp"
mv "$SRC.tmp" "$SRC"
echo "==> logit_bias_eog substitution OK"
else
echo "==> $SRC has no ctx_server.get_meta().logit_bias_eog call, skipping logit_bias_eog patch"
fi
if grep -q 'get_media_marker()' "$SRC"; then
echo "==> patching $SRC to replace get_media_marker() with legacy \"<__media__>\" literal"
# Only one call site today (ModelMetadata), but replace all occurrences to