mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-04-29 11:37:40 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix(ik-llama-cpp): adapt to common_grammar struct in sampling.h (#9512)

Browse Source 
Upstream ik_llama.cpp commit e0596bf6 ("Autoparser") changed
common_params_sampling::grammar from std::string to a common_grammar
struct (type + grammar), which broke our two direct accesses:

 - JSON ingest fed the field through json_value<common_grammar>(...),
   for which nlohmann has no from_json adapter.
 - JSON export emitted the struct directly, for which nlohmann has no
   to_json adapter.

Wrap the incoming JSON string in common_grammar{COMMON_GRAMMAR_TYPE_USER, ...}
and serialize via the inner .grammar member, mirroring upstream's
examples/server/server-context.cpp.

Also bump IK_LLAMA_VERSION to 286ce324baed17c95faec77792eaa6bdb1c7a5f5
so the local-ai side lines up with the dependency bump in #9496.

Assisted-by: Claude-Code:claude-opus-4-7
This commit is contained in:
Ettore Di Giacinto
2026-04-23 13:45:06 +02:00
committed by GitHub
parent 181ebb6df4
commit 04f1a0285d
2 changed files with 12 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
backend/cpp/ik-llama-cpp/grpc-server.cpp
View File

@@ -686,7 +686,16 @@ struct llama_server_context
slot->sparams.mirostat_eta = json_value(data, "mirostat_eta", default_sparams.mirostat_eta);
slot->params.n_keep = json_value(data, "n_keep", slot->params.n_keep);
slot->sparams.seed = json_value(data, "seed", default_sparams.seed);
slot->sparams.grammar = json_value(data, "grammar", default_sparams.grammar);
{
// upstream changed common_params_sampling::grammar from std::string to
// the common_grammar struct (type + grammar). The incoming JSON still
// carries a plain string, so build the user-provided grammar here and
// fall back to the server default when the request omits it.
std::string grammar_str = json_value(data, "grammar", std::string());
slot->sparams.grammar = grammar_str.empty()
? default_sparams.grammar
: common_grammar{COMMON_GRAMMAR_TYPE_USER, std::move(grammar_str)};
}
slot->sparams.n_probs = json_value(data, "n_probs", default_sparams.n_probs);
slot->sparams.min_keep = json_value(data, "min_keep", default_sparams.min_keep);
slot->sparams.grammar_triggers = grammar_triggers;
@@ -1232,7 +1241,7 @@ struct llama_server_context
// {"logit_bias", slot.sparams.logit_bias},
{"n_probs", slot.sparams.n_probs},
{"min_keep", slot.sparams.min_keep},
{"grammar", slot.sparams.grammar},
{"grammar", slot.sparams.grammar.grammar},
{"samplers", samplers}
};
}