mirror/LocalAI
1
0 Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-10-28 17:18:49 -04:00
Code Issues Packages Projects Releases Wiki Activity

chore: ⬆️ Update ggml-org/llama.cpp to 78a8ae14be559b45f5f13ddee82c188b683d5d8d (#12741)

Browse Source 
* ⬆️ Update ggml-org/llama.cpp

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix(llama-cpp): adapt to the multimodal rerank signature

Upstream llama.cpp added query_files and doc_files parameters to
format_prompt_rerank() (ggml-org/llama.cpp#29625). The gRPC rerank
call passed the init options in their place, so grpc-server.cpp no
longer compiled against the new pin.

prepare.sh now probes server-common.h for query_files and writes
LOCALAI_HAS_RERANK_FILES into llama_compat.h. When it is set, the
rerank call passes two empty media lists. The rerank API is text
only, so behaviour is unchanged. Forks that still use the old
signature (turboquant, bonsai) compile as before.

Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
localai-org-maint-botmudlerEttore Di Giacinto
 authored and GitHub committed 2026-10-25 10:54:23 +01:00
1 parent 49f0a30df2
commit 2200489044
3 files changed
+18 -2
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

backend/cpp/llama-cpp/grpc-server.cpp
+9 -1
View File
@@ -107,6 +107,14 @@ using grpc::Status;
#define LOCALAI_MTMD_INIT_OPT_ARG(value)
#endif
// Newer format_prompt_rerank() takes the decoded media of the query and of the
// document. The gRPC rerank API is text only, so pass empty lists.
#if LOCALAI_HAS_RERANK_FILES
#define LOCALAI_RERANK_FILES_ARGS , std::vector<raw_buffer>(), std::vector<raw_buffer>()
#else
#define LOCALAI_RERANK_FILES_ARGS
#endif
// gRPC bearer token auth for distributed mode.
// Reads LOCALAI_GRPC_AUTH_TOKEN from the environment. When set, rejects
// requests without a matching "authorization: Bearer <token>" metadata header.
@@ -3114,7 +3122,7 @@ public:
tasks.reserve(documents.size());
for (size_t i = 0; i < documents.size(); i++) {
auto tmp = format_prompt_rerank(ctx_server.impl->model_tgt, ctx_server.impl->vocab, ctx_server.impl->mctx, request->query(), documents[i] LOCALAI_MTMD_INIT_OPT_ARG(ctx_server.impl->init_opt));
auto tmp = format_prompt_rerank(ctx_server.impl->model_tgt, ctx_server.impl->vocab, ctx_server.impl->mctx, request->query(), documents[i] LOCALAI_RERANK_FILES_ARGS LOCALAI_MTMD_INIT_OPT_ARG(ctx_server.impl->init_opt));
server_task task = server_task(SERVER_TASK_TYPE_RERANK);
task.id = rd.queue_tasks.get_new_id();
task.index = i;