mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 09:08:54 -04:00
chore: ⬆️ Update ggml-org/llama.cpp to f2cc7282ce0042641df55e0e563b75e824e2b647 (#12601)
* ⬆️ Update ggml-org/llama.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix(llama-cpp): preserve chat session state Newer llama.cpp parsers return template state outside the request JSON. Pass that state to both inference tasks and response readers so grammar, tool calls, and reasoning remain available in both chat paths. Detect the parser API during preparation to preserve older fork builds. Assisted-by: Codex:gpt-6 --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
This commit is contained in:
4 files changed
+44
-2
No files matched your search
@@ -548,7 +548,10 @@ It is sufficient to copy the `ggml` or `gguf` model files in the `models` folder
|
||||
|
||||
[You can optionally create an associated YAML]({{%relref "advanced" %}}) model config file to tune the model's parameters or apply a template to the prompt.
|
||||
|
||||
Prompt templates are useful for models that are fine-tuned towards a specific prompt.
|
||||
Prompt templates are useful for models that are fine-tuned towards a specific prompt.
|
||||
|
||||
With the llama.cpp backend, streaming and non-streaming chat requests preserve
|
||||
the template's grammar, stop strings, and message boundaries.
|
||||
|
||||
##### Automatic setup
|
||||
|
||||
|
||||
Reference in new issue
Block a user