mirror of
https://github.com/mudler/LocalAI.git
synced 2026-05-16 20:52:08 -04:00
969005b2a1
* feat: Rework VRAM estimation and use known_usecases in gallery Signed-off-by: Richard Palethorpe <io@richiejp.com> Assisted-by: Claude:claude-opus-4-7[1m] [Claude Code] * chore(gallery): regenerate gallery index and add known_usecases to model entries Signed-off-by: Richard Palethorpe <io@richiejp.com> --------- Signed-off-by: Richard Palethorpe <io@richiejp.com>
50 lines
2.0 KiB
YAML
50 lines
2.0 KiB
YAML
config_file: |
|
|
backend: llama-cpp
|
|
context_size: 8192
|
|
f16: true
|
|
known_usecases:
|
|
- chat
|
|
- completion
|
|
mmap: true
|
|
stopwords:
|
|
- <|im_end|>
|
|
- <dummy32000>
|
|
- </s>
|
|
- <|end_of_text|>
|
|
template:
|
|
chat: |
|
|
{{.Input -}}
|
|
<|start_of_role|>assistant<|end_of_role|>
|
|
chat_message: |
|
|
<|start_of_role|>{{ .RoleName }}<|end_of_role|>
|
|
{{ if .FunctionCall -}}
|
|
<tool_call>
|
|
{{ else if eq .RoleName "tool" -}}
|
|
<tool_response>
|
|
{{ end -}}
|
|
{{ if .Content -}}
|
|
{{.Content }}
|
|
{{ end -}}
|
|
{{ if eq .RoleName "tool" -}}
|
|
</tool_response>
|
|
{{ end -}}
|
|
{{ if .FunctionCall -}}
|
|
{{toJson .FunctionCall}}
|
|
</tool_call>
|
|
{{ end -}}
|
|
<|end_of_text|>
|
|
completion: |
|
|
{{.Input}}
|
|
function: |
|
|
<|start_of_role|>system<|end_of_role|>
|
|
You are a helpful AI assistant with access to the following tools. When a tool is required to answer the user's query, respond with <|tool_call|> followed by a JSON list of tools used. If a tool does not exist in the provided list of tools, notify the user that you do not have the ability to fulfill the request.
|
|
|
|
Write the response to the user's input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data.
|
|
{{range .Functions}}
|
|
{'type': 'function', 'function': {'name': '{{.Name}}', 'description': '{{.Description}}', 'parameters': {{toJson .Parameters}} }}
|
|
{{end}}
|
|
For each function call return a json object with function name and arguments
|
|
{{.Input -}}
|
|
<|start_of_role|>assistant<|end_of_role|>
|
|
name: granite-3.2
|