6af83592d0
A no-ai-slop detect pass over all 85 docs pages and the 8 website content files. The docs came out clean on every pattern that actually got the blog post criticized on HN: zero faux-insight setups, zero unearned framing, zero ledger metaphors, zero pre-chewed numbers, zero importance puffery, zero weasel attribution, zero recap endings, zero rhetorical setups. What was left was vocabulary, so that is all this changes. Ten edits in eight files, no links or code blocks touched: - overview.md: "In today's AI landscape, privacy, control, and flexibility are paramount" and "Ready to dive in?" - architecture.md: "seamlessly integrate ... effortlessly implemented" - customize-model.md: "is utilized", "utilizes a shorthand format" - advanced/_index: "fully leverage LocalAI's capabilities beyond basic usage" - agents.md, object-detection.md, text-to-audio.md, faq.md: leverage/seamless used as filler. text-to-audio also had "before the api provide its response". Deliberately left alone: "GPU utilization", "KV utilization" and "highest-leverage knob" are the correct technical terms, not filler. The docs are reference material and read like it. They do not need the treatment the blog posts got. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3.1 KiB
+++ disableToc = false title = "Customizing the Model" weight = 5 url = "/getting-started/customize-model" icon = "rocket_launch"
+++
To customize the prompt template or the default settings of the model, a configuration file is used. This file must adhere to the LocalAI YAML configuration standards. For comprehensive syntax details, refer to the [advanced documentation]({{%relref "advanced" %}}). The configuration file can be located either remotely (such as in a Github Gist) or within the local filesystem or a remote URL.
LocalAI can be initiated using either its container image or binary, with a command that includes URLs of model config files or uses a shorthand format (like huggingface:// or github://), which is then expanded into complete URLs.
The configuration can also be set via an environment variable. For instance:
local-ai github://owner/repo/file.yaml@branch
MODELS="github://owner/repo/file.yaml@branch,github://owner/repo/file.yaml@branch" local-ai
Here's an example to initiate the phi-2 model:
docker run -p 8080:8080 localai/localai:{{< version >}} https://gist.githubusercontent.com/mudler/ad601a0488b497b69ec549150d9edd18/raw/a8a8869ef1bb7e3830bf5c0bae29a0cce991ff8d/phi-2.yaml
You can also check all the embedded models configurations here.
{{% notice tip %}} The model configurations used in the quickstart are accessible here: https://github.com/mudler/LocalAI/tree/master/embedded/models. Contributions are welcome; please feel free to submit a Pull Request.
The phi-2 model configuration from the quickstart is expanded from https://github.com/mudler/LocalAI/blob/master/examples/configurations/phi-2.yaml.
{{% /notice %}}
Example: Customizing the Prompt Template
To modify the prompt template, create a Github gist or a Pastebin file, and copy the content from https://github.com/mudler/LocalAI/blob/master/examples/configurations/phi-2.yaml. Alter the fields as needed:
name: phi-2
context_size: 2048
f16: true
threads: 11
gpu_layers: 90
mmap: true
parameters:
# Reference any HF model or a local file here
model: huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
temperature: 0.2
top_k: 40
top_p: 0.95
template:
chat: &template |
Instruct: {{.Input}}
Output:
# Modify the prompt template here ^^^ as per your requirements
completion: *template
Then, launch LocalAI using your gist's URL:
## Important! Substitute with your gist's URL!
docker run -p 8080:8080 localai/localai:{{< version >}} https://gist.githubusercontent.com/xxxx/phi-2.yaml
Next Steps
- Visit the [advanced section]({{%relref "advanced" %}}) for more insights on prompt templates and configuration files.
- To learn about fine-tuning an LLM model, check out the [fine-tuning section]({{%relref "features/fine-tuning" %}}).