diff --git a/.agents/adding-gallery-models.md b/.agents/adding-gallery-models.md new file mode 100644 index 000000000..fe5a57e87 --- /dev/null +++ b/.agents/adding-gallery-models.md @@ -0,0 +1,111 @@ +# Adding GGUF Models from HuggingFace to the Gallery + +When adding a GGUF model from HuggingFace to the LocalAI model gallery, follow this guide. + +## Gallery file + +All models are defined in `gallery/index.yaml`. Find the appropriate section (embedding models near other embeddings, chat models near similar chat models) and add a new entry. + +## Getting the SHA256 + +GGUF files on HuggingFace expose their SHA256 via the `x-linked-etag` HTTP header. Fetch it with: + +```bash +curl -sI "https://huggingface.co///resolve/main/.gguf" | grep -i x-linked-etag +``` + +The value (without quotes) is the SHA256 hash. Example: + +```bash +curl -sI "https://huggingface.co/ggml-org/embeddinggemma-300m-qat-q8_0-GGUF/resolve/main/embeddinggemma-300m-qat-Q8_0.gguf" | grep -i x-linked-etag +# x-linked-etag: "6fa0c02a9c302be6f977521d399b4de3a46310a4f2621ee0063747881b673f67" +``` + +**Important**: Pay attention to exact filename casing — HuggingFace filenames are case-sensitive (e.g., `Q8_0` vs `q8_0`). Check the repo's file listing to get the exact name. + +## Entry format — Embedding models + +Embedding models use `gallery/virtual.yaml` as the base config and set `embeddings: true`: + +```yaml +- name: "model-name" + url: github:mudler/LocalAI/gallery/virtual.yaml@master + urls: + - https://huggingface.co// + - https://huggingface.co// + description: | + Short description of the model, its size, and capabilities. + tags: + - embeddings + overrides: + backend: llama-cpp + embeddings: true + parameters: + model: .gguf + files: + - filename: .gguf + uri: huggingface:////.gguf + sha256: +``` + +## Entry format — Chat/LLM models + +Chat models typically reference a template config (e.g., `gallery/gemma.yaml`, `gallery/chatml.yaml`) that defines the prompt format. Use YAML anchors (`&name` / `*name`) if adding multiple quantization variants of the same model: + +```yaml +- &model-anchor + url: "github:mudler/LocalAI/gallery/