fix(gallery): configure EmbeddingGemma 2 variants (#12667)

* chore(model gallery): 🤖 add new models via gallery agent Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): give the Q6 build a unique name The Q8 entry already uses embeddinggemma-2. Include the quantization in the new entry name so both builds remain reachable. Assisted-by: Codex:gpt-6 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): configure EmbeddingGemma 2 variants Serve the Unsloth builds as text embeddings and remove chat settings.\nGroup Q4, Q5, and Q6 with the existing family and pin verified assets. Assisted-by: Codex:gpt-6 --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
2026-10-28 00:59:28 -04:00 · 2026-10-17 22:45:07 +02:00
1 parent e7cc993b0f
commit 216c85374e
@@ -331,6 +331,24 @@ When browsing the gallery or importing a model by URI, LocalAI can show **estima
 - **Hardware fit indicator**: When your system reports GPU or RAM capacity, each row shows whether the estimated memory at the chosen context length fits, using a 95% headroom rule. A model that is too big for the GPU but would run from system RAM is marked as spilling to the CPU, with how much; a model too big for both is marked as over by the shortfall.
 - Estimates are best-effort and may be missing if the server does not support HEAD/Range or the request times out.

+## EmbeddingGemma 2
+
+The `embeddinggemma-2` family serves text embeddings through `llama-cpp`.
+It offers Q8_0 and BF16 weights, plus smaller Unsloth UD-Q4_K_XL,
+UD-Q5_K_XL, and UD-Q6_K_XL variants. These entries use the 270M-parameter
+text backbone with an 8192-token context and 768-dimensional output.
+
+To select a smaller build explicitly:
+
+```sh
+local-ai models install embeddinggemma-2 --variant embeddinggemma-2-ud-q4-k-xl
+```
+
+Send text to `/v1/embeddings`. For retrieval, prefix queries with
+`task: search result | query: ` and documents with `title: {title} | text: `.
+See the [model card](https://huggingface.co/google/embeddinggemma-2)
+for other task prefixes. These gallery entries support text input only.
+
 ## Useful Links and resources

 - [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) - here you can find a list of the most performing models on the Open LLM benchmark. Keep in mind models compatible with LocalAI must be quantized in the `gguf` format.
@@ -38545,6 +38545,9 @@
 - name: embeddinggemma-2
  variants:
    - model: embeddinggemma-2-bf16
+    - model: embeddinggemma-2-ud-q4-k-xl
+    - model: embeddinggemma-2-ud-q5-k-xl
+    - model: embeddinggemma-2-ud-q6-k-xl
  url: github:mudler/LocalAI/gallery/virtual.yaml@master
  urls:
    - https://huggingface.co/google/embeddinggemma-2
@@ -38602,6 +38605,93 @@
    - filename: llama-cpp/models/embeddinggemma-2/embeddinggemma-2-BF16.gguf
      sha256: 68bae29d62fb8c7d23e98d21fd4662753ddd636e6b62b8a70dfe059a9844f216
      uri: https://huggingface.co/ggml-org/embeddinggemma-2-GGUF/resolve/bfcd298762cc34d0357ece5ebdd31791a3a374d8/embeddinggemma-2-BF16.gguf
+- name: embeddinggemma-2-ud-q4-k-xl
+  url: github:mudler/LocalAI/gallery/virtual.yaml@master
+  urls:
+    - https://huggingface.co/google/embeddinggemma-2
+    - https://huggingface.co/unsloth/embeddinggemma-2-GGUF
+  description: |
+    EmbeddingGemma 2 is Google's multilingual embedding model.
+    This Unsloth UD-Q4_K_XL GGUF serves the 270M-parameter text backbone through llama.cpp
+    and produces 768-dimensional embeddings for retrieval and semantic similarity.
+    This entry supports text input with an 8192-token context.
+    Use the model's task prefixes for queries and documents.
+  license: apache-2.0
+  tags:
+    - embeddings
+    - gguf
+    - cpu
+    - gpu
+  overrides:
+    backend: llama-cpp
+    embeddings: true
+    context_size: 8192
+    known_usecases:
+      - embeddings
+    parameters:
+      model: llama-cpp/models/embeddinggemma-2/embeddinggemma-2-UD-Q4_K_XL.gguf
+  files:
+    - filename: llama-cpp/models/embeddinggemma-2/embeddinggemma-2-UD-Q4_K_XL.gguf
+      sha256: ea905fd08e8061db77a0031cbb5096d459d7cfd2bdd0ad2ba66e483f16719493
+      uri: https://huggingface.co/unsloth/embeddinggemma-2-GGUF/resolve/031f0d4b35536f69ab3509d4893c923264fcf253/embeddinggemma-2-UD-Q4_K_XL.gguf
+- name: embeddinggemma-2-ud-q5-k-xl
+  url: github:mudler/LocalAI/gallery/virtual.yaml@master
+  urls:
+    - https://huggingface.co/google/embeddinggemma-2
+    - https://huggingface.co/unsloth/embeddinggemma-2-GGUF
+  description: |
+    EmbeddingGemma 2 is Google's multilingual embedding model.
+    This Unsloth UD-Q5_K_XL GGUF serves the 270M-parameter text backbone through llama.cpp
+    and produces 768-dimensional embeddings for retrieval and semantic similarity.
+    This entry supports text input with an 8192-token context.
+    Use the model's task prefixes for queries and documents.
+  license: apache-2.0
+  tags:
+    - embeddings
+    - gguf
+    - cpu
+    - gpu
+  overrides:
+    backend: llama-cpp
+    embeddings: true
+    context_size: 8192
+    known_usecases:
+      - embeddings
+    parameters:
+      model: llama-cpp/models/embeddinggemma-2/embeddinggemma-2-UD-Q5_K_XL.gguf
+  files:
+    - filename: llama-cpp/models/embeddinggemma-2/embeddinggemma-2-UD-Q5_K_XL.gguf
+      sha256: a9d7a3b7eb421113f05e964e10882676f10fff83811b7876d5bb983db7c7ac41
+      uri: https://huggingface.co/unsloth/embeddinggemma-2-GGUF/resolve/031f0d4b35536f69ab3509d4893c923264fcf253/embeddinggemma-2-UD-Q5_K_XL.gguf
+- name: embeddinggemma-2-ud-q6-k-xl
+  url: github:mudler/LocalAI/gallery/virtual.yaml@master
+  urls:
+    - https://huggingface.co/google/embeddinggemma-2
+    - https://huggingface.co/unsloth/embeddinggemma-2-GGUF
+  description: |
+    EmbeddingGemma 2 is Google's multilingual embedding model.
+    This Unsloth UD-Q6_K_XL GGUF serves the 270M-parameter text backbone through llama.cpp
+    and produces 768-dimensional embeddings for retrieval and semantic similarity.
+    This entry supports text input with an 8192-token context.
+    Use the model's task prefixes for queries and documents.
+  license: apache-2.0
+  tags:
+    - embeddings
+    - gguf
+    - cpu
+    - gpu
+  overrides:
+    backend: llama-cpp
+    embeddings: true
+    context_size: 8192
+    known_usecases:
+      - embeddings
+    parameters:
+      model: llama-cpp/models/embeddinggemma-2/embeddinggemma-2-UD-Q6_K_XL.gguf
+  files:
+    - filename: llama-cpp/models/embeddinggemma-2/embeddinggemma-2-UD-Q6_K_XL.gguf
+      sha256: dc84f042bc3ffbe6122b3b38b2fb299d66f5f37b276c9d458db85d4a4f1e6ba3
+      uri: https://huggingface.co/unsloth/embeddinggemma-2-GGUF/resolve/031f0d4b35536f69ab3509d4893c923264fcf253/embeddinggemma-2-UD-Q6_K_XL.gguf
 - name: embeddinggemma-300m
  url: github:mudler/LocalAI/gallery/virtual.yaml@master
  urls: