diff --git a/docs/content/features/model-gallery.md b/docs/content/features/model-gallery.md index b665be21c..01617f460 100644 --- a/docs/content/features/model-gallery.md +++ b/docs/content/features/model-gallery.md @@ -23,6 +23,36 @@ GPT and text generation models might have a license which is not permissive for ![output](https://github.com/mudler/LocalAI/assets/2420543/7b16676e-d5b1-4c97-89bd-9fa5065c21ad) +## EmbeddingGemma 2 text embeddings + +[EmbeddingGemma 2](https://huggingface.co/google/embeddinggemma-2) is available +for text embeddings through llama.cpp, with Q8_0 and BF16 builds. +The entries use the 270M-parameter text backbone, return 768-dimensional vectors, +and set an 8192-token context. They do not configure image, audio, or video input. + +Install with automatic variant selection: + +```bash +local-ai models install embeddinggemma-2 +``` + +To select a specific build, add `--variant embeddinggemma-2` for Q8_0 +or `--variant embeddinggemma-2-bf16` for BF16. + +Include the task prefix in each text input. For retrieval, use +`task: search result | query: {query}` for queries and +`title: {title} | text: {content}` for documents. +Use `none` as the title when a document has no title. +LocalAI does not add these prefixes automatically. + +For example, after installing the Q8_0 entry: + +```bash +curl http://localhost:8080/v1/embeddings \ + -H 'Content-Type: application/json' \ + -d '{"model":"embeddinggemma-2","input":"task: search result | query: How does local inference work?"}' +``` + ## How it works Open **Models** in the WebUI. It is the canonical page for a model's complete diff --git a/gallery/index.yaml b/gallery/index.yaml index 17a7cdfa1..ea4cc23c2 100644 --- a/gallery/index.yaml +++ b/gallery/index.yaml @@ -34529,6 +34529,66 @@ - filename: granite-embedding-125m-english-f16.gguf sha256: e2950cf0228514e0e81c6f0701a62a9e4763990ce660b4a3c0329cd6a4acd4b9 uri: huggingface://bartowski/granite-embedding-125m-english-GGUF/granite-embedding-125m-english-f16.gguf +- name: embeddinggemma-2 + variants: + - model: embeddinggemma-2-bf16 + url: github:mudler/LocalAI/gallery/virtual.yaml@master + urls: + - https://huggingface.co/google/embeddinggemma-2 + - https://huggingface.co/ggml-org/embeddinggemma-2-GGUF + description: | + EmbeddingGemma 2 is Google's multilingual embedding model. + This Q8_0 GGUF serves the 270M-parameter text backbone through llama.cpp + and produces 768-dimensional embeddings for retrieval and semantic similarity. + This entry supports text input with an 8192-token context. + Use the model's task prefixes for queries and documents. + license: apache-2.0 + tags: + - embeddings + - gguf + - cpu + - gpu + overrides: + backend: llama-cpp + embeddings: true + context_size: 8192 + known_usecases: + - embeddings + parameters: + model: llama-cpp/models/embeddinggemma-2/embeddinggemma-2-Q8_0.gguf + files: + - filename: llama-cpp/models/embeddinggemma-2/embeddinggemma-2-Q8_0.gguf + sha256: 2188ac1deca4b77dffefd603c2776a9d76d9d74ec01841392982ebb840b09135 + uri: https://huggingface.co/ggml-org/embeddinggemma-2-GGUF/resolve/bfcd298762cc34d0357ece5ebdd31791a3a374d8/embeddinggemma-2-Q8_0.gguf +- name: embeddinggemma-2-bf16 + url: github:mudler/LocalAI/gallery/virtual.yaml@master + urls: + - https://huggingface.co/google/embeddinggemma-2 + - https://huggingface.co/ggml-org/embeddinggemma-2-GGUF + description: | + EmbeddingGemma 2 is Google's multilingual embedding model. + This BF16 GGUF serves the 270M-parameter text backbone through llama.cpp + and produces 768-dimensional embeddings for retrieval and semantic similarity. + This entry supports text input with an 8192-token context. + Use the model's task prefixes for queries and documents. + license: apache-2.0 + tags: + - embeddings + - gguf + - cpu + - gpu + overrides: + backend: llama-cpp + embeddings: true + context_size: 8192 + known_usecases: + - embeddings + parameters: + model: llama-cpp/models/embeddinggemma-2/embeddinggemma-2-BF16.gguf + files: + - filename: llama-cpp/models/embeddinggemma-2/embeddinggemma-2-BF16.gguf + sha256: 68bae29d62fb8c7d23e98d21fd4662753ddd636e6b62b8a70dfe059a9844f216 + uri: https://huggingface.co/ggml-org/embeddinggemma-2-GGUF/resolve/bfcd298762cc34d0357ece5ebdd31791a3a374d8/embeddinggemma-2-BF16.gguf - name: embeddinggemma-300m url: github:mudler/LocalAI/gallery/virtual.yaml@master urls: