mirror/LocalAI
1
0 Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-10-28 09:08:54 -04:00
Code Issues Packages Projects Releases Wiki Activity

feat(gallery): add BGE base English variants

Browse Source 
Add F16, Q8_0 and Q4_K_M builds with CLS pooling and retrieval usage documentation. Pin the GGUF revision and verify SHA256 values against Hugging Face metadata and download headers.

Assisted-by: Codex:gpt-6
This commit is contained in:
localai-org-maint-bot committed 2026-10-23 12:03:44 +00:00
1 parent 0a44303c21
commit e61c44f317
3 files changed
+207
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/content/features/embeddings.md
+9
View File
@@ -33,6 +33,15 @@ Example gallery models:
- `qwen3-embedding-8b` - Qwen3 Embedding 8B model
- `qwen3-embedding-0.6b` - Qwen3 Embedding 0.6B model
### BGE-base-en-v1.5
Install `bge-base-en-v1.5-f16` for English retrieval with llama.cpp. The gallery
also offers `bge-base-en-v1.5-q8-0` and `bge-base-en-v1.5-q4-k-m` variants.
These builds produce 768-dimensional embeddings with a 512-token context.
For retrieval, prefix search queries with
`Represent this sentence for searching relevant passages: `, including the trailing
space. Embed documents without this prefix.
### Example: Using Qwen3-Embedding-4B from Gallery
```bash
gallery/index.yaml
+87
View File
@@ -3,6 +3,93 @@
# written by scripts/models/cmd/modelcatalog and stays only so releases before
# the cutover keep working. CI fails when it differs from the generated copy:
# go run ./scripts/models/cmd/modelcatalog --library gallery/models/index.yaml --out gallery/index.yaml
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: bge-base-en-v1.5-f16
description: |
BGE-base-en-v1.5 is a 109M English embedding model for semantic search
and retrieval. This F16 GGUF build produces 768-dimensional embeddings
with a 512-token context. For retrieval, prefix queries with
"Represent this sentence for searching relevant passages: "; leave passages unprefixed.
license: mit
urls:
- https://huggingface.co/BAAI/bge-base-en-v1.5
- https://huggingface.co/CompendiumLabs/bge-base-en-v1.5-gguf
tags:
- embeddings
- gguf
- cpu
- gpu
files:
- filename: bge-base-en-v1.5-f16.gguf
sha256: 88360fdf8521af0ac08d43818bd272da679ab97c685d9b273c48efd01a4187c2
uri: https://huggingface.co/CompendiumLabs/bge-base-en-v1.5-gguf/resolve/24914efdfa9ee54e815c3fcaa78a617031251c5c/bge-base-en-v1.5-f16.gguf
overrides:
backend: llama-cpp
context_size: 512
embeddings: true
options:
- pooling:cls
parameters:
model: bge-base-en-v1.5-f16.gguf
variants:
- model: bge-base-en-v1.5-q8-0
- model: bge-base-en-v1.5-q4-k-m
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: bge-base-en-v1.5-q8-0
description: |
BGE-base-en-v1.5 is a 109M English embedding model for semantic search
and retrieval. This Q8_0 GGUF build produces 768-dimensional embeddings
with a 512-token context. For retrieval, prefix queries with
"Represent this sentence for searching relevant passages: "; leave passages unprefixed.
license: mit
urls:
- https://huggingface.co/BAAI/bge-base-en-v1.5
- https://huggingface.co/CompendiumLabs/bge-base-en-v1.5-gguf
tags:
- embeddings
- gguf
- cpu
- gpu
files:
- filename: bge-base-en-v1.5-q8_0.gguf
sha256: ad1afe72cd6654a558667a3db10878b049a75bfd72912e1dabb91310d671173c
uri: https://huggingface.co/CompendiumLabs/bge-base-en-v1.5-gguf/resolve/24914efdfa9ee54e815c3fcaa78a617031251c5c/bge-base-en-v1.5-q8_0.gguf
overrides:
backend: llama-cpp
context_size: 512
embeddings: true
options:
- pooling:cls
parameters:
model: bge-base-en-v1.5-q8_0.gguf
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: bge-base-en-v1.5-q4-k-m
description: |
BGE-base-en-v1.5 is a 109M English embedding model for semantic search
and retrieval. This Q4_K_M GGUF build produces 768-dimensional embeddings
with a 512-token context. For retrieval, prefix queries with
"Represent this sentence for searching relevant passages: "; leave passages unprefixed.
license: mit
urls:
- https://huggingface.co/BAAI/bge-base-en-v1.5
- https://huggingface.co/CompendiumLabs/bge-base-en-v1.5-gguf
tags:
- embeddings
- gguf
- cpu
- gpu
files:
- filename: bge-base-en-v1.5-q4_k_m.gguf
sha256: 74aebb552ea73b271d3b9c709923b4b7633b304fbc897a0498e52a180c3a9da9
uri: https://huggingface.co/CompendiumLabs/bge-base-en-v1.5-gguf/resolve/24914efdfa9ee54e815c3fcaa78a617031251c5c/bge-base-en-v1.5-q4_k_m.gguf
overrides:
backend: llama-cpp
context_size: 512
embeddings: true
options:
- pooling:cls
parameters:
model: bge-base-en-v1.5-q4_k_m.gguf
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: adavar-3b-q4
description: |
gallery/models/index.yaml
+111
View File
@@ -4,6 +4,117 @@
# changes and never edit it by hand. CI fails when it is stale. A build from
# master reads it as the default model source; a tagged build reads the gallery
# until the release cutover.
- kind: model
id: bge-base-en-v1.5-f16
name: bge-base-en-v1.5-f16
version: 0.0.0-gallery.e45654707d11
description: |
BGE-base-en-v1.5 is a 109M English embedding model for semantic search
and retrieval. This F16 GGUF build produces 768-dimensional embeddings
with a 512-token context. For retrieval, prefix queries with
"Represent this sentence for searching relevant passages: "; leave passages unprefixed.
license: mit
urls:
- https://huggingface.co/BAAI/bge-base-en-v1.5
- https://huggingface.co/CompendiumLabs/bge-base-en-v1.5-gguf
tags:
- embeddings
- gguf
- cpu
- gpu
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: bge-base-en-v1.5-f16.gguf
sha256: 88360fdf8521af0ac08d43818bd272da679ab97c685d9b273c48efd01a4187c2
uri: https://huggingface.co/CompendiumLabs/bge-base-en-v1.5-gguf/resolve/24914efdfa9ee54e815c3fcaa78a617031251c5c/bge-base-en-v1.5-f16.gguf
overrides:
backend: llama-cpp
context_size: 512
embeddings: true
options:
- pooling:cls
parameters:
model: bge-base-en-v1.5-f16.gguf
variants:
- model: bge-base-en-v1.5-q8-0
- model: bge-base-en-v1.5-q4-k-m
- kind: model
id: bge-base-en-v1.5-q8-0
name: bge-base-en-v1.5-q8-0
version: 0.0.0-gallery.c8e222453322
description: |
BGE-base-en-v1.5 is a 109M English embedding model for semantic search
and retrieval. This Q8_0 GGUF build produces 768-dimensional embeddings
with a 512-token context. For retrieval, prefix queries with
"Represent this sentence for searching relevant passages: "; leave passages unprefixed.
license: mit
urls:
- https://huggingface.co/BAAI/bge-base-en-v1.5
- https://huggingface.co/CompendiumLabs/bge-base-en-v1.5-gguf
tags:
- embeddings
- gguf
- cpu
- gpu
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: bge-base-en-v1.5-q8_0.gguf
sha256: ad1afe72cd6654a558667a3db10878b049a75bfd72912e1dabb91310d671173c
uri: https://huggingface.co/CompendiumLabs/bge-base-en-v1.5-gguf/resolve/24914efdfa9ee54e815c3fcaa78a617031251c5c/bge-base-en-v1.5-q8_0.gguf
overrides:
backend: llama-cpp
context_size: 512
embeddings: true
options:
- pooling:cls
parameters:
model: bge-base-en-v1.5-q8_0.gguf
- kind: model
id: bge-base-en-v1.5-q4-k-m
name: bge-base-en-v1.5-q4-k-m
version: 0.0.0-gallery.1dfbb38a60d2
description: |
BGE-base-en-v1.5 is a 109M English embedding model for semantic search
and retrieval. This Q4_K_M GGUF build produces 768-dimensional embeddings
with a 512-token context. For retrieval, prefix queries with
"Represent this sentence for searching relevant passages: "; leave passages unprefixed.
license: mit
urls:
- https://huggingface.co/BAAI/bge-base-en-v1.5
- https://huggingface.co/CompendiumLabs/bge-base-en-v1.5-gguf
tags:
- embeddings
- gguf
- cpu
- gpu
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: bge-base-en-v1.5-q4_k_m.gguf
sha256: 74aebb552ea73b271d3b9c709923b4b7633b304fbc897a0498e52a180c3a9da9
uri: https://huggingface.co/CompendiumLabs/bge-base-en-v1.5-gguf/resolve/24914efdfa9ee54e815c3fcaa78a617031251c5c/bge-base-en-v1.5-q4_k_m.gguf
overrides:
backend: llama-cpp
context_size: 512
embeddings: true
options:
- pooling:cls
parameters:
model: bge-base-en-v1.5-q4_k_m.gguf
- kind: model
id: adavar-3b-q4
name: adavar-3b-q4