diff --git a/docs/content/features/model-gallery.md b/docs/content/features/model-gallery.md index f0088f583..4031f0210 100644 --- a/docs/content/features/model-gallery.md +++ b/docs/content/features/model-gallery.md @@ -64,6 +64,19 @@ The [source model](https://huggingface.co/XHToken/Spark-X2.5-1.7B) supports up t 1,048,576 tokens; larger contexts require more memory. Use a current LocalAI llama.cpp backend with Spark-X2.5 support. +## MiniCPM5-2B + +MiniCPM5-2B offers the official Q4_K_M, Q8_0, and F16 GGUF builds through +llama.cpp. To install the F16 build explicitly: + +```bash +local-ai models install minicpm5-2b --variant minicpm5-2b-f16 +``` + +The F16 weights require a 5.04 GB download, plus additional memory for inference. +This entry uses the embedded chat template and an 8,192-token context. +See the [official GGUF repository](https://huggingface.co/openbmb/MiniCPM5-2B-GGUF). + ## VRAM and download size estimates When browsing the gallery or importing a model by URI, LocalAI can show **estimated download size** and **estimated VRAM** for models. diff --git a/gallery/index.yaml b/gallery/index.yaml index 104827cc3..baa711d99 100644 --- a/gallery/index.yaml +++ b/gallery/index.yaml @@ -27155,6 +27155,7 @@ Q4_K_M GGUF quantization and the model's embedded chat template. variants: - model: minicpm5-2b-q8 + - model: minicpm5-2b-f16 overrides: backend: llama-cpp known_usecases: @@ -27196,6 +27197,30 @@ - filename: llama-cpp/models/minicpm5-2b/MiniCPM5-2B-Q8_0.gguf uri: huggingface://openbmb/MiniCPM5-2B-GGUF/MiniCPM5-2B-Q8_0.gguf sha256: c5415f8989bf88a8288f1b55a3cc371af53c07b0faa220a63bd7a990cfaba078 +- !!merge <<: *minicpm5-2b + name: minicpm5-2b-f16 + description: | + MiniCPM5-2B is OpenBMB's compact Llama-based language model for English + and Chinese chat, coding, and reasoning. This entry uses the official + F16 GGUF build and the model's embedded chat template. + variants: null + overrides: + backend: llama-cpp + context_size: 8192 + known_usecases: + - chat + options: + - use_jinja:true + template: + use_tokenizer_template: true + parameters: + model: llama-cpp/models/minicpm5-2b/MiniCPM5-2B-F16.gguf + temperature: 1.0 + top_p: 0.95 + files: + - filename: llama-cpp/models/minicpm5-2b/MiniCPM5-2B-F16.gguf + uri: https://huggingface.co/openbmb/MiniCPM5-2B-GGUF/resolve/2079a22f3beaa4e306449978533478fe0522f4b3/MiniCPM5-2B-F16.gguf + sha256: 0ffba3682a853295566b98bc38c0ab755d6b2d91b994bc031ed727a08cfcdf25 - &minicpm5-1b name: minicpm5-1b url: github:mudler/LocalAI/gallery/chatml.yaml@master