mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 09:08:54 -04:00
feat(gallery): add Hy-Memory 30B A3B variants
Add official Q4_K_M and Q8_0 builds for memory-writing workflows. Use the embedded template with no-think mode and pinned downloads. Document installation and variant selection. Assisted-by: Codex:gpt-6
This commit is contained in:
1 parent
f1ff707b97
commit
a1259642c5
2 files changed
+107
No files matched your search
@@ -44,6 +44,29 @@ Both views store the view, search, filter, and selection in the URL. Installing
|
||||
from Explore does not move you away from the catalog; the entry updates in place
|
||||
when the operation finishes.
|
||||
|
||||
### Hy-Memory 30B A3B
|
||||
|
||||
[Hy-Memory](https://huggingface.co/tencent/Hy-Memory-30B-A3B) is Tencent's
|
||||
English and Chinese memory-writing model, with about 3B active parameters.
|
||||
It extracts and updates memories for external agent frameworks. The framework
|
||||
must provide the task prompts and persistent storage.
|
||||
|
||||
The gallery offers official Q4_K_M and Q8_0 builds for llama.cpp. Both use the
|
||||
embedded chat template, `reasoning_effort: no_think`, and an 8,192-token default
|
||||
context. Increase `context_size` in the model configuration for longer inputs
|
||||
if enough memory is available; the model supports up to 262,144 tokens.
|
||||
|
||||
Install with automatic variant selection:
|
||||
|
||||
```bash
|
||||
local-ai models install hy-memory-30b-a3b-q4
|
||||
```
|
||||
|
||||
To select a build explicitly, add `--variant hy-memory-30b-a3b-q4` or
|
||||
`--variant hy-memory-30b-a3b-q8`. See the
|
||||
[publisher's GGUF guide](https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF)
|
||||
for memory-framework examples and quantization evaluations.
|
||||
|
||||
### MiniCPM-V 4.7 1B
|
||||
|
||||
[MiniCPM-V 4.7 1B](https://huggingface.co/openbmb/MiniCPM-V-4.7-1B)
|
||||
|
||||
@@ -12975,6 +12975,90 @@
|
||||
model: llama-cpp/models/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-Q8_0/MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-Q8_0.gguf
|
||||
template:
|
||||
use_tokenizer_template: true
|
||||
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
|
||||
name: hy-memory-30b-a3b-q4
|
||||
license: apache-2.0
|
||||
urls:
|
||||
- https://huggingface.co/tencent/Hy-Memory-30B-A3B
|
||||
- https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF
|
||||
description: |
|
||||
Tencent Hy-Memory is a 30B MoE model with about 3B active parameters
|
||||
for writing and updating long-term memory in external agent frameworks.
|
||||
This Q4_K_M build uses llama.cpp with the embedded chat template and
|
||||
no-think mode. It supports English and Chinese; the memory framework
|
||||
must supply the task prompts and persistent storage.
|
||||
tags:
|
||||
- llm
|
||||
- gguf
|
||||
- cpu
|
||||
- gpu
|
||||
- memory
|
||||
- english
|
||||
- chinese
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
context_size: 8192
|
||||
chat_template_kwargs:
|
||||
reasoning_effort: no_think
|
||||
known_usecases:
|
||||
- chat
|
||||
options:
|
||||
- use_jinja:true
|
||||
parameters:
|
||||
model: Hy-Memory-30B-A3B-Q4_K_M.gguf
|
||||
temperature: 0.7
|
||||
top_k: 0
|
||||
top_p: 1.0
|
||||
repeat_penalty: 1.0
|
||||
template:
|
||||
use_tokenizer_template: true
|
||||
files:
|
||||
- filename: Hy-Memory-30B-A3B-Q4_K_M.gguf
|
||||
sha256: e417fc86d0a2bb058ed253028f4617d144aa874b8af35711cca0c7009bd63446
|
||||
uri: https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF/resolve/38e10a37701e1caff4a6dc3831448aaf8c2e15d1/Hy-Memory-30B-A3B-Q4_K_M.gguf
|
||||
variants:
|
||||
- model: hy-memory-30b-a3b-q8
|
||||
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
|
||||
name: hy-memory-30b-a3b-q8
|
||||
license: apache-2.0
|
||||
urls:
|
||||
- https://huggingface.co/tencent/Hy-Memory-30B-A3B
|
||||
- https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF
|
||||
description: |
|
||||
Tencent Hy-Memory is a 30B MoE model with about 3B active parameters
|
||||
for writing and updating long-term memory in external agent frameworks.
|
||||
This Q8_0 build uses llama.cpp with the embedded chat template and
|
||||
no-think mode. It supports English and Chinese; the memory framework
|
||||
must supply the task prompts and persistent storage.
|
||||
tags:
|
||||
- llm
|
||||
- gguf
|
||||
- cpu
|
||||
- gpu
|
||||
- memory
|
||||
- english
|
||||
- chinese
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
context_size: 8192
|
||||
chat_template_kwargs:
|
||||
reasoning_effort: no_think
|
||||
known_usecases:
|
||||
- chat
|
||||
options:
|
||||
- use_jinja:true
|
||||
parameters:
|
||||
model: Hy-Memory-30B-A3B-Q8_0.gguf
|
||||
temperature: 0.7
|
||||
top_k: 0
|
||||
top_p: 1.0
|
||||
repeat_penalty: 1.0
|
||||
template:
|
||||
use_tokenizer_template: true
|
||||
files:
|
||||
- filename: Hy-Memory-30B-A3B-Q8_0.gguf
|
||||
sha256: 3ba2897dbb7e21e618cb65cec4b31498e59eca2960b381c5e0a0590e7ce8526b
|
||||
uri: https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF/resolve/38e10a37701e1caff4a6dc3831448aaf8c2e15d1/Hy-Memory-30B-A3B-Q8_0.gguf
|
||||
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
|
||||
name: hy3
|
||||
description: "中文 | English\n\n[](#license)\n \n[](https://huggingface.co/tencent/Hy3)\n \n[](https://modelscope.cn/models/Tencent-Hunyuan/Hy3)\n \n[](https://cnb.cool/ai-models/tencent/Hy3)\n \n[](https://ai.gitcode.com/tencent_hunyuan/Hy3)\n\n\U0001F5A5️ Official Website | \n\U0001F4AC GitHub\n\n## Table of Contents\n\n - Model Introduction\n - Stronger Agent Capabilities\n - More Reliable Product Experiences\n - Benchmark Appendix\n - News\n - Model Links\n - Quickstart\n - Deployment\n - vLLM\n - SGLang\n - Finetuning\n - RL Post-training\n - Quantization\n - License\n - Contact Us\n\n## Model Introduction\n\n**Hy3** is a 295B-parameter Mixture-of-Experts (MoE) model with 21B active parameters and 3.8B MTP layer parameters, developed by the Tencent Hy Team. Following the Hy3 Preview launch in late April, we gathered feedback from 50+ products and scaled up post-training with higher quality data. Today, we introduce Hy3, which outperforms similar-size models and rivals flagship open-source models with 2-5x parameters. It also shows significant gains in utility across various products and productivity tasks.\n\n## Stronger Agent Capabilities\n\n...\n"
|
||||
|
||||
Reference in new issue
Block a user