Offer intermediate quantizations between the existing Q4 and Q8 builds.
Pin both to the same publisher revision to preserve the step-576 merge.
Document explicit variant selection.
Assisted-by: Codex:gpt-6
(cherry picked from commit e78c44dc7b, mudler/LocalAI#12591)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Fill the gap between the existing Q4 and Q8 builds with the
publisher's Q5_K_M and Q6_K GGUF files. Preserve the runtime settings
and license notice, and document explicit variant installation.
Assisted-by: Codex:gpt-6
(cherry picked from commit 3b73ecd1e2, mudler/LocalAI#12589)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The Qwopus 3.5 9B Coder MTP projector URL returns 404. Use the
publisher's mmproj-F32.gguf, which has the same SHA256, and pin
all downloads to the verified repository revision.
Offer Q6_K and Q8_0 variants alongside Q4_K_M and document explicit
selection. Keep the existing MTP and chat-template settings.
Assisted-by: Codex:gpt-6
(cherry picked from commit 01c5e044f2, mudler/LocalAI#12586)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add JetBrains Q4_K_M and Q8_0 GGUF builds for llama.cpp, with
variant selection and installation documentation. Pin both downloads
and verify their SHA256 values against LFS metadata and HTTP headers.
Assisted-by: Codex:gpt-6
(cherry picked from commit a5accc7488, mudler/LocalAI#12573)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add Q8_0 and BF16 builds for text embeddings through llama.cpp.
Document installation and retrieval task prefixes.
Assisted-by: Codex:gpt-6
(cherry picked from commit b166f881e9, mudler/LocalAI#12565)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add the mixed IQ2 GGUF with its F16 vision projector for llama.cpp.
Pin both downloads and cross-check SHA256 values against HF LFS metadata.
Document installation and the default context size.
Assisted-by: Codex:gpt-6
(cherry picked from commit 3657d221a0, mudler/LocalAI#12564)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add the Japanese Spark-X2.5-4B adaptation in Q4_K_M and Q8_0.
Pin both downloads and document variant selection and output budgets.
Assisted-by: Codex:gpt-6
(cherry picked from commit 592a913442, mudler/LocalAI#12563)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add Q4_K_M and Q8_0 builds with the BF16 vision projector for llama.cpp.
Pin all three assets to a Hugging Face revision and document installation.
Assisted-by: Codex:gpt-6
(cherry picked from commit 215ec2f0f9, mudler/LocalAI#12561)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Offer Q4_K_M and Q8_0 builds of the Qwen3.5 tool-routing fine-tune.
Keep the entries text-only because the quantization repository does
not publish a vision projector. Pin downloads and document selection.
Assisted-by: Codex:gpt-6
(cherry picked from commit 98cad4a19f, mudler/LocalAI#12559)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add Q4_K_M and Q6_K builds with the BF16 vision projector.
Pin all files to a verified Hugging Face revision and document usage.
Assisted-by: Codex:gpt-6
(cherry picked from commit 689aca7f03, mudler/LocalAI#12552)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add Q4_K_M and Q8_0 builds with the F16 vision projector. Pin the
assets and document the prompts needed for adaptive visual reasoning.
Assisted-by: Codex:gpt-6
(cherry picked from commit 8c64079b01, mudler/LocalAI#12547)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add Q4_K_M and Q8_0 builds with the F16 vision projector for document
visual question answering. Pin the assets and document the external
agent loop needed to reproduce the publisher's agentic results.
Assisted-by: Codex:gpt-6
(cherry picked from commit 50f2715967, mudler/LocalAI#12545)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Fill the quantization gap between the existing Q4 and Q8 builds. Pin both downloads to the same publisher revision and verify SHA256 values against LFS metadata and x-linked-etag headers.
Assisted-by: Codex:gpt-6
(cherry picked from commit 1c17d6fe9d, mudler/LocalAI#12543)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add the published four-shard UD-Q4_K_XL build for llama.cpp with
revision-pinned downloads and verified SHA256 checksums. Document
installation, download size, text-only input, and sampling defaults.
Assisted-by: Codex:gpt-6
(cherry picked from commit e32d0f50de, mudler/LocalAI#12542)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add Q4_K_M and Q8_0 GGUF builds for screenshot interpretation and GUI
planning with llama.cpp. Include the F16 vision projector in both builds
and document installation and the separate action executor requirement.
Pin all downloads and verify SHA256 values against Hugging Face LFS
metadata and download headers.
Assisted-by: Codex:gpt-6
(cherry picked from commit c4273485d1, mudler/LocalAI#12541)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Offer Q4_K_M and Q6_K builds for character roleplay with llama.cpp.
Pin downloads and document explicit variant selection.
Assisted-by: Codex:gpt-6
(cherry picked from commit 2aaae6cfd3, mudler/LocalAI#12523)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Offer Q4_K_M and Q8_0 builds with the matching vision projector.
Pin downloads and verify their SHA256 against Hugging Face metadata
and response headers. Document installation and variant selection.
Assisted-by: Codex:gpt-6
(cherry picked from commit 0e8ebcde37, mudler/LocalAI#12516)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add Q4_K_M and Q8_0 builds with the matching vision projector for
single-step screenshot grounding on Kayak interfaces.
Assisted-by: Codex:gpt-6-astra
(cherry picked from commit 2f54c3216a, mudler/LocalAI#12513)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add official Q4_K_M and Q8_0 builds with pinned downloads and verified
SHA256 values. Document installation and text translation prompts.
Assisted-by: Codex:gpt-6
(cherry picked from commit e50820d9d0, mudler/LocalAI#12512)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Offer Q4_K_M and Q8_0 builds for cited research reports from supplied
literature excerpts. Use the embedded Qwen3 template and document the
upstream prompt. Pin both downloads and cross-check their SHA256 values
against Hugging Face LFS metadata and download headers.
Assisted-by: Codex:gpt-6
(cherry picked from commit b14169ac2a, mudler/LocalAI#12509)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add three llama.cpp quantizations with the matching vision projector,
MTP decoding, and installation examples. Pin downloads to one Hugging
Face revision and verify SHA256 values against LFS metadata and headers.
Assisted-by: Codex:gpt-6
(cherry picked from commit b5f4a2f958, mudler/LocalAI#12498)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Update stable-diffusion.cpp to its merged Iris support commit and add the original F32 Iris weights with the Qwen3-VL Q4_K_M text encoder to the model gallery.
Assisted-by: nib:gpt-5.6-sol
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The Qwen3 template nests context_size, f16, and mmap under parameters,
where the typed model configuration ignores them. Move these options to
the root and correct three model-specific context overrides.
Test the real gallery data and generated qwen3-vl-4b-thinking YAML.
Document manual repair because template updates do not rewrite installed
configurations.
Fixes#12528
Assisted-by: nib:gpt-6-astra
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* feat(diarization): return sound events with include_sounds
A client that wants text, speakers, voice prints and sound events had to
make a diarization call and a separate sound call. Add an include_sounds
request field to /v1/audio/diarization that adds a sounds array of closed
events {start, end, label, confidence}, in seconds.
The parakeet-cpp backend runs a tagger-only scene stream over the clip,
the same stream and thresholds the live path uses, so a clip gives the
same events offline and live. A model with no sound_model companion, or a
backend that does not report sound events, fails with 501 and the stable
code include_sounds_unsupported instead of an empty list. The proto
carries sounds_included so an empty list still means "nothing heard".
The localai-proxy backend forwards the field. Swagger, docs and the
e2e mock backend are updated.
Assisted-by: Claude:claude-sonnet-5-5 [protoc swag go]
* feat(gallery): add parakeet-cpp-multilingual-diarization-speakers-sounds
Same as parakeet-cpp-multilingual-diarization-speakers (TDT 0.6B v3,
Nemotron-3-Diarization, WeSpeaker) plus a CED-Tiny sound_model, so one
model name serves /v1/audio/diarization with include_text,
include_speaker_profiles and include_sounds. It declares the
sound_classification usecase like the realtime scene entries.
Assisted-by: Claude:claude-sonnet-5-5
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The English-only 110M ASR in parakeet-cpp-nemotron-3-diarization-asr-speakers
garbles other languages. Add a bundle that pairs Parakeet TDT 0.6B v3
(25 European languages) with the Nemotron-3-Diarization model and the
WeSpeaker ResNet34 encoder, so one /v1/audio/diarization call with
include_text and include_speaker_profiles returns turns, multilingual text
and one voice-print embedding per speaker.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The small and standard bundles have a voice component, and
parakeet-cpp-realtime-scene-speakers loads a speaker encoder, so they can
serve /v1/voice/* and the realtime voice_recognition stage. Add the usecase
to their known_usecases and pin it in the gallery tests.
Assisted-by: Claude Code:claude-sonnet-5-5
* feat(parakeet-cpp): load bundle GGUF files and use their components by role
A bundle GGUF holds several models (ASR, VAD, diarization, sound events,
speaker encoder) in one file, each with its own licence. Detect a bundle
at load through parakeet_capi_bundle_components_json and open components
with parakeet_capi_load_component. The three symbols are probed together,
so an older libparakeet.so still loads plain files as before.
The only ASR component is the primary model; bundle_asr:<name> picks one
when there are several. A Silero VAD component of the primary bundle is
loaded without an option and serves /v1/vad and vad:true. The diar, ced
and voice components load on request: diar_component, sound_component and
speaker_component, or a companion option (diarization_model, sound_model,
speaker_model, vad_model) that names a bundle, even the model file itself.
vad_component picks a VAD component and implies vad:true.
A role the bundle cannot fill fails the load with the component list, and
a diarization or sound request on a model without that role names the
bundle components. Every existing option and single-file model behaves as
before.
Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
* chore(parakeet-cpp): bump parakeet.cpp to 781a973
Brings in the bundle GGUF format and its C-API (parakeet_capi_load_component,
parakeet_capi_bundle_components_json, parakeet_capi_load_error).
Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
* feat(parakeet-cpp): gallery entries for the bundle GGUF files, docs
Add parakeet-cpp-bundle-small (338 MB: Parakeet TDT+CTC 110M, Nemotron-3-
Diarization, CED-Small, WeSpeaker ResNet34-LM, Silero VAD), -standard
(1.1 GB, Parakeet TDT 0.6B v3 instead of the 110M model) and
-moondream-redux (215 MB: packed Redux and Silero VAD, CPU only). One
install serves transcription, VAD, diarization, sound events and speaker
naming through the component options. The existing single-purpose entries
stay.
A bundle has no single licence, so the entries use license: other and
state the licence and credit of each component in the description, with
the upstream inconsistency of the CED licence. The docs get a section on
bundles in audio-to-text with the entries, the roles, the options and the
licence notice, and pointers from the VAD, diarization and sound
classification pages. A gallery test checks the file names, checksums,
usecases and options.
Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* feat(parakeet-cpp): add gallery entries for the VAD-only Moondream slices
Add parakeet-cpp-vad-moondream-redux and parakeet-cpp-vad-moondream-ultra.
They install the VAD head of Moondream Redux and Ultra (Q8_0) as small
files of 10 MB and 6 MB, cut out of the full models without retraining,
for the VAD endpoint. The files cannot transcribe, and a transcription
request fails with a clear error.
The files load only with a parakeet.cpp build that has VAD-only GGUF
support (parakeet.cpp pull request 87). The backend pin must move to a
commit that includes it before these entries work in a released image.
The parakeet-cpp-vad entry keeps installing Silero.
The docs list the files with the size, load time and memory compared
with loading a whole model. A gallery test checks the usecase, the file
name and the checksum of each entry.
Assisted-by: Claude Code:claude-sonnet-5-5 [golangci-lint]
* chore(parakeet-cpp): bump parakeet.cpp to e53a253
Brings in the VAD-only GGUF loader.
Assisted-by: Claude Code:claude-sonnet-5-5 [git] [gh]
* docs(gallery): link the parakeet.cpp VAD docs instead of the merged PR
Assisted-by: Claude Code:claude-sonnet-5-5 [git]
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* fix(schema): preserve SystemOne image inputs
Assisted-by: OpenAI
* test(schema): follow Ginkgo conventions for decision inputs
Assisted-by: OpenAI
* feat(llama-cpp): dispatch native decisions through Score
Upgrade the stock dependency and reconcile Score/TTS patches. Reuse native decision parsing, tasks, formatting and response-reader cleanup; preserve ordinary scoring admission and guard older dependencies.
Assisted-by: OpenAI
* refactor(systemone): share request and model validation
Assisted-by: OpenAI:gpt-5
* fix(systemone): preserve HTTP wire-byte validation limit
Keep structural validation separate from the serialized internal request bound so HTML escaping cannot reject valid HTTP payloads.
Assisted-by: OpenAI:gpt-5
* feat(systemone): bound images and account native decisions
Preserve public wire limits independently from router serialization. Reject unsupported NER images, map native request/capability errors, and stamp explicit usage once. Advertise decisions for stock llama-cpp.
Assisted-by: OpenAI
* fix(systemone): record usage on registered native route
Exercise real registration and billing with a mock native backend. Reject empty native responses, malformed image URLs, trailing JSON, and wire overflow including whitespace.
Assisted-by: OpenAI
* feat(router): add lazy native decision transport
Bind named models through internal ModelSystemOne calls with shared validation and bounded abandoned operations. Remove request and echoed-error contents from decision traces.
Assisted-by: OpenAI:gpt-5
* feat(router): classify overlapping policies with native decisions
Ask independent noul questions, validate probabilities and preserve first-superset routing. Wire the central factory with config-sensitive invalidation and cancellation-safe resolution. Document native framing and bounded operation limits.
Assisted-by: OpenAI:gpt-5
* feat(gallery): add pinned Julia-1 native decision model
Add a separate text-only llama-cpp Q8 entry with pinned Apache-2.0 source provenance and checksum. Installed using the gallery installer and exercised choice, score and noul on CPU.
Assisted-by: OpenAI
* test(router): verify native decisions through central factory
Add an opt-in real-model Ginkgo integration covering the native Go loader and C++ transport, token usage, independent overlapping labels, and candidate selection. Document owned-server execution and the intentionally non-quality threshold.
Assisted-by: Codex:gpt-5
* fix(llama-cpp): align upstream pin and preserve decision signatures
Advance to bed0a856 without losing the automated upstream bump. Detect full-request fill_task support at compile time and forward every question for Nimble framing while retaining the earlier native signature. Preserve reconciled SCORE/TTS patches; add standalone compatibility coverage.
Assisted-by: Codex:gpt-5
* feat(gallery): add native decision family defaults
Pin Laya, Kev-4B, lev, OpenJev and Nimble artifacts. Verify Laya/Kev/lev gallery installs and CPU contracts on both native pins; clearly mark OpenJev/Nimble runtime validation pending and their noncommercial licenses.
Assisted-by: OpenAI
* docs(decisions): clarify integrated Nimble prerequisite
Record the exact combined backend pin while retaining pending OpenJev and Nimble installation/runtime validation status.
Assisted-by: Codex:gpt-5
* fix(gallery): indent native decision model sequences
Match repository yamllint indentation for Laya, Kev, lev and OpenJev list fields. Parsed gallery data is unchanged; reproduce CI gallery lint failure before the whitespace-only fix and pass the same command afterward.
Assisted-by: Codex:gpt-5
* docs(decisions): record OpenJev and Nimble CPU validation
Record gallery installation, checksum/metadata verification and multiquestion native smoke results on bed0a856. Retain noncommercial and text-only limitations without accuracy or deterministic-output claims.
Assisted-by: OpenAI
* fix(ui): expose native Decisions router classifiers
Select classifier models using metadata-driven capability routing, retain tuned thresholds, and validate native decision selections before saving. Cover both native backends and create/save/reopen in the real React editor.
Assisted-by: Codex:gpt-5
* fix(router): exclude aliases from native decision discovery
Check the originally named config before advertising native Decisions eligibility. Retain target capability inheritance for ordinary generation aliases. Exercise the actual capabilities endpoint with native models on both backends, aliases, and disabled models.
Assisted-by: Codex:gpt-5
* feat(systemone): share bounded multimodal input validation
Preserve text wire limits while admitting bounded PNG/JPEG decision input. Share collection and header validation across internal and public callers and keep the native runner response budget independent.
Assisted-by: OpenAI:API-assistant
* fix(systemone): bound admission lifetimes and validate complete images
Retain shared admission leases through actual work completion, including abandoned internal operations. Decode bounded image pixels, cap public native responses before usage stamping, and preserve oversized malformed text status precedence.
Assisted-by: OpenAI:API-assistant
* fix(router): classify images before media fetching
Preserve ordered structured probes for native decisions. Defer OpenAI
media preparation until routing selects the served model, so rejected
decision URLs cannot trigger downloads before shared validation.
Guard direct image collection with context-aware shared admission.
Keep text classifiers and embedding caches from discarding image input.
Retain fail-closed classifier configuration and runtime fallback policy.
Add middleware, typed-content, admission, cancellation and cache tests.
Assisted-by: OpenAI:API-assistant
* fix(router): bound extraction before serialization
Check probe budgets before copying text or marshaling message state.
Count JSON escaping so oversized internal inputs fail before allocation.
Preserve typed Anthropic blocks through selected-model conversion and
fallback. Keep retry coverage in Ginkgo without global test registration.
Assisted-by: OpenAI
* fix(router): bound supported probe serialization
Arbitrary structs can bypass the probe budget through pointer marshalers,
string tags, and promoted fields. Accept concrete chat schema types and
plain JSON values instead of emulating arbitrary struct serialization.
Budget escaped direct prompts before marshaling so raw length cannot hide
serialized expansion. Preserve runtime fallback and reject oversized
input before invoking the decision runner.
Add Ginkgo allocation, boundary, and marshaler invocation regressions.
Six-package tests, three-package race tests, and full-T2 delta lint pass.
Assisted-by: OpenAI:GPT-5 golangci-lint
* feat(decisions): enable bounded OpenJev images
Validate native decision images before permissive media parsing and pixel
allocation. Require both decision image support and a vision projector;
missing or audio-only projectors cannot silently become text decisions.
Pin the OpenJev Q8 projector and document its license and disk footprint.
Add native safety tests, canonical limit parity, gallery and load-option
checks, and a reproducible CPU direct-RPC contrasting-image smoke.
Assisted-by: OpenAI:GPT-5
* fix(decisions): reject incomplete image streams
stb accepts corrupt PNG Adler checksums and truncated JPEG scans.
Use bounded zlib validation and strict libjpeg decoding before parsing.
Keep dimension and aggregate pixel checks ahead of decoder allocations.
Wire decoder dependencies into native builds and runtime packaging.
Add regressions for appended EOI and embedded marker bypasses.
Assisted-by: OpenAI:GPT-5
* fix(ci): gate native decision image validation
Run the decoder security tests outside the stdlib-only native suite.
Fetch vendor headers at the backend pin and provision decoder dependencies.
Gate Go limit parity and production CMake wiring without model downloads.
Assisted-by: OpenAI:GPT-5
* test(decisions): cover multimodal public API paths
Exercise shared image contracts through the registered HTTP routes and
external mock backend. Add opt-in cached gallery installation and real
OpenJev image decisions through SystemOne and both routing APIs.
Assisted-by: Codex:gpt-5
* test(decisions): assert isolation and cache bypass
Observe external RPC calls and compare complete classifier history.
Winner-only and cache-miss checks could hide dropped history or cache use.
Give real inference its own application and model directory so shared
backend mappings and loaded processes cannot affect mixed suite order.
Assisted-by: OpenAI:ChatGPT
* test(decisions): isolate fixture globals
Disable optional global services in the isolated HTTP fixture and register
cleanup before setup assertions. Verify meter provider identity survives
fixture creation and destruction.
Snapshot observed usage before assertions so failures cannot retain the
mutex. Require a successful usage stamp before checking error responses.
Assisted-by: Codex:gpt-5 golangci-lint
* fix(application): honor optional telemetry controls
Skip failover gauge registration when metrics are disabled. Register
against the application meter rather than looking up the global provider.
Allow embedders to retain the bounded routing log without billing stats.
Keep the existing default when stats are disabled. The isolated HTTP
fixture uses this option without losing its native router assertions.
Assisted-by: Codex:gpt-5 golangci-lint
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* feat(parakeet-cpp): implement the VAD call and add the vad_model option
The backend now serves the VAD gRPC call (POST /vad and /v1/vad) with the
standalone VAD of libparakeet. It accepts a Silero VAD GGUF as the model
file, or an ASR model with a VAD head (Moondream Ultra and Redux). The
request audio is float32 PCM at 16 kHz; the response lists the speech
segments in seconds, like the silero-vad backend. A model with neither
fails the request with the library message.
The vad_threshold, vad_min_pause, vad_min_speech, vad_speech_pad and
vad_max_segment options tune the segmenter. Unset values keep the
defaults of the detector in use, and a bad value fails the load.
The vad_model option names a Silero GGUF, resolved against the models
directory like the other companion files. It lets any ASR model cut long
audio at pauses through parakeet_capi_transcribe_path_json_vad_with, and
it implies vad. vad:true alone still uses the model's own head.
The new symbols are probed like the existing optional ones. A library
without them still loads; the feature that needs one fails with a clear
message only when it is used.
Assisted-by: Claude:claude-sonnet-5-5 [go test]
* feat(gallery): add parakeet-cpp VAD entries and a v3 plus Silero example
Add VAD-only entries for the parakeet-cpp backend: the VAD heads of
Moondream Redux (packed, CPU) and Ultra (Q8_0), which share their files
with the existing ASR entries, and Silero VAD v6.2.3 as a GGUF (MIT,
Silero Team). The parakeet-cpp-vad entry installs Silero; it has no variants,
because variant ranking prefers the larger build that fits and these are
different detectors.
Add parakeet-cpp-tdt-0.6b-v3-silero-vad, a v3 entry that sets vad_model
so long audio is cut at pauses by Silero.
The Silero GGUF entries point at the intended Hugging Face URL of the
file; the existing silero-vad entries are unchanged. A test checks the
usecases, the shared files and the default entry and the vad_model reference.
Assisted-by: Claude:claude-sonnet-5-5 [go test]
* docs: describe parakeet-cpp VAD and the vad_model option
Document the VAD endpoint on the parakeet-cpp backend (Silero GGUF and
the VAD heads of Moondream Ultra and Redux), the vad_* tuning options,
and the vad_model option that lets an ASR model without a VAD head cut
long audio with Silero.
Assisted-by: Claude:claude-sonnet-5-5
* chore(parakeet-cpp): bump parakeet.cpp to 6165e3d
Pin the release that adds the standalone VAD (Ultra/Redux head and
Silero) and the C API calls the backend now uses.
Assisted-by: Claude:claude-sonnet-5-5
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* chore(parakeet-cpp): bump parakeet.cpp to 11c1a0f
Picks up support for the Moondream Ultra and Redux models and the
VAD-segmented transcription entry point (C-API ABI is still 10).
Assisted-by: Claude:claude-sonnet-5-5
* feat(parakeet-cpp): add vad option for long-audio transcription
Models with a VAD head (Moondream Ultra and Redux) can cut long audio at
pauses. Setting vad:true in the model options routes offline
transcription through parakeet_capi_transcribe_path_json_vad. The symbol
is probed at startup like the other optional entry points, and vad:true
fails the load with a clear message when the library lacks it. A model
without a VAD head fails the request with the library's own message.
The option is off by default and does not affect streaming.
Also say in the load error that a packed ternary Redux model is CPU only,
because the library reports its refusal on a GPU backend through its log,
not through the C API.
Assisted-by: Claude:claude-sonnet-5-5
* feat(gallery): add Moondream Ultra and Redux for parakeet-cpp
Add five entries from the public parakeet-cpp GGUF repository: Ultra in
F16 and Q8_0, and Redux as packed ternary (CPU only, offline only) and
as dequantized F16 and Q8_0 (any backend). The entries enable vad:true so
long audio is cut at pauses. Checksums come from the repository's LFS
metadata. The weights are CC-BY-4.0.
Assisted-by: Claude:claude-sonnet-5-5
* docs(parakeet-cpp): document Moondream Ultra, Redux and the vad option
Assisted-by: Claude:claude-sonnet-5-5
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* feat(gallery): add nimble-9b-vllm-cpp decision model
Add Bespoke Nimble 9B, converted for vllm.cpp and pinned to the weights
commit 52eead25 of mudler/Bespoke-Nimble-9B-vllm-cpp (HEAD only adds the
model card). It is a redistribution of bespokelabs/Bespoke-Nimble-9B
with the LoRA merged into Qwen3.5-9B; config.json names NimbleModel, so
no hf_overrides are needed.
The artifact sits under overrides, where the installer reads it. The
entry sets an 8192-token context, Nimble's own prompt limit, and a KV
pool of 1024 blocks of 32 tokens for 4 sequences (about 1 GiB at 32 KiB
per token for the 8 full-attention layers).
Installed with local-ai models install and served on CPU through the
vllm-cpp backend: the model card's billing request gives billing
(0.986), refund 0.998 and urgency 0.33. Peak resident memory was
18.4 GB, so the description asks for about 20 GB of free RAM.
List the entry in the decisions gallery table. CLM stays out of the
gallery: the pinned engine cannot load the published head layout.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-sonnet-5-5
* feat(gallery): add clm-v0.1-8b-vllm-cpp and bump vllm.cpp for CLM
The CLM checkpoint on mudler/CLM-v0.1-8B-vllm-cpp stores the heads with
the reference's own tensor names (state_head.inp, hidden.N, norms.N,
out). The pinned vllm.cpp 96788348 still expects the old .0/.2/.4/.6
layout and refuses the load with "head.safetensors incomplete for
state_head". vllm.cpp a19294a9 matches the reference layout and adds the
converter that produced the upload, so move the pin there. The ABI stays
at v30.
Add the CLM entry, pinned to the weights commit 0d1903b1 (HEAD only adds
the model card), with a 4096-token context and a KV pool for 4 sequences
(about 2.25 GiB at 144 KiB per token for Qwen3-8B).
Installed with local-ai models install and served on CPU against a
libvllm built at a19294a9: the model card example (john works at google,
entity type) gives person 0.950, the same as the card. Peak resident
memory was 17.9 GB.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-sonnet-5-5
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
