mirror of
https://github.com/mudler/LocalAI.git
synced 2026-06-27 09:57:14 -04:00
NVFP4 (GGML_TYPE_NVFP4=40 / MOSTLY_NVFP4) GGUFs are portable: full CPU/CUDA-DP4A/ generic-MMA/Vulkan dequant coverage. FP4-MMA is a runtime Blackwell-only speed tier (mmq.cu use_native_fp4 flag), not a load/run gate. Off-Blackwell = runs via dequant, correct-but-slow, never fail/garbage. Gallery has no microarch-gating primitive (tags are search-only, capabilities map is family-level nvidia/amd/ metal, model struct has no hardware field), so the 6 -paged entries can only express Blackwell-targeting via description prose + tags. Signed-off-by: Ettore Di Giacinto <mudler@localai.io>