LocalAI · Quantization

From HF model to quantized GGUF

GGUF

q4..q8

Convert first, then quantize - tracked as a job from queued to completed.

localai.io/features/quantization