Compare commits

..

1 Commits

Author SHA1 Message Date
Ettore Di Giacinto
ea456e7027 fix(ci): treat .docker/ as a shared build input
#11346 changed how ROCm llama.cpp compiles and built nothing. It touched
.docker/llama-cpp-build-target.sh and scripts/build/llama-cpp-build-target_test.sh,
no SHARED_BUILD_INPUTS rule matched either (the second is carved out on
purpose), so the filter selected zero entries and all six backend job
groups reported "skipping". The fix shipped to master unvalidated.

.docker/ holds the per-backend compile and build-target scripts for
llama-cpp, turboquant, bonsai and ik-llama-cpp, plus inputs every
Dockerfile consumes (apt-mirror.sh, install-base-deps.sh). Those files
decide how a backend is compiled, so a change to one can only be
validated by rebuilding it.

Take the same posture the scripts/build/ catch-all already takes, and for
the reason its comment already gives: a shared build input silently
shipping to nothing is the failure this list exists to prevent. A rule
sees only the matrix entry, not which file matched it, so narrowing
.docker/<name>-compile.sh to the backend named by its prefix would mean
threading the filename through matchedSharedRules. Not worth it for files
edited a handful of times a release.

Two regression tests, both verified red against the unfixed filter:
one asserting a .docker/ compile script selects the backend it compiles,
one asserting a shared .docker/ input takes the full matrix.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
2026-08-04 21:44:08 +00:00
12 changed files with 70 additions and 151 deletions

View File

@@ -9,7 +9,7 @@
# recipe is a make target (not a prepare.sh) so 'make purge && make' is a clean
# rebuild and so the bump bot can see the pin.
AUDIO_CPP_VERSION?=238ab6a9e321c17de8e120559f57efeedaeb1345
AUDIO_CPP_VERSION?=4e3aea2fd99aeaa5924e71c51eb2793846045332
AUDIO_CPP_REPO?=https://github.com/0xShug0/audio.cpp
CURRENT_MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))

View File

@@ -1,10 +1,10 @@
# ds4 backend Makefile.
#
# Upstream pin lives below as DS4_VERSION?=6747e7718dd08f00b680d0c16231f2d59ec3747e
# Upstream pin lives below as DS4_VERSION?=b7e9f0091139999b6c070a57590c447c5741da5c
# (.github/bump_deps.sh) can find and update it - matches the
# llama-cpp / ik-llama-cpp / turboquant convention.
DS4_VERSION?=6747e7718dd08f00b680d0c16231f2d59ec3747e
DS4_VERSION?=b7e9f0091139999b6c070a57590c447c5741da5c
DS4_REPO?=https://github.com/antirez/ds4
CURRENT_MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))

View File

@@ -1,5 +1,5 @@
IK_LLAMA_VERSION?=6b55d2c7504f482e7c8ec6cbf22a19f3778c522b
IK_LLAMA_VERSION?=60389410a1ff01f9d37dcc6261db33b3183bdea2
LLAMA_REPO?=https://github.com/ikawrakow/ik_llama.cpp
CMAKE_ARGS?=

View File

@@ -8,7 +8,7 @@ JOBS?=$(shell nproc --ignore=1)
# CrispASR version (release tag)
CRISPASR_REPO?=https://github.com/CrispStrobe/CrispASR
CRISPASR_VERSION?=ec730908a418b6032f9e69ded6186d3f042a7747
CRISPASR_VERSION?=fe3caf8e363b27572dbdd1a9d37083f25e6decda
SO_TARGET?=libgocrispasr.so
CMAKE_ARGS+=-DBUILD_SHARED_LIBS=OFF

View File

@@ -8,7 +8,7 @@ JOBS?=$(shell nproc --ignore=1)
# stablediffusion.cpp (ggml)
STABLEDIFFUSION_GGML_REPO?=https://github.com/leejet/stable-diffusion.cpp
STABLEDIFFUSION_GGML_VERSION?=ea7f0c87cfe4c673263b4c201c596c7f1cbe2528
STABLEDIFFUSION_GGML_VERSION?=db99efdd6d2a43c7937fd55b3359206c680a75b0
CMAKE_ARGS+=-DGGML_MAX_NAME=128

View File

@@ -11,7 +11,7 @@ JOBS?=$(shell nproc --ignore=1 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || e
# vllm.cpp version
VLLM_CPP_REPO?=https://github.com/mudler/vllm.cpp
VLLM_CPP_VERSION?=0757cac231ecd571a83c4fd2f50805c9251fc225
VLLM_CPP_VERSION?=9d1fad3cde0acb95eb0bb0a1025f40a0eb614147
# MLX GEMM provider (darwin/metal only; see the metal branch below for why).
# Consumed as the prebuilt pip wheel: building MLX from source needs `xcrun

View File

@@ -8,7 +8,7 @@ JOBS?=$(shell nproc --ignore=1)
# whisper.cpp version
WHISPER_REPO?=https://github.com/ggml-org/whisper.cpp
WHISPER_CPP_VERSION?=306c88f4d1286aec1bf96e544632897886af5501
WHISPER_CPP_VERSION?=64d57d3df5c8dacee098577257edcaa154bf5ef3
SO_TARGET?=libgowhisper.so
CMAKE_ARGS+=-DBUILD_SHARED_LIBS=OFF

View File

@@ -193,22 +193,12 @@
alias: "vllm-cpp"
license: apache-2.0
description: |
ALPHA development builds. Try it, but llama-cpp stays the recommendation for
production use.
vllm.cpp is an Apache-2.0 C++20 inference engine maintained by the LocalAI team,
developed in its own repository and usable without LocalAI. It began as a port of
vLLM and keeps vLLM as its reference implementation, checking output against it and
benchmarking against it, while growing a featureset of its own. It implements vLLM's
V1 architecture (paged KV cache, continuous batching, prefix caching, scheduler,
sampler) on a portable tensor runtime with no Python, PyTorch or ggml at inference
time. It loads GGUF as well as Hugging Face safetensors, supports structured output
(JSON schema / regex / choice / GBNF grammar) enforced in-engine, ships speculative
decoding and KV offload, and runs on CPU, NVIDIA CUDA (Blackwell-family), Apple
Metal and Vulkan.
The project is expected to be renamed as it diverges further from vLLM; the new
name is still to be decided.
vllm.cpp is a from-scratch C++20 port of vLLM created and maintained by the LocalAI team.
It mirrors vLLM's V1 architecture (paged KV cache, continuous batching, prefix caching,
scheduler, sampler) on a portable tensor runtime with no Python, PyTorch or ggml at
inference time. It loads Hugging Face safetensors and GGUF checkpoints, supports
structured output (JSON schema / regex / choice / GBNF grammar) enforced in-engine,
and runs on CPU, NVIDIA CUDA (Blackwell-family), Apple Metal and Vulkan.
urls:
- https://github.com/mudler/vllm.cpp
tags:

View File

@@ -1992,7 +1992,7 @@
files:
- filename: ds4flash.gguf
uri: https://huggingface.co/unsloth/DeepSeek-V4-Flash-GGUF
sha256: ba1d64ad8d77038124839956b614db2e889daa1a4ddc83060bb06ccb5a1d7461
sha256: 1bfdafd1c288eb1b2bcb629ee9e1b7567dcf0abbe4d20995905a3c3465e9bd1e
- name: "qwopus3.6-35b-a3b-coder-mtp"
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
urls:
@@ -2614,83 +2614,6 @@
- filename: llama-cpp/models/LFM2.5-1.2B-Instruct-GGUF/LFM2.5-1.2B-Instruct-Q4_K_M.gguf
sha256: b1b3de114215d9507409a662a501a631095a479a419584e8a2ded6304b19b4f5
uri: https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF/resolve/main/LFM2.5-1.2B-Instruct-Q4_K_M.gguf
- &lfm2-5-2-6b
name: "lfm2.5-2.6b"
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
urls:
- https://huggingface.co/LiquidAI/LFM2.5-2.6B
- https://huggingface.co/LiquidAI/LFM2.5-2.6B-GGUF
description: |
LFM2.5-2.6B is LiquidAI's compact, text-only reasoning model for on-device
agentic workloads. It has 2.69B parameters, a 128K-token context window,
multilingual support, and post-training for tool use, instruction following,
data extraction, RAG, and multi-step agents. This entry uses the recommended
Q4_K_M GGUF quantization from LiquidAI's official repository.
license: "other"
tags:
- llm
- gguf
- reasoning
- cpu
- gpu
icon: https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/2b08LKpev0DNEk6DlnWkY.png
variants:
- model: lfm2.5-2.6b-q8
overrides:
backend: llama-cpp
context_size: 131072
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- completion
options:
- use_jinja:true
parameters:
model: llama-cpp/models/LFM2.5-2.6B-GGUF/LFM2.5-2.6B-Q4_K_M.gguf
repeat_penalty: 1.1
temperature: 0.1
top_k: 50
template:
use_tokenizer_template: true
files:
- filename: llama-cpp/models/LFM2.5-2.6B-GGUF/LFM2.5-2.6B-Q4_K_M.gguf
sha256: 79fdf00351b46cf26f020aead28d01889886be87c55fa0eb907e6f9b00bfee14
uri: https://huggingface.co/LiquidAI/LFM2.5-2.6B-GGUF/resolve/main/LFM2.5-2.6B-Q4_K_M.gguf
- !!merge <<: *lfm2-5-2-6b
name: "lfm2.5-2.6b-q8"
description: |
LFM2.5-2.6B is LiquidAI's compact, text-only reasoning model for on-device
agentic workloads. It has 2.69B parameters, a 128K-token context window,
multilingual support, and post-training for tool use, instruction following,
data extraction, RAG, and multi-step agents. This entry uses the higher-quality
Q8_0 GGUF quantization from LiquidAI's official repository.
variants: null
overrides:
backend: llama-cpp
context_size: 131072
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- completion
options:
- use_jinja:true
parameters:
model: llama-cpp/models/LFM2.5-2.6B-GGUF/LFM2.5-2.6B-Q8_0.gguf
repeat_penalty: 1.1
temperature: 0.1
top_k: 50
template:
use_tokenizer_template: true
files:
- filename: llama-cpp/models/LFM2.5-2.6B-GGUF/LFM2.5-2.6B-Q8_0.gguf
sha256: 36587fdf27bdfc69caf2637273679a0870ec155162161bde6fd16e8c70bdb757
uri: https://huggingface.co/LiquidAI/LFM2.5-2.6B-GGUF/resolve/main/LFM2.5-2.6B-Q8_0.gguf
- name: "qwopus3.6-27b-coder-compat-mtp"
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
urls:

View File

@@ -436,6 +436,27 @@ export const SHARED_BUILD_INPUTS = [
linux: always,
darwin: always,
},
{
// Same posture as the scripts/build/ catch-all above, and for the same
// reason. .docker/ holds the per-backend compile and build-target scripts
// (llama-cpp, turboquant, bonsai, ik-llama-cpp) plus inputs every
// Dockerfile consumes (apt-mirror.sh, install-base-deps.sh). Nothing
// matched any of them before, which is how #11346 shipped without a single
// backend job: it changed how ROCm llama.cpp compiles, touching only
// .docker/llama-cpp-build-target.sh and a `*_test.sh` that the rule above
// deliberately carves out, so the filter selected zero entries and every
// backend job reported "skipping".
//
// A rule cannot see which file matched it, only the matrix entry, so
// narrowing `.docker/<name>-compile.sh` to the backend named by its prefix
// would mean threading the filename through matchedSharedRules. Until
// someone wants that, take the full matrix: these files are edited a
// handful of times a release, and a shared build input silently shipping
// to nothing is the failure this list exists to prevent.
matches: file => file.startsWith(".docker/"),
linux: always,
darwin: always,
},
];
// The matrix stores dockerfiles as "./backend/Dockerfile.python"; changed-file

View File

@@ -204,6 +204,29 @@ test("an unclassified scripts/build/ file conservatively rebuilds everything", (
assert.equal(filteredDarwin.length, includesDarwin.length);
});
// #11346 changed how ROCm llama.cpp compiles and built nothing: it touched only
// .docker/llama-cpp-build-target.sh and a *_test.sh, no rule matched either, so
// every backend job reported "skipping".
test("a .docker/ compile script rebuilds the backends it compiles", () => {
const { filtered } = run([
".docker/llama-cpp-build-target.sh",
"scripts/build/llama-cpp-build-target_test.sh",
]);
assert.ok(filtered.length > 0, ".docker/ change selected no entries");
assert.ok(
filtered.some(e => e.backend === "llama-cpp"),
"llama-cpp was not selected by a change to its own compile script",
);
});
test("a shared .docker/ input rebuilds everything", () => {
const { filtered, filteredDarwin } = run([".docker/apt-mirror.sh"]);
assert.equal(filtered.length, includes.length);
assert.equal(filteredDarwin.length, includesDarwin.length);
});
test("tests for the packaging scripts do not rebuild anything", () => {
const { filtered, filteredDarwin } = run([
"scripts/build/package-gpu-libs_test.sh",

View File

@@ -1,14 +1,14 @@
---
title: "What landed in LocalAI 4.8"
date: 2026-08-04
date: 2026-08-01
author: "Ettore Di Giacinto"
category: "Release"
tags: ["release", "vllm.cpp", "audio.cpp", "3d", "gallery", "distributed", "performance"]
summary: "A new inference engine, 3D generation, one backend that serves six audio endpoints, and a web interface 3.48x lighter. 374 pull requests in twenty-one days."
summary: "A new inference engine, 3D generation, one backend that serves six audio endpoints, and a web interface 3.48x lighter. 321 pull requests in eighteen days."
extracss: ["blog.css"]
---
LocalAI 4.8.0 is out, after twenty-one days and 374 merged pull requests. There are three new things LocalAI can do, and a lot of repair work on things it already did.
LocalAI 4.8.0 is out. It took eighteen days and 321 merged pull requests, and it pulls in two directions at once: three new things LocalAI can do that it could not do before, and a long list of places where it now does the old things without lying to you.
The full notes list everything. This post covers the parts that change what you do day to day, with the pull request numbers so you can read the diffs.
@@ -55,35 +55,12 @@ Every surface can override the choice: `variant` on `POST /models/apply`, `local
One gap worth knowing about: in distributed mode `InstallModel` resolves against the frontend rather than the worker that will serve the model, so a cluster with a small frontend and large workers selects conservatively. PRs [#10943](https://github.com/mudler/LocalAI/pull/10943), [#10983](https://github.com/mudler/LocalAI/pull/10983), [#10992](https://github.com/mudler/LocalAI/pull/10992), [#11027](https://github.com/mudler/LocalAI/pull/11027) and [#11139](https://github.com/mudler/LocalAI/pull/11139).
## A new engine: vllm.cpp (alpha)
## A new engine: vllm.cpp
[vllm.cpp](https://github.com/mudler/vllm.cpp) is Apache-2.0 and maintained by the LocalAI team. We want it community-first rather than a LocalAI-only engine, so it lives in its own repository with its own docs, benchmark record and issue tracker, and it runs without LocalAI anywhere in the picture. It began as a C++20 port of vLLM. It ships here as the `vllm-cpp` backend ([#11100](https://github.com/mudler/LocalAI/pull/11100)). It implements vLLM's V1 architecture, so paged KV cache, continuous batching, prefix caching, scheduler and sampler, on a portable tensor runtime with no Python, no PyTorch and no ggml at inference. vLLM stays its reference implementation: correctness is checked by comparing output against it, and the benchmark scoreboard is kept against it.
It has grown features vLLM does not have, which is most of the reason the port exists. It loads GGUF as well as safetensors, runs on CPU, Apple Metal and Vulkan alongside CUDA 12 and 13 and L4T for GB10, and ships speculative decoding and KV offload. Its benchmark page now measures against llama.cpp, MLX-LM and DwarfStar as well as vLLM, because on that hardware those are the engines it competes with. The project is expected to be renamed, with the new name still to be decided; it is drifting far enough that vllm.cpp will eventually mislead.
[vllm.cpp](https://github.com/mudler/vllm.cpp) is a from-scratch C++20 port of vLLM, written and maintained by the LocalAI team under Apache-2.0, and it ships here as the `vllm-cpp` backend ([#11100](https://github.com/mudler/LocalAI/pull/11100)). It mirrors vLLM's V1 architecture, so paged KV cache, continuous batching, prefix caching, scheduler and sampler, on a portable tensor runtime with no Python, no PyTorch and no ggml at inference. It loads Hugging Face safetensors and GGUF, enforces structured output inside the engine (JSON schema, regex, choice, GBNF), and builds for CPU amd64 and arm64, CUDA 12 and 13 including Blackwell, L4T for GB10, Vulkan and Darwin Metal.
Tool calling is at llama.cpp parity by construction, because chat deliberately reuses the same autoparser path: full minja chat templates, `tool_choice: auto` lowered to a lazy structural-tag decode constraint, 30 tool dialects, 7 reasoning parsers, and streamed `ChatDelta` and `ToolCallDelta`.
Numbers from the project's own [scoreboard](https://github.com/mudler/vllm.cpp/blob/master/docs/BENCHMARKS.md), which calls ties ties and losses losses. Above 1.0 means vllm.cpp is ahead:
<div class="tw">
<table>
<thead><tr><th>Reference</th><th>Workload</th><th>Result</th></tr></thead>
<tbody>
<tr><td>vLLM</td><td>Qwen3.6-27B NVFP4, GB10</td><td>1.045x at concurrency 1, 1.007x to 1.017x from c2 to c32, output token-for-token identical</td></tr>
<tr><td>vLLM</td><td>Qwen3.6-35B-A3B NVFP4, GB10</td><td>1.010x at c16 and 1.013x at c32, behind from c1 to c8 (0.817x at c1)</td></tr>
<tr><td>llama.cpp</td><td>Qwen3.5-2B GGUF, CPU aarch64</td><td>prefill 1.18x, decode a tie, memory parity</td></tr>
<tr><td>MLX-LM</td><td>Qwen3-0.6B, Apple M4</td><td>97.6% of warm total, prefill ahead</td></tr>
<tr><td>DwarfStar (ds4)</td><td>DeepSeek-V4-Flash IQ2_XXS, one DGX Spark</td><td>16.28 vs 16.33 tok/s decode, 0.997x, a parity result</td></tr>
</tbody>
</table>
</div>
The upstream page is careful about its own noise: on the 27B grid the run-to-run spread is 0.5% and c2 through c32 land between 0.7% and 1.7%, so it calls those five ties rather than wins. The concurrency-1 result is the one it stands behind.
The DeepSeek-V4-Flash row is the one that shows how far this has moved from being a vLLM port. It runs DeepSeek-V4-Flash at roughly 2-bit (IQ2_XXS mixed, about 80 GB) on a single DGX Spark, decoding at 16.28 tok/s against DwarfStar's 16.33. At 300B+ total parameters even a 4-bit checkpoint is 156 GB or more, so a 2-bit GGUF is what fits inside the Spark's 119 GiB unified pool, and reading GGUF is what makes that possible.
Speculative decoding is in similar shape: MTP on Qwen3.6-27B NVFP4 is token-identical to vLLM's MTP and about 4% faster at concurrency 1.
Configuration is a normal backend install:
```yaml
@@ -96,24 +73,9 @@ options:
- max_num_seqs:16 # also: block_size:<n>, num_blocks:<n>
```
**Treat these as alpha development builds, not a released backend.** vllm.cpp is early, and shipping it in 4.8 is about getting it in front of people who want to try it, not about recommending it for anything you care about. `llama-cpp` stays the default for real use.
The CPU path is verified end to end against `Qwen3.5-2B-UD-Q8_K_XL.gguf` with the full Ginkgo suite, covering blocking and streaming byte-parity, greedy determinism, stop words, GBNF-constrained generation, concurrent streams, reasoning split and both `required` and `auto` tool calls. The maturity statement from the release notes is worth repeating in full:
The CPU path is verified end to end against `Qwen3.5-2B-UD-Q8_K_XL.gguf` with the full Ginkgo suite, covering blocking and streaming byte-parity, greedy determinism, stop words, GBNF-constrained generation, concurrent streams, reasoning split and both `required` and `auto` tool calls. The GPU images build and ship, but their runtime behavior has not been through that gate. No throughput comparison against upstream vLLM is claimed. Expect rough edges, and please report what breaks.
On Apple Silicon the image now ships vllm.cpp's MLX GEMM provider ([#11137](https://github.com/mudler/LocalAI/pull/11137)). Upstream keeps it off by default because it adds about 124 MB, so we measured before turning it on. Qwen3-1.7B-bf16 on an M4, p=512 g=128, both arms toggled on one binary so a build difference cannot explain the gap:
<div class="tw">
<table>
<thead><tr><th>Batch</th><th>MLX tok/s</th><th>native tok/s</th><th>speedup</th><th>MLX TTFT</th><th>native TTFT</th></tr></thead>
<tbody>
<tr><td>1</td><td>5.79</td><td>3.08</td><td><b>1.88x</b></td><td>3.32 s</td><td>7.68 s</td></tr>
<tr><td>4</td><td>15.75</td><td>10.24</td><td><b>1.54x</b></td><td>9.63 s</td><td>18.77 s</td></tr>
<tr><td>16</td><td>38.65</td><td>17.69</td><td><b>2.19x</b></td><td>18.33 s</td><td>54.48 s</td></tr>
</tbody>
</table>
</div>
Two reps, with rep spread reaching 9.4%, so treat the multipliers as +/-10%. Time to first token roughly halves across the range.
> The GPU images build and ship, but their runtime behavior has not been through the same e2e gate yet. This is a first release of a young engine: no throughput comparison against upstream vLLM is claimed here, and `llama-cpp` remains the default recommendation for general use. Try it, and please report what breaks.
<figure>
<video src="/media/vllm-race.mp4" muted loop playsinline preload="none" data-lazy aria-label="vllm.cpp generating tokens"></video>
@@ -122,7 +84,7 @@ Two reps, with rep spread reaching 9.4%, so treat the multipliers as +/-10%. Tim
## LocalAI generates 3D models now
3D generation is a new modality, so it had to be wired through the whole stack: a `Generate3D` RPC in `backend.proto`, a `FLAG_3D` capability so the loader knows which backends can serve it, and `POST /v1/3d/generations`.
This is a new modality rather than a new backend under an existing one, so it goes through the whole stack: a `Generate3D` RPC in `backend.proto`, a `FLAG_3D` capability so the loader knows which backends can serve it, and `POST /v1/3d/generations`.
The first engine behind it is `trellis2cpp`, an image-to-3D backend over TRELLIS.2. You give it an image, you get a GLB back. The web UI has a page for it with a native GLB viewer, so you can turn the result around in the browser instead of downloading it to find out whether it worked, history kept in IndexedDB so a reload does not lose your generations, and previewable print remeshing for output you actually intend to send to a printer ([#10979](https://github.com/mudler/LocalAI/pull/10979)).
@@ -133,7 +95,7 @@ The first engine behind it is `trellis2cpp`, an image-to-3D backend over TRELLIS
## One backend, six audio endpoints
The usual shape for audio is one backend per model family, which means a process per capability and a config file for each. `audio-cpp` wraps [audio.cpp](https://github.com/0xShug0/audio.cpp), a multi-family ggml audio engine. One backend process serves several unrelated families through a single runtime vocabulary, and works out which family a checkpoint belongs to from the GGUF's own `audiocpp.model_spec.family` metadata key. There is nothing backend-specific to write in the model config.
The usual shape for audio is one backend per model family, which means a process per capability and a config file for each. `audio-cpp` wraps [audio.cpp](https://github.com/0xShug0/audio.cpp), a multi-family ggml audio engine, and inverts that: one backend process serves several unrelated families through a single runtime vocabulary, and works out which family a checkpoint belongs to from the GGUF's own `audiocpp.model_spec.family` metadata key. There is nothing backend-specific to write in the model config.
<div class="tw">
<table>
@@ -168,7 +130,7 @@ The `bonsai` backend serves the 1-bit (Q1_0) and ternary (Q2_0) Bonsai quantizat
## The operations bar became a page
The old operations bar rendered one row per in-flight operation above every page. Queue four model installs and a backend and it took most of the viewport, on every route, until the last one finished. It was doing two jobs at once. A global "something is happening" signal only needs one line, and the detail of what is happening needs a page of its own.
The old operations bar rendered one row per in-flight operation above every page. Queue four model installs and a backend and it took most of the viewport, on every route, until the last one finished. Two things were conflated there: a global "something is happening" signal, which needs one line, and the detail of what is happening, which needs somewhere to put it.
The strip is now one line, permanently, showing a failure first and otherwise the least-advanced running operation, with a `+N more` pill. Its `✕` hides the strip and no longer cancels anything. That is a deliberate behavior change worth knowing about before you click it out of habit: the same glyph used to cancel a 17 GB download in one row and dismiss a message in the next. Cancelling moved to the new page, behind a button that says so.
@@ -217,6 +179,6 @@ Valkey Search joins the vector store options as the `valkey-store` backend ([#11
This is also the release where localai.io split in two: the project site at the root, and the documentation under `/docs/`. Every URL that was published before still resolves, through 214 generated redirect stubs, because GitHub Pages has no server-side rewrites to do it properly ([#11243](https://github.com/mudler/LocalAI/pull/11243)).
Twenty-five people contributed to this release, eleven of them for the first time. The gallery went from 1,221 entries to 1,515.
Twenty-four people contributed to this release, eleven of them for the first time. The gallery went from 1,221 entries to 1,505.
To upgrade, pull `localai/localai:latest` or re-run the install script. The [full changelog](https://github.com/mudler/LocalAI/compare/v4.7.1...v4.8.0) has everything this post left out.