mudler-agentandEttore Di Giacinto
38a2aa45fe
* feat(gallery): add nimble-9b-vllm-cpp decision model Add Bespoke Nimble 9B, converted for vllm.cpp and pinned to the weights commit 52eead25 of mudler/Bespoke-Nimble-9B-vllm-cpp (HEAD only adds the model card). It is a redistribution of bespokelabs/Bespoke-Nimble-9B with the LoRA merged into Qwen3.5-9B; config.json names NimbleModel, so no hf_overrides are needed. The artifact sits under overrides, where the installer reads it. The entry sets an 8192-token context, Nimble's own prompt limit, and a KV pool of 1024 blocks of 32 tokens for 4 sequences (about 1 GiB at 32 KiB per token for the 8 full-attention layers). Installed with local-ai models install and served on CPU through the vllm-cpp backend: the model card's billing request gives billing (0.986), refund 0.998 and urgency 0.33. Peak resident memory was 18.4 GB, so the description asks for about 20 GB of free RAM. List the entry in the decisions gallery table. CLM stays out of the gallery: the pinned engine cannot load the published head layout. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-sonnet-5-5 * feat(gallery): add clm-v0.1-8b-vllm-cpp and bump vllm.cpp for CLM The CLM checkpoint on mudler/CLM-v0.1-8B-vllm-cpp stores the heads with the reference's own tensor names (state_head.inp, hidden.N, norms.N, out). The pinned vllm.cpp 96788348 still expects the old .0/.2/.4/.6 layout and refuses the load with "head.safetensors incomplete for state_head". vllm.cpp a19294a9 matches the reference layout and adds the converter that produced the upload, so move the pin there. The ABI stays at v30. Add the CLM entry, pinned to the weights commit 0d1903b1 (HEAD only adds the model card), with a 4096-token context and a KV pool for 4 sequences (about 2.25 GiB at 144 KiB per token for Qwen3-8B). Installed with local-ai models install and served on CPU against a libvllm built at a19294a9: the model card example (john works at google, entity type) gives person 0.950, the same as the card. Peak resident memory was 17.9 GB. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-sonnet-5-5 --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
LocalAI website
LocalAI documentation website
Requirement
In this project, the Docsy theme component is pulled in as a Hugo module, together with other module dependencies:
$ hugo mod graph
hugo: collected modules in 566 ms
hugo: collected modules in 578 ms
github.com/google/docsy-example github.com/google/docsy@v0.5.1-0.20221017155306-99eacb09ffb0
github.com/google/docsy-example github.com/google/docsy/dependencies@v0.5.1-0.20221014161617-be5da07ecff1
github.com/google/docsy/dependencies@v0.5.1-0.20221014161617-be5da07ecff1 github.com/twbs/bootstrap@v4.6.2+incompatible
github.com/google/docsy/dependencies@v0.5.1-0.20221014161617-be5da07ecff1 github.com/FortAwesome/Font-Awesome@v0.0.0-20220831210243-d3a7818c253f
If you want to do SCSS edits and want to publish these, you need to install PostCSS
npm install
Running the website locally
Building and running the site locally requires a recent extended version of Hugo.
You can find out more about how to install Hugo for your environment in our
Getting started guide.
From the LocalAI repository root, run:
make docs
The Hugo configuration lives in the docs directory. To invoke Hugo
directly instead, run:
cd docs
hugo server
Running a container locally
You can run docsy-example inside a Docker
container, the container runs with a volume bound to the docsy-example
folder. This approach doesn't require you to install any dependencies other
than Docker Desktop on
Windows and Mac, and Docker Compose
on Linux.
-
Build the docker image
docker-compose build -
Run the built image
docker-compose upNOTE: You can run both commands at once with
docker-compose up --build. -
Verify that the service is working.
Open your web browser and type
http://localhost:1313in your navigation bar, This opens a local instance of the docsy-example homepage. You can now make changes to the docsy example and those changes will immediately show up in your browser after you save.
Cleanup
To stop Docker Compose, on your terminal window, press Ctrl + C.
To remove the produced images run:
docker-compose rm
For more information see the Docker Compose documentation.
Troubleshooting
As you run the website locally, you may run into the following error:
➜ hugo server
INFO 2021/01/21 21:07:55 Using config file:
Building sites … INFO 2021/01/21 21:07:55 syncing static files to /
Built in 288 ms
Error: Error building site: TOCSS: failed to transform "scss/main.scss" (text/x-scss): resource "scss/scss/main.scss_9fadf33d895a46083cdd64396b57ef68" not found in file cache
This error occurs if you have not installed the extended version of Hugo. See this section of the user guide for instructions on how to install Hugo.
Or you may encounter the following error:
➜ hugo server
Error: failed to download modules: binary with name "go" not found
This error occurs if you have not installed the go programming language on your system.
See this section of the user guide for instructions on how to install go.