Commit Graph
5 Commits
Author SHA1 Message Date
localai-org-maint-bot 4ad5e4d1c5 fix(compose): request NVIDIA compute capability
The legacy NVIDIA device reservation requests utility without compute.
Docker derives driver capabilities from that list, leaving CUDA libraries
unavailable even when monitoring works.

Include compute in the legacy example and clarify the matching docs.

Assisted-by: Codex:GPT-6
2026-09-13 03:02:48 +00:00
mudler's LocalAI [bot]andEttore Di Giacinto 40d35c0385 docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)
* docs: fix CPU image tag (latest, not latest-cpu)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: use canonical localai/localai registry in models guide

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: replace dead llama-stable backend with llama-cpp

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: correct mitm-proxy intercept config and redaction tier

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: fix text-to-audio endpoint and broken notice block

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: fix VAD example, stale FAQ, broken link, CLI list, whats-new dump

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: render advanced/reference section indexes (consolidate _index)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: remove duplicate getting-started build/kubernetes pages

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: fold container image reference into installation/containers

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: remove stale advanced fine-tuning page (superseded by features/fine-tuning)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: fold distribution/longcat/sound pages into their parents

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: make getting-started index accurate and complete

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: carry one concrete model through the getting-started path

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: add end-to-end 'build your first agent' walkthrough

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: add runtime errors reference; consolidate troubleshooting from FAQ

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: add agent actions catalog

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: agent-scoped MCP, skills walkthrough, agentic disambiguation

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: add concrete gallery install lines to media feature pages

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: merge installation into getting-started (URLs preserved via aliases)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: add Operations section; move operator pages and P2P API reference

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: journey-ordered top nav and grouped feature sections

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: add docs-with-code process gate (PR template + agent instructions)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: remove em/en dashes from documentation prose

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-07-17 22:08:20 +02:00
Ettore Di Giacinto 551ebdb57a fix(distributed): correct VRAM/RAM reporting on NVIDIA unified-memory hosts (#9545)
Workers on NVIDIA unified-memory hardware (DGX Spark / GB10, Jetson AGX Thor,
Jetson Orin/Xavier/Nano) were reporting `available_vram=0` back to the frontend,
so the Nodes UI showed the node as fully used even when most of the unified
memory was actually free.

Three causes addressed:

* `isTegraDevice` only matched `/sys/devices/soc0/family == "Tegra"`. DGX Spark
  (SBSA) reports JEDEC codes there instead — `jep106:0426` for the NVIDIA
  manufacturer — so the Tegra/unified-memory fallback never ran. Renamed to
  `isNVIDIAIntegratedGPU` and extended to also match `jep106:0426[:*]` via
  `/sys/devices/soc0/soc_id`.

* The unified-iGPU code defaulted the device name to `"NVIDIA Jetson"` when
  `/proc/device-tree/model` was missing. That's what happens for Thor inside a
  docker container, and always on DGX Spark. New `nvidiaIntegratedGPUName`
  resolves via dt-model → `/sys/devices/soc0/machine` → `soc_id` lookup
  (`jep106:0426:8901` → `"NVIDIA GB10"`) so the Nodes UI labels the box
  correctly.

* Worker heartbeat sent `available_vram=0` (or total-as-available) when VRAM
  usage was momentarily unknown — e.g. when `nvidia-smi` intermittently failed
  with `waitid: no child processes` under containers without `--init`. Each
  such heartbeat overwrote the DB and made the UI flip to "fully used".
  `heartbeatBody` now omits `available_vram` in that case so the DB keeps its
  last good value.

Also updates the commented GPU blocks in both compose files with
`NVIDIA_DRIVER_CAPABILITIES=compute,utility`, `capabilities: [gpu, utility]`,
and `init: true`, and documents the requirement in the distributed-mode and
nvidia-l4t pages. Without `utility`, NVML/`nvidia-smi` are absent inside the
container, which is what put the DGX Spark worker into the buggy fallback in
the first place.

Detection verified on live hardware (dgx.casa / GB10 and 192.168.68.23 / Thor)
by running a cross-compiled probe of the new helpers on both host and inside
the worker container.

Assisted-by: Claude:opus-4.7 [Claude Code]
2026-04-24 22:02:23 +02:00
Ettore Di Giacinto bf2f95c684 chore(docs): update docs with cuda 13 instructions and the new vibevoice backend
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2025-12-25 10:00:07 +01:00
Ettore Di Giacinto 2cc4809b0d feat: docs revamp (#7313)
* docs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Small enhancements

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Enhancements

* Default to zen-dark

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2025-11-19 22:21:20 +01:00