From a776d74c079b04f3034c7f7379d03b063aa5c208 Mon Sep 17 00:00:00 2001 From: manunicholasjacob Date: Thu, 15 Oct 2026 17:11:05 -0500 Subject: [PATCH] docs: explain the automatic thread count Document the GOMAXPROCS cap and CPU quota floor introduced by #12691. Assisted-by: Codex:gpt-6 Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> Signed-off-by: manunicholasjacob (cherry picked from commit 123b1342e8ceb19df6e6c4d70c12d1753b7a354b) --- docs/content/reference/cli-reference.md | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/docs/content/reference/cli-reference.md b/docs/content/reference/cli-reference.md index 1a9793023..7c1d39037 100644 --- a/docs/content/reference/cli-reference.md +++ b/docs/content/reference/cli-reference.md @@ -121,9 +121,24 @@ For more information on VRAM management, see [VRAM and Memory Management]({{%rel | Parameter | Default | Description | Environment Variable | |-----------|---------|-------------|----------------------| | `--f16` | `false` | Enable GPU acceleration | `$LOCALAI_F16`, `$F16` | -| `-t, --threads` | | Number of threads used for parallel computation. Usage of the number of physical cores in the system is suggested | `$LOCALAI_THREADS`, `$THREADS` | +| `-t, --threads` | Automatic | Number of threads used for parallel computation. See the default thread count below | `$LOCALAI_THREADS`, `$THREADS` | | `--context-size` | | Default context size for models (`-1` = each model's full trained context from GGUF metadata) | `$LOCALAI_CONTEXT_SIZE`, `$CONTEXT_SIZE` | +### Default thread count + +When the server thread setting is unset or zero, LocalAI selects the smaller +of the physical core count and Go's current `GOMAXPROCS` value at startup. +The result is at least one thread. + +By default, Go accounts for CPU affinity and Linux container CPU quotas. +For example, `docker run --cpus=2` limits the automatic default to two threads. +Go normally applies a minimum of two for CPU quotas, so `--cpus=1` can still +produce two threads. Set `--threads=1` if you need one inference thread. + +An explicit `GOMAXPROCS` value overrides Go's automatic CPU detection. +A nonzero server thread setting from the CLI, environment, or saved runtime +settings bypasses this automatic default. + ## API Flags | Parameter | Default | Description | Environment Variable |