docs: explain the automatic thread count

Document the GOMAXPROCS cap and CPU quota floor introduced by #12691.

Assisted-by: Codex:gpt-6
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Signed-off-by: manunicholasjacob <manunicholasjacob@gmail.com>
This commit is contained in:
manunicholasjacobandlocalai-org-maint-bot committed 2026-10-15 19:58:31 -05:00
1 parent aca800d003
commit 123b1342e8
1 file changed
+16 -1
+16 -1
View File
@@ -115,9 +115,24 @@ For more information on VRAM management, see [VRAM and Memory Management]({{%rel
| Parameter | Default | Description | Environment Variable |
|-----------|---------|-------------|----------------------|
| `--f16` | `false` | Enable GPU acceleration | `$LOCALAI_F16`, `$F16` |
| `-t, --threads` | | Number of threads used for parallel computation. Usage of the number of physical cores in the system is suggested | `$LOCALAI_THREADS`, `$THREADS` |
| `-t, --threads` | Automatic | Number of threads used for parallel computation. See the default thread count below | `$LOCALAI_THREADS`, `$THREADS` |
| `--context-size` | | Default context size for models (`-1` = each model's full trained context from GGUF metadata) | `$LOCALAI_CONTEXT_SIZE`, `$CONTEXT_SIZE` |
### Default thread count
When the server thread setting is unset or zero, LocalAI selects the smaller
of the physical core count and Go's current `GOMAXPROCS` value at startup.
The result is at least one thread.
By default, Go accounts for CPU affinity and Linux container CPU quotas.
For example, `docker run --cpus=2` limits the automatic default to two threads.
Go normally applies a minimum of two for CPU quotas, so `--cpus=1` can still
produce two threads. Set `--threads=1` if you need one inference thread.
An explicit `GOMAXPROCS` value overrides Go's automatic CPU detection.
A nonzero server thread setting from the CLI, environment, or saved runtime
settings bypasses this automatic default.
## API Flags
| Parameter | Default | Description | Environment Variable |