mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-07-13 09:44:26 -04:00
Code Issues Packages Projects Releases Wiki Activity

feat(config): context_size: -1 to auto-use model's full trained context (#10752)

Browse Source 
* feat(config): clamp negative context_size to default in EffectiveContextSize

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

* feat(config): resolve context_size=-1 to model trained max with VRAM warn

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

* fix(config): treat negative context_size as unset when GGUF is unparseable

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

* docs(config): document context_size=-1 auto-max

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

* docs(backend): drop em dashes from EffectiveContextSize comment

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
LocalAI [bot]
2026-07-09 09:03:40 +02:00
committed by GitHub
parent c9f73f40ff
commit 5569b2de56
7 changed files with 123 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/content/advanced/model-configuration.md
View File

@@ -145,7 +145,7 @@ These settings apply to most LLM backends (llama.cpp, vLLM, etc.):
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `threads` | int | `processor count` | Number of threads for parallel computation |
| `context_size` | int | `512` | Maximum context size (number of tokens) |
| `context_size` | int | `512` | Maximum context size in tokens. Set to `-1` to auto-use the model's full trained context from GGUF metadata (raw max, no VRAM capping; a warning is logged if it may not fit detected VRAM). |
| `f16` | bool | `false` | Enable 16-bit floating point precision (GPU acceleration) |
| `gpu_layers` | int | `0` | Number of layers to offload to GPU (0 = CPU only) |

2
docs/content/reference/cli-reference.md
View File

@@ -73,7 +73,7 @@ For more information on VRAM management, see [VRAM and Memory Management]({{%rel
|-----------|---------|-------------|----------------------|
| `--f16` | `false` | Enable GPU acceleration | `$LOCALAI_F16`, `$F16` |
| `-t, --threads` | | Number of threads used for parallel computation. Usage of the number of physical cores in the system is suggested | `$LOCALAI_THREADS`, `$THREADS` |
| `--context-size` | | Default context size for models | `$LOCALAI_CONTEXT_SIZE`, `$CONTEXT_SIZE` |
| `--context-size` | | Default context size for models (`-1` = each model's full trained context from GGUF metadata) | `$LOCALAI_CONTEXT_SIZE`, `$CONTEXT_SIZE` |
## API Flags