mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 17:18:49 -04:00
mudler-agentandEttore Di Giacinto
0e0e91f95c
* feat(diagnostics): add private opt-in profiler Keep profiling on a dedicated loopback listener rather than the public API. Validate settings and bind synchronously so startup reports configuration and socket failures. Own sampling and shutdown for the run process. Cover private handlers, blocked cmdline access, sampling cleanup, and serving failures with focused Ginkgo tests. Assisted-by: nib:gpt-6-astra * feat(diagnostics): add sanitized phase timing Carry generated request and reload identities through private contexts. Keep timing disabled without allocations and emit only validated fields at Info level when enabled. Cover identity isolation, nested durations, inheritance, and sanitized logging with focused race-tested Ginkgo specs. Assisted-by: nib:gpt-6-astra * fix(diagnostics): retain request observations Reuse an existing request observation across recorders to avoid splitting correlation. Keep the original context, identity, and sink without new allocations. Preserve nil-recorder behavior and fresh reload identities. Verify the regression fails before the fix and passes with the full Diagnostics race suite afterward. Assisted-by: nib:gpt-6-astra * feat(cli): own opt-in diagnostics lifecycle Expose profiling and phase timing flags without enabling them for debug. Bind the private profiler before application initialization and propagate serving failures through the run lifecycle. Bound signal cleanup and preserve preload-only backend startup. Cover flag parsing, preload isolation, cancellation, and once-only shutdown. Document initialization and shutdown cancellation limitations. Assisted-by: nib:gpt-6-astra * feat(config): measure filter and reload phases Separate request lock waits from filtering and held work. Aggregate reload filesystem and parsing durations without logging inside the config lock. Preserve caller option execution under the lock after metadata collection. Extract only the internal diagnostics option before waiting for the lock. Keep direct callers uninstrumented unless they supply diagnostics options. Verify focused Diagnostics examples with the race detector, including option ordering, gated contention, errors, panic propagation, and parity. Assisted-by: nib:gpt-6-astra * fix(config): separate early reload observation Opaque callbacks cannot select timing before their execution point. Carry the startup recorder explicitly through loader construction. Apply late options once under the lock and honor composed last writes. Document early and late phase boundaries and observation identities. Report malformed gallery syntax as an internal parse failure without changing strict or non-strict loader returns. Assisted-by: nib:gpt-6-astra * fix(config): preserve per-model option callbacks The resolved snapshot suppresses existing callbacks during model defaults, even when diagnostics are disabled. Pass original options through the existing defaults pathway without adding diagnostics-driven replay. Test stateful options with two models in enabled and disabled modes. Preserve three calls and thread defaults of two and three. Assisted-by: nib:gpt-6-astra * feat(model): observe discovery phases Carry request diagnostics through config and loose-file discovery to separate directory enumeration from in-memory filtering and fallback. Keep existing APIs as background-context wrappers and preserve discovery policies, ordering, duplicates, errors, and filesystem operation counts. Add focused parity, privacy, phase, and disabled-allocation tests. The focused diagnostics race gate passes for model and galleryop. Assisted-by: nib:gpt-6-astra * feat(http): observe extraction phases Generate diagnostic identity before request discovery, independently of client correlation headers. End each extraction segment before inference so downstream latency does not inflate discovery measurements. Carry observations through discovery helpers and rebuilt request contexts without changing cancellation or model selection. Cover disabled parity, error outcomes, privacy, concurrent identity isolation and route contexts. Assisted-by: nib:gpt-6-astra * fix(http): retain diagnostics across failover Failover restores its entry request before retrying. Initialize the observation before that capture so direct extraction retains one ID. Keep extraction spans per attempt and end them before downstream work. Exercise two real targets without default middleware, including replay, shared identity, phase boundaries, and timing-off response parity. Assisted-by: nib:gpt-6-astra * feat(model): observe initialization and routing Mark loader entry before cache and admission checks. Measure only the ModelRouter callback, without attributing transport or worker latency. Test inherited identity and sanitized events with fake backends. Exercise the actual Echo router to keep profiles private across debug and timing options. Cover capture shutdown, forced close, and lifecycle failures. Assisted-by: nib:gpt-6-astra * docs(diagnostics): guide bounded investigations Document private profiling, sampling ownership, and phase boundaries so operators can interpret captures without assuming request attribution. Add Ginkgo drift checks for flags, defaults, phases, links, and capture warnings. These checks do not certify the operator prose. The focused documentation race test passes after four expected failures. The consolidated Diagnostics race gate passes across all eight packages. This gate does not validate the full suite, auth, or production UI. Assisted-by: nib:gpt-6-astra * refactor(cli): restore inline run lifecycle Remove diagnostic startup hooks so application startup stays readable in Run. Keep the profiler as an optional sidecar with bounded cleanup and an error watcher, without a general lifecycle runner. Replace hook simulations with real CLI startup checks and component capture tests. Document that application cleanup can exceed the private listener budget. The eight-package Diagnostics race gate passes. A cleanup mutation fails the real CLI listener-release regression as expected. Assisted-by: nib:gpt-6-astra * fix(cli): retain concurrent startup errors Cancellation must not hide a public API bind failure. Join the public serve result, cancellation cause, and HTTP shutdown errors inline. Ignore ErrServerClosed only after intentional shutdown. Exercise real startup with an occupied API port and a failed private listener. Isolate the Linux socket fixture in a subprocess and wait for cancellation acknowledgment without timing sleeps. Assisted-by: nib:gpt-6-astra * refactor(http): make request context primary Move context preparation into the request extractor. Diagnostics remain an optional addition, not the entry point for request processing. Preserve context initialization before failover captures the request. Assisted-by: nib:gpt-6-astra --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
23 KiB
23 KiB
+++ disableToc = false title = "CLI Reference" weight = 25 url = '/reference/cli-reference' +++
Complete reference for all LocalAI command-line interface (CLI) parameters and environment variables.
For client-side text inference measurements, see [Benchmark text models]({{% relref "features/benchmark" %}}).
These options configure the LocalAI server process. To configure an individual model, see [Model Configuration]({{% relref "advanced/model-configuration" %}}).
Note: All CLI flags can also be set via environment variables. Environment variables take precedence over CLI flags. See [.env files]({{%relref "advanced/advanced-usage#env-files" %}}) for configuration file support.
Global Flags
| Parameter | Default | Description | Environment Variable |
|---|---|---|---|
-h, --help |
Show context-sensitive help | ||
--log-level |
info |
Set the level of logs to output [error,warn,info,debug,trace] | $LOCALAI_LOG_LEVEL |
--credentials-file |
YAML file with credentials for private registries, galleries and download hosts. See [Private Registries and Galleries]({{% relref "advanced/private-sources" %}}) | $LOCALAI_CREDENTIALS_FILE |
|
--debug |
false |
DEPRECATED - Use --log-level=debug instead. Enable debug logging |
$LOCALAI_DEBUG, $DEBUG |
Diagnostics flags
See [Profiling and request diagnostics]({{% relref "features/diagnostics" %}}) for sampling semantics, phase boundaries, and protected pod captures.
These options apply only to local-ai run, including distributed frontends.
Diagnostics state is per-instance, not shared between replicas. Debug logging enables neither feature.
The profiling listener is separate from the public API and accepts only numeric loopback addresses with explicit ports.
--preload-backend-only validates these options but does not start profiling or sampling.
| Parameter | Default | Description | Environment Variable |
|---|---|---|---|
--pprof |
false |
Enable the private loopback profiling listener | $LOCALAI_PPROF |
--pprof-address |
127.0.0.1:6060 |
Numeric loopback IP and port (1–65535), including bracketed IPv6; validated even when disabled | $LOCALAI_PPROF_ADDRESS |
--pprof-mutex-profile-fraction |
0 |
Nonnegative mutex sampling fraction; nonzero requires pprof | $LOCALAI_PPROF_MUTEX_PROFILE_FRACTION |
--pprof-block-profile-rate |
0 |
Nonnegative block sampling rate in nanoseconds; nonzero requires pprof | $LOCALAI_PPROF_BLOCK_PROFILE_RATE |
--request-phase-timing |
false |
Emit sanitized phase timing at Info, independently of pprof; warn/error filtering suppresses events | $LOCALAI_REQUEST_PHASE_TIMING |
Sampling adds overhead. The process applies configured rates once and resets them to zero on shutdown.
There is no runtime toggle, continuous capture, or public API route for diagnostics.
The private listener has no authentication: other processes in the same network namespace can access profiles.
Profiles can contain secrets; protect captured files. The /debug/pprof/cmdline endpoint is unavailable.
Shutdown allows five seconds for cleanup, then force-closes profiling connections.
Existing application initialization operations may not promptly observe cancellation.
The signal callback still returns after its cleanup budget expires.
If initialization fails without returning an application handle, this lifecycle cannot shut down resources created inside that failed initialization.
Application shutdown has no context parameter; work that ignores cancellation can outlive the cleanup budget.
Storage Flags
| Parameter | Default | Description | Environment Variable |
|---|---|---|---|
--models-path |
BASEPATH/models |
Path containing models used for inferencing | $LOCALAI_MODELS_PATH, $MODELS_PATH |
--download-staging-path |
BASEPATH/downloading |
Path where in-flight downloads are staged before extraction | $LOCALAI_DOWNLOAD_STAGING_PATH, $DOWNLOAD_STAGING_PATH |
--data-path |
BASEPATH/data |
Path for persistent data (collectiondb, agent state, tasks, jobs). Separates mutable data from configuration | $LOCALAI_DATA_PATH |
--generated-content-path |
TMPDIR/localai-UID/generated/content |
Location for assets generated by backends (e.g. stablediffusion, images, audio, videos). Defaults under the OS temp dir ($TMPDIR, falling back to /tmp), scoped to the current user's UID so accounts sharing a host never collide. |
$LOCALAI_GENERATED_CONTENT_PATH, $GENERATED_CONTENT_PATH |
--upload-path |
TMPDIR/localai-UID/upload |
Path to store uploads from files API. Defaults under the OS temp dir ($TMPDIR, falling back to /tmp), scoped to the current user's UID. |
$LOCALAI_UPLOAD_PATH, $UPLOAD_PATH |
--localai-config-dir |
BASEPATH/configuration |
Directory for dynamic loading of certain configuration files (currently runtime_settings.json, api_keys.json, and external_backends.json). See [Runtime Settings]({{%relref "features/runtime-settings" %}}) for web-based configuration. | $LOCALAI_CONFIG_DIR |
--localai-config-dir-poll-interval |
Time duration to poll the LocalAI Config Dir if your system has broken fsnotify events (example: 1m) |
$LOCALAI_CONFIG_DIR_POLL_INTERVAL |
| --models-config-file | | YAML file containing a list of model backend configs (alias: --config-file) | $LOCALAI_MODELS_CONFIG_FILE, $CONFIG_FILE |
| --artifact-download-concurrency | 1 | How many files of a model artifact to download at once. 1 downloads sequentially. Raising it helps artifacts split into many files on a fast link, at the cost of more concurrent load on the models volume. Whole files only — a single file is never split, so resume and per-file checksum verification are unaffected | $LOCALAI_ARTIFACT_DOWNLOAD_CONCURRENCY |
Backend processes receive a private scratch directory through TMPDIR, TMP,
and TEMP. LocalAI removes that directory when the backend exits and removes
abandoned directories left by a LocalAI crash before starting another backend.
Set $LOCALAI_BACKEND_TEMP_DIR to choose their base volume. LocalAI always
appends localai-UID/backend-runtime; the default base is TMPDIR.
Backend Flags
| Parameter | Default | Description | Environment Variable |
|---|---|---|---|
--backends-path |
BASEPATH/backends |
Path containing backends used for inferencing | $LOCALAI_BACKENDS_PATH, $BACKENDS_PATH |
--backends-system-path |
/var/lib/local-ai/backends |
Path containing system backends used for inferencing | $LOCALAI_BACKENDS_SYSTEM_PATH, $BACKEND_SYSTEM_PATH |
--external-backends |
A list of external backends to load from gallery on boot | $LOCALAI_EXTERNAL_BACKENDS, $EXTERNAL_BACKENDS |
|
--external-grpc-backends |
A list of external gRPC backends (format: BACKEND_NAME:URI) |
$LOCALAI_EXTERNAL_GRPC_BACKENDS, $EXTERNAL_GRPC_BACKENDS |
|
--backend-galleries |
JSON list of backend galleries | $LOCALAI_BACKEND_GALLERIES, $BACKEND_GALLERIES |
|
--autoload-backend-galleries |
true |
Automatically load backend galleries on startup | $LOCALAI_AUTOLOAD_BACKEND_GALLERIES, $AUTOLOAD_BACKEND_GALLERIES |
--max-active-backends |
0 |
Maximum number of active backends (loaded models). When exceeded, the least recently used model is evicted. Set to 0 for unlimited, 1 for single-backend mode |
$LOCALAI_MAX_ACTIVE_BACKENDS, $MAX_ACTIVE_BACKENDS |
--single-active-backend |
false |
DEPRECATED - Use --max-active-backends=1 instead. Allow only one backend to be run at a time |
$LOCALAI_SINGLE_ACTIVE_BACKEND, $SINGLE_ACTIVE_BACKEND |
--preload-backend-only |
false |
Do not launch the API services, only the preloaded models/backends are started (useful for multi-node setups) | $LOCALAI_PRELOAD_BACKEND_ONLY, $PRELOAD_BACKEND_ONLY |
--enable-watchdog-idle |
false |
Enable watchdog for stopping backends that are idle longer than the watchdog-idle-timeout | $LOCALAI_WATCHDOG_IDLE, $WATCHDOG_IDLE |
--watchdog-idle-timeout |
15m |
Threshold beyond which an idle backend should be stopped | $LOCALAI_WATCHDOG_IDLE_TIMEOUT, $WATCHDOG_IDLE_TIMEOUT |
--enable-watchdog-busy |
false |
Enable watchdog for stopping backends that are busy longer than the watchdog-busy-timeout | $LOCALAI_WATCHDOG_BUSY, $WATCHDOG_BUSY |
--watchdog-busy-timeout |
5m |
Threshold beyond which a busy backend should be stopped | $LOCALAI_WATCHDOG_BUSY_TIMEOUT, $WATCHDOG_BUSY_TIMEOUT |
--watchdog-interval |
500ms |
Interval between watchdog checks (e.g., 500ms, 5s, 1m) |
$LOCALAI_WATCHDOG_INTERVAL, $WATCHDOG_INTERVAL |
--force-eviction-when-busy |
false |
Force eviction even when models have active API calls (default: false for safety). Warning: Enabling this can interrupt active requests | $LOCALAI_FORCE_EVICTION_WHEN_BUSY, $FORCE_EVICTION_WHEN_BUSY |
--lru-eviction-max-retries |
30 |
Maximum number of retries when waiting for busy models to become idle before eviction | $LOCALAI_LRU_EVICTION_MAX_RETRIES, $LRU_EVICTION_MAX_RETRIES |
--lru-eviction-retry-interval |
1s |
Interval between retries when waiting for busy models to become idle (e.g., 1s, 2s) |
$LOCALAI_LRU_EVICTION_RETRY_INTERVAL, $LRU_EVICTION_RETRY_INTERVAL |
--model-load-failure-cooldown |
10s |
After a model load fails, refuse new load attempts for that model for this long (HTTP 503 + Retry-After) so a client polling a broken model doesn't respawn a crashing backend every request. Doubles per consecutive failure up to 5m; reset on success. 0 disables |
$LOCALAI_MODEL_LOAD_FAILURE_COOLDOWN, $MODEL_LOAD_FAILURE_COOLDOWN |
For more information on VRAM management, see [VRAM and Memory Management]({{%relref "advanced/vram-management" %}}).
Models Flags
| Parameter | Default | Description | Environment Variable |
|---|---|---|---|
--galleries |
JSON list of galleries | $LOCALAI_GALLERIES, $GALLERIES |
|
--autoload-galleries |
true |
Automatically load galleries on startup | $LOCALAI_AUTOLOAD_GALLERIES, $AUTOLOAD_GALLERIES |
--vram-persistent-cache |
true |
Persist successful remote VRAM metadata probes across restarts | $LOCALAI_VRAM_PERSISTENT_CACHE, $VRAM_PERSISTENT_CACHE |
--preload-models |
A list of models to apply in JSON at start | $LOCALAI_PRELOAD_MODELS, $PRELOAD_MODELS |
|
--models |
A list of model configuration URLs to load | $LOCALAI_MODELS, $MODELS |
|
--preload-models-config |
A list of models to apply at startup. Path to a YAML config file | $LOCALAI_PRELOAD_MODELS_CONFIG, $PRELOAD_MODELS_CONFIG |
|
--load-to-memory |
A list of models to load into memory at startup | $LOCALAI_LOAD_TO_MEMORY, $LOAD_TO_MEMORY |
Note: You can also pass model configuration URLs as positional arguments:
local-ai run MODEL_URL1 MODEL_URL2 ...
Performance Flags
| Parameter | Default | Description | Environment Variable |
|---|---|---|---|
--f16 |
false |
Enable GPU acceleration | $LOCALAI_F16, $F16 |
-t, --threads |
Number of threads used for parallel computation. Usage of the number of physical cores in the system is suggested | $LOCALAI_THREADS, $THREADS |
|
--context-size |
Default context size for models (-1 = each model's full trained context from GGUF metadata) |
$LOCALAI_CONTEXT_SIZE, $CONTEXT_SIZE |
API Flags
| Parameter | Default | Description | Environment Variable |
|---|---|---|---|
--address |
:8080 |
Bind address for the API server | $LOCALAI_ADDRESS, $ADDRESS |
--max-concurrent-backend-requests |
1024 |
Process-wide ceiling for concurrent backend inference operations. Excess inference receives HTTP 429 with Retry-After; UI and administrative endpoints remain available |
$LOCALAI_MAX_CONCURRENT_BACKEND_REQUESTS, $MAX_CONCURRENT_BACKEND_REQUESTS |
--cors |
false |
Enable CORS (Cross-Origin Resource Sharing) | $LOCALAI_CORS, $CORS |
--cors-allow-origins |
Comma-separated list of allowed CORS origins | $LOCALAI_CORS_ALLOW_ORIGINS, $CORS_ALLOW_ORIGINS |
|
--disable-csrf |
false |
Disable CSRF middleware (enabled by default) | $LOCALAI_DISABLE_CSRF |
--disable-http-compression |
false |
Disable gzip compression of HTTP responses. Compression is enabled by default; streaming endpoints (streaming chat completions, SSE bridges, WebSocket upgrades) and already-compressed formats are never compressed | $LOCALAI_DISABLE_HTTP_COMPRESSION |
--http-compression-min-length |
1024 |
Minimum response size in bytes before gzip compression is applied. Smaller responses are sent as-is because the gzip envelope would outweigh the saving | $LOCALAI_HTTP_COMPRESSION_MIN_LENGTH |
--upload-limit |
15 |
Default upload-limit in MB | $LOCALAI_UPLOAD_LIMIT, $UPLOAD_LIMIT |
--api-keys |
List of API Keys to enable API authentication. When this is set, all requests must be authenticated with one of these API keys | $LOCALAI_API_KEY, $API_KEY |
|
--disable-webui |
false |
Disables the web user interface. When set to true, the server will only expose API endpoints without serving the web interface | $LOCALAI_DISABLE_WEBUI, $DISABLE_WEBUI |
--disable-runtime-settings |
false |
Disables the runtime settings feature. When set to true, the server will not load runtime settings from the runtime_settings.json file and the settings web interface will be disabled |
$LOCALAI_DISABLE_RUNTIME_SETTINGS, $DISABLE_RUNTIME_SETTINGS |
--disable-gallery-endpoint |
false |
Disable the gallery endpoints | $LOCALAI_DISABLE_GALLERY_ENDPOINT, $DISABLE_GALLERY_ENDPOINT |
--disable-metrics-endpoint |
false |
Disable the /metrics endpoint |
$LOCALAI_DISABLE_METRICS_ENDPOINT, $DISABLE_METRICS_ENDPOINT |
--machine-tag |
If not empty, add that string to Machine-Tag header in each response. Useful to track response from different machines using multiple P2P federated nodes | $LOCALAI_MACHINE_TAG, $MACHINE_TAG |
Hardening Flags
| Parameter | Default | Description | Environment Variable |
|---|---|---|---|
--disable-predownload-scan |
false |
If true, disables the best-effort security scanner before downloading any files | $LOCALAI_DISABLE_PREDOWNLOAD_SCAN |
--opaque-errors |
false |
If true, all error responses are replaced with blank 500 errors. This is intended only for hardening against information leaks and is normally not recommended | $LOCALAI_OPAQUE_ERRORS |
--use-subtle-key-comparison |
false |
If true, API Key validation comparisons will be performed using constant-time comparisons rather than simple equality. This trades off performance on each request for resilience against timing attacks | $LOCALAI_SUBTLE_KEY_COMPARISON |
--disable-api-key-requirement-for-http-get |
false |
If true, a valid API key is not required to issue GET requests to portions of the web UI. This should only be enabled in secure testing environments | $LOCALAI_DISABLE_API_KEY_REQUIREMENT_FOR_HTTP_GET |
--http-get-exempted-endpoints |
^/$,^/app(/.*)?$,^/browse(/.*)?$,^/login/?$,^/explorer/?$,^/assets/.*$,^/static/.*$,^/swagger.*$ |
If --disable-api-key-requirement-for-http-get is overridden to true, this is the list of endpoints to exempt. Only adjust this in case of a security incident or as a result of a personal security posture review |
$LOCALAI_HTTP_GET_EXEMPTED_ENDPOINTS |
Authentication Flags
| Parameter | Default | Description | Environment Variable |
|---|---|---|---|
--auth-enabled |
false |
Enable user authentication and authorization | $LOCALAI_AUTH |
--auth-database-url |
{DataPath}/database.db |
Database URL for auth - postgres://... for PostgreSQL, or a file path for SQLite |
$LOCALAI_AUTH_DATABASE_URL, $DATABASE_URL |
--github-client-id |
GitHub OAuth App Client ID (auto-enables auth when set) | $GITHUB_CLIENT_ID |
|
--github-client-secret |
GitHub OAuth App Client Secret | $GITHUB_CLIENT_SECRET |
|
--oidc-issuer |
OIDC issuer URL for auto-discovery | $LOCALAI_OIDC_ISSUER |
|
--oidc-client-id |
OIDC Client ID (auto-enables auth when set) | $LOCALAI_OIDC_CLIENT_ID |
|
--oidc-client-secret |
OIDC Client Secret | $LOCALAI_OIDC_CLIENT_SECRET |
|
--auth-base-url |
Base URL for OAuth callbacks (e.g. http://localhost:8080) |
$LOCALAI_BASE_URL |
|
--auth-admin-email |
Email address to auto-promote to admin role on login | $LOCALAI_ADMIN_EMAIL |
|
--auth-registration-mode |
open |
Registration mode: open, approval, or invite |
$LOCALAI_REGISTRATION_MODE |
--disable-local-auth |
false |
Disable local email/password registration and login (for OAuth/OIDC-only setups) | $LOCALAI_DISABLE_LOCAL_AUTH |
See [Authentication & Authorization]({{%relref "features/authentication" %}}) for full documentation.
Chat Flags
Use local-ai chat to run the built-in terminal agent against a LocalAI server.
The agent can run shell commands, delegate to sub-agents, and use MCP tools. Read-only
calls run on their own; everything else goes through an approval prompt you answer.
See [Terminal agent]({{% relref "features/terminal-agent" %}}) for the full feature page.
| Parameter | Default | Description | Environment Variable |
|---|---|---|---|
--endpoint |
http://127.0.0.1:8080 |
LocalAI server endpoint. The /v1 path is added automatically when omitted. |
$LOCALAI_CHAT_ENDPOINT |
--model |
Model to use. Defaults to the only model the server offers, or asks when there are several. | ||
--api-key |
API key to use when the LocalAI server requires authentication. | $LOCALAI_API_KEY, $API_KEY |
|
--config-dir |
~/.config/localai/chat |
Directory holding the agent's config, plugins, and skills. | $LOCALAI_CHAT_CONFIG_DIR |
--trace-dir |
Write a session LLM trace (NDJSON) to this directory. | $LOCALAI_CHAT_TRACE_DIR |
|
--cli |
false |
Plain CLI mode instead of the full-screen interface. Pipe stdin for one-shot use. | |
--tui |
false |
Force the full-screen interface. | |
--height |
Run as an inline drop-down of this height, e.g. 40%. |
||
--tmux / --no-tmux |
Control the tmux split. | ||
--init |
Print the shell integration script for Ctrl+Space (zsh, bash, or fish). |
||
--yolo |
false |
Auto-approve every tool call. Use with care. | $LOCALAI_CHAT_YOLO |
The agent's own subcommands are reached by passing them through. LocalAI's flags must come first, because everything after the first positional argument is forwarded verbatim:
local-ai chat plugin install https://github.com/user/plugin
local-ai chat skill list
local-ai chat mcp add my-server -- npx -y @modelcontextprotocol/server-filesystem /tmp
local-ai chat --config-dir /srv/agent plugin list # flags first
{{% notice warning %}}
In a script, pass --yes to those subcommands. Without it, a non-interactive
plugin install installs the plugin, leaves it disabled, and still exits 0.
{{% /notice %}}
Summon the agent from any shell prompt with Ctrl+Space:
echo 'eval "$(local-ai chat --init zsh)"' >> ~/.zshrc
Inside a session:
/modelslists the models the server offers, marking the current one./model <name>switches model, keeping the conversation./compactsummarizes the conversation so far to free up context./skill <name>,/agent <name> <task>,/attach <file>,/goal <text>.
If no server is reachable, the agent offers to start one for the session
(interactive terminals only) and otherwise points you at local-ai run.
Piped use needs --cli, and exits 0 when it answers:
echo "what is 2+2" | local-ai chat --cli
Read-only tools still run in a piped session. A tool call that is not read-only has
nobody to approve it, so it is denied and the session exits 3, a code chosen to be
distinct from the 1 a failure reports.
P2P Flags
| Parameter | Default | Description | Environment Variable |
|---|---|---|---|
--p2p |
false |
Enable P2P mode | $LOCALAI_P2P, $P2P |
--p2p-dht-interval |
360 |
Interval for DHT refresh (used during token generation) | $LOCALAI_P2P_DHT_INTERVAL, $P2P_DHT_INTERVAL |
--p2p-otp-interval |
9000 |
Interval for OTP refresh (used during token generation) | $LOCALAI_P2P_OTP_INTERVAL, $P2P_OTP_INTERVAL |
--p2ptoken |
Token for P2P mode (optional) | $LOCALAI_P2P_TOKEN, $P2P_TOKEN, $TOKEN |
|
--p2p-network-id |
Network ID for P2P mode, can be set arbitrarily by the user for grouping a set of instances | $LOCALAI_P2P_NETWORK_ID, $P2P_NETWORK_ID |
|
--federated |
false |
Enable federated instance | $LOCALAI_FEDERATED, $FEDERATED |
Other Commands
LocalAI supports several subcommands beyond run:
local-ai models- Manage LocalAI models and definitionslocal-ai backends- Manage LocalAI backends and definitionslocal-ai chat- Run the built-in terminal agent against a LocalAI serverlocal-ai tts- Convert text to speechlocal-ai sound-generation- Generate audio files from text or audiolocal-ai transcript- Convert audio to textlocal-ai agent- Run agents standalone without the full LocalAI serverlocal-ai mcp-server- Run the LocalAI admin tool surface as a stdio MCP server (controls a remote LocalAI instance over HTTP)local-ai worker- Start a worker for distributed mode (generic, backend-agnostic)local-ai p2p-worker- Run workers to distribute workload via p2p (llama.cpp-only)local-ai agent-worker- Start an agent worker for distributed mode (executes agent chats via NATS)local-ai util- Utility commandslocal-ai explorer- Run P2P explorerlocal-ai federated- Run LocalAI in federated modelocal-ai completion- Generate shell completion scripts for bash, zsh, or fish
Use local-ai <command> --help for more information on each command.
Examples
Basic Usage
./local-ai run
./local-ai run --models-path /path/to/models --address :9090
./local-ai run --f16
Environment Variables
export LOCALAI_MODELS_PATH=/path/to/models
export LOCALAI_ADDRESS=:9090
export LOCALAI_F16=true
./local-ai run
Advanced Configuration
./local-ai run \
--models model1.yaml model2.yaml \
--enable-watchdog-idle \
--watchdog-idle-timeout=10m \
--p2p \
--federated
Related Documentation
- See [Advanced Usage]({{%relref "advanced/advanced-usage" %}}) for configuration examples
- See [VRAM and Memory Management]({{%relref "advanced/vram-management" %}}) for memory management options