LocalAI/docs/content/reference/cli-reference.md

+++ disableToc = false title = "CLI Reference" weight = 25 url = '/reference/cli-reference' +++

Complete reference for all LocalAI command-line interface (CLI) parameters and environment variables.

For client-side text inference measurements, see [Benchmark text models]({{% relref "features/benchmark" %}}).

These options configure the LocalAI server process. To configure an individual model, see [Model Configuration]({{% relref "advanced/model-configuration" %}}).

Note: All CLI flags can also be set via environment variables. Environment variables take precedence over CLI flags. See [.env files]({{%relref "advanced/advanced-usage#env-files" %}}) for configuration file support.

Global Flags

Parameter	Default	Description	Environment Variable
-h, --help		Show context-sensitive help
--log-level	info	Set the level of logs to output [error,warn,info,debug,trace]	$LOCALAI_LOG_LEVEL
--credentials-file		YAML file with credentials for private registries, galleries and download hosts. See [Private Registries and Galleries]({{% relref "advanced/private-sources" %}})	$LOCALAI_CREDENTIALS_FILE
--debug	false	DEPRECATED - Use --log-level=debug instead. Enable debug logging	$LOCALAI_DEBUG, $DEBUG

Diagnostics flags

See [Profiling and request diagnostics]({{% relref "features/diagnostics" %}}) for sampling semantics, phase boundaries, and protected pod captures.

These options apply only to local-ai run, including distributed frontends. Diagnostics state is per-instance, not shared between replicas. Debug logging enables neither feature. The profiling listener is separate from the public API and accepts only numeric loopback addresses with explicit ports. --preload-backend-only validates these options but does not start profiling or sampling.

Parameter	Default	Description	Environment Variable
--pprof	false	Enable the private loopback profiling listener	$LOCALAI_PPROF
--pprof-address	127.0.0.1:6060	Numeric loopback IP and port (1–65535), including bracketed IPv6; validated even when disabled	$LOCALAI_PPROF_ADDRESS
--pprof-mutex-profile-fraction	0	Nonnegative mutex sampling fraction; nonzero requires pprof	$LOCALAI_PPROF_MUTEX_PROFILE_FRACTION
--pprof-block-profile-rate	0	Nonnegative block sampling rate in nanoseconds; nonzero requires pprof	$LOCALAI_PPROF_BLOCK_PROFILE_RATE
--request-phase-timing	false	Emit sanitized phase timing at Info, independently of pprof; warn/error filtering suppresses events	$LOCALAI_REQUEST_PHASE_TIMING

Sampling adds overhead. The process applies configured rates once and resets them to zero on shutdown. There is no runtime toggle, continuous capture, or public API route for diagnostics. The private listener has no authentication: other processes in the same network namespace can access profiles. Profiles can contain secrets; protect captured files. The /debug/pprof/cmdline endpoint is unavailable. Shutdown allows five seconds for cleanup, then force-closes profiling connections. Existing application initialization operations may not promptly observe cancellation. The signal callback still returns after its cleanup budget expires. If initialization fails without returning an application handle, this lifecycle cannot shut down resources created inside that failed initialization. Application shutdown has no context parameter; work that ignores cancellation can outlive the cleanup budget.

Storage Flags

Parameter	Default	Description	Environment Variable
--models-path	BASEPATH/models	Path containing models used for inferencing	$LOCALAI_MODELS_PATH, $MODELS_PATH
--download-staging-path	BASEPATH/downloading	Path where in-flight downloads are staged before extraction	$LOCALAI_DOWNLOAD_STAGING_PATH, $DOWNLOAD_STAGING_PATH
--data-path	BASEPATH/data	Path for persistent data (collectiondb, agent state, tasks, jobs). Separates mutable data from configuration	$LOCALAI_DATA_PATH
--generated-content-path	TMPDIR/localai-UID/generated/content	Location for assets generated by backends (e.g. stablediffusion, images, audio, videos). Defaults under the OS temp dir ($TMPDIR, falling back to /tmp), scoped to the current user's UID so accounts sharing a host never collide.	$LOCALAI_GENERATED_CONTENT_PATH, $GENERATED_CONTENT_PATH
--upload-path	TMPDIR/localai-UID/upload	Path to store uploads from files API. Defaults under the OS temp dir ($TMPDIR, falling back to /tmp), scoped to the current user's UID.	$LOCALAI_UPLOAD_PATH, $UPLOAD_PATH
--localai-config-dir	BASEPATH/configuration	Directory for dynamic loading of certain configuration files (currently runtime_settings.json, api_keys.json, and external_backends.json). See [Runtime Settings]({{%relref "features/runtime-settings" %}}) for web-based configuration.	$LOCALAI_CONFIG_DIR
--localai-config-dir-poll-interval		Time duration to poll the LocalAI Config Dir if your system has broken fsnotify events (example: 1m)	$LOCALAI_CONFIG_DIR_POLL_INTERVAL

| --models-config-file | | YAML file containing a list of model backend configs (alias: --config-file) | $LOCALAI_MODELS_CONFIG_FILE, $CONFIG_FILE | | --addon-host-ops | fine-tuning=finetune.,quantization=quantize. | Add-ons that may call host operations, as a comma list of addon-id=operation-prefix. An add-on that is not listed gets no host operation channel; an empty value lists none. See [Host operations]({{% relref "features/addons" %}}) | $LOCALAI_ADDON_HOST_OPS | | --addons-role | combined | How this process takes part in the add-on owner: combined (the node is its own owner), owner or frontend. See [Turning the owner on or off]({{% relref "features/addons#turning-the-owner-on-or-off" %}}) | $LOCALAI_ADDONS_ROLE | | --addons-owner-address | | https://host:port of the add-on owner carrier, with no path. Required for the owner and frontend roles | $LOCALAI_ADDONS_OWNER_ADDRESS | | --addons-owner-listen | | owner role only: the address the carrier listens on. Empty listens on all interfaces, on the port of --addons-owner-address | $LOCALAI_ADDONS_OWNER_LISTEN | | --addons-carrier-dir | | Directory with the carrier key set from local-ai util addons-carrier-keys. Required for the owner and frontend roles | $LOCALAI_ADDONS_CARRIER_DIR | | --artifact-download-concurrency | 1 | How many files of a model artifact to download at once. 1 downloads sequentially. Raising it helps artifacts split into many files on a fast link, at the cost of more concurrent load on the models volume. Whole files only — a single file is never split, so resume and per-file checksum verification are unaffected | $LOCALAI_ARTIFACT_DOWNLOAD_CONCURRENCY |

Backend processes receive a private scratch directory through TMPDIR, TMP, and TEMP. LocalAI removes that directory when the backend exits and removes abandoned directories left by a LocalAI crash before starting another backend. Set $LOCALAI_BACKEND_TEMP_DIR to choose their base volume. LocalAI always appends localai-UID/backend-runtime; the default base is TMPDIR.

Backend Flags

Parameter	Default	Description	Environment Variable
--backends-path	BASEPATH/backends	Path containing backends used for inferencing	$LOCALAI_BACKENDS_PATH, $BACKENDS_PATH
--backends-system-path	/var/lib/local-ai/backends	Path containing system backends used for inferencing	$LOCALAI_BACKENDS_SYSTEM_PATH, $BACKEND_SYSTEM_PATH
--external-backends		A list of external backends to load from gallery on boot	$LOCALAI_EXTERNAL_BACKENDS, $EXTERNAL_BACKENDS
--external-grpc-backends		A list of external gRPC backends (format: BACKEND_NAME:URI)	$LOCALAI_EXTERNAL_GRPC_BACKENDS, $EXTERNAL_GRPC_BACKENDS
--backend-galleries		JSON list of backend galleries	$LOCALAI_BACKEND_GALLERIES, $BACKEND_GALLERIES
--autoload-backend-galleries	true	Automatically load backend galleries on startup	$LOCALAI_AUTOLOAD_BACKEND_GALLERIES, $AUTOLOAD_BACKEND_GALLERIES
--max-active-backends	0	Maximum number of active backends (loaded models). When exceeded, the least recently used model is evicted. Set to 0 for unlimited, 1 for single-backend mode	$LOCALAI_MAX_ACTIVE_BACKENDS, $MAX_ACTIVE_BACKENDS
--single-active-backend	false	DEPRECATED - Use --max-active-backends=1 instead. Allow only one backend to be run at a time	$LOCALAI_SINGLE_ACTIVE_BACKEND, $SINGLE_ACTIVE_BACKEND
--preload-backend-only	false	Do not launch the API services, only the preloaded models/backends are started (useful for multi-node setups)	$LOCALAI_PRELOAD_BACKEND_ONLY, $PRELOAD_BACKEND_ONLY
--enable-watchdog-idle	false	Enable watchdog for stopping backends that are idle longer than the watchdog-idle-timeout	$LOCALAI_WATCHDOG_IDLE, $WATCHDOG_IDLE
--watchdog-idle-timeout	15m	Threshold beyond which an idle backend should be stopped	$LOCALAI_WATCHDOG_IDLE_TIMEOUT, $WATCHDOG_IDLE_TIMEOUT
--enable-watchdog-busy	false	Enable watchdog for stopping backends that are busy longer than the watchdog-busy-timeout	$LOCALAI_WATCHDOG_BUSY, $WATCHDOG_BUSY
--watchdog-busy-timeout	5m	Threshold beyond which a busy backend should be stopped	$LOCALAI_WATCHDOG_BUSY_TIMEOUT, $WATCHDOG_BUSY_TIMEOUT
--watchdog-interval	500ms	Interval between watchdog checks (e.g., 500ms, 5s, 1m)	$LOCALAI_WATCHDOG_INTERVAL, $WATCHDOG_INTERVAL
--force-eviction-when-busy	false	Force eviction even when models have active API calls (default: false for safety). Warning: Enabling this can interrupt active requests	$LOCALAI_FORCE_EVICTION_WHEN_BUSY, $FORCE_EVICTION_WHEN_BUSY
--lru-eviction-max-retries	30	Maximum number of retries when waiting for busy models to become idle before eviction	$LOCALAI_LRU_EVICTION_MAX_RETRIES, $LRU_EVICTION_MAX_RETRIES
--lru-eviction-retry-interval	1s	Interval between retries when waiting for busy models to become idle (e.g., 1s, 2s)	$LOCALAI_LRU_EVICTION_RETRY_INTERVAL, $LRU_EVICTION_RETRY_INTERVAL
--model-load-failure-cooldown	10s	After a model load fails, refuse new load attempts for that model for this long (HTTP 503 + Retry-After) so a client polling a broken model doesn't respawn a crashing backend every request. Doubles per consecutive failure up to 5m; reset on success. 0 disables	$LOCALAI_MODEL_LOAD_FAILURE_COOLDOWN, $MODEL_LOAD_FAILURE_COOLDOWN

For more information on VRAM management, see [VRAM and Memory Management]({{%relref "advanced/vram-management" %}}).

Models Flags

Parameter	Default	Description	Environment Variable
--galleries		JSON list of galleries	$LOCALAI_GALLERIES, $GALLERIES
--autoload-galleries	true	Automatically load galleries on startup	$LOCALAI_AUTOLOAD_GALLERIES, $AUTOLOAD_GALLERIES
--vram-persistent-cache	true	Persist successful remote VRAM metadata probes across restarts	$LOCALAI_VRAM_PERSISTENT_CACHE, $VRAM_PERSISTENT_CACHE
--preload-models		A list of models to apply in JSON at start	$LOCALAI_PRELOAD_MODELS, $PRELOAD_MODELS
--models		A list of model configuration URLs to load	$LOCALAI_MODELS, $MODELS
--preload-models-config		A list of models to apply at startup. Path to a YAML config file	$LOCALAI_PRELOAD_MODELS_CONFIG, $PRELOAD_MODELS_CONFIG
--load-to-memory		A list of models to load into memory at startup	$LOCALAI_LOAD_TO_MEMORY, $LOAD_TO_MEMORY

Note: You can also pass model configuration URLs as positional arguments: local-ai run MODEL_URL1 MODEL_URL2 ...

Performance Flags

Parameter	Default	Description	Environment Variable
--f16	false	Enable GPU acceleration	$LOCALAI_F16, $F16
-t, --threads		Number of threads used for parallel computation. Usage of the number of physical cores in the system is suggested	$LOCALAI_THREADS, $THREADS
--context-size		Default context size for models (-1 = each model's full trained context from GGUF metadata)	$LOCALAI_CONTEXT_SIZE, $CONTEXT_SIZE

API Flags

Parameter	Default	Description	Environment Variable
--address	:8080	Bind address for the API server	$LOCALAI_ADDRESS, $ADDRESS
--max-concurrent-backend-requests	1024	Process-wide ceiling for concurrent backend inference operations. Excess inference receives HTTP 429 with Retry-After; UI and administrative endpoints remain available	$LOCALAI_MAX_CONCURRENT_BACKEND_REQUESTS, $MAX_CONCURRENT_BACKEND_REQUESTS
--cors	false	Enable CORS (Cross-Origin Resource Sharing)	$LOCALAI_CORS, $CORS
--cors-allow-origins		Comma-separated list of allowed CORS origins	$LOCALAI_CORS_ALLOW_ORIGINS, $CORS_ALLOW_ORIGINS
--disable-csrf	false	Disable CSRF middleware (enabled by default)	$LOCALAI_DISABLE_CSRF
--disable-http-compression	false	Disable gzip compression of HTTP responses. Compression is enabled by default; streaming endpoints (streaming chat completions, SSE bridges, WebSocket upgrades) and already-compressed formats are never compressed	$LOCALAI_DISABLE_HTTP_COMPRESSION
--http-compression-min-length	1024	Minimum response size in bytes before gzip compression is applied. Smaller responses are sent as-is because the gzip envelope would outweigh the saving	$LOCALAI_HTTP_COMPRESSION_MIN_LENGTH
--upload-limit	15	Default upload-limit in MB	$LOCALAI_UPLOAD_LIMIT, $UPLOAD_LIMIT
--api-keys		List of API Keys to enable API authentication. When this is set, all requests must be authenticated with one of these API keys	$LOCALAI_API_KEY, $API_KEY
--disable-webui	false	Disables the web user interface. When set to true, the server will only expose API endpoints without serving the web interface	$LOCALAI_DISABLE_WEBUI, $DISABLE_WEBUI
--disable-runtime-settings	false	Disables the runtime settings feature. When set to true, the server will not load runtime settings from the runtime_settings.json file and the settings web interface will be disabled	$LOCALAI_DISABLE_RUNTIME_SETTINGS, $DISABLE_RUNTIME_SETTINGS
--disable-gallery-endpoint	false	Disable the gallery endpoints	$LOCALAI_DISABLE_GALLERY_ENDPOINT, $DISABLE_GALLERY_ENDPOINT
--disable-metrics-endpoint	false	Disable the /metrics endpoint	$LOCALAI_DISABLE_METRICS_ENDPOINT, $DISABLE_METRICS_ENDPOINT
--machine-tag		If not empty, add that string to Machine-Tag header in each response. Useful to track response from different machines using multiple P2P federated nodes	$LOCALAI_MACHINE_TAG, $MACHINE_TAG

Hardening Flags

Parameter	Default	Description	Environment Variable
--disable-predownload-scan	false	If true, disables the best-effort security scanner before downloading any files	$LOCALAI_DISABLE_PREDOWNLOAD_SCAN
--disable-backend-install	false	If true, refuse to install or upgrade backends from galleries, OCI images, URLs or paths, including automatic installs, so the running backend set stays exactly what was deployed and tested; only backends already present or provided via --backends-system-path are used	$LOCALAI_DISABLE_BACKEND_INSTALL, $DISABLE_BACKEND_INSTALL
--opaque-errors	false	If true, all error responses are replaced with blank 500 errors. This is intended only for hardening against information leaks and is normally not recommended	$LOCALAI_OPAQUE_ERRORS
--use-subtle-key-comparison	false	If true, API Key validation comparisons will be performed using constant-time comparisons rather than simple equality. This trades off performance on each request for resilience against timing attacks	$LOCALAI_SUBTLE_KEY_COMPARISON
--disable-api-key-requirement-for-http-get	false	If true, a valid API key is not required to issue GET requests to portions of the web UI. This should only be enabled in secure testing environments	$LOCALAI_DISABLE_API_KEY_REQUIREMENT_FOR_HTTP_GET
--http-get-exempted-endpoints	^/$,^/app(/.*)?$,^/browse(/.*)?$,^/login/?$,^/explorer/?$,^/assets/.*$,^/static/.*$,^/swagger.*$	If --disable-api-key-requirement-for-http-get is overridden to true, this is the list of endpoints to exempt. Only adjust this in case of a security incident or as a result of a personal security posture review	$LOCALAI_HTTP_GET_EXEMPTED_ENDPOINTS

Authentication Flags

Parameter	Default	Description	Environment Variable
--auth-enabled	false	Enable user authentication and authorization	$LOCALAI_AUTH
--auth-database-url	{DataPath}/database.db	Database URL for auth - postgres://... for PostgreSQL, or a file path for SQLite	$LOCALAI_AUTH_DATABASE_URL, $DATABASE_URL
--github-client-id		GitHub OAuth App Client ID (auto-enables auth when set)	$GITHUB_CLIENT_ID
--github-client-secret		GitHub OAuth App Client Secret	$GITHUB_CLIENT_SECRET
--oidc-issuer		OIDC issuer URL for auto-discovery	$LOCALAI_OIDC_ISSUER
--oidc-client-id		OIDC Client ID (auto-enables auth when set)	$LOCALAI_OIDC_CLIENT_ID
--oidc-client-secret		OIDC Client Secret	$LOCALAI_OIDC_CLIENT_SECRET
--auth-base-url		Base URL for OAuth callbacks (e.g. http://localhost:8080)	$LOCALAI_BASE_URL
--auth-admin-email		Email address to auto-promote to admin role on login	$LOCALAI_ADMIN_EMAIL
--auth-registration-mode	open	Registration mode: open, approval, or invite	$LOCALAI_REGISTRATION_MODE
--disable-local-auth	false	Disable local email/password registration and login (for OAuth/OIDC-only setups)	$LOCALAI_DISABLE_LOCAL_AUTH

See [Authentication & Authorization]({{%relref "features/authentication" %}}) for full documentation.

Chat Flags

Use local-ai chat to run the built-in terminal agent against a LocalAI server. The agent can run shell commands, delegate to sub-agents, and use MCP tools. Read-only calls run on their own; everything else goes through an approval prompt you answer. See [Terminal agent]({{% relref "features/terminal-agent" %}}) for the full feature page.

Parameter	Default	Description	Environment Variable
--endpoint	http://127.0.0.1:8080	LocalAI server endpoint. The /v1 path is added automatically when omitted.	$LOCALAI_CHAT_ENDPOINT
--model		Model to use. Defaults to the only model the server offers, or asks when there are several.
--api-key		API key to use when the LocalAI server requires authentication.	$LOCALAI_API_KEY, $API_KEY
--config-dir	~/.config/localai/chat	Directory holding the agent's config, plugins, and skills.	$LOCALAI_CHAT_CONFIG_DIR
--trace-dir		Write a session LLM trace (NDJSON) to this directory.	$LOCALAI_CHAT_TRACE_DIR
--cli	false	Plain CLI mode instead of the full-screen interface. Pipe stdin for one-shot use.
--tui	false	Force the full-screen interface.
--height		Run as an inline drop-down of this height, e.g. 40%.
--tmux / --no-tmux		Control the tmux split.
--init		Print the shell integration script for Ctrl+Space (zsh, bash, or fish).
--yolo	false	Auto-approve every tool call. Use with care.	$LOCALAI_CHAT_YOLO

The agent's own subcommands are reached by passing them through. LocalAI's flags must come first, because everything after the first positional argument is forwarded verbatim:

local-ai chat plugin install https://github.com/user/plugin
local-ai chat skill list
local-ai chat mcp add my-server -- npx -y @modelcontextprotocol/server-filesystem /tmp
local-ai chat --config-dir /srv/agent plugin list   # flags first

{{% notice warning %}} In a script, pass --yes to those subcommands. Without it, a non-interactive plugin install installs the plugin, leaves it disabled, and still exits 0. {{% /notice %}}

Summon the agent from any shell prompt with Ctrl+Space:

echo 'eval "$(local-ai chat --init zsh)"' >> ~/.zshrc

Inside a session:

• /models lists the models the server offers, marking the current one.
• /model <name> switches model, keeping the conversation.
• /compact summarizes the conversation so far to free up context.
• /skill <name>, /agent <name> <task>, /attach <file>, /goal <text>.

If no server is reachable, the agent offers to start one for the session (interactive terminals only) and otherwise points you at local-ai run.

Piped use needs --cli, and exits 0 when it answers:

echo "what is 2+2" | local-ai chat --cli

Read-only tools still run in a piped session. A tool call that is not read-only has nobody to approve it, so it is denied and the session exits 3, a code chosen to be distinct from the 1 a failure reports.

P2P Flags

Parameter	Default	Description	Environment Variable
--p2p	false	Enable P2P mode	$LOCALAI_P2P, $P2P
--p2p-dht-interval	360	Interval for DHT refresh (used during token generation)	$LOCALAI_P2P_DHT_INTERVAL, $P2P_DHT_INTERVAL
--p2p-otp-interval	9000	Interval for OTP refresh (used during token generation)	$LOCALAI_P2P_OTP_INTERVAL, $P2P_OTP_INTERVAL
--p2ptoken		Token for P2P mode (optional)	$LOCALAI_P2P_TOKEN, $P2P_TOKEN, $TOKEN
--p2p-network-id		Network ID for P2P mode, can be set arbitrarily by the user for grouping a set of instances	$LOCALAI_P2P_NETWORK_ID, $P2P_NETWORK_ID
--federated	false	Enable federated instance	$LOCALAI_FEDERATED, $FEDERATED

Library sources

local-ai library manages the sources LocalAI reads add-ons, backends and models from. The sources are stored under the data path (addons/sources.json), the file the server reads, so the commands work with the server stopped.

Command	What it does
local-ai library list	List every source with its kind, id, URL, and whether it is enabled, a shipped default, or removed
local-ai library add <url> --kind addon|backend|model [--id ID] [--name NAME] [--format library|gallery]	Add a source of your own
local-ai library remove <id>	Remove a source. A removed default leaves a tombstone, so it does not come back at the next start
local-ai library enable <id>	Turn a source on. This also brings back a removed default
local-ai library disable <id>	Turn a source off without removing it

All local-ai library commands accept --official-libraries (LOCALAI_OFFICIAL_LIBRARIES, default true). With --official-libraries=false the list shows the shipped default sources as off. The same switch is the runtime setting official_libraries and the toggle "Use official libraries" in Settings. See Library sources.

When two enabled sources list the same entry (same kind and id), your own source wins over a shipped default. If two of your own sources list it, the entry is refused and the message names both sources: disable or remove one of them. The --galleries and --backend-galleries flags keep working: each gallery they name is added as a source of your own (kind model or backend, format gallery) at startup, unless it is already a source.

Other Commands

LocalAI supports several subcommands beyond run:

• local-ai models - Manage LocalAI models and definitions
• local-ai backends - Manage LocalAI backends and definitions
• local-ai chat - Run the built-in terminal agent against a LocalAI server
• local-ai tts - Convert text to speech
• local-ai sound-generation - Generate audio files from text or audio
• local-ai transcript - Convert audio to text
• local-ai agent - Run agents standalone without the full LocalAI server
• local-ai mcp-server - Run the LocalAI admin tool surface as a stdio MCP server (controls a remote LocalAI instance over HTTP)
• local-ai worker - Start a worker for distributed mode (generic, backend-agnostic)
• local-ai p2p-worker - Run workers to distribute workload via p2p (llama.cpp-only)
• local-ai agent-worker - Start an agent worker for distributed mode (executes agent chats via NATS)
• local-ai util - Utility commands
• local-ai explorer - Run P2P explorer
• local-ai federated - Run LocalAI in federated mode
• local-ai completion - Generate shell completion scripts for bash, zsh, or fish

Use local-ai <command> --help for more information on each command.

Add-on carrier keys

local-ai util addons-carrier-keys makes the key set that a separate add-on owner and its frontends share. See [Running the owner as its own workload]({{% relref "features/addons#running-the-owner-as-its-own-workload" %}}) for how the set is used.

Flag	Description
--out	Directory to write the key set to. Required. The command fails if the directory already holds any of the files, and it never overwrites a file.
--owner-host	DNS name or IP address of the add-on owner, as the frontends dial it. Required. Repeat the flag for more than one.

local-ai util addons-carrier-keys --out ./carrier-keys --owner-host addons.owner.example

The command prints which files go to the owner and which go to the frontends. It writes the files with mode 0600 in a directory with mode 0700, and it does not keep the CA key.

Examples

Basic Usage

./local-ai run

./local-ai run --models-path /path/to/models --address :9090

./local-ai run --f16

Environment Variables

export LOCALAI_MODELS_PATH=/path/to/models
export LOCALAI_ADDRESS=:9090
export LOCALAI_F16=true
./local-ai run

Advanced Configuration

./local-ai run \
  --models model1.yaml model2.yaml \
  --enable-watchdog-idle \
  --watchdog-idle-timeout=10m \
  --p2p \
  --federated

Related Documentation

• See [Advanced Usage]({{%relref "advanced/advanced-usage" %}}) for configuration examples
• See [VRAM and Memory Management]({{%relref "advanced/vram-management" %}}) for memory management options