mudler's LocalAI [bot]andEttore Di Giacinto
8052c950cf
A container engine started from a socket-activated system unit leaks a bare
LISTEN_PID into every container it spawns, with no matching LISTEN_FDS. LocalAI
read that as a malformed activation attempt and refused to start:
ERROR Error running the application error=loading systemd socket
activation listeners: invalid LISTEN_FDS ""
systemd's own sd_listen_fds() treats either variable being absent as "not
activated" rather than as an error, so do the same and fall back to ordinary
--address binding. A value that is present but malformed is still rejected, so
a real activation attempt cannot silently bind the wrong socket.
Fixes #11390
Assisted-by: Claude:claude-opus-5 [golangci-lint]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
3.4 KiB
title, description, weight, url
| title | description | weight | url |
|---|---|---|---|
| Linux Installation | Install LocalAI on Linux using binaries | 9 | /installation/linux/ |
Manual Installation
Download Binary
You can manually download the appropriate binary for your system from the releases page:
- Go to GitHub Releases
- Download the binary for your architecture (amd64, arm64, etc.)
- Make it executable:
chmod +x local-ai-*
- Run LocalAI:
./local-ai-*
Run your first model
Starting the binary on its own gives you an empty server. To get a working chat right away, run LocalAI with a model name and it will download and serve it from the gallery:
./local-ai-* run qwen3-4b
Once it is ready, open the WebUI at http://localhost:8080 or send a request to the API:
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "qwen3-4b",
"messages": [{"role": "user", "content": "Hello!"}]
}'
System Requirements
Hardware requirements vary based on:
- Model size
- Quantization method
- Backend used
For performance benchmarks with different backends like llama.cpp, visit this link.
Configuration
After installation, you can:
- Access the WebUI at
http://localhost:8080 - Configure models in the models directory
- Customize settings via environment variables or config files
Start LocalAI on demand with systemd
LocalAI accepts a single TCP listener passed through the systemd socket activation protocol. This lets systemd listen on the public port and start LocalAI only when the first client connects.
Create /etc/systemd/system/local-ai.socket:
[Unit]
Description=LocalAI API socket
[Socket]
ListenStream=8080
NoDelay=true
[Install]
WantedBy=sockets.target
Create the matching /etc/systemd/system/local-ai.service:
[Unit]
Description=LocalAI
[Service]
Type=simple
User=localai
Group=localai
ExecStart=/usr/local/bin/local-ai run
WorkingDirectory=/var/lib/local-ai
Adjust the user, binary path, working directory, and model configuration for your installation. Then enable the socket, not the service:
sudo systemctl daemon-reload
sudo systemctl enable --now local-ai.socket
The first connection to port 8080 starts local-ai.service; systemd holds that
connection until LocalAI is ready to accept it. LOCALAI_ADDRESS and
--address are ignored while an inherited listener is present. LocalAI
rejects activation with multiple stream listeners so it cannot silently choose
the wrong endpoint.
For a Podman-managed container, configure Podman to preserve and pass the systemd socket file descriptor into the container. The LocalAI process inside the container consumes the same activation protocol.
Activation needs both LISTEN_PID and LISTEN_FDS. If only one of them is set,
LocalAI ignores them and binds --address as usual. A container engine started
from a socket-activated system unit can leak a bare LISTEN_PID into every
container it spawns, and that is not an activation attempt.