mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-04-04 23:14:41 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix(p2p): adapt to backend changes, general improvements (#5889)

Browse Source 
The binary is now named "llama-cpp-rpc-server" for p2p workers.

We also decrease the default token rotation interval, in this way
peer discovery is much more responsive.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto
2025-07-23 12:40:32 +02:00
committed by GitHub
parent c717b8d800
commit 5f7ece3e94
5 changed files with 63 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/content/docs/features/distributed_inferencing.md
View File

@@ -152,6 +152,14 @@ LocalAI uses https://github.com/libp2p/go-libp2p under the hood, the same projec
The weights are split proportional to the memory when running into worker mode, when in federation mode each request is split to every node which have to load the model fully.
## Debugging
To debug, it's often useful to run in debug mode, for instance:
```
LOCALAI_P2P_LOGLEVEL=debug LOCALAI_P2P_LIB_LOGLEVEL=debug LOCALAI_P2P_ENABLE_LIMITS=true LOCALAI_P2P_DISABLE_DHT=true LOCALAI_P2P_TOKEN="<TOKEN>" ./local-ai ...
```
## Notes
- If running in p2p mode with container images, make sure you start the container with `--net host` or `network_mode: host` in the docker-compose file.