LocalAI · Distributed

Federated vs worker mode

TWO

modes

Federated routes whole requests to one node; worker shards one model across machines.

localai.io/features/distributed_inferencing