LocalAI · SmartRouter

How the router places a request

IDLE

first

Idle-first placement with preemptive least-recently-used eviction.

localai.io/features/distributed-mode