Ettore Di Giacinto fc5b9ebfcc feat(loader): enhance single active backend to support LRU eviction (#7535) ...

* feat(loader): refactor single active backend support to LRU

This changeset introduces LRU management of loaded backends. Users can
set now a maximum number of models to be loaded concurrently, and, when
setting LocalAI in single active backend mode we set LRU to 1 for
backward compatibility.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore: add tests

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Update docs

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Fixups

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

2025-12-12 12:28:38 +01:00

..

_index.en.md

feat: docs revamp (#7313)

2025-11-19 22:21:20 +01:00

architecture.md

feat: docs revamp (#7313)

2025-11-19 22:21:20 +01:00

binaries.md

feat: docs revamp (#7313)

2025-11-19 22:21:20 +01:00

cli-reference.md

feat(loader): enhance single active backend to support LRU eviction (#7535)

2025-12-12 12:28:38 +01:00

compatibility-table.md

feat: docs revamp (#7313)

2025-11-19 22:21:20 +01:00

nvidia-l4t.md

feat: docs revamp (#7313)

2025-11-19 22:21:20 +01:00