An eviction untracks the backend before it stops it. When the stop failed,
the error was only logged: the backend kept running and kept its VRAM, but
the watchdog had already forgotten it. Group and LRU enforcement then saw no
conflict, and the next load started next to it and ran out of memory.
Stopping a model takes its exclusive lifecycle lock, which a model that is
still loading holds for the whole load. The stop then times out ("waiting to
shut down model: context deadline exceeded"), and the watchdog no longer
knows the model is there.
The tracking state is now captured before the backend is untracked, and put
back when the stop fails. This applies to every eviction path that untracks
first: group, LRU, idle timeout and busy timeout. A model that is already
gone (ErrModelNotFound) still counts as stopped, and a replacement
registered on the same address in the meantime is left alone. EvictedCount
only counts backends that really stopped, and NeedMore is set when one did
not, so the loader keeps retrying instead of treating the room as made. The
idle and busy checks retry the stop on their next tick.
The state is only put back while the loader still holds the model at that
address (ModelLoader.HasModelAtAddress). deleteProcess drops a remote-only
model, or the local half of a local+remote placement, before it reports a
failed remote unload; tracking that model again would leave a phantom entry
counted as loaded and as a group conflict. The memory reclaimer applies the
same rule and untracks a model the loader already dropped.
The untrack-before-stop ordering dates from #7535, so this is not a
regression of 4.12.0.
Assisted-by: Claude:claude-sonnet-5.5 go vet
Signed-off-by: Hugues CHARBONNIER <hugues.charbonnier@gmail.com>
(cherry picked from commit 97390f7754)
