mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-01-16 18:30:46 -05:00
Code Issues Packages Projects Releases Wiki Activity
5,379 Commits 73 Branches 133 Tags
master
Commit Graph

14 Commits

Author SHA1 Message Date
Ettore Di Giacinto
ffb2dc4666 chore(detection): detect GPU vendor from files present in the system (#7908) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2026-01-07 16:18:27 +01:00
Ettore Di Giacinto
185a685211 fix(amd-gpu): correctly show total and used vram (#7761) 
An example output of `rocm-smi --showproductname --showmeminfo vram --showuniqueid --csv`:

```
device,Unique ID,VRAM Total Memory (B),VRAM Total Used Memory (B),Card Series,Card Model,Card Vendor,Card SKU,Subsystem ID,Device Rev,Node ID,GUID,GFX Version
card0,0x9246____________,17163091968,692142080,Navi 21 [Radeon RX 6800/6800 XT / 6900 XT],0x73bf,Advanced Micro Devices Inc. [AMD/ATI],001,0x2406,0xc1,1,45534,gfx1030
card1,N/A,67108864,26079232,Raphael,0x164e,Advanced Micro Devices Inc. [AMD/ATI],RAPHAEL,0x364e,0xc6,2,52156,gfx1036
```

Total memory is actually showed before the total used memory as can be seen in https://github.com/LostRuins/koboldcpp/issues/1104#issuecomment-2321143507.

This PR fixes https://github.com/mudler/LocalAI/issues/7724

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-12-29 07:57:07 +01:00
Ettore Di Giacinto
c37785b78c chore(refactor): move logging to common package based on slog (#7668) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-12-21 19:33:13 +01:00
Ettore Di Giacinto
3ca90876f1 chore(memory detection): do not use go-sigar as requires CGO on darwin (#7618) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-12-16 23:10:42 +01:00
Ettore Di Giacinto
e3e5f59965 fix(ram): do not read from cgroup (#7606) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-12-16 13:28:11 +01:00
Ettore Di Giacinto
878c9d46d5 fix: improve ram estimation (#7603) 
* fix: default to 10seconds of watchdog if runtime setting is malformed

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix: use gosigar for RAM estimation

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-12-16 10:18:36 +01:00
Ettore Di Giacinto
50f9c9a058 feat(watchdog): add Memory resource reclaimer (#7583) 
* feat(watchdog): add GPU reclaimer

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Handle vram calculation for unified memory devices

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Support RAM eviction, set watchdog interval from runtime settings

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-12-16 09:15:18 +01:00
Ettore Di Giacinto
b034cff149 feat: improve RAM estimation by using values from summary (#5525) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-06-05 19:16:26 +02:00
Ettore Di Giacinto
159388cce8 chore: memoize detected GPUs (#5385) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-05-18 08:55:44 +02:00
Ettore Di Giacinto
72111c597d fix(gpu): do not assume gpu being returned has node and mem (#5310) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-05-03 19:00:24 +02:00
Ettore Di Giacinto
5c6cd50ed6 feat(llama.cpp): estimate vram usage (#5299) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-05-02 17:40:26 +02:00
Ettore Di Giacinto
9628860c0e feat(llama.cpp/clip): inject gpu options if we detect GPUs (#5243) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2025-04-26 00:04:47 +02:00
Ettore Di Giacinto
bdd6769b2d feat(default): use number of physical cores as default (#2483) 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2024-06-04 15:23:29 +02:00
Ettore Di Giacinto
b69ff46c7e feat(startup): show CPU/GPU information with --debug (#2241) 
Signed-off-by: mudler <mudler@localai.io>
 2024-05-05 09:10:23 +02:00