mirror/LocalAI
1
0 Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-09-25 23:54:56 -04:00
Code Issues Packages Projects Releases Wiki Activity
Files
fix_eos
LocalAI/backend/python/common-env/transformers
T
History
fakezeta 3882130911 feat: Add Bitsandbytes quantization for transformer backend enhancement #1775 and fix: Transformer backend error on CUDA #1774 (#1823) 
* fixes #1775 and #1774

Add BitsAndBytes Quantization and fixes embedding on CUDA devices

* Manage 4bit and 8 bit quantization

Manage different BitsAndBytes options with the quantization: parameter in yaml

* fix compilation errors on non CUDA environment
2024-03-14 23:06:30 +01:00
..
install.sh
feat(intel): add diffusers/transformers support (#1746)
2024-03-07 14:37:45 +01:00
Makefile
feat(intel): add diffusers/transformers support (#1746)
2024-03-07 14:37:45 +01:00
transformers-nvidia.yml
feat: Add Bitsandbytes quantization for transformer backend enhancement #1775 and fix: Transformer backend error on CUDA #1774 (#1823)
2024-03-14 23:06:30 +01:00
transformers-rocm.yml
Update Backend Dependancies (#1797)
2024-03-05 10:10:00 +00:00
transformers.yml
Update Backend Dependancies (#1797)
2024-03-05 10:10:00 +00:00