feat(transformers): add embeddings with Automodel (#1308)

* Update huggingface.py

Switch SentenceTransformer for AutoModel in order to set trust_remote_code needed to use the encode method with embeddings models like jinai-v2

Signed-off-by: Lucas Hänke de Cansino <lhc@next-boss.eu>

* feat(transformers): split in separate backend

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Lucas Hänke de Cansino <lhc@next-boss.eu>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Lucas Hänke de Cansino <lhc@next-boss.eu>
This commit is contained in:
Ettore Di GiacintoandLucas Hänke de Cansino authored and GitHub committed 2023-11-20 21:21:17 +01:00
1 parent ff9afdb0fe
commit 92cbc4d516
23 files changed
+783 -34

No files matched your search

+5
View File
@@ -0,0 +1,5 @@
# Creating a separate environment for the transformers project
```
make transformers
```