Wyatt Neal
4076ea0494
fix: vllm missing logprobs ( #5279 )
...
* working to address missing items
referencing #3436 , #2930 - if i could test it, this might show that the
output from the vllm backend is processed and returned to the user
Signed-off-by: Wyatt Neal <wyatt.neal+git@gmail.com >
* adding in vllm tests to test-extras
Signed-off-by: Wyatt Neal <wyatt.neal+git@gmail.com >
* adding in tests to pipeline for execution
Signed-off-by: Wyatt Neal <wyatt.neal+git@gmail.com >
* removing todo block, test via pipeline
Signed-off-by: Wyatt Neal <wyatt.neal+git@gmail.com >
---------
Signed-off-by: Wyatt Neal <wyatt.neal+git@gmail.com >
2025-04-30 12:55:07 +00:00
Ettore Di Giacinto
8abecb4a18
chore: bump grpc limits to 50MB ( #5212 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2025-04-19 08:53:24 +02:00
Brandon Beiler
6a6e1a0ea9
feat(vllm): Additional vLLM config options (Disable logging, dtype, and Per-Prompt media limits) ( #4855 )
...
* Adding the following vLLM config options: disable_log_status, dtype, limit_mm_per_prompt
Signed-off-by: TheDropZone <brandonbeiler@gmail.com >
* using " marks in the config.yaml file
Signed-off-by: TheDropZone <brandonbeiler@gmail.com >
* adding in missing colon
Signed-off-by: TheDropZone <brandonbeiler@gmail.com >
---------
Signed-off-by: TheDropZone <brandonbeiler@gmail.com >
2025-02-18 19:27:58 +01:00
Ettore Di Giacinto
ae1ec4e096
feat(vllm): expose 'load_format' ( #3943 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-10-23 15:34:57 +02:00
Ettore Di Giacinto
26c4058be4
fix(vllm): do not set videos if we don't have any ( #3885 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-10-20 11:44:28 +02:00
Ettore Di Giacinto
9db068388b
fix(vllm): images and videos are base64 by default ( #3867 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-10-17 17:32:57 +02:00
Ettore Di Giacinto
2553de0187
feat(vllm): add support for image-to-text and video-to-text ( #3729 )
...
* feat(vllm): add support for image-to-text
Related to https://github.com/mudler/LocalAI/issues/3670
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* feat(vllm): add support for video-to-text
Closes: https://github.com/mudler/LocalAI/issues/2318
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* feat(vllm): support CPU installations
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* feat(vllm): add bnb
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* chore: add docs reference
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* Apply suggestions from code review
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2024-10-04 23:42:05 +02:00
Ettore Di Giacinto
68fc014c6d
feat(vllm): add support for embeddings ( #3440 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-09-02 21:44:32 +02:00
cryptk
e2de8a88f7
feat: create bash library to handle install/run/test of python backends ( #2286 )
...
* feat: create bash library to handle install/run/test of python backends
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* chore: minor cleanup
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* fix: remove incorrect LIMIT_TARGETS from parler-tts
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* fix: update runUnitests to handle running tests from a custom test file
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* chore: document runUnittests
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
---------
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
2024-05-11 18:32:46 +02:00
