mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-01-25 15:57:51 -05:00
Code Issues Packages Projects Releases Wiki Activity
1,574 Commits 6 Branches 194 Tags
a67ccdfd314158ccde3a5a3678622b6da279b1aa
Commit Graph

53 Commits

Author SHA1 Message Date
Aaron Pham
e9e46b2cc7 chore: update examples and readme 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2024-05-29 00:41:32 -04:00
paperspace
ef11e54a6d chore: update docs and base instruction [skip ci] 
Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>
 2024-05-29 03:19:47 +00:00
Aaron Pham
97d76eec85 tests: add additional basic testing (#982) 
* chore: update rebase tests

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

* chore: update partial clients before removing

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

* fix: update clients parsing logics to work with 0.5

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

* chore: ignore ci runs as to run locally

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

* chore: update async client tests

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

* chore: update pre-commit

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>
 2024-05-23 10:02:23 -04:00
paperspace
1d2e554a94 chore: disable progressbar for cleaner log trace 
Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>
 2024-05-10 03:11:47 +00:00
Aaron Pham
d53cf234bd fix(api-server): correct set generation from LLM class 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 10:38:36 +00:00
Aaron Pham
2821e172ef fix(examples): use non-chat models 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 10:12:48 +00:00
Aaron Pham
41c857f292 fix: set correct type annotations 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 07:17:38 +00:00
Aaron Pham
539f250c0f feat(vllm): bump to 0.2.2 (#695) 
* feat(vllm): bump to 0.2.2

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: move up to CUDA 12.1

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: remove auto-gptq installation

since the builder image doesn't have access to GPU

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: update containerization warning

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-19 02:52:32 -05:00
Aaron Pham
099cc22a94 chore: update documentation (#693) 
* chore: update documentation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update readme

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update documentations for configuration

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-18 19:44:52 -05:00
Aaron Pham
e01f93f0c3 examples: improve instructions and cleanup simple API server (#684) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-17 11:53:56 -05:00
Aaron Pham
c9daf4b5cb fix(examples): add support for streaming feature (#677) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-17 08:32:28 -05:00
xianxian.zhang
c6264f3af7 fix(examples): update notebook with new API (#662) 2023-11-15 22:28:40 -05:00
Aaron
cbdcfc87a2 infra: remove cohere examples 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 23:15:31 -05:00
Aaron
9eddae83a6 infra: update cohere client 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 01:52:38 -05:00
Aaron Pham
b0ab8ccdf6 experimental: Cohere compatible endpoints. (#644) 
* feat: add generate endpoint

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update generation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix(cohere): generate endpoints

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: --wip--

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat: update testing clients and chat implementation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: disable schemas for easter eggs

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 01:07:43 -05:00
Aaron Pham
e77a7fb2a4 chore: update jupyter notebooks with new API (#623) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 21:07:28 -05:00
Aaron Pham
c41828f68f feat(client): support authentication token and shim implementation (#605) 
* chore: synch generate_iterator to be the same as server

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* --wip--

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* wip

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat: cleanup shim implementation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

* chore: fix pre-commit

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update check with tuple

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-10 17:44:31 -05:00
Aaron Pham
ac377fe490 infra: using ruff formatter (#594) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-09 12:44:05 -05:00
Aaron Pham
655a4071de fix: unbound variable for completion client (#587) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-08 05:37:39 -05:00
Aaron Pham
c40d4c1016 fix(examples): correct dependencies in requirements.txt [skip ci] (#575) 
Update requirements.txt

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-07 23:02:53 -05:00
Aaron Pham
30d8981916 chore(openai): simplify client examples (#574) 
* chore(openai): simplify client examples

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: fix spacing

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-07 23:01:14 -05:00
Aaron Pham
cfd09bfc47 chore(runner): yield the outputs directly (#573) 
update openai client examples to >1

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-07 22:34:11 -05:00
Aaron Pham
272b185ee7 chore(examples): update openai client (#568) 
chore: update openai client examples

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-07 16:42:32 -05:00
Aaron
8aca432df5 chore(examples): add logprobs check on PyTorch 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-06 20:52:49 -05:00
Aaron Pham
e2029c934b perf: unify LLM interface (#518) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-06 20:39:43 -05:00
xianxian.zhang
e1e365ddc1 fix: update llama2 notebook example (#516) 
* fix: update llama2 notebook example

* fix: signing commit test
 2023-10-31 15:41:43 +08:00
XunchaoZ
d9183267dc feat: openai.Model.list() (#499) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-10-14 16:33:49 -04:00
xianxian.zhang
bdfb28d412 feat: add llama2 notebook demo (#479) 2023-10-12 12:03:12 -04:00
Aaron
625b82a0fc fix(style): remove weird break on split item 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-10-07 02:21:31 -04:00
XunchaoZ
04bb29a264 feat: OpenAI-compatible API (#417) 
Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-10-07 00:50:03 -04:00
Aaron Pham
fddd0bf95e feat: bootstrap documentation site (#252) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: GutZuFusss <leon.ikinger@googlemail.com>
Co-authored-by: GutZuFusss <leon.ikinger@googlemail.com>
 2023-09-12 12:28:29 -04:00
aarnphm-ec2-dev
f917898867 chore: update langchain examples 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-09-08 03:35:10 +00:00
Aaron Pham
b7af7765d4 fix(yapf): align weird new lines break [generated] [skip ci] (#284) 
fix(yapf): align weird new lines break

Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-09-01 05:34:22 -04:00
Aaron
b545ad2ad1 style: google 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-08-30 13:52:35 -04:00
Aaron Pham
c9cef1d773 fix: persistent styling between ruff and yapf (#279) 2023-08-30 11:37:41 -04:00
aarnphm-ec2-dev
806a663e4a chore(style): add one blank line 
to conform with Google style

Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-08-26 11:36:57 +00:00
Aaron Pham
46c8904806 cron(style): run formatter [generated] [skip ci] (#257) 2023-08-25 06:38:59 -04:00
Aaron
6b0ab17018 chore: remove unnecessary headers 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-08-15 18:15:54 -04:00
Aaron Pham
5329853b10 perf: compiled modules and enable lazyeval (#200) 2023-08-11 05:53:45 -04:00
Aaron Pham
8c2867d26d style: define experimental guidelines (#168) 2023-07-31 07:54:26 -04:00
Aaron Pham
ef94c6b98a feat(container): vLLM build and base image strategies (#142) 2023-07-31 02:44:52 -04:00
Aaron Pham
c7f4dc7bb2 feat(test): snapshot testing (#107) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-07-10 17:23:19 -04:00
Aaron Pham
5a4df53490 fix(load): tokenizer and adapter within a BentoLLM (#88) 2023-06-28 15:45:25 -04:00
Aaron Pham
98328be394 peft(models): improve implementation (#60) 
If you have a local Dolly-V2 version, please do `openllm prune`
 2023-06-24 05:22:18 -04:00
Aaron Pham
03758a5487 fix(tools): adhere to style guidelines (#31) 2023-06-18 20:03:17 -04:00
Aaron Pham
ded8a9f809 feat: quantization (#27) 2023-06-16 18:10:50 -04:00
Aaron Pham
19bc7e3116 feat: fine-tuning [part 1] (#23) 2023-06-16 00:19:01 -04:00
Chaoyu
dc50a2e7e5 docs: add LangChain and BentoML Examples (#25) 
Co-authored-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-06-15 06:14:37 -04:00
Aaron
ec941c95d5 chore: add license header 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-06-04 16:22:37 -07:00
Aaron
78358dbb8d fix(type): configuration and dependencies 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 06:01:30 -07:00