mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-01-17 03:47:54 -05:00
Code Issues Packages Projects Releases Wiki Activity
1,058 Commits 4 Branches 194 Tags
e0632a85ede6205621561ba6f4115ee4fa5e76ce
Commit Graph

37 Commits

Author SHA1 Message Date
Aaron Pham
c41828f68f feat(client): support authentication token and shim implementation (#605) 
* chore: synch generate_iterator to be the same as server

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* --wip--

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* wip

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat: cleanup shim implementation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

* chore: fix pre-commit

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update check with tuple

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-10 17:44:31 -05:00
Aaron Pham
ac377fe490 infra: using ruff formatter (#594) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-09 12:44:05 -05:00
Aaron Pham
655a4071de fix: unbound variable for completion client (#587) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-08 05:37:39 -05:00
Aaron Pham
c40d4c1016 fix(examples): correct dependencies in requirements.txt [skip ci] (#575) 
Update requirements.txt

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-07 23:02:53 -05:00
Aaron Pham
30d8981916 chore(openai): simplify client examples (#574) 
* chore(openai): simplify client examples

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: fix spacing

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-07 23:01:14 -05:00
Aaron Pham
cfd09bfc47 chore(runner): yield the outputs directly (#573) 
update openai client examples to >1

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-07 22:34:11 -05:00
Aaron Pham
272b185ee7 chore(examples): update openai client (#568) 
chore: update openai client examples

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-07 16:42:32 -05:00
Aaron
8aca432df5 chore(examples): add logprobs check on PyTorch 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-06 20:52:49 -05:00
Aaron Pham
e2029c934b perf: unify LLM interface (#518) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-06 20:39:43 -05:00
xianxian.zhang
e1e365ddc1 fix: update llama2 notebook example (#516) 
* fix: update llama2 notebook example

* fix: signing commit test
 2023-10-31 15:41:43 +08:00
XunchaoZ
d9183267dc feat: openai.Model.list() (#499) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-10-14 16:33:49 -04:00
xianxian.zhang
bdfb28d412 feat: add llama2 notebook demo (#479) 2023-10-12 12:03:12 -04:00
Aaron
625b82a0fc fix(style): remove weird break on split item 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-10-07 02:21:31 -04:00
XunchaoZ
04bb29a264 feat: OpenAI-compatible API (#417) 
Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-10-07 00:50:03 -04:00
Aaron Pham
fddd0bf95e feat: bootstrap documentation site (#252) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: GutZuFusss <leon.ikinger@googlemail.com>
Co-authored-by: GutZuFusss <leon.ikinger@googlemail.com>
 2023-09-12 12:28:29 -04:00
aarnphm-ec2-dev
f917898867 chore: update langchain examples 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-09-08 03:35:10 +00:00
Aaron Pham
b7af7765d4 fix(yapf): align weird new lines break [generated] [skip ci] (#284) 
fix(yapf): align weird new lines break

Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-09-01 05:34:22 -04:00
Aaron
b545ad2ad1 style: google 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-08-30 13:52:35 -04:00
Aaron Pham
c9cef1d773 fix: persistent styling between ruff and yapf (#279) 2023-08-30 11:37:41 -04:00
aarnphm-ec2-dev
806a663e4a chore(style): add one blank line 
to conform with Google style

Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-08-26 11:36:57 +00:00
Aaron Pham
46c8904806 cron(style): run formatter [generated] [skip ci] (#257) 2023-08-25 06:38:59 -04:00
Aaron
6b0ab17018 chore: remove unnecessary headers 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-08-15 18:15:54 -04:00
Aaron Pham
5329853b10 perf: compiled modules and enable lazyeval (#200) 2023-08-11 05:53:45 -04:00
Aaron Pham
8c2867d26d style: define experimental guidelines (#168) 2023-07-31 07:54:26 -04:00
Aaron Pham
ef94c6b98a feat(container): vLLM build and base image strategies (#142) 2023-07-31 02:44:52 -04:00
Aaron Pham
c7f4dc7bb2 feat(test): snapshot testing (#107) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-07-10 17:23:19 -04:00
Aaron Pham
5a4df53490 fix(load): tokenizer and adapter within a BentoLLM (#88) 2023-06-28 15:45:25 -04:00
Aaron Pham
98328be394 peft(models): improve implementation (#60) 
If you have a local Dolly-V2 version, please do `openllm prune`
 2023-06-24 05:22:18 -04:00
Aaron Pham
03758a5487 fix(tools): adhere to style guidelines (#31) 2023-06-18 20:03:17 -04:00
Aaron Pham
ded8a9f809 feat: quantization (#27) 2023-06-16 18:10:50 -04:00
Aaron Pham
19bc7e3116 feat: fine-tuning [part 1] (#23) 2023-06-16 00:19:01 -04:00
Chaoyu
dc50a2e7e5 docs: add LangChain and BentoML Examples (#25) 
Co-authored-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-06-15 06:14:37 -04:00
Aaron
ec941c95d5 chore: add license header 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-06-04 16:22:37 -07:00
Aaron
78358dbb8d fix(type): configuration and dependencies 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 06:01:30 -07:00
Aaron
b7f3a10910 refactor: migrate __init_subclass__ to Metaclass 
LLMMetaclass will now responsible for generate internal attributes

add llm_type and identifying_params to Runnable class

subclass of openllm.LLM now can set a class attribute
__openllm_internal__ to let openllm knows that this is an internal class
implementation, instead of providing a _internal in the class
initialization.

support for preprocess_parameters and postprocess_parameters on client
side for better client UX

Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 03:09:45 +00:00
Aaron
d31d450526 feat: Adding central service definition and init openllm_client 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-15 00:33:05 -07:00
Chaoyu
dd8b6050b2 feat: FLAN-T5 supports 
- add infrastructure, to be implemented: cache, chat history

- Base Runnable Implementation, that fits LangChain API

- Added a Prompt descriptor and utils.

feat: license headers and auto factory impl and CLI

Auto construct args from pydantic config

Add auto factory for ease of use

only provide `/generate` to streamline UX experience

CLI > envvar > input contract for configuration

fix: serve from a thread

fix CLI args

chore: cleanup names and refactor imports

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-03 17:50:14 -07:00