mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-03-05 07:36:15 -05:00
Code Issues Packages Projects Releases Wiki Activity
1,118 Commits 6 Branches 194 Tags
8fdfd0491fc6713e02f27816b657f82ea45eb37e
Commit Graph

1118 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Aaron Pham
8fdfd0491f perf(build): locking and improve build speed (#669) 
* revert(build): not locking packages

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* perf: improve svars generation and unifying envvar parsing

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* docs: update changelog

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: update stubs check for mypy

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-16 06:27:45 -05:00
Aaron Pham
fce8f223f3 perf: reduce footprint (#668) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-16 04:45:49 -05:00
Aaron Pham
9e3f0fea15 types: update stubs for remaining entrypoints (#667) 
* perf(type): static OpenAI types definition

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat: add hf types

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* types: update remaining missing stubs

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-16 04:26:13 -05:00
Aaron Pham
6102a67a83 infra: makes huggingface-hub requirements on fine-tune (#665) 
infra: makes huggingface-hub core deps

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-16 03:12:52 -05:00
Aaron Pham
86d23fd6f5 feat(llm): respect warnings environment for dtype warning (#664) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-16 03:05:58 -05:00
Aaron Pham
4a6f13ddd2 feat(type): provide structured annotations stubs (#663) 
* feat(type): provide client stubs

separation of concern for more brevity code base

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* docs: update changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-16 02:58:45 -05:00
xianxian.zhang
c6264f3af7 fix(examples): update notebook with new API (#662) 2023-11-15 22:28:40 -05:00
Kuan-Chun Wang
af88b9b077 fix(runner): remove keyword args for attrs.get() (#661) 2023-11-15 04:59:01 -05:00
Aaron Pham
c05f405163 infra: bump to homebrew tap release to 0.4.9 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 08:02:38 +00:00
Aaron Pham
7c64ffea0f infra: bump to dev version of 0.4.10.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 08:01:40 +00:00
Aaron Pham
68c6a9dac6 infra: prepare for release 0.4.9 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.9 		2023-11-15 07:51:30 +00:00
Aaron Pham
876586a30e fix(falcon): remove early_stopping default arguments (#660) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 02:49:54 -05:00
Aaron Pham
9e6df0df89 chore: update requirements in README.md (#659) 
chore: update requirements

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 02:32:36 -05:00
Aaron Pham
034e08cf08 infra: update scripts to run update readme automatically (#658) 
* infra: update scripts to run update readme automatically

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: cleanup mirror

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore(dropdown): correctly format noteblock and important block

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: whitespace aware

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 02:22:49 -05:00
Aaron Pham
2ea2f3fd4f infra: bump to homebrew tap release to 0.4.8 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 05:31:05 +00:00
Aaron Pham
b90e44f679 infra: bump to dev version of 0.4.9.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 05:29:44 +00:00
Aaron Pham
625afd0c3b infra: prepare for release 0.4.8 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.8 		2023-11-15 05:19:22 +00:00
Aaron Pham
a58d947bc8 perf: improve build logics and cleanup speed (#657) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 00:18:31 -05:00
Aaron Pham
103156cd71 chore(cli): move playground to CLI components (#655) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 23:20:50 -05:00
Aaron
cbdcfc87a2 infra: remove cohere examples 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 23:15:31 -05:00
Aaron Pham
c5f8602d4c docs: update instruction adding new models and remove command docstring (#654) 
docs: update instruction adding new models and remove command
docstring

as start will just support model_id directly, there is no need to
support custom docstring anymore

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 23:11:16 -05:00
Aaron Pham
ac0de3e44e infra: bump to homebrew tap release to 0.4.7 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 03:56:04 +00:00
Aaron Pham
88c898ffad infra: bump to dev version of 0.4.8.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 03:54:34 +00:00
Aaron Pham
b40874149e infra: prepare for release 0.4.7 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.7 		2023-11-15 03:44:09 +00:00
xianxian.zhang
ea02aaaa23 fix: correct OPENLLM_DEV_BUILD check (#653) 2023-11-14 22:21:37 -05:00
Aaron Pham
6a6d689a77 feat: Yi models (#651) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 21:55:24 -05:00
Aaron Pham
b4b70e2f20 fix(cli): update context name parsing correctly (#652) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 21:53:56 -05:00
Aaron
9eddae83a6 infra: update cohere client 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 01:52:38 -05:00
Aaron Pham
31a799ff61 refactor: use DEBUG env-var instead of OPENLLMDEVDEBUG (#647) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 01:39:58 -05:00
Aaron Pham
d63da6a5cb infra: bump to homebrew tap release to 0.4.6 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 06:31:46 +00:00
Aaron Pham
2462c1ad73 infra: bump to dev version of 0.4.7.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 06:30:29 +00:00
Aaron Pham
145aafd386 infra: prepare for release 0.4.6 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.6 		2023-11-14 06:18:14 +00:00
Aaron Pham
00d6016bcb chore(openapi): unify inject param (#645) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 01:16:20 -05:00
Aaron Pham
b0ab8ccdf6 experimental: Cohere compatible endpoints. (#644) 
* feat: add generate endpoint

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update generation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix(cohere): generate endpoints

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: --wip--

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat: update testing clients and chat implementation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: disable schemas for easter eggs

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 01:07:43 -05:00
Aaron Pham
0bf6ec7537 fix(dependencies): lock build < 1 for now (#643) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 00:36:08 -05:00
Aaron Pham
b30a412398 fix(cli): set default dtype to auto infer (#642) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 23:05:27 -05:00
Aaron Pham
99a5d26527 fix(service): to yield out correct JSON objects (#640) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 22:41:52 -05:00
Aaron Pham
2d428f12da fix(cpu): more verbose definition for dtype casting (#639) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 20:40:50 -05:00
Aaron Pham
b20c7d1c1d fix(generation): compatibility dtype with CPU (#638) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 20:32:07 -05:00
Zhao Shenyang
ae69524749 doc: update adding new model guide (#637) 
* update

* Update openllm-python/ADDING_NEW_MODEL.md

Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Zhao Shenyang <dev@zsy.im>

* Update openllm-python/ADDING_NEW_MODEL.md

Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Zhao Shenyang <dev@zsy.im>

* Update openllm-python/ADDING_NEW_MODEL.md

Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Zhao Shenyang <dev@zsy.im>

* move ADDING_NEW_MODEL.md to git root directory

---------

Signed-off-by: Zhao Shenyang <dev@zsy.im>
Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 18:30:44 -05:00
Aaron Pham
853def95cd fix(client): correct destructor the httpx object boht sync and async (#636) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 18:11:17 -05:00
Aaron Pham
de06557844 docs: update README.md (#635) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 17:37:27 -05:00
Aaron Pham
af84462f27 infra: remove unused postprocess_generate (#634) 
This is currently a noop

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 17:35:39 -05:00
Zhao Shenyang
f202fddce8 perf(model): update mistral inference parameters and prompt format (#632) 
* feat(model): add initial mistral support

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

* chore: update with recent refactor

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 17:32:16 -05:00
Aaron Pham
a6387d1d15 chore: cleanup unused code path (#633) 
we now rely on tokenizer.chat_templates to format prompts correctly

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 17:23:07 -05:00
Aaron Pham
67ee492715 infra: bump to homebrew tap release to 0.4.5 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 19:01:38 +00:00
Aaron Pham
16a644411f infra: bump to dev version of 0.4.6.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 19:00:32 +00:00
Aaron Pham
a0d74017cd infra: prepare for release 0.4.5 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.5 		2023-11-13 18:49:31 +00:00
Aaron Pham
d358e68539 fix(torch_dtype): load eagerly (#631) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 13:48:04 -05:00
Aaron Pham
0924c0b34d infra: removing clojure frontend from infra cycle (#630) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 13:29:34 -05:00