mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-01-12 17:40:54 -05:00
Code Issues Packages Projects Releases Wiki Activity
1,118 Commits 4 Branches 194 Tags
8fdfd0491fc6713e02f27816b657f82ea45eb37e
Commit Graph

154 Commits

Author SHA1 Message Date
Aaron Pham
8fdfd0491f perf(build): locking and improve build speed (#669) 
* revert(build): not locking packages

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* perf: improve svars generation and unifying envvar parsing

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* docs: update changelog

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: update stubs check for mypy

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-16 06:27:45 -05:00
Aaron Pham
fce8f223f3 perf: reduce footprint (#668) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-16 04:45:49 -05:00
Aaron Pham
9e3f0fea15 types: update stubs for remaining entrypoints (#667) 
* perf(type): static OpenAI types definition

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat: add hf types

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* types: update remaining missing stubs

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-16 04:26:13 -05:00
Aaron Pham
6102a67a83 infra: makes huggingface-hub requirements on fine-tune (#665) 
infra: makes huggingface-hub core deps

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-16 03:12:52 -05:00
Aaron Pham
86d23fd6f5 feat(llm): respect warnings environment for dtype warning (#664) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-16 03:05:58 -05:00
Aaron Pham
4a6f13ddd2 feat(type): provide structured annotations stubs (#663) 
* feat(type): provide client stubs

separation of concern for more brevity code base

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* docs: update changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-16 02:58:45 -05:00
Kuan-Chun Wang
af88b9b077 fix(runner): remove keyword args for attrs.get() (#661) 2023-11-15 04:59:01 -05:00
Aaron Pham
68c6a9dac6 infra: prepare for release 0.4.9 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 07:51:30 +00:00
Aaron Pham
9e6df0df89 chore: update requirements in README.md (#659) 
chore: update requirements

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 02:32:36 -05:00
Aaron Pham
034e08cf08 infra: update scripts to run update readme automatically (#658) 
* infra: update scripts to run update readme automatically

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: cleanup mirror

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore(dropdown): correctly format noteblock and important block

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: whitespace aware

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 02:22:49 -05:00
Aaron Pham
625afd0c3b infra: prepare for release 0.4.8 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 05:19:22 +00:00
Aaron Pham
a58d947bc8 perf: improve build logics and cleanup speed (#657) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 00:18:31 -05:00
Aaron Pham
103156cd71 chore(cli): move playground to CLI components (#655) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 23:20:50 -05:00
Aaron Pham
b40874149e infra: prepare for release 0.4.7 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 03:44:09 +00:00
xianxian.zhang
ea02aaaa23 fix: correct OPENLLM_DEV_BUILD check (#653) 2023-11-14 22:21:37 -05:00
Aaron Pham
6a6d689a77 feat: Yi models (#651) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 21:55:24 -05:00
Aaron Pham
b4b70e2f20 fix(cli): update context name parsing correctly (#652) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 21:53:56 -05:00
Aaron
9eddae83a6 infra: update cohere client 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 01:52:38 -05:00
Aaron Pham
31a799ff61 refactor: use DEBUG env-var instead of OPENLLMDEVDEBUG (#647) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 01:39:58 -05:00
Aaron Pham
145aafd386 infra: prepare for release 0.4.6 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 06:18:14 +00:00
Aaron Pham
00d6016bcb chore(openapi): unify inject param (#645) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 01:16:20 -05:00
Aaron Pham
b0ab8ccdf6 experimental: Cohere compatible endpoints. (#644) 
* feat: add generate endpoint

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update generation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix(cohere): generate endpoints

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: --wip--

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat: update testing clients and chat implementation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: disable schemas for easter eggs

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 01:07:43 -05:00
Aaron Pham
0bf6ec7537 fix(dependencies): lock build < 1 for now (#643) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 00:36:08 -05:00
Aaron Pham
b30a412398 fix(cli): set default dtype to auto infer (#642) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 23:05:27 -05:00
Aaron Pham
99a5d26527 fix(service): to yield out correct JSON objects (#640) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 22:41:52 -05:00
Aaron Pham
2d428f12da fix(cpu): more verbose definition for dtype casting (#639) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 20:40:50 -05:00
Aaron Pham
b20c7d1c1d fix(generation): compatibility dtype with CPU (#638) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 20:32:07 -05:00
Zhao Shenyang
ae69524749 doc: update adding new model guide (#637) 
* update

* Update openllm-python/ADDING_NEW_MODEL.md

Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Zhao Shenyang <dev@zsy.im>

* Update openllm-python/ADDING_NEW_MODEL.md

Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Zhao Shenyang <dev@zsy.im>

* Update openllm-python/ADDING_NEW_MODEL.md

Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Zhao Shenyang <dev@zsy.im>

* move ADDING_NEW_MODEL.md to git root directory

---------

Signed-off-by: Zhao Shenyang <dev@zsy.im>
Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 18:30:44 -05:00
Aaron Pham
a6387d1d15 chore: cleanup unused code path (#633) 
we now rely on tokenizer.chat_templates to format prompts correctly

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 17:23:07 -05:00
Aaron Pham
a0d74017cd infra: prepare for release 0.4.5 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 18:49:31 +00:00
Aaron Pham
d358e68539 fix(torch_dtype): load eagerly (#631) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 13:48:04 -05:00
pre-commit-ci[bot]
52367d1e8b ci: pre-commit autoupdate [pre-commit.ci] (#629) 
* ci: pre-commit autoupdate [pre-commit.ci]

updates:
- [github.com/pre-commit/mirrors-prettier: v3.0.3 → v3.1.0](https://github.com/pre-commit/mirrors-prettier/compare/v3.0.3...v3.1.0)

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-13 13:07:53 -05:00
Aaron Pham
852cd863a9 fix(cli): make sure to pass the dtype to subprocess service (#628) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 05:32:17 -05:00
Aaron Pham
099c0dc31b feat(cli): --dtype arguments (#627) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 05:25:50 -05:00
Aaron Pham
22eaaf3ce1 feat(vllm): support passing specific dtype (#626) 
* feat(vllm): support passing specific dtype

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* fix: correctly cached the item

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-13 05:08:33 -05:00
Aaron Pham
126e6c9d63 fix(ruff): correct consistency between isort and formatter (#624) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 21:12:50 -05:00
Aaron Pham
de04de7136 fix(sdk): make sure build to quiet out stdout (#622) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 18:59:48 -05:00
Aaron Pham
e667dac82f chore(cli): always show available models (#621) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 18:36:19 -05:00
Aaron Pham
c50a7db80d fix(cli): correct set working_dir (#620) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 18:34:11 -05:00
Aaron Pham
e0632a85ed refactor(cli): move out to its own packages (#619) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 18:25:44 -05:00
Aaron Pham
b1f3a7297b infra: prepare for release 0.4.4 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 22:39:52 +00:00
Aaron Pham
c3416c0afd feat(llm): update warning envvar and add embedded mode (#618) 
* chore: unify warning envvar and update type inference

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore; update documentation about embedded

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 17:39:06 -05:00
Aaron Pham
7e1fb35a71 chore(llm): expose quantise and lazy load heavy imports (#617) 
* chore(llm): expose quantise and lazy load heavy imports

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: move transformers to TYPE_CHECKING block

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 14:55:37 -05:00
Aaron Pham
f3b16a4db0 infra: prepare for release 0.4.3 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 07:39:56 +00:00
Aaron Pham
fad4186dbc feat(server): helpers endpoints for conversation format (#613) 
* feat: add support for helpers conversation conversion endpoint

also correct schema generation for openllm client

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update clients to reuse `openllm-core` logics

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: add changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 01:02:27 -05:00
Aaron Pham
4d4df66188 infra: prepare for release 0.4.2 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 03:38:52 +00:00
Aaron Pham
7438005c04 refactor(config): simplify configuration and update start CLI output (#611) 
* chore(config): simplify configuration and update start CLI output
handling

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: remove state and message sent after server lifecycle

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update color stream and refactor reusable logic

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update documentations and mypy

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-11 22:36:10 -05:00
Aaron Pham
c41828f68f feat(client): support authentication token and shim implementation (#605) 
* chore: synch generate_iterator to be the same as server

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* --wip--

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* wip

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat: cleanup shim implementation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

* chore: fix pre-commit

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update check with tuple

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-10 17:44:31 -05:00
Aaron Pham
f89bec261c fix: correct importmodules locally (#601) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-10 03:32:12 -05:00
Aaron Pham
fa2038f4e2 fix: loading correct local models (#599) 
* fix(model): loading local correctly

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: update repr and correct bentomodel processor

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

* chore: cleanup transformers implementation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: ruff to ignore I001 on all stubs

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-10 02:36:12 -05:00