mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-01-15 10:57:49 -05:00
Code Issues Packages Projects Releases Wiki Activity
1,292 Commits 4 Branches 194 Tags
59e8ef93dce2f0caa7b5c9bf9cd5f97a4f7ddbdf
Commit Graph

244 Commits

Author SHA1 Message Date
Aaron
59e8ef93dc chore(deps): lock vLLM to 0.2.4 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-12 00:17:18 -05:00
Aaron Pham
08114410bc fix(openai): logprobs when echo is enabled (#761) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-10 18:09:25 -05:00
Aaron Pham
c3a0b5c39f feat(openai): supports echo (#760) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-10 13:19:40 -05:00
Aaron
bb4ed8b53c fix(llm): correct annotations definitions 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-09 09:59:02 -05:00
Aaron Pham
8019fd84c8 infra: prepare for release 0.4.35 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-12-07 08:38:13 +00:00
Aaron
9a7e0cecf0 fix(types): makes sures mypy is running strict 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-30 09:42:24 -05:00
Aaron
55a0b2f825 fix(style): setup correct block format 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-30 07:58:35 -05:00
Aaron
b53559de6f fix(setter): correct item with the same kwargs with stubs 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-30 07:36:34 -05:00
Aaron Pham
81688e0949 infra: prepare for release 0.4.34 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-30 12:17:48 +00:00
yansheng
3cb7f14fc1 feat(models): Support qwen (#742) 
* support qwen

* support qwen

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

* Update openllm-core/src/openllm_core/config/configuration_qwen.py

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: update correct readme and supports qwen models

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: root <yansheng105@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-30 06:54:17 -05:00
Aaron Pham
9fa0dee406 infra: prepare for release 0.4.33 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-29 18:01:13 +00:00
Aaron Pham
0909e08e3c fix(llm): remove unecessary parsing 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-29 18:00:08 +00:00
Aaron Pham
69deedd9b8 infra: prepare for release 0.4.32 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-29 07:39:42 +00:00
Aaron Pham
9706228956 chore(vllm): add arguments for gpu memory utilization 
Probably not going to fix anything, just delaying the problem.

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-29 06:45:14 +00:00
Aaron Pham
f0fa06004b chore: revert back previous backend support PyTorch (#739) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-29 01:44:41 -05:00
Aaron Pham
d04309188b chore(style): 2.7k 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-28 07:04:27 +00:00
Aaron
ce6efc2a9e chore(style): cleanup bytes 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-28 01:27:27 -05:00
Aaron Pham
77af72ed2a infra: prepare for release 0.4.31 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-26 23:49:04 +00:00
Aaron Pham
e157d3aa9e infra: prepare for release 0.4.30 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-26 09:54:42 +00:00
Aaron
96318b65ee fix(sdk): remove broken sdk 
codespace now around 2.8k lines

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-26 04:53:36 -05:00
Aaron Pham
f7a803dfa2 infra: prepare for release 0.4.29 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-26 07:50:48 +00:00
Aaron
43a96dab2c fix(gpus): disable slots for now to enable cached_property 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-26 02:49:48 -05:00
Aaron
69aae34cf4 fix(style): reduce boilerplate and format to custom logics 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-26 01:44:59 -05:00
Aaron Pham
e27764fe6b infra: prepare for release 0.4.28 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 07:09:06 +00:00
MingLiangDai
7b8d9024c4 fix(baichuan): supported from baichuan 2 from now on. (#728) 
* config support multiple architectures

* chore: only support baichuan2 from now on

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update notes

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: run script [skip ci]

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 02:07:06 -05:00
Aaron Pham
d8a783772d infra: prepare for release 0.4.27 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 06:25:16 +00:00
Aaron
b4c9971678 fix(build): explicitly not lock packages 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:21:29 -05:00
Aaron
d0e12b1fb8 fix(metadata): remove unused packages 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:19:09 -05:00
Aaron
7dd4e3ac4b fix(build): don't lock packages for now, but do lock base requirements 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:17:45 -05:00
Aaron
7beaa92c2b fix(types): using correct refactored literal 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:14:29 -05:00
Aaron Pham
aab173cd99 refactor: focus (#730) 
* perf: remove based images

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: move dockerifle to run on release only

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: cleanup unused types

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:11:31 -05:00
Aaron Pham
52a44b1bfa chore: cleanup loader (#729) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 21:51:51 -05:00
Aaron Pham
5442d9cd10 fix(trust_remote_code): handle args correctly (#727) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 17:03:13 -05:00
Aaron Pham
7eae50377d infra: prepare for release 0.4.26 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 11:50:50 +00:00
Aaron Pham
b28b5269b5 feat(openai): chat templates and complete control of prompt generation (#725) 
* feat(openai): chat templates and complete control of prompt generation

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* fix: correctly use base chat templates

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* fix: remove symlink

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 06:49:14 -05:00
Aaron Pham
f83f64ffd7 fix(infra): setup higher timer for building container images (#723) 
* fix(infra): setup higher timer for building container images

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: remove invalid tests

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 05:00:33 -05:00
Aaron Pham
0189342730 infra: prepare for release 0.4.25 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 09:22:45 +00:00
Aaron Pham
63d86faa32 fix(openai): correct stop tokens and finish_reason state (#722) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 04:21:13 -05:00
Aaron Pham
7f09f9daf2 infra: prepare for release 0.4.24 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 06:34:30 +00:00
Aaron
d697ea3903 fix(image): setup correct installation 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 01:33:26 -05:00
Aaron Pham
85e03a4b92 infra: prepare for release 0.4.23 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 06:16:49 +00:00
Aaron Pham
38b7c44df0 fix(base-image): update base image to include cuda for now (#720) 
* fix(base-image): update base image to include cuda for now

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: build core and client on release images

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: cleanup style changes

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 01:15:19 -05:00
Aaron Pham
8bb2742a9a chore(types): append additional types change (#719) 
* chore(types): append additional types change

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: add arguments for parsing dir

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 22:38:20 -05:00
Aaron Pham
04ef08a7f8 chore(strategy): compact and add stubs (#718) 
generate service_vars automatically inline without reading from files

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 21:49:28 -05:00
Aaron Pham
909db8c3bf refactor: reduce compiled cacheline 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 02:27:42 +00:00
Aaron Pham
77bd6f090a chore(logger): fix warnings and streamline style (#717) 
Sorry but there are too much wasted spacing in `_llm.py`, and I'm unhappy and not productive anytime I look or want to do anything with it

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-21 18:54:51 -05:00
Aaron
14242a7ab8 fix(utils): correct import 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 05:03:20 -05:00
Aaron Pham
c33b071ee4 refactor: delete unused code (#716) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 04:39:48 -05:00
Aaron Pham
a8a9f154ce fix(ci): tests (#715) 
* fix: tests

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: remove broken tests

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 03:05:22 -05:00
Aaron Pham
e70246ca5d feat(generation): add support for eos_token_id (#714) 
chore: add support for custom eos_token_id

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 02:01:36 -05:00