mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-01-14 02:17:48 -05:00
Code Issues Packages Projects Releases Wiki Activity
1,353 Commits 4 Branches 194 Tags
2bb97f8ba2b95d3383aeeced865da11ddfb3f40a
Commit Graph

262 Commits

Author SHA1 Message Date
Aaron Pham
2bb97f8ba2 chore: update discord link (#838) 
* Update pyproject.toml

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* Update pyproject.toml

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* Update pyproject.toml

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* Update pyproject.toml

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2024-01-08 19:09:51 -05:00
Aaron Pham
79da419d87 chore(deps): bump vllm to 0.2.7 (#837) 
* chore(deps): bump vllm to 0.2.7

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2024-01-08 14:41:58 -05:00
Aaron Pham
7e0c9180fe chore(script): run vendored scripts (#808) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-12-22 10:46:15 -05:00
Aaron Pham
b09bd20750 infra: prepare for release 0.4.41 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-12-18 18:08:46 +00:00
Aaron Pham
8d63afc9ce feat(vllm): support GPTQ with 0.2.6 (#797) 
* feat(vllm): GPTQ support passthrough

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: run scripts

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* fix(install): set order of xformers before vllm

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat: support GPTQ with vLLM

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-12-18 12:41:19 -05:00
Aaron Pham
5d27337e82 fix(cli): avoid runtime __origin__ check for older Python (#798) 
fix(cli): avoid runtime __origin__ on older Python

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-18 12:33:36 -05:00
Aaron Pham
2e8fc284f5 infra: prepare for release 0.4.40 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-12-15 16:46:12 +00:00
Aaron Pham
88b6d3d6de perf: upgrade mixtral to use expert parallelism (#783) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-15 11:45:08 -05:00
Aaron Pham
c8c9663d06 fix(infra): conform ruff to 150 LL (#781) 
Generally correctly format it with ruff format and manual style

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-14 17:27:32 -05:00
Aaron Pham
d4fbbcee34 infra: prepare for release 0.4.39 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-12-14 19:20:01 +00:00
Aaron Pham
44383528b5 fix(logprobs): correct check logprobs (#779) 
* fix(logprobs): correct check logprobs

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changlog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-14 14:19:01 -05:00
Aaron Pham
1dbae67172 infra: prepare for release 0.4.38 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-12-13 23:27:41 +00:00
Aaron Pham
0d83cefcb6 fix(mixtral): setup hack atm to load weights from pt specifically instead of safetensors (#776) 
fix(mixtral): setup hack atm to load weights from pt specifically
instead of safetensors

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-13 18:18:51 -05:00
Aaron Pham
2dbcfa8a0c fix(cli): correct set arguments for openllm import and openllm build (#775) 
* fix(cli): correct set arguments for `openllm import` and `openllm build`

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-13 15:52:59 -05:00
Aaron Pham
8d9d212d61 infra: prepare for release 0.4.37 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-12-13 14:07:33 +00:00
Aaron Pham
3ab78cd105 feat(mixtral): correct support for mixtral (#772) 
feat(mixtral): support inference with pt

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-13 09:03:56 -05:00
Aaron Pham
9cd1e44b1e infra: prepare for release 0.4.36 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-12-12 06:34:39 +00:00
Aaron Pham
d3328343d7 feat: mixtral support (#770) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-12-12 01:33:13 -05:00
Aaron
59e8ef93dc chore(deps): lock vLLM to 0.2.4 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-12 00:17:18 -05:00
Aaron Pham
08114410bc fix(openai): logprobs when echo is enabled (#761) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-10 18:09:25 -05:00
Aaron Pham
c3a0b5c39f feat(openai): supports echo (#760) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-10 13:19:40 -05:00
Aaron
bb4ed8b53c fix(llm): correct annotations definitions 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-09 09:59:02 -05:00
Aaron Pham
8019fd84c8 infra: prepare for release 0.4.35 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-12-07 08:38:13 +00:00
Aaron
9a7e0cecf0 fix(types): makes sures mypy is running strict 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-30 09:42:24 -05:00
Aaron
55a0b2f825 fix(style): setup correct block format 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-30 07:58:35 -05:00
Aaron
b53559de6f fix(setter): correct item with the same kwargs with stubs 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-30 07:36:34 -05:00
Aaron Pham
81688e0949 infra: prepare for release 0.4.34 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-30 12:17:48 +00:00
yansheng
3cb7f14fc1 feat(models): Support qwen (#742) 
* support qwen

* support qwen

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

* Update openllm-core/src/openllm_core/config/configuration_qwen.py

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: update correct readme and supports qwen models

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: root <yansheng105@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-30 06:54:17 -05:00
Aaron Pham
9fa0dee406 infra: prepare for release 0.4.33 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-29 18:01:13 +00:00
Aaron Pham
0909e08e3c fix(llm): remove unecessary parsing 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-29 18:00:08 +00:00
Aaron Pham
69deedd9b8 infra: prepare for release 0.4.32 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-29 07:39:42 +00:00
Aaron Pham
9706228956 chore(vllm): add arguments for gpu memory utilization 
Probably not going to fix anything, just delaying the problem.

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-29 06:45:14 +00:00
Aaron Pham
f0fa06004b chore: revert back previous backend support PyTorch (#739) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-29 01:44:41 -05:00
Aaron Pham
d04309188b chore(style): 2.7k 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-28 07:04:27 +00:00
Aaron
ce6efc2a9e chore(style): cleanup bytes 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-28 01:27:27 -05:00
Aaron Pham
77af72ed2a infra: prepare for release 0.4.31 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-26 23:49:04 +00:00
Aaron Pham
e157d3aa9e infra: prepare for release 0.4.30 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-26 09:54:42 +00:00
Aaron
96318b65ee fix(sdk): remove broken sdk 
codespace now around 2.8k lines

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-26 04:53:36 -05:00
Aaron Pham
f7a803dfa2 infra: prepare for release 0.4.29 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-26 07:50:48 +00:00
Aaron
43a96dab2c fix(gpus): disable slots for now to enable cached_property 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-26 02:49:48 -05:00
Aaron
69aae34cf4 fix(style): reduce boilerplate and format to custom logics 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-26 01:44:59 -05:00
Aaron Pham
e27764fe6b infra: prepare for release 0.4.28 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 07:09:06 +00:00
MingLiangDai
7b8d9024c4 fix(baichuan): supported from baichuan 2 from now on. (#728) 
* config support multiple architectures

* chore: only support baichuan2 from now on

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update notes

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: run script [skip ci]

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 02:07:06 -05:00
Aaron Pham
d8a783772d infra: prepare for release 0.4.27 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 06:25:16 +00:00
Aaron
b4c9971678 fix(build): explicitly not lock packages 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:21:29 -05:00
Aaron
d0e12b1fb8 fix(metadata): remove unused packages 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:19:09 -05:00
Aaron
7dd4e3ac4b fix(build): don't lock packages for now, but do lock base requirements 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:17:45 -05:00
Aaron
7beaa92c2b fix(types): using correct refactored literal 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:14:29 -05:00
Aaron Pham
aab173cd99 refactor: focus (#730) 
* perf: remove based images

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: move dockerifle to run on release only

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: cleanup unused types

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:11:31 -05:00
Aaron Pham
52a44b1bfa chore: cleanup loader (#729) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 21:51:51 -05:00