mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-03-05 07:36:15 -05:00
Code Issues Packages Projects Releases Wiki Activity
1,244 Commits 6 Branches 194 Tags
69aae34cf4e6995edf2e2dc1a669fc4bdecf959a
Commit Graph

1244 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Aaron
69aae34cf4 fix(style): reduce boilerplate and format to custom logics 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-26 01:44:59 -05:00
Aaron Pham
b4ea4b3e99 infra: bump to homebrew tap release to 0.4.28 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 07:21:13 +00:00
Aaron Pham
753b49c647 infra: bump to dev version of 0.4.29.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 07:20:18 +00:00
Aaron Pham
e27764fe6b infra: prepare for release 0.4.28 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.28 		2023-11-24 07:09:06 +00:00
MingLiangDai
7b8d9024c4 fix(baichuan): supported from baichuan 2 from now on. (#728) 
* config support multiple architectures

* chore: only support baichuan2 from now on

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update notes

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: run script [skip ci]

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 02:07:06 -05:00
Aaron
39ecc73a50 infra: bump to dev version of 0.4.28.dev0 [generated] [skip ci] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:54:46 -05:00
Aaron Pham
d8a783772d infra: prepare for release 0.4.27 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.27 		2023-11-24 06:25:16 +00:00
Aaron
b4c9971678 fix(build): explicitly not lock packages 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:21:29 -05:00
Aaron
d0e12b1fb8 fix(metadata): remove unused packages 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:19:09 -05:00
Aaron
7dd4e3ac4b fix(build): don't lock packages for now, but do lock base requirements 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:17:45 -05:00
Aaron
7beaa92c2b fix(types): using correct refactored literal 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:14:29 -05:00
Aaron Pham
aab173cd99 refactor: focus (#730) 
* perf: remove based images

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: move dockerifle to run on release only

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: cleanup unused types

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 01:11:31 -05:00
Aaron Pham
52a44b1bfa chore: cleanup loader (#729) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 21:51:51 -05:00
Aaron Pham
5442d9cd10 fix(trust_remote_code): handle args correctly (#727) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 17:03:13 -05:00
Aaron Pham
79c9608735 infra: reduce wait time to around 7 mins (#726) 
Seems like the release process for PyPI usually takes from 4-7 minutes

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 07:28:36 -05:00
Aaron Pham
831bb8c497 infra: bump to homebrew tap release to 0.4.26 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 11:59:36 +00:00
Aaron Pham
80842ad501 infra: bump to dev version of 0.4.27.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 11:58:42 +00:00
Aaron Pham
7eae50377d infra: prepare for release 0.4.26 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.26 		2023-11-22 11:50:50 +00:00
Aaron Pham
b28b5269b5 feat(openai): chat templates and complete control of prompt generation (#725) 
* feat(openai): chat templates and complete control of prompt generation

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* fix: correctly use base chat templates

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* fix: remove symlink

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 06:49:14 -05:00
Aaron Pham
7aa0918a6f fix(client): correct schemas parser from correct response output (#724) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 05:01:35 -05:00
Aaron Pham
f83f64ffd7 fix(infra): setup higher timer for building container images (#723) 
* fix(infra): setup higher timer for building container images

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: remove invalid tests

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 05:00:33 -05:00
Aaron Pham
6dd07580e2 infra: bump to homebrew tap release to 0.4.25 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 09:35:41 +00:00
Aaron Pham
1876609a67 infra: bump to dev version of 0.4.26.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 09:34:22 +00:00
Aaron Pham
0189342730 infra: prepare for release 0.4.25 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.25 		2023-11-22 09:22:45 +00:00
Aaron Pham
63d86faa32 fix(openai): correct stop tokens and finish_reason state (#722) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 04:21:13 -05:00
Aaron Pham
06626e7d1e infra: bump to homebrew tap release to 0.4.24 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 06:51:39 +00:00
Aaron Pham
c9b23638a5 infra: bump to dev version of 0.4.25.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 06:50:35 +00:00
Aaron Pham
7f09f9daf2 infra: prepare for release 0.4.24 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.24 		2023-11-22 06:34:30 +00:00
Aaron
d697ea3903 fix(image): setup correct installation 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 01:33:26 -05:00
Aaron Pham
9f84b8b945 infra: bump to homebrew tap release to 0.4.23 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 06:26:20 +00:00
Aaron Pham
1df549f76a infra: bump to dev version of 0.4.24.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 06:25:32 +00:00
Aaron Pham
85e03a4b92 infra: prepare for release 0.4.23 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.23 		2023-11-22 06:16:49 +00:00
Aaron Pham
38b7c44df0 fix(base-image): update base image to include cuda for now (#720) 
* fix(base-image): update base image to include cuda for now

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: build core and client on release images

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: cleanup style changes

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 01:15:19 -05:00
Aaron Pham
8bb2742a9a chore(types): append additional types change (#719) 
* chore(types): append additional types change

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: add arguments for parsing dir

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 22:38:20 -05:00
Aaron Pham
04ef08a7f8 chore(strategy): compact and add stubs (#718) 
generate service_vars automatically inline without reading from files

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 21:49:28 -05:00
Aaron Pham
909db8c3bf refactor: reduce compiled cacheline 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 02:27:42 +00:00
Aaron Pham
77bd6f090a chore(logger): fix warnings and streamline style (#717) 
Sorry but there are too much wasted spacing in `_llm.py`, and I'm unhappy and not productive anytime I look or want to do anything with it

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-21 18:54:51 -05:00
Aaron Pham
d53cf234bd fix(api-server): correct set generation from LLM class 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 10:38:36 +00:00
Aaron Pham
2821e172ef fix(examples): use non-chat models 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 10:12:48 +00:00
Aaron
93709f1d66 fix(infra): remove unless exclude 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 05:04:47 -05:00
Aaron
14242a7ab8 fix(utils): correct import 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 05:03:20 -05:00
Aaron Pham
c33b071ee4 refactor: delete unused code (#716) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 04:39:48 -05:00
Aaron Pham
a8a9f154ce fix(ci): tests (#715) 
* fix: tests

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: remove broken tests

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 03:05:22 -05:00
Aaron Pham
e70246ca5d feat(generation): add support for eos_token_id (#714) 
chore: add support for custom eos_token_id

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 02:01:36 -05:00
Aaron Pham
fde78a2c78 chore: cleanup unused prompt templates (#713) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 01:56:51 -05:00
Aaron Pham
e6b9a749a4 infra: bump to homebrew tap release to 0.4.22 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 01:50:09 +00:00
Aaron Pham
be7a4bf576 infra: bump to dev version of 0.4.23.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 01:49:02 +00:00
Aaron Pham
f3fd32d596 infra: prepare for release 0.4.22 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.22 		2023-11-21 01:38:46 +00:00
Aaron Pham
ad4f388c98 refactor: update runner helpers and add max_model_len (#712) 
* chore(runner): cleanup unecessary checks for runnable backend

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: saving llm reference to runner

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: correct inject item

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update support for max_seq_len

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: correct max_model_len

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update and warning backward compatibility

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: remove unused sets

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 20:37:15 -05:00
Aaron Pham
8fc5f1f70c infra: bump to homebrew tap release to 0.4.21 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 22:50:00 +00:00