mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-01-28 17:31:57 -05:00
Code Issues Packages Projects Releases Wiki Activity
80 Commits 6 Branches 194 Tags
d5f213e49cdbb097f13ff6256972caadb7dcf48d
Commit Graph

68 Commits

Author SHA1 Message Date
Aaron
d5f213e49c compat: provide shim for pydantic 1 and 2 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 19:14:51 -07:00
Aaron
41706eee5b feat(save_model): passing tag 
update with upstream bentoml

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 17:04:15 -07:00
aarnphm-ec2-dev
f68c764f9b chore: disable bettertransformer when bitsandbytes is used 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 20:09:57 +00:00
aarnphm-ec2-dev
b4b65fc609 chore(stablelm): tuning half when 8bit is not loaded 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 19:55:26 +00:00
Aaron Pham
ceb9f7bded feat: BetterTransformer during inference (#6) 2023-05-28 11:56:49 -07:00
Aaron
ac710dfd54 revert(perf): remove group alias 
There is no need for this feature

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 10:04:33 -07:00
Aaron
454b1edc32 infra: bump to dev version of 0.0.10.dev0 [generated] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 10:00:34 -07:00
Aaron
ef0ddebd60 infra: prepare for release 0.0.9 [generated] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 10:00:08 -07:00
Aaron
435129372e perf(cli): lazily cached start commands 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 09:59:43 -07:00
aarnphm-ec2-dev
9c1c4ca0bf perf(cli): using click instead of rich console 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 09:32:56 -07:00
aarnphm-ec2-dev
3f36d81744 infra: docs and normalize formatting 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 15:00:17 +00:00
aarnphm-ec2-dev
8ca488d8fc fix(stablelm): Ensure passing EOS_TOKEN_ID for generation 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 14:43:00 +00:00
Aaron
b4403c24b0 fix(model): Make sure we download the model before starting the 
service

This will ensure we don't deadlock among processes

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 14:01:49 +00:00
Aaron
8e8d95cae9 feat: StableLM tuned and alpha 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 06:36:06 -07:00
Aaron
78358dbb8d fix(type): configuration and dependencies 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 06:01:30 -07:00
Aaron
0df8d8b9a6 perf: reduce unecessary object creation for config class 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 05:22:22 -07:00
Aaron
3fb1e5338a feat(dependencies): add optional for model 
pretty print failed models loading due to missing dependencies

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 00:11:36 -07:00
Aaron
89a714b8c2 infra: bump to dev version of 0.0.9.dev0 [generated] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 23:42:56 -07:00
Aaron
ffc441a227 infra: prepare for release 0.0.8 [generated] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 23:42:39 -07:00
Aaron
6aca092bfc infra: prepare for 0.0.8 release 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 23:42:06 -07:00
aarnphm-ec2-dev
1e254a4cf3 feat: FalconLM 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 06:39:37 +00:00
Aaron
c84f653b77 feat(cli): add output for build and bundle 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 22:37:10 -07:00
Aaron
f24f13e6e4 chore(cli): consistency between table and json format 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 22:29:54 -07:00
Aaron
186658be63 docs: delay GPU to model check, 
allow users to package and interact with models that requires GPU even on device without GPU

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 22:24:18 -07:00
Aaron
e0fc37e47f fix(docs): update docs about saving custom fine-tuned 
and update annotations for client

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 21:15:44 -07:00
Aaron
fd48cbdeb2 infra: bump to dev version of 0.0.8.dev0 [generated] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 20:57:57 -07:00
Aaron
98cd06a686 infra: prepare for release 0.0.7 [generated] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 20:57:38 -07:00
Aaron
52d65f999f feat(telemetry): add support for usage tracking 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 20:39:13 -07:00
Aaron
a55817d647 feat(cli): update nicely formatted commands with shared output logics 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 15:51:03 -07:00
Aaron
fa895c329c feat: pre-commit setup 
also sync JS release with Python version

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 06:54:22 -07:00
Aaron
775ddab941 fix(generation): correct type of top_k to int instead of float 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 05:23:44 -07:00
Aaron
c595b63736 infra: bump to dev version of 0.0.7.dev0 [generated] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 05:20:45 -07:00
Aaron
b9e1ca5514 infra: prepare for release 0.0.6 [generated] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 05:20:27 -07:00
Aaron
cf4e55c36f fix(client): implement per client framework and model_name getters 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 05:20:07 -07:00
Aaron
4da7aa4911 infra: bump to dev version of 0.0.6.dev0 [generated] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 04:48:52 -07:00
Aaron
6439ad55fc infra: prepare for release 0.0.5 [generated] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 04:48:28 -07:00
Aaron Pham
0a37bac25d feat(codegen): using black parser (#5) 2023-05-27 04:46:54 -07:00
Aaron
70bdaf9c05 infra: bump to dev version of 0.0.5.dev0 [generated] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 01:26:19 -07:00
Aaron
8ab7e87083 infra: prepare for release 0.0.4 [generated] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 01:21:34 -07:00
Aaron
c73732db6f fix(configuration): Make sure GenerationInput dumped the correct 
dictionary for llm_config

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 01:01:32 -07:00
aarnphm-ec2-dev
8ee5b048f3 feat(client): Async and Sync client 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-26 22:51:21 -07:00
aarnphm-ec2-dev
7b5ddc1efa fix(configuration): make sure we don't override per-request 
configuration

Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 03:13:40 +00:00
Aaron
b7f3a10910 refactor: migrate __init_subclass__ to Metaclass 
LLMMetaclass will now responsible for generate internal attributes

add llm_type and identifying_params to Runnable class

subclass of openllm.LLM now can set a class attribute
__openllm_internal__ to let openllm knows that this is an internal class
implementation, instead of providing a _internal in the class
initialization.

support for preprocess_parameters and postprocess_parameters on client
side for better client UX

Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 03:09:45 +00:00
aarnphm-ec2-dev
150ea33d2d infra: bump to dev version [generated] 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-05-26 23:18:30 +00:00
Aaron
ee3d17fdfa infra: prepare for 0.0.3 release 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-26 14:10:06 -07:00
Aaron
85252f13c4 fix(cli): simplify register code for start 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-26 01:45:03 -07:00
aarnphm-ec2-dev
4127961c5c feat: openllm.client 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-05-26 07:17:28 +00:00
aarnphm-ec2-dev
ac933d60f1 fix(cli): Make sure to skips models that only runs on GPU 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-05-26 00:43:14 +00:00
aarnphm-ec2-dev
fed17fafdc migrate(configuration): remove deprecated max_length in favor of 
max_new_tokens

Preparation for transformers 5

Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-25 16:22:07 -07:00
aarnphm-ec2-dev
b502703f67 fix(chatglm): make sure to check for required dependencies cpm_kernels 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-25 16:22:07 -07:00