mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-04-23 08:28:24 -04:00
Code Issues Packages Projects Releases Wiki Activity
240 Commits 6 Branches 194 Tags
bb37f7e2380abe3f39428428bbbca82cc0da4c4c
Commit Graph

36 Commits

Author SHA1 Message Date
aarnphm-ec2-dev
bb37f7e238 feat(utils): lazy load modules and fix typo 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-06-10 22:18:37 +00:00
Aaron
05fa34f9e6 refactor: pretrained => model_id 
I think model_id makes more sense than calling it pretrained

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-06-10 17:36:02 -04:00
Aaron
afddaed08c fix(perf): respect per request information 
remove use_default_prompt_template options

add pretrained to list of start help docstring

fix flax generation config

improve flax and tensorflow implementation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-06-10 02:14:13 -04:00
Aaron
aa50b5279e fix(falcon): loading based on model registration 
remove duplicate events

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-06-06 22:42:28 -04:00
Aaron
f78d55f0fd fix(cli): type handling for specific container types 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-06-06 17:18:25 -04:00
Aaron
ed54a0b746 chore: fix type issue on 3.8 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-06-06 15:53:02 -04:00
Aaron
8ef4c9cb19 fix(types): broken import and add hints for client 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-06-05 00:10:44 -07:00
Aaron
ec941c95d5 chore: add license header 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-06-04 16:22:37 -07:00
Aaron
e66340aaba chore: using typing.get_origin 
since we only support 3.8+, get_origin and get_args are available

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-06-04 14:24:08 -07:00
Aaron
49cb02d2f2 perf(cli): improve printing speed that respect terminal_size 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-06-02 06:58:11 -07:00
aarnphm-ec2-dev
c3aeb43997 fix: generation serde 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-06-02 07:06:04 +00:00
aarnphm-ec2-dev
a94294bc65 fix: generate attrs class internally to conform with interface 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-06-01 19:06:06 +00:00
Aaron
e86dc35ec5 chore: migrate service to use JSON 
until we have attrs io descriptor, this should do it

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-06-01 00:28:43 -07:00
Aaron
33e7004e66 format: consistent CLI outputs 
vendorred type-related module from bentoml._internal.types

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-30 14:56:11 -07:00
Aaron Pham
01517e37c6 migration: attrs (#7) 
Move configuration to attrs

Depends on https://github.com/bentoml/BentoML/pull/3906
 2023-05-30 11:59:21 -07:00
Aaron
d5f213e49c compat: provide shim for pydantic 1 and 2 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 19:14:51 -07:00
Aaron Pham
ceb9f7bded feat: BetterTransformer during inference (#6) 2023-05-28 11:56:49 -07:00
Aaron
8e8d95cae9 feat: StableLM tuned and alpha 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 06:36:06 -07:00
Aaron
78358dbb8d fix(type): configuration and dependencies 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 06:01:30 -07:00
Aaron
0df8d8b9a6 perf: reduce unecessary object creation for config class 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 05:22:22 -07:00
Aaron
3fb1e5338a feat(dependencies): add optional for model 
pretty print failed models loading due to missing dependencies

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 00:11:36 -07:00
Aaron
6aca092bfc infra: prepare for 0.0.8 release 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 23:42:06 -07:00
aarnphm-ec2-dev
1e254a4cf3 feat: FalconLM 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-05-28 06:39:37 +00:00
Aaron
52d65f999f feat(telemetry): add support for usage tracking 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 20:39:13 -07:00
Aaron
fa895c329c feat: pre-commit setup 
also sync JS release with Python version

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-27 06:54:22 -07:00
Aaron Pham
0a37bac25d feat(codegen): using black parser (#5) 2023-05-27 04:46:54 -07:00
aarnphm-ec2-dev
ac933d60f1 fix(cli): Make sure to skips models that only runs on GPU 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-05-26 00:43:14 +00:00
aarnphm-ec2-dev
b502703f67 fix(chatglm): make sure to check for required dependencies cpm_kernels 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-25 16:22:07 -07:00
aarnphm-ec2-dev
5c416fa218 feat: StarCoder 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-25 16:22:07 -07:00
aarnphm-ec2-dev
83a8a7cb4f docs(codegen): make sure the generated dostring is correct 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-25 16:22:06 -07:00
Aaron
2676085b59 feat: chatglm and configuration naming type 
by default, it is dasherize, but for cases like chatglm, it can be
lowercase as well

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-24 04:20:24 -07:00
Aaron
b1c07946c1 feat: dolly-v2 and general cleanup 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-23 14:27:27 -07:00
Aaron
549b0c54e9 feat: codegen and bundle build 
fix configuration generation for runnable

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-15 18:22:25 -07:00
Aaron
d31d450526 feat: Adding central service definition and init openllm_client 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-15 00:33:05 -07:00
Aaron
426a61713f feat: start and start_grpc API 
with_options listen from environment variable for said models.

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-05 11:07:52 -07:00
Chaoyu
dd8b6050b2 feat: FLAN-T5 supports 
- add infrastructure, to be implemented: cache, chat history

- Base Runnable Implementation, that fits LangChain API

- Added a Prompt descriptor and utils.

feat: license headers and auto factory impl and CLI

Auto construct args from pydantic config

Add auto factory for ease of use

only provide `/generate` to streamline UX experience

CLI > envvar > input contract for configuration

fix: serve from a thread

fix CLI args

chore: cleanup names and refactor imports

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-05-03 17:50:14 -07:00