mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-06-12 18:39:16 -04:00
Code Issues Packages Projects Releases Wiki Activity
786 Commits 6 Branches 194 Tags
2a1bcb6dc2008bc1f008116400b1d426e9c1d708
Commit Graph

121 Commits

Author SHA1 Message Date
aarnphm-ec2-dev
7d893e6cd2 chore: ignore new lines split [skip ci] 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-09-01 17:00:49 +00:00
Aaron Pham
b7af7765d4 fix(yapf): align weird new lines break [generated] [skip ci] (#284) 
fix(yapf): align weird new lines break

Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-09-01 05:34:22 -04:00
Aaron Pham
3e45530abd refactor(breaking): unify LLM API (#283) 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-09-01 05:15:19 -04:00
Aaron
b545ad2ad1 style: google 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-08-30 13:52:35 -04:00
Aaron Pham
c9cef1d773 fix: persistent styling between ruff and yapf (#279) 2023-08-30 11:37:41 -04:00
aarnphm-ec2-dev
806a663e4a chore(style): add one blank line 
to conform with Google style

Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-08-26 11:36:57 +00:00
Aaron
f5dd9be122 fix: correct format consistency between ruff and yapf [skip ci] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-08-25 06:28:09 -04:00
Aaron Pham
08dc6ed2ba chore: ignore peft and fix adapter loading issue (#255) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-08-25 04:36:35 -04:00
Aaron
787ce1b3b6 chore(style): synchronized style across packages [skip ci] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-08-23 08:46:22 -04:00
aarnphm-ec2-dev
eddbc06374 chore(style): reduce line length and truncate compression 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-08-22 17:02:00 +00:00
pre-commit-ci[bot]
bc851b1d13 ci: update pre-commit dependencies (#246) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-08-22 10:25:09 -04:00
aarnphm-ec2-dev
1488fbb167 chore(style): enable yapf to match with style guidelines 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-08-22 14:03:06 +00:00
Aaron Pham
3ffb25a872 refactor: packages (#249) 2023-08-22 08:55:46 -04:00
Aaron
099d63d712 chore(changelog): add refactor section [skip ci] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-08-20 07:43:57 -04:00
Aaron Pham
9e205b4963 feat: token streaming and SSE support (#240) 2023-08-20 07:32:49 -04:00
Aaron Pham
4140d160b8 feat(embedding): Adding generic endpoint (#227) 2023-08-17 15:17:00 -04:00
Aaron Pham
ccca49af04 fix(ci): remove broken build hooks (#216) 2023-08-16 04:49:12 -04:00
Aaron
af8cb73832 fix: latest vllm build 
sync changelog with monorepo for sdist installation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-08-16 04:03:34 -04:00
Aaron
6b0ab17018 chore: remove unnecessary headers 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-08-15 18:15:54 -04:00
Aaron
43740aca8b fix(metadata): include hatch-fancy-pypi-readme into subdir [skip ci] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-08-15 05:06:48 -04:00
Aaron Pham
cd872ef631 refactor: monorepo (#203) 2023-08-15 02:11:14 -04:00
Aaron Pham
f6317d8003 infra: enable compiled wheels for all supported Python (#201) 2023-08-12 04:54:50 -04:00
Aaron
785c1db237 fix(client): include openllm.client into main module [skip ci] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-08-11 06:19:56 -04:00
Aaron Pham
5329853b10 perf: compiled modules and enable lazyeval (#200) 2023-08-11 05:53:45 -04:00
Aaron Pham
c083990edd infra: migrate to initial openllm-node library (#199) 2023-08-10 18:54:00 -04:00
Aaron Pham
8c93b781b8 fix(release): fix exclude options within compiled wheels (#197) 2023-08-10 18:48:58 -04:00
aarnphm-ec2-dev
dfc4b489c5 feat(build): notes on compiled wheels for Bento 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-08-09 21:52:34 +00:00
Aaron Pham
b1445c6516 refactor(cli): compiled wheels and extension modules (#191) 2023-08-09 17:10:15 -04:00
Aaron Pham
b9dd54f634 feat: homebrew tap (#190) 2023-08-08 22:11:48 -04:00
Aaron Pham
2541a0f8dc infra: initial work on compiling mypyc wheels (#182) 2023-08-04 10:20:03 -04:00
Aaron
db8e47bc5b fix(build): correct module type for stubs and strip assert [skip ci] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-08-03 04:15:55 -04:00
Aaron Pham
cfc7f3888d chore(vllm): add all supported models (#179) 2023-08-02 17:42:02 -04:00
Aaron Pham
8c2867d26d style: define experimental guidelines (#168) 2023-07-31 07:54:26 -04:00
Aaron Pham
ef94c6b98a feat(container): vLLM build and base image strategies (#142) 2023-07-31 02:44:52 -04:00
aarnphm-ec2-dev
56bf84a760 fix(ci): make sure to exclude generated _version.py 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-07-25 09:55:24 +00:00
Aaron Pham
dcd34bd381 fix(build): running bento insider container (#141) 
Behaviour of `docker run` should be the same with `openllm start`
 2023-07-25 04:24:28 -04:00
Aaron Pham
c391717226 feat(ci): automatic release semver + git archival installation (#143) 2023-07-25 04:18:49 -04:00
Aaron Pham
7eabcd4355 feat: vLLM integration for PagedAttention (#134) 2023-07-24 15:42:17 -04:00
dependabot[bot]
9afbdc5198 chore(deps): update bitsandbytes requirement from <0.40 to <0.42 (#137) 
Updates the requirements on [bitsandbytes](https://github.com/TimDettmers/bitsandbytes) to permit the latest version.
- [Release notes](https://github.com/TimDettmers/bitsandbytes/releases)
- [Changelog](https://github.com/TimDettmers/bitsandbytes/blob/main/CHANGELOG.md)
- [Commits](https://github.com/TimDettmers/bitsandbytes/compare/0.32.0...0.41.0)

---
updated-dependencies:
- dependency-name: bitsandbytes
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-24 07:59:50 +00:00
Aaron Pham
693631958a feat(service): provisional API (#133) 2023-07-23 02:15:39 -04:00
aarnphm-ec2-dev
e4ac0ed8b7 fix(cuda): support loading in single GPU 
add available_devices for getting # of available GPUs

Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-07-21 08:10:01 +00:00
Aaron Pham
f56f8ee782 feat: fine-tuning script for LlaMA 2 (#128) 2023-07-20 20:44:51 -04:00
aarnphm-ec2-dev
3e50f0a851 fix(cli): implement latest bentoml 1.0.25 features 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-07-20 20:51:27 +00:00
Aaron Pham
c1ddb9ed7c feat: GPTQ + vLLM and LlaMA (#113) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-07-19 18:12:12 -04:00
Aaron Pham
fc963c42ce fix: build isolation (#116) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-07-16 01:52:21 -04:00
HeTaoPKU
fd9ae56812 fix(baichuan): add "cpm-kernel" as additional requirements (#117) 
This is to support the 13b variant of baichuan

Co-authored-by: the <tao.he@hulu.com>
Co-authored-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-07-15 23:16:05 -04:00
HeTaoPKU
09b0787306 feat(models): Baichuan (#115) 
Co-authored-by: the <tao.he@hulu.com>
Co-authored-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-07-15 22:01:37 -04:00
aarnphm-ec2-dev
d37d14e52b fix(tests): mark package on CI to xfail 
XXX: @aarnphm to solve build isolation when have bandwidth. Currently
this is not a problem when running locally.

`openllm build` just works, where as `openllm.build` won't work
sequentially.

Address some type stubs for jupytext

Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-07-15 12:48:28 +00:00
Aaron Pham
b2dba6143f fix(resource): correctly parse CUDA_VISIBLE_DEVICES (#114) 2023-07-15 07:19:35 -04:00
aarnphm-ec2-dev
e2ae24b74c fix(tests): building not being isolated 
We will need to fix this from BentoML

Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-07-11 17:28:00 +00:00