mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-03-06 08:08:03 -05:00
Code Issues Packages Projects Releases Wiki Activity
1,071 Commits 6 Branches 194 Tags
a0d74017cd96045859c2bb7d26bfcea3a44cb383
Commit Graph

1071 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Aaron Pham
a0d74017cd infra: prepare for release 0.4.5 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.5 		2023-11-13 18:49:31 +00:00
Aaron Pham
d358e68539 fix(torch_dtype): load eagerly (#631) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 13:48:04 -05:00
Aaron Pham
0924c0b34d infra: removing clojure frontend from infra cycle (#630) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 13:29:34 -05:00
pre-commit-ci[bot]
52367d1e8b ci: pre-commit autoupdate [pre-commit.ci] (#629) 
* ci: pre-commit autoupdate [pre-commit.ci]

updates:
- [github.com/pre-commit/mirrors-prettier: v3.0.3 → v3.1.0](https://github.com/pre-commit/mirrors-prettier/compare/v3.0.3...v3.1.0)

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-13 13:07:53 -05:00
Aaron Pham
852cd863a9 fix(cli): make sure to pass the dtype to subprocess service (#628) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 05:32:17 -05:00
Aaron Pham
099c0dc31b feat(cli): --dtype arguments (#627) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 05:25:50 -05:00
dependabot[bot]
f288b8a276 chore(deps): bump taiki-e/install-action from 2.21.8 to 2.21.11 (#625) 
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.21.8 to 2.21.11.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](b4f94d4449...4d8504289a)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-11-13 05:08:45 -05:00
Aaron Pham
22eaaf3ce1 feat(vllm): support passing specific dtype (#626) 
* feat(vllm): support passing specific dtype

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* fix: correctly cached the item

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-13 05:08:33 -05:00
Aaron Pham
126e6c9d63 fix(ruff): correct consistency between isort and formatter (#624) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 21:12:50 -05:00
Aaron Pham
e77a7fb2a4 chore: update jupyter notebooks with new API (#623) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 21:07:28 -05:00
Aaron Pham
de04de7136 fix(sdk): make sure build to quiet out stdout (#622) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 18:59:48 -05:00
Aaron Pham
e667dac82f chore(cli): always show available models (#621) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 18:36:19 -05:00
Aaron Pham
c50a7db80d fix(cli): correct set working_dir (#620) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 18:34:11 -05:00
Aaron Pham
e0632a85ed refactor(cli): move out to its own packages (#619) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 18:25:44 -05:00
Aaron Pham
38a7d2a5b5 infra: bump to homebrew tap release to 0.4.4 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 22:53:17 +00:00
Aaron Pham
b4a18f8337 infra: bump to dev version of 0.4.5.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 22:51:38 +00:00
Aaron Pham
b1f3a7297b infra: prepare for release 0.4.4 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.4 		2023-11-12 22:39:52 +00:00
Aaron Pham
c3416c0afd feat(llm): update warning envvar and add embedded mode (#618) 
* chore: unify warning envvar and update type inference

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore; update documentation about embedded

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 17:39:06 -05:00
Aaron Pham
7e1fb35a71 chore(llm): expose quantise and lazy load heavy imports (#617) 
* chore(llm): expose quantise and lazy load heavy imports

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: move transformers to TYPE_CHECKING block

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 14:55:37 -05:00
Aaron Pham
106e8617c1 chore(config): no need compat workaround for setting cell_contents (#616) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 04:29:27 -05:00
Aaron Pham
a3c2cdcdc9 infra: bump to homebrew tap release to 0.4.3 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 07:51:23 +00:00
Aaron Pham
f510668392 infra: bump to dev version of 0.4.4.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 07:50:31 +00:00
Aaron Pham
f3b16a4db0 infra: prepare for release 0.4.3 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.3 		2023-11-12 07:39:56 +00:00
Aaron Pham
8534870bd5 feat(client): add helpers subclass (#615) 
* feat(client): add helpers subclass

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* docs: add changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 01:44:12 -05:00
Aaron Pham
bbd20aed89 feat(client): support return response_cls to string (#614) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 01:15:35 -05:00
Aaron Pham
fad4186dbc feat(server): helpers endpoints for conversation format (#613) 
* feat: add support for helpers conversation conversion endpoint

also correct schema generation for openllm client

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update clients to reuse `openllm-core` logics

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: add changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 01:02:27 -05:00
Aaron Pham
3afa204264 infra: bump to homebrew tap release to 0.4.2 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 03:48:48 +00:00
Aaron Pham
e975eb7917 infra: bump to dev version of 0.4.3.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 03:47:42 +00:00
Aaron Pham
4d4df66188 infra: prepare for release 0.4.2 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.2 		2023-11-12 03:38:52 +00:00
Aaron Pham
0b26badc9b docs: update supported feature set (#612) 
Update README.md

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-11 22:37:38 -05:00
Aaron Pham
7438005c04 refactor(config): simplify configuration and update start CLI output (#611) 
* chore(config): simplify configuration and update start CLI output
handling

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: remove state and message sent after server lifecycle

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update color stream and refactor reusable logic

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update documentations and mypy

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-11 22:36:10 -05:00
Aaron Pham
36559a5ab5 chore: remove generated stubs for now (#610) 
Once we have more bandwidth and request for support gRPC streaming

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-10 18:11:35 -05:00
Aaron Pham
08466dd389 chore(client): remove ununsed state enum (#609) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-10 18:10:06 -05:00
Aaron Pham
b65b1bbc52 fix(client): check for should retry header (#606) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-10 17:50:12 -05:00
Aaron Pham
c41828f68f feat(client): support authentication token and shim implementation (#605) 
* chore: synch generate_iterator to be the same as server

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* --wip--

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* wip

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat: cleanup shim implementation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

* chore: fix pre-commit

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update check with tuple

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-10 17:44:31 -05:00
Aaron Pham
af0b1b9a7f fix(config): overload flattened dict (#602) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-10 03:45:52 -05:00
Aaron Pham
f89bec261c fix: correct importmodules locally (#601) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-10 03:32:12 -05:00
Aaron Pham
fa2038f4e2 fix: loading correct local models (#599) 
* fix(model): loading local correctly

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: update repr and correct bentomodel processor

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

* chore: cleanup transformers implementation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: ruff to ignore I001 on all stubs

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-10 02:36:12 -05:00
Aaron Pham
5e45245457 package: add openllm core dependencies to labels (#600) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-10 02:33:55 -05:00
Aaron Pham
665a41940e revert: configuration not to dump flatten (#597) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-09 14:48:23 -05:00
Aaron Pham
d60f2fb909 infra: remove tsconfig (#595) 
* infra: remove tsconfig

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

* chore: filter only ec python and jsx

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update pnpm lock

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: run vendor

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: ignore blame

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: ignore on CI

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-09 13:06:31 -05:00
Aaron Pham
ac377fe490 infra: using ruff formatter (#594) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-09 12:44:05 -05:00
Aaron Pham
021fd453b9 infra: move out clojure to external (#593) 
As we don't write this

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-09 12:24:18 -05:00
Aaron Pham
b8a2e8cf91 refactor(cli): cleanup API (#592) 
* chore: remove unused imports

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* refactor(cli): update to only need model_id

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat: `openllm start model-id`

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: add changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changelog notice

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update correct config and running tools

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update backward compat options and treat JSON outputs
corespondingly

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-09 11:40:17 -05:00
Aaron Pham
86f7acafa9 infra: bump to homebrew tap release to 0.4.1 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-08 13:36:07 +00:00
Aaron Pham
12858605a0 infra: bump to dev version of 0.4.2.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-08 13:34:43 +00:00
Aaron Pham
0d88370127 infra: prepare for release 0.4.1 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.1 		2023-11-08 13:24:46 +00:00
Aaron Pham
e87830ef0a container: update tracing dependencies (#591) 
* chore: update build message

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: add tracing dependencies to container

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-08 08:08:40 -05:00
Aaron Pham
0ea025da5a fix(cli): append model-id instruction to build (#590) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-08 07:44:36 -05:00
Aaron Pham
d47b985e5d docs: update quantization notes (#589) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-08 07:40:12 -05:00