mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-04-21 23:47:23 -04:00
Code Issues Packages Projects Releases Wiki Activity
1,104 Commits 6 Branches 194 Tags
2ea2f3fd4ffd59ac4f0b23fbbd503772fb41bd30
Commit Graph

1104 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Aaron Pham
2ea2f3fd4f infra: bump to homebrew tap release to 0.4.8 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 05:31:05 +00:00
Aaron Pham
b90e44f679 infra: bump to dev version of 0.4.9.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 05:29:44 +00:00
Aaron Pham
625afd0c3b infra: prepare for release 0.4.8 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.8 		2023-11-15 05:19:22 +00:00
Aaron Pham
a58d947bc8 perf: improve build logics and cleanup speed (#657) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 00:18:31 -05:00
Aaron Pham
103156cd71 chore(cli): move playground to CLI components (#655) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 23:20:50 -05:00
Aaron
cbdcfc87a2 infra: remove cohere examples 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 23:15:31 -05:00
Aaron Pham
c5f8602d4c docs: update instruction adding new models and remove command docstring (#654) 
docs: update instruction adding new models and remove command
docstring

as start will just support model_id directly, there is no need to
support custom docstring anymore

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 23:11:16 -05:00
Aaron Pham
ac0de3e44e infra: bump to homebrew tap release to 0.4.7 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 03:56:04 +00:00
Aaron Pham
88c898ffad infra: bump to dev version of 0.4.8.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 03:54:34 +00:00
Aaron Pham
b40874149e infra: prepare for release 0.4.7 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.7 		2023-11-15 03:44:09 +00:00
xianxian.zhang
ea02aaaa23 fix: correct OPENLLM_DEV_BUILD check (#653) 2023-11-14 22:21:37 -05:00
Aaron Pham
6a6d689a77 feat: Yi models (#651) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 21:55:24 -05:00
Aaron Pham
b4b70e2f20 fix(cli): update context name parsing correctly (#652) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 21:53:56 -05:00
Aaron
9eddae83a6 infra: update cohere client 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 01:52:38 -05:00
Aaron Pham
31a799ff61 refactor: use DEBUG env-var instead of OPENLLMDEVDEBUG (#647) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 01:39:58 -05:00
Aaron Pham
d63da6a5cb infra: bump to homebrew tap release to 0.4.6 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 06:31:46 +00:00
Aaron Pham
2462c1ad73 infra: bump to dev version of 0.4.7.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 06:30:29 +00:00
Aaron Pham
145aafd386 infra: prepare for release 0.4.6 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.6 		2023-11-14 06:18:14 +00:00
Aaron Pham
00d6016bcb chore(openapi): unify inject param (#645) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 01:16:20 -05:00
Aaron Pham
b0ab8ccdf6 experimental: Cohere compatible endpoints. (#644) 
* feat: add generate endpoint

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update generation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix(cohere): generate endpoints

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: --wip--

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat: update testing clients and chat implementation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: disable schemas for easter eggs

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 01:07:43 -05:00
Aaron Pham
0bf6ec7537 fix(dependencies): lock build < 1 for now (#643) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 00:36:08 -05:00
Aaron Pham
b30a412398 fix(cli): set default dtype to auto infer (#642) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 23:05:27 -05:00
Aaron Pham
99a5d26527 fix(service): to yield out correct JSON objects (#640) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 22:41:52 -05:00
Aaron Pham
2d428f12da fix(cpu): more verbose definition for dtype casting (#639) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 20:40:50 -05:00
Aaron Pham
b20c7d1c1d fix(generation): compatibility dtype with CPU (#638) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 20:32:07 -05:00
Zhao Shenyang
ae69524749 doc: update adding new model guide (#637) 
* update

* Update openllm-python/ADDING_NEW_MODEL.md

Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Zhao Shenyang <dev@zsy.im>

* Update openllm-python/ADDING_NEW_MODEL.md

Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Zhao Shenyang <dev@zsy.im>

* Update openllm-python/ADDING_NEW_MODEL.md

Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Zhao Shenyang <dev@zsy.im>

* move ADDING_NEW_MODEL.md to git root directory

---------

Signed-off-by: Zhao Shenyang <dev@zsy.im>
Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 18:30:44 -05:00
Aaron Pham
853def95cd fix(client): correct destructor the httpx object boht sync and async (#636) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 18:11:17 -05:00
Aaron Pham
de06557844 docs: update README.md (#635) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 17:37:27 -05:00
Aaron Pham
af84462f27 infra: remove unused postprocess_generate (#634) 
This is currently a noop

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 17:35:39 -05:00
Zhao Shenyang
f202fddce8 perf(model): update mistral inference parameters and prompt format (#632) 
* feat(model): add initial mistral support

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

* chore: update with recent refactor

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 17:32:16 -05:00
Aaron Pham
a6387d1d15 chore: cleanup unused code path (#633) 
we now rely on tokenizer.chat_templates to format prompts correctly

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 17:23:07 -05:00
Aaron Pham
67ee492715 infra: bump to homebrew tap release to 0.4.5 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 19:01:38 +00:00
Aaron Pham
16a644411f infra: bump to dev version of 0.4.6.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 19:00:32 +00:00
Aaron Pham
a0d74017cd infra: prepare for release 0.4.5 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.5 		2023-11-13 18:49:31 +00:00
Aaron Pham
d358e68539 fix(torch_dtype): load eagerly (#631) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 13:48:04 -05:00
Aaron Pham
0924c0b34d infra: removing clojure frontend from infra cycle (#630) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 13:29:34 -05:00
pre-commit-ci[bot]
52367d1e8b ci: pre-commit autoupdate [pre-commit.ci] (#629) 
* ci: pre-commit autoupdate [pre-commit.ci]

updates:
- [github.com/pre-commit/mirrors-prettier: v3.0.3 → v3.1.0](https://github.com/pre-commit/mirrors-prettier/compare/v3.0.3...v3.1.0)

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-13 13:07:53 -05:00
Aaron Pham
852cd863a9 fix(cli): make sure to pass the dtype to subprocess service (#628) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 05:32:17 -05:00
Aaron Pham
099c0dc31b feat(cli): --dtype arguments (#627) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 05:25:50 -05:00
dependabot[bot]
f288b8a276 chore(deps): bump taiki-e/install-action from 2.21.8 to 2.21.11 (#625) 
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.21.8 to 2.21.11.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](b4f94d4449...4d8504289a)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-11-13 05:08:45 -05:00
Aaron Pham
22eaaf3ce1 feat(vllm): support passing specific dtype (#626) 
* feat(vllm): support passing specific dtype

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* fix: correctly cached the item

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-13 05:08:33 -05:00
Aaron Pham
126e6c9d63 fix(ruff): correct consistency between isort and formatter (#624) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 21:12:50 -05:00
Aaron Pham
e77a7fb2a4 chore: update jupyter notebooks with new API (#623) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 21:07:28 -05:00
Aaron Pham
de04de7136 fix(sdk): make sure build to quiet out stdout (#622) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 18:59:48 -05:00
Aaron Pham
e667dac82f chore(cli): always show available models (#621) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 18:36:19 -05:00
Aaron Pham
c50a7db80d fix(cli): correct set working_dir (#620) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 18:34:11 -05:00
Aaron Pham
e0632a85ed refactor(cli): move out to its own packages (#619) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 18:25:44 -05:00
Aaron Pham
38a7d2a5b5 infra: bump to homebrew tap release to 0.4.4 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 22:53:17 +00:00
Aaron Pham
b4a18f8337 infra: bump to dev version of 0.4.5.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 22:51:38 +00:00
Aaron Pham
b1f3a7297b infra: prepare for release 0.4.4 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.4 		2023-11-12 22:39:52 +00:00