mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-06-12 18:39:16 -04:00
Code Issues Packages Projects Releases Wiki Activity
1,574 Commits 6 Branches 194 Tags
a67ccdfd314158ccde3a5a3678622b6da279b1aa
Commit Graph

62 Commits

Author SHA1 Message Date
Aaron Pham
fe17a235ce chore(ci): prepare for releases 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2024-06-03 17:09:18 -04:00
Aaron Pham
a2746a6ff2 fix(client): generate config from model_name to avoid private model 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2024-06-03 16:53:08 -04:00
paperspace
2b15aaee96 fix: remove breakpoint 
Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>
 2024-06-02 22:29:12 +00:00
Aaron Pham
c60398c45b chore: add more info to metadata 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2024-06-02 17:57:51 -04:00
Aaron Pham
3193190b94 chore: update configuration to yield objects instead 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2024-06-02 17:48:03 -04:00
Aaron Pham
9d3ddae520 fix(client): remove circular dependency 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2024-06-02 12:31:53 -04:00
Aaron Pham
12f0d45a9d fix(client): make sure to initialised helpers class correctly 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2024-05-30 17:26:09 -04:00
Aaron Pham (mbp16)
f4f7f16e81 chore(releases): remove deadcode 
Signed-off-by: Aaron Pham (mbp16) <29749331+aarnphm@users.noreply.github.com>
 2024-05-27 12:37:50 -04:00
Aaron Pham
f248ea25cd feat(ci): running CI on paperspace (#998) 
* chore: update tiny script

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

* feat(ci): running on paperspace machines

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update models and increase timeout readiness

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: schema validation for inputs and update client supporting stop

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

* chore: update coverage config

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

* chore: remove some non-essentials

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

* chore: update locks

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2024-05-26 13:14:54 -04:00
Aaron Pham
97d76eec85 tests: add additional basic testing (#982) 
* chore: update rebase tests

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

* chore: update partial clients before removing

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

* fix: update clients parsing logics to work with 0.5

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

* chore: ignore ci runs as to run locally

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

* chore: update async client tests

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

* chore: update pre-commit

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>
 2024-05-23 10:02:23 -04:00
Yuchen Cheng
7c9fd85205 fix(compat): use annotated type from typing_compat (#943) 
fix: use annotated type from typing_compat instead for compatibility
 2024-04-02 00:49:00 -04:00
Aaron
6d2fd1bb8b fix(cli): HTTP debug log format 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2024-03-20 11:57:59 -04:00
Aaron
727361ced7 chore: running updated ruff formatter [skip ci] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2024-03-15 05:35:24 -04:00
Aaron Pham
072b3e97ec feat: 1.2 APIs (#821) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-03-15 03:49:19 -04:00
Aaron
e3392476be revert: "ci: pre-commit autoupdate [pre-commit.ci] (#931)" 
This reverts commit 7b00c84c2a.
 2024-03-15 03:47:23 -04:00
pre-commit-ci[bot]
7b00c84c2a ci: pre-commit autoupdate [pre-commit.ci] (#931) 
* ci: pre-commit autoupdate [pre-commit.ci]

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.2.2 → v0.3.2](https://github.com/astral-sh/ruff-pre-commit/compare/v0.2.2...v0.3.2)
- [github.com/pre-commit/mirrors-eslint: v9.0.0-beta.0 → v9.0.0-beta.2](https://github.com/pre-commit/mirrors-eslint/compare/v9.0.0-beta.0...v9.0.0-beta.2)

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-03-15 03:46:28 -04:00
Zhao Shenyang
16d8caf2ee chore: bump up bentoml version to 1.1.11 (#883) 2024-02-04 21:31:14 +08:00
Zhao Shenyang
9d0e292076 fix: limit BentoML version range (#881) 2024-02-04 16:59:21 +08:00
Aaron Pham
2bb97f8ba2 chore: update discord link (#838) 
* Update pyproject.toml

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* Update pyproject.toml

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* Update pyproject.toml

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* Update pyproject.toml

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2024-01-08 19:09:51 -05:00
Aaron Pham
c8c9663d06 fix(infra): conform ruff to 150 LL (#781) 
Generally correctly format it with ruff format and manual style

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-14 17:27:32 -05:00
Aaron
7bb53a7aa4 fix(client): include ability to getitem for sync client 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-26 18:48:01 -05:00
Aaron Pham
52a44b1bfa chore: cleanup loader (#729) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 21:51:51 -05:00
Aaron Pham
38b7c44df0 fix(base-image): update base image to include cuda for now (#720) 
* fix(base-image): update base image to include cuda for now

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: build core and client on release images

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: cleanup style changes

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 01:15:19 -05:00
Aaron Pham
8bb2742a9a chore(types): append additional types change (#719) 
* chore(types): append additional types change

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: add arguments for parsing dir

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 22:38:20 -05:00
Aaron Pham
c33b071ee4 refactor: delete unused code (#716) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 04:39:48 -05:00
Aaron Pham
fde78a2c78 chore: cleanup unused prompt templates (#713) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 01:56:51 -05:00
Aaron Pham
12b2b8ed21 fix: remove prompt template 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 08:04:40 +00:00
Aaron Pham
816c1ee80e feat(engine): CTranslate2 (#698) 
* chore: update instruction for dependencies

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat(experimental): CTranslate2

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-19 10:25:08 -05:00
Aaron Pham
206521e02d feat(ctranslate): initial infrastructure support (#694) 
* perf: compact and improve speed and agility

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* --wip--

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: cleanup infrastructure

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update styles notes and autogen mypy configuration

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-19 01:48:33 -05:00
Aaron Pham
80ed400646 fix(build): lock lower version based on each release and update infra (#686) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-17 15:57:31 -05:00
Aaron Pham
853def95cd fix(client): correct destructor the httpx object boht sync and async (#636) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 18:11:17 -05:00
Aaron Pham
126e6c9d63 fix(ruff): correct consistency between isort and formatter (#624) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 21:12:50 -05:00
Aaron Pham
8534870bd5 feat(client): add helpers subclass (#615) 
* feat(client): add helpers subclass

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* docs: add changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 01:44:12 -05:00
Aaron Pham
bbd20aed89 feat(client): support return response_cls to string (#614) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 01:15:35 -05:00
Aaron Pham
fad4186dbc feat(server): helpers endpoints for conversation format (#613) 
* feat: add support for helpers conversation conversion endpoint

also correct schema generation for openllm client

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update clients to reuse `openllm-core` logics

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: add changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-12 01:02:27 -05:00
Aaron Pham
36559a5ab5 chore: remove generated stubs for now (#610) 
Once we have more bandwidth and request for support gRPC streaming

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-10 18:11:35 -05:00
Aaron Pham
08466dd389 chore(client): remove ununsed state enum (#609) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-10 18:10:06 -05:00
Aaron Pham
b65b1bbc52 fix(client): check for should retry header (#606) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-10 17:50:12 -05:00
Aaron Pham
c41828f68f feat(client): support authentication token and shim implementation (#605) 
* chore: synch generate_iterator to be the same as server

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* --wip--

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* wip

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat: cleanup shim implementation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

* chore: fix pre-commit

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update check with tuple

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-10 17:44:31 -05:00
Aaron Pham
ac377fe490 infra: using ruff formatter (#594) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-09 12:44:05 -05:00
Aaron Pham
7f46aa3475 fix(stubs): update initialisation types (#577) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-08 01:30:46 -05:00
Aaron Pham
97d7c38fea refactor: cleanup typing to expose correct API (#576) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-08 01:24:03 -05:00
Aaron Pham
d9a7b6a147 fix(client): one-shot generation construction (#570) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-07 17:14:39 -05:00
Aaron Pham
e2029c934b perf: unify LLM interface (#518) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-06 20:39:43 -05:00
Abhishek
f2639879af feat: support toggle TLS verification (#532) 
Signed-off-by: Abhishek <59995387+ABHISHEK03312@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: abhishek <abhishek_vaidyanathan@ensigninfosecurity.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-06 13:34:53 -05:00
Aaron Pham
c1ca7ccd3b fix(breaking): remove embeddings and update client implementation (#500) 2023-10-14 16:04:35 -04:00
Aaron Pham
1539c3f7dc feat(client): simple implementation and streaming (#256) 2023-10-12 17:21:54 -04:00
Aaron
60bc0bd4a0 infra: make github recognize this as a Pip packages [skip ci] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-10-12 06:32:07 -04:00
Sauyon Lee
8977f32339 fix: support client HTTPS (#480) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-10-12 05:25:32 -04:00
Aaron
625b82a0fc fix(style): remove weird break on split item 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-10-07 02:21:31 -04:00