mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-04-18 22:18:34 -04:00
Code Issues Packages Projects Releases Wiki Activity
1,481 Commits 6 Branches 194 Tags
cec0aa5487349cd94175ccbfa904f72bef036708
Commit Graph

122 Commits

Author SHA1 Message Date
Dennis Rall
362c099f97 fix(docs): update correct BentoML links (#995) 
Signed-off-by: Dennis Rall <56480601+dennisrall@users.noreply.github.com>
 2024-05-22 11:14:08 -04:00
Chaoyu
f9221879f0 docs: Update README.md (#964) 
* Update README.md

Signed-off-by: Chaoyu <paranoyang@gmail.com>

* Update README.md

Co-authored-by: Sherlock Xu <65327072+Sherlock113@users.noreply.github.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Chaoyu <paranoyang@gmail.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: Sherlock Xu <65327072+Sherlock113@users.noreply.github.com>
 2024-04-24 13:23:24 -07:00
Sherlock Xu
e696be01e1 docs: Update high-level messaging (#949) 
Update high-level messaging

Signed-off-by: Sherlock113 <sherlockxu07@gmail.com>
 2024-04-11 21:57:05 -04:00
Aaron Pham
5c0d2787c0 feat: add dbrx support 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2024-04-02 04:10:19 +00:00
Aaron Pham
072b3e97ec feat: 1.2 APIs (#821) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-03-15 03:49:19 -04:00
Aaron
7edbcf8a2d infra: migrate to uv 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2024-02-21 04:27:40 -05:00
Fazli Sapuan
1c0ff115a4 docs: update README.md telemetry code link (#842) 2024-01-15 11:41:49 -05:00
Sherlock Xu
291fac36ae docs: Add Llamaindex to features (#826) 
Add Llamaindex in freedom to build

Signed-off-by: Sherlock113 <sherlockxu07@gmail.com>
 2024-01-03 01:21:20 -08:00
weibeu
2d7c53ccde docs: README.md typo (#819) 
Fix typo

Signed-off-by: weibeu <deepakrajko14@gmail.com>
 2023-12-25 16:50:39 -08:00
Sherlock Xu
7e24c3037c docs: Update opt example to ms-phi (#805) 
Update opt example to ms-phi

Signed-off-by: Sherlock113 <sherlockxu07@gmail.com>
 2023-12-22 02:47:48 -05:00
Aaron Pham
1b3d737f8c docs: add notes about dtypes usage. (#786) 
Update README.md

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-12-16 06:56:14 -05:00
Aaron Pham
3ab78cd105 feat(mixtral): correct support for mixtral (#772) 
feat(mixtral): support inference with pt

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-12-13 09:03:56 -05:00
yansheng
3cb7f14fc1 feat(models): Support qwen (#742) 
* support qwen

* support qwen

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

* Update openllm-core/src/openllm_core/config/configuration_qwen.py

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: update correct readme and supports qwen models

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: root <yansheng105@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-30 06:54:17 -05:00
Aaron Pham
739eff3d21 fix(docs): remove invalid options (#733) 
fix(langchain): remove invalid options

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-26 05:04:54 -05:00
MingLiangDai
7b8d9024c4 fix(baichuan): supported from baichuan 2 from now on. (#728) 
* config support multiple architectures

* chore: only support baichuan2 from now on

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update notes

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: run script [skip ci]

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-24 02:07:06 -05:00
Aaron Pham
5442d9cd10 fix(trust_remote_code): handle args correctly (#727) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 17:03:13 -05:00
Aaron Pham
d80c392661 chore: update documentation about runtime (#699) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-19 10:27:07 -05:00
Aaron Pham
816c1ee80e feat(engine): CTranslate2 (#698) 
* chore: update instruction for dependencies

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat(experimental): CTranslate2

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-19 10:25:08 -05:00
Aaron Pham
539f250c0f feat(vllm): bump to 0.2.2 (#695) 
* feat(vllm): bump to 0.2.2

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: move up to CUDA 12.1

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: remove auto-gptq installation

since the builder image doesn't have access to GPU

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: update containerization warning

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-19 02:52:32 -05:00
Aaron Pham
206521e02d feat(ctranslate): initial infrastructure support (#694) 
* perf: compact and improve speed and agility

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* --wip--

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: cleanup infrastructure

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update styles notes and autogen mypy configuration

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-19 01:48:33 -05:00
Aaron Pham
099cc22a94 chore: update documentation (#693) 
* chore: update documentation

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update readme

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update documentations for configuration

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-18 19:44:52 -05:00
Aaron Pham
60b60ed29a infra: update cbfmt options (#676) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-17 07:51:33 -05:00
Aaron Pham
1a38de9b1f fix(docs): chatglm support on vLLM (#673) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-16 17:54:06 -05:00
Aaron Pham
c850d76ccd feat(models): Phi 1.5 (#672) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-16 17:48:10 -05:00
Aaron Pham
0fdfe786f3 docs: add LlamaIndex integration (#646) 
* docs: add LlamaIndex integration

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-16 16:24:43 -05:00
Aaron Pham
9e6df0df89 chore: update requirements in README.md (#659) 
chore: update requirements

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 02:32:36 -05:00
Aaron Pham
034e08cf08 infra: update scripts to run update readme automatically (#658) 
* infra: update scripts to run update readme automatically

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: cleanup mirror

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore(dropdown): correctly format noteblock and important block

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: whitespace aware

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-15 02:22:49 -05:00
Aaron Pham
c5f8602d4c docs: update instruction adding new models and remove command docstring (#654) 
docs: update instruction adding new models and remove command
docstring

as start will just support model_id directly, there is no need to
support custom docstring anymore

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-14 23:11:16 -05:00
Zhao Shenyang
ae69524749 doc: update adding new model guide (#637) 
* update

* Update openllm-python/ADDING_NEW_MODEL.md

Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Zhao Shenyang <dev@zsy.im>

* Update openllm-python/ADDING_NEW_MODEL.md

Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Zhao Shenyang <dev@zsy.im>

* Update openllm-python/ADDING_NEW_MODEL.md

Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Zhao Shenyang <dev@zsy.im>

* move ADDING_NEW_MODEL.md to git root directory

---------

Signed-off-by: Zhao Shenyang <dev@zsy.im>
Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 18:30:44 -05:00
Aaron Pham
de06557844 docs: update README.md (#635) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-13 17:37:27 -05:00
Aaron Pham
0b26badc9b docs: update supported feature set (#612) 
Update README.md

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-11 22:37:38 -05:00
Aaron Pham
7438005c04 refactor(config): simplify configuration and update start CLI output (#611) 
* chore(config): simplify configuration and update start CLI output
handling

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: remove state and message sent after server lifecycle

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update color stream and refactor reusable logic

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update documentations and mypy

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-11 22:36:10 -05:00
Aaron Pham
021fd453b9 infra: move out clojure to external (#593) 
As we don't write this

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-09 12:24:18 -05:00
Aaron Pham
b8a2e8cf91 refactor(cli): cleanup API (#592) 
* chore: remove unused imports

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* refactor(cli): update to only need model_id

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* feat: `openllm start model-id`

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: add changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changelog notice

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update correct config and running tools

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update backward compat options and treat JSON outputs
corespondingly

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-09 11:40:17 -05:00
Aaron Pham
d47b985e5d docs: update quantization notes (#589) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-08 07:40:12 -05:00
Aaron Pham
4d356f4b72 feat: Mistral support (#571) 
* feat: Mistral support

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

* chore: fix style

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update README docs about mistral

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-07 17:28:02 -05:00
Aaron Pham
8fade070f3 infra: update docs on serving fine-tuning layers (#567) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-06 21:34:44 -05:00
Aaron
b158609e95 docs: update BentoML integration 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-06 20:57:51 -05:00
Aaron
4550b2548b docs: update readme for HF agent 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-06 20:55:10 -05:00
Aaron Pham
e2029c934b perf: unify LLM interface (#518) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-06 20:39:43 -05:00
Aaron Pham
c1ca7ccd3b fix(breaking): remove embeddings and update client implementation (#500) 2023-10-14 16:04:35 -04:00
Sherlock Xu
00442997a6 docs: Add OpenLLM Google Colab link (#497) 2023-10-13 15:28:44 -04:00
Aaron Pham
fddd0bf95e feat: bootstrap documentation site (#252) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: GutZuFusss <leon.ikinger@googlemail.com>
Co-authored-by: GutZuFusss <leon.ikinger@googlemail.com>
 2023-09-12 12:28:29 -04:00
aarnphm-ec2-dev
c7f915fa71 chore: update documentation wrt to envvar correctness 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
 2023-09-08 17:43:03 +00:00
Sherlock Xu
6b6cf7a9cb dos: correct typo on headline (#305) 
fix typo

Signed-off-by: Sherlock113 <sherlockxu07@gmail.com>
 2023-09-07 10:35:12 -04:00
Sherlock Xu
c59b3a03b5 docs: cleanup README.md landing page (#302) 
* Update the readme

Signed-off-by: Sherlock113 <sherlockxu07@gmail.com>

* ci: auto fixes from pre-commit.ci

For more information, see https://pre-commit.ci

* fix type

Signed-off-by: Sherlock113 <sherlockxu07@gmail.com>

* Update README.md

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Sherlock113 <sherlockxu07@gmail.com>
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-09-05 07:51:40 -04:00
Aaron Pham
3e45530abd refactor(breaking): unify LLM API (#283) 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-09-01 05:15:19 -04:00
Aaron
b545ad2ad1 style: google 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-08-30 13:52:35 -04:00
Aaron Pham
4140d160b8 feat(embedding): Adding generic endpoint (#227) 2023-08-17 15:17:00 -04:00
Aaron Pham
e13d2800cc chore(readme): update nightly badge [skip ci] (#224) 2023-08-16 07:38:58 -04:00