mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-03-05 15:46:16 -05:00
Code Issues Packages Projects Releases Wiki Activity
1,212 Commits 6 Branches 194 Tags
38b7c44df08f232eff02acb359c69cbaeb310ef3
Commit Graph

1212 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Aaron Pham
38b7c44df0 fix(base-image): update base image to include cuda for now (#720) 
* fix(base-image): update base image to include cuda for now

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: build core and client on release images

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: cleanup style changes

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 01:15:19 -05:00
Aaron Pham
8bb2742a9a chore(types): append additional types change (#719) 
* chore(types): append additional types change

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: add arguments for parsing dir

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 22:38:20 -05:00
Aaron Pham
04ef08a7f8 chore(strategy): compact and add stubs (#718) 
generate service_vars automatically inline without reading from files

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 21:49:28 -05:00
Aaron Pham
909db8c3bf refactor: reduce compiled cacheline 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-22 02:27:42 +00:00
Aaron Pham
77bd6f090a chore(logger): fix warnings and streamline style (#717) 
Sorry but there are too much wasted spacing in `_llm.py`, and I'm unhappy and not productive anytime I look or want to do anything with it

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-21 18:54:51 -05:00
Aaron Pham
d53cf234bd fix(api-server): correct set generation from LLM class 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 10:38:36 +00:00
Aaron Pham
2821e172ef fix(examples): use non-chat models 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 10:12:48 +00:00
Aaron
93709f1d66 fix(infra): remove unless exclude 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 05:04:47 -05:00
Aaron
14242a7ab8 fix(utils): correct import 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 05:03:20 -05:00
Aaron Pham
c33b071ee4 refactor: delete unused code (#716) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 04:39:48 -05:00
Aaron Pham
a8a9f154ce fix(ci): tests (#715) 
* fix: tests

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* chore: remove broken tests

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 03:05:22 -05:00
Aaron Pham
e70246ca5d feat(generation): add support for eos_token_id (#714) 
chore: add support for custom eos_token_id

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 02:01:36 -05:00
Aaron Pham
fde78a2c78 chore: cleanup unused prompt templates (#713) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 01:56:51 -05:00
Aaron Pham
e6b9a749a4 infra: bump to homebrew tap release to 0.4.22 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 01:50:09 +00:00
Aaron Pham
be7a4bf576 infra: bump to dev version of 0.4.23.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-21 01:49:02 +00:00
Aaron Pham
f3fd32d596 infra: prepare for release 0.4.22 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.22 		2023-11-21 01:38:46 +00:00
Aaron Pham
ad4f388c98 refactor: update runner helpers and add max_model_len (#712) 
* chore(runner): cleanup unecessary checks for runnable backend

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: saving llm reference to runner

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: correct inject item

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update support for max_seq_len

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* fix: correct max_model_len

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update and warning backward compatibility

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: remove unused sets

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 20:37:15 -05:00
Aaron Pham
8fc5f1f70c infra: bump to homebrew tap release to 0.4.21 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 22:50:00 +00:00
Aaron Pham
aac92f40ac infra: bump to dev version of 0.4.22.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 22:48:59 +00:00
Aaron Pham
4c4bc82a47 infra: prepare for release 0.4.21 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.21 		2023-11-20 22:32:44 +00:00
Aaron
00e2666e48 fix(build): contraint packages for bentoml >1.1.10 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 17:30:38 -05:00
dependabot[bot]
1f2cdc8021 chore(deps): bump github/codeql-action from 2.22.5 to 2.22.7 (#707) 
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.22.5 to 2.22.7.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](74483a38d3...66b90a5db1)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-11-20 17:22:49 -05:00
dependabot[bot]
62da3a7fe5 chore(deps): bump docker/build-push-action from 5.0.0 to 5.1.0 (#708) 
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 5.0.0 to 5.1.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](0565240e2d...4a13e500e5)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-11-20 17:22:41 -05:00
dependabot[bot]
b1ee1ce2a1 chore(deps): bump taiki-e/install-action from 2.21.11 to 2.21.17 (#709) 
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.21.11 to 2.21.17.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](4d8504289a...ccc14bdc8d)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-11-20 17:22:30 -05:00
pre-commit-ci[bot]
d7bb0ab197 ci: pre-commit autoupdate [pre-commit.ci] (#711) 
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.1.5 → v0.1.6](https://github.com/astral-sh/ruff-pre-commit/compare/v0.1.5...v0.1.6)
- [github.com/pre-commit/mirrors-eslint: v8.53.0 → v8.54.0](https://github.com/pre-commit/mirrors-eslint/compare/v8.53.0...v8.54.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-11-20 17:22:19 -05:00
Aaron Pham
c25f69fddc infra: bump to homebrew tap release to 0.4.20 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 22:19:49 +00:00
Aaron Pham
5491fe30b0 infra: bump to dev version of 0.4.21.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 22:18:14 +00:00
Aaron Pham
204cbd43d2 infra: prepare for release 0.4.20 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.20 		2023-11-20 22:09:47 +00:00
Aaron
f753662ae6 fix(build): only load model when eager is True 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 17:06:25 -05:00
Aaron
5b92e848e2 fix: raises error if backend is not supported 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 17:03:30 -05:00
Aaron Pham
3769ca73a9 infra: bump to homebrew tap release to 0.4.19 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 08:19:58 +00:00
Aaron Pham
82ee9e845a infra: bump to dev version of 0.4.20.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 08:19:02 +00:00
Aaron Pham
46d6fcca98 infra: prepare for release 0.4.19 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.19 		2023-11-20 08:06:53 +00:00
Aaron Pham
12b2b8ed21 fix: remove prompt template 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 08:04:40 +00:00
Aaron Pham
41c857f292 fix: set correct type annotations 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 07:17:38 +00:00
Aaron Pham
7e2aa80a8c infra: bump to homebrew tap release to 0.4.18 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 05:25:42 +00:00
Aaron Pham
d82896a80c infra: bump to dev version of 0.4.19.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 05:24:31 +00:00
Aaron Pham
c1f86bda16 infra: prepare for release 0.4.18 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.18 		2023-11-20 05:15:14 +00:00
Aaron Pham
513c08ccda feat(openai): dynamic model_type registration (#704) 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 00:13:45 -05:00
Aaron Pham
6505abdb44 chore: update lower bound version of bentoml to avoid breakage (#703) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-19 23:09:14 -05:00
Aaron Pham
ed01080c1b infra: bump to homebrew tap release to 0.4.17 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 03:55:05 +00:00
Aaron Pham
7b89fde99e infra: bump to dev version of 0.4.18.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-20 03:54:07 +00:00
Aaron Pham
d1915d7a9e infra: prepare for release 0.4.17 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.17 		2023-11-20 03:43:21 +00:00
Aaron Pham
4491aa54d0 fix(backend): correct use variable for backend when initialisation (#702) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-19 22:42:25 -05:00
Aaron Pham
44f05da845 infra: update generate notes and better local handle (#701) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-19 17:50:23 -05:00
Aaron Pham
83f95c74ff infra: bump to homebrew tap release to 0.4.16 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-19 15:53:11 +00:00
Aaron Pham
184c6739d1 infra: bump to dev version of 0.4.17.dev0 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
 2023-11-19 15:50:55 +00:00
Aaron Pham
e9207ff683 infra: prepare for release 0.4.16 [generated] [skip ci] 
Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
v0.4.16 		2023-11-19 15:41:03 +00:00
Aaron
cb4386b013 fix(release): remove unecessary check for client dependencies [skip ci] 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-19 10:39:38 -05:00
Aaron Pham
1968704b71 chore: update changelog [skip ci] (#700) 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-11-19 10:30:48 -05:00