mirror of
https://github.com/bentoml/OpenLLM.git
synced 2026-02-19 07:06:02 -05:00
38b7c44df0
* fix(base-image): update base image to include cuda for now Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com> * fix: build core and client on release images Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com> * chore: cleanup style changes Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com> --------- Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
10 lines
261 B
Python
10 lines
261 B
Python
'''OpenLLM Python client.
|
|
|
|
```python
|
|
client = openllm.client.HTTPClient("http://localhost:8080")
|
|
client.query("What is the difference between gather and scatter?")
|
|
```
|
|
'''
|
|
|
|
from openllm_client import AsyncHTTPClient as AsyncHTTPClient, HTTPClient as HTTPClient
|