mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-04-21 07:29:41 -04:00
Code Issues Packages Projects Releases Wiki Activity

chore: update readme [skip ci]

Browse Source 
Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>
This commit is contained in:
paperspace
2024-05-29 04:43:50 +00:00
parent e9e46b2cc7
commit 7fca472a66
2 changed files with 260 additions and 64 deletions
Download Patch File Download Diff File Expand all files Collapse all files

162
openllm-python/README.md generated
View File

@@ -98,20 +98,22 @@ OpenLLM currently supports the following models. By default, OpenLLM doesn't inc
<summary>Baichuan</summary>
### Quickstart
> **Note:** Baichuan requires to install with:
>
> ```bash
> pip install "openllm[baichuan]"
> ```
Run the following command to quickly spin up a Baichuan server:
```bash
TRUST_REMOTE_CODE=True openllm start baichuan-inc/baichuan-7b
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -119,12 +121,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any Baichuan variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=baichuan) to see more Baichuan-compatible models.
### Supported models
You can specify any of the following Baichuan models via `openllm start`:
- [baichuan-inc/baichuan2-7b-base](https://huggingface.co/baichuan-inc/baichuan2-7b-base)
- [baichuan-inc/baichuan2-7b-chat](https://huggingface.co/baichuan-inc/baichuan2-7b-chat)
- [baichuan-inc/baichuan2-13b-base](https://huggingface.co/baichuan-inc/baichuan2-13b-base)
@@ -136,20 +142,22 @@ You can specify any of the following Baichuan models via `openllm start`:
<summary>ChatGLM</summary>
### Quickstart
> **Note:** ChatGLM requires to install with:
>
> ```bash
> pip install "openllm[chatglm]"
> ```
Run the following command to quickly spin up a ChatGLM server:
```bash
TRUST_REMOTE_CODE=True openllm start thudm/chatglm-6b
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -157,12 +165,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any ChatGLM variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=chatglm) to see more ChatGLM-compatible models.
### Supported models
You can specify any of the following ChatGLM models via `openllm start`:
- [thudm/chatglm-6b](https://huggingface.co/thudm/chatglm-6b)
- [thudm/chatglm-6b-int8](https://huggingface.co/thudm/chatglm-6b-int8)
- [thudm/chatglm-6b-int4](https://huggingface.co/thudm/chatglm-6b-int4)
@@ -176,20 +188,22 @@ You can specify any of the following ChatGLM models via `openllm start`:
<summary>Dbrx</summary>
### Quickstart
> **Note:** Dbrx requires to install with:
>
> ```bash
> pip install "openllm[dbrx]"
> ```
Run the following command to quickly spin up a Dbrx server:
```bash
TRUST_REMOTE_CODE=True openllm start databricks/dbrx-instruct
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -197,12 +211,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any Dbrx variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=dbrx) to see more Dbrx-compatible models.
### Supported models
You can specify any of the following Dbrx models via `openllm start`:
- [databricks/dbrx-instruct](https://huggingface.co/databricks/dbrx-instruct)
- [databricks/dbrx-base](https://huggingface.co/databricks/dbrx-base)
@@ -212,6 +230,7 @@ You can specify any of the following Dbrx models via `openllm start`:
<summary>DollyV2</summary>
### Quickstart
Run the following command to quickly spin up a DollyV2 server:
@@ -219,7 +238,6 @@ Run the following command to quickly spin up a DollyV2 server:
```bash
TRUST_REMOTE_CODE=True openllm start databricks/dolly-v2-3b
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -227,12 +245,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any DollyV2 variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=dolly_v2) to see more DollyV2-compatible models.
### Supported models
You can specify any of the following DollyV2 models via `openllm start`:
- [databricks/dolly-v2-3b](https://huggingface.co/databricks/dolly-v2-3b)
- [databricks/dolly-v2-7b](https://huggingface.co/databricks/dolly-v2-7b)
- [databricks/dolly-v2-12b](https://huggingface.co/databricks/dolly-v2-12b)
@@ -243,20 +265,22 @@ You can specify any of the following DollyV2 models via `openllm start`:
<summary>Falcon</summary>
### Quickstart
> **Note:** Falcon requires to install with:
>
> ```bash
> pip install "openllm[falcon]"
> ```
Run the following command to quickly spin up a Falcon server:
```bash
TRUST_REMOTE_CODE=True openllm start tiiuae/falcon-7b
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -264,12 +288,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any Falcon variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=falcon) to see more Falcon-compatible models.
### Supported models
You can specify any of the following Falcon models via `openllm start`:
- [tiiuae/falcon-7b](https://huggingface.co/tiiuae/falcon-7b)
- [tiiuae/falcon-40b](https://huggingface.co/tiiuae/falcon-40b)
- [tiiuae/falcon-7b-instruct](https://huggingface.co/tiiuae/falcon-7b-instruct)
@@ -281,20 +309,22 @@ You can specify any of the following Falcon models via `openllm start`:
<summary>Gemma</summary>
### Quickstart
> **Note:** Gemma requires to install with:
>
> ```bash
> pip install "openllm[gemma]"
> ```
Run the following command to quickly spin up a Gemma server:
```bash
TRUST_REMOTE_CODE=True openllm start google/gemma-7b
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -302,12 +332,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any Gemma variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=gemma) to see more Gemma-compatible models.
### Supported models
You can specify any of the following Gemma models via `openllm start`:
- [google/gemma-7b](https://huggingface.co/google/gemma-7b)
- [google/gemma-7b-it](https://huggingface.co/google/gemma-7b-it)
- [google/gemma-2b](https://huggingface.co/google/gemma-2b)
@@ -319,6 +353,7 @@ You can specify any of the following Gemma models via `openllm start`:
<summary>GPTNeoX</summary>
### Quickstart
Run the following command to quickly spin up a GPTNeoX server:
@@ -326,7 +361,6 @@ Run the following command to quickly spin up a GPTNeoX server:
```bash
TRUST_REMOTE_CODE=True openllm start eleutherai/gpt-neox-20b
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -334,12 +368,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any GPTNeoX variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=gpt_neox) to see more GPTNeoX-compatible models.
### Supported models
You can specify any of the following GPTNeoX models via `openllm start`:
- [eleutherai/gpt-neox-20b](https://huggingface.co/eleutherai/gpt-neox-20b)
</details>
@@ -348,20 +386,22 @@ You can specify any of the following GPTNeoX models via `openllm start`:
<summary>Llama</summary>
### Quickstart
> **Note:** Llama requires to install with:
>
> ```bash
> pip install "openllm[llama]"
> ```
Run the following command to quickly spin up a Llama server:
```bash
TRUST_REMOTE_CODE=True openllm start NousResearch/llama-2-7b-hf
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -369,12 +409,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any Llama variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=llama) to see more Llama-compatible models.
### Supported models
You can specify any of the following Llama models via `openllm start`:
- [meta-llama/Llama-2-70b-chat-hf](https://huggingface.co/meta-llama/Llama-2-70b-chat-hf)
- [meta-llama/Llama-2-13b-chat-hf](https://huggingface.co/meta-llama/Llama-2-13b-chat-hf)
- [meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf)
@@ -394,20 +438,22 @@ You can specify any of the following Llama models via `openllm start`:
<summary>Mistral</summary>
### Quickstart
> **Note:** Mistral requires to install with:
>
> ```bash
> pip install "openllm[mistral]"
> ```
Run the following command to quickly spin up a Mistral server:
```bash
TRUST_REMOTE_CODE=True openllm start mistralai/Mistral-7B-Instruct-v0.1
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -415,12 +461,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any Mistral variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=mistral) to see more Mistral-compatible models.
### Supported models
You can specify any of the following Mistral models via `openllm start`:
- [HuggingFaceH4/zephyr-7b-alpha](https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha)
- [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta)
- [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2)
@@ -433,20 +483,22 @@ You can specify any of the following Mistral models via `openllm start`:
<summary>Mixtral</summary>
### Quickstart
> **Note:** Mixtral requires to install with:
>
> ```bash
> pip install "openllm[mixtral]"
> ```
Run the following command to quickly spin up a Mixtral server:
```bash
TRUST_REMOTE_CODE=True openllm start mistralai/Mixtral-8x7B-Instruct-v0.1
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -454,12 +506,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any Mixtral variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=mixtral) to see more Mixtral-compatible models.
### Supported models
You can specify any of the following Mixtral models via `openllm start`:
- [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1)
- [mistralai/Mixtral-8x7B-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1)
@@ -469,20 +525,22 @@ You can specify any of the following Mixtral models via `openllm start`:
<summary>MPT</summary>
### Quickstart
> **Note:** MPT requires to install with:
>
> ```bash
> pip install "openllm[mpt]"
> ```
Run the following command to quickly spin up a MPT server:
```bash
TRUST_REMOTE_CODE=True openllm start mosaicml/mpt-7b-instruct
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -490,12 +548,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any MPT variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=mpt) to see more MPT-compatible models.
### Supported models
You can specify any of the following MPT models via `openllm start`:
- [mosaicml/mpt-7b](https://huggingface.co/mosaicml/mpt-7b)
- [mosaicml/mpt-7b-instruct](https://huggingface.co/mosaicml/mpt-7b-instruct)
- [mosaicml/mpt-7b-chat](https://huggingface.co/mosaicml/mpt-7b-chat)
@@ -510,20 +572,22 @@ You can specify any of the following MPT models via `openllm start`:
<summary>OPT</summary>
### Quickstart
> **Note:** OPT requires to install with:
>
> ```bash
> pip install "openllm[opt]"
> ```
Run the following command to quickly spin up a OPT server:
```bash
openllm start facebook/opt-1.3b
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -531,12 +595,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any OPT variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=opt) to see more OPT-compatible models.
### Supported models
You can specify any of the following OPT models via `openllm start`:
- [facebook/opt-125m](https://huggingface.co/facebook/opt-125m)
- [facebook/opt-350m](https://huggingface.co/facebook/opt-350m)
- [facebook/opt-1.3b](https://huggingface.co/facebook/opt-1.3b)
@@ -550,20 +618,22 @@ You can specify any of the following OPT models via `openllm start`:
<summary>Phi</summary>
### Quickstart
> **Note:** Phi requires to install with:
>
> ```bash
> pip install "openllm[phi]"
> ```
Run the following command to quickly spin up a Phi server:
```bash
TRUST_REMOTE_CODE=True openllm start microsoft/Phi-3-mini-4k-instruct
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -571,12 +641,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any Phi variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=phi) to see more Phi-compatible models.
### Supported models
You can specify any of the following Phi models via `openllm start`:
- [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)
- [microsoft/Phi-3-mini-128k-instruct](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct)
- [microsoft/Phi-3-small-8k-instruct](https://huggingface.co/microsoft/Phi-3-small-8k-instruct)
@@ -590,20 +664,22 @@ You can specify any of the following Phi models via `openllm start`:
<summary>Qwen</summary>
### Quickstart
> **Note:** Qwen requires to install with:
>
> ```bash
> pip install "openllm[qwen]"
> ```
Run the following command to quickly spin up a Qwen server:
```bash
TRUST_REMOTE_CODE=True openllm start qwen/Qwen-7B-Chat
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -611,12 +687,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any Qwen variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=qwen) to see more Qwen-compatible models.
### Supported models
You can specify any of the following Qwen models via `openllm start`:
- [qwen/Qwen-7B-Chat](https://huggingface.co/qwen/Qwen-7B-Chat)
- [qwen/Qwen-7B-Chat-Int8](https://huggingface.co/qwen/Qwen-7B-Chat-Int8)
- [qwen/Qwen-7B-Chat-Int4](https://huggingface.co/qwen/Qwen-7B-Chat-Int4)
@@ -630,20 +710,22 @@ You can specify any of the following Qwen models via `openllm start`:
<summary>StableLM</summary>
### Quickstart
> **Note:** StableLM requires to install with:
>
> ```bash
> pip install "openllm[stablelm]"
> ```
Run the following command to quickly spin up a StableLM server:
```bash
TRUST_REMOTE_CODE=True openllm start stabilityai/stablelm-tuned-alpha-3b
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -651,12 +733,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any StableLM variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=stablelm) to see more StableLM-compatible models.
### Supported models
You can specify any of the following StableLM models via `openllm start`:
- [stabilityai/stablelm-tuned-alpha-3b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-3b)
- [stabilityai/stablelm-tuned-alpha-7b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b)
- [stabilityai/stablelm-base-alpha-3b](https://huggingface.co/stabilityai/stablelm-base-alpha-3b)
@@ -668,20 +754,22 @@ You can specify any of the following StableLM models via `openllm start`:
<summary>StarCoder</summary>
### Quickstart
> **Note:** StarCoder requires to install with:
>
> ```bash
> pip install "openllm[starcoder]"
> ```
Run the following command to quickly spin up a StarCoder server:
```bash
TRUST_REMOTE_CODE=True openllm start bigcode/starcoder
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -689,12 +777,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any StarCoder variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=starcoder) to see more StarCoder-compatible models.
### Supported models
You can specify any of the following StarCoder models via `openllm start`:
- [bigcode/starcoder](https://huggingface.co/bigcode/starcoder)
- [bigcode/starcoderbase](https://huggingface.co/bigcode/starcoderbase)
@@ -704,20 +796,22 @@ You can specify any of the following StarCoder models via `openllm start`:
<summary>Yi</summary>
### Quickstart
> **Note:** Yi requires to install with:
>
> ```bash
> pip install "openllm[yi]"
> ```
Run the following command to quickly spin up a Yi server:
```bash
TRUST_REMOTE_CODE=True openllm start 01-ai/Yi-6B
```
In a different terminal, run the following command to interact with the server:
```bash
@@ -725,12 +819,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'What are large language models?'
```
> **Note:** Any Yi variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=yi) to see more Yi-compatible models.
### Supported models
You can specify any of the following Yi models via `openllm start`:
- [01-ai/Yi-6B](https://huggingface.co/01-ai/Yi-6B)
- [01-ai/Yi-34B](https://huggingface.co/01-ai/Yi-34B)
- [01-ai/Yi-6B-200K](https://huggingface.co/01-ai/Yi-6B-200K)