From 7fca472a6656595f083f741d27a0aa7e3758fd78 Mon Sep 17 00:00:00 2001 From: paperspace <29749331+aarnphm@users.noreply.github.com> Date: Wed, 29 May 2024 04:43:50 +0000 Subject: [PATCH] chore: update readme [skip ci] Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com> --- README.md | 162 +++++++++++++++++++++++++++++++-------- openllm-python/README.md | 162 +++++++++++++++++++++++++++++++-------- 2 files changed, 260 insertions(+), 64 deletions(-) diff --git a/README.md b/README.md index 9bc439ca..47b81e8a 100644 --- a/README.md +++ b/README.md @@ -98,20 +98,22 @@ OpenLLM currently supports the following models. By default, OpenLLM doesn't inc Baichuan + ### Quickstart + + > **Note:** Baichuan requires to install with: -> > ```bash > pip install "openllm[baichuan]" > ``` + Run the following command to quickly spin up a Baichuan server: ```bash TRUST_REMOTE_CODE=True openllm start baichuan-inc/baichuan-7b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -119,12 +121,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Baichuan variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=baichuan) to see more Baichuan-compatible models. + + ### Supported models You can specify any of the following Baichuan models via `openllm start`: + - [baichuan-inc/baichuan2-7b-base](https://huggingface.co/baichuan-inc/baichuan2-7b-base) - [baichuan-inc/baichuan2-7b-chat](https://huggingface.co/baichuan-inc/baichuan2-7b-chat) - [baichuan-inc/baichuan2-13b-base](https://huggingface.co/baichuan-inc/baichuan2-13b-base) @@ -136,20 +142,22 @@ You can specify any of the following Baichuan models via `openllm start`: ChatGLM + ### Quickstart + + > **Note:** ChatGLM requires to install with: -> > ```bash > pip install "openllm[chatglm]" > ``` + Run the following command to quickly spin up a ChatGLM server: ```bash TRUST_REMOTE_CODE=True openllm start thudm/chatglm-6b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -157,12 +165,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any ChatGLM variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=chatglm) to see more ChatGLM-compatible models. + + ### Supported models You can specify any of the following ChatGLM models via `openllm start`: + - [thudm/chatglm-6b](https://huggingface.co/thudm/chatglm-6b) - [thudm/chatglm-6b-int8](https://huggingface.co/thudm/chatglm-6b-int8) - [thudm/chatglm-6b-int4](https://huggingface.co/thudm/chatglm-6b-int4) @@ -176,20 +188,22 @@ You can specify any of the following ChatGLM models via `openllm start`: Dbrx + ### Quickstart + + > **Note:** Dbrx requires to install with: -> > ```bash > pip install "openllm[dbrx]" > ``` + Run the following command to quickly spin up a Dbrx server: ```bash TRUST_REMOTE_CODE=True openllm start databricks/dbrx-instruct ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -197,12 +211,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Dbrx variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=dbrx) to see more Dbrx-compatible models. + + ### Supported models You can specify any of the following Dbrx models via `openllm start`: + - [databricks/dbrx-instruct](https://huggingface.co/databricks/dbrx-instruct) - [databricks/dbrx-base](https://huggingface.co/databricks/dbrx-base) @@ -212,6 +230,7 @@ You can specify any of the following Dbrx models via `openllm start`: DollyV2 + ### Quickstart Run the following command to quickly spin up a DollyV2 server: @@ -219,7 +238,6 @@ Run the following command to quickly spin up a DollyV2 server: ```bash TRUST_REMOTE_CODE=True openllm start databricks/dolly-v2-3b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -227,12 +245,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any DollyV2 variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=dolly_v2) to see more DollyV2-compatible models. + + ### Supported models You can specify any of the following DollyV2 models via `openllm start`: + - [databricks/dolly-v2-3b](https://huggingface.co/databricks/dolly-v2-3b) - [databricks/dolly-v2-7b](https://huggingface.co/databricks/dolly-v2-7b) - [databricks/dolly-v2-12b](https://huggingface.co/databricks/dolly-v2-12b) @@ -243,20 +265,22 @@ You can specify any of the following DollyV2 models via `openllm start`: Falcon + ### Quickstart + + > **Note:** Falcon requires to install with: -> > ```bash > pip install "openllm[falcon]" > ``` + Run the following command to quickly spin up a Falcon server: ```bash TRUST_REMOTE_CODE=True openllm start tiiuae/falcon-7b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -264,12 +288,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Falcon variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=falcon) to see more Falcon-compatible models. + + ### Supported models You can specify any of the following Falcon models via `openllm start`: + - [tiiuae/falcon-7b](https://huggingface.co/tiiuae/falcon-7b) - [tiiuae/falcon-40b](https://huggingface.co/tiiuae/falcon-40b) - [tiiuae/falcon-7b-instruct](https://huggingface.co/tiiuae/falcon-7b-instruct) @@ -281,20 +309,22 @@ You can specify any of the following Falcon models via `openllm start`: Gemma + ### Quickstart + + > **Note:** Gemma requires to install with: -> > ```bash > pip install "openllm[gemma]" > ``` + Run the following command to quickly spin up a Gemma server: ```bash TRUST_REMOTE_CODE=True openllm start google/gemma-7b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -302,12 +332,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Gemma variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=gemma) to see more Gemma-compatible models. + + ### Supported models You can specify any of the following Gemma models via `openllm start`: + - [google/gemma-7b](https://huggingface.co/google/gemma-7b) - [google/gemma-7b-it](https://huggingface.co/google/gemma-7b-it) - [google/gemma-2b](https://huggingface.co/google/gemma-2b) @@ -319,6 +353,7 @@ You can specify any of the following Gemma models via `openllm start`: GPTNeoX + ### Quickstart Run the following command to quickly spin up a GPTNeoX server: @@ -326,7 +361,6 @@ Run the following command to quickly spin up a GPTNeoX server: ```bash TRUST_REMOTE_CODE=True openllm start eleutherai/gpt-neox-20b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -334,12 +368,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any GPTNeoX variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=gpt_neox) to see more GPTNeoX-compatible models. + + ### Supported models You can specify any of the following GPTNeoX models via `openllm start`: + - [eleutherai/gpt-neox-20b](https://huggingface.co/eleutherai/gpt-neox-20b) @@ -348,20 +386,22 @@ You can specify any of the following GPTNeoX models via `openllm start`: Llama + ### Quickstart + + > **Note:** Llama requires to install with: -> > ```bash > pip install "openllm[llama]" > ``` + Run the following command to quickly spin up a Llama server: ```bash TRUST_REMOTE_CODE=True openllm start NousResearch/llama-2-7b-hf ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -369,12 +409,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Llama variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=llama) to see more Llama-compatible models. + + ### Supported models You can specify any of the following Llama models via `openllm start`: + - [meta-llama/Llama-2-70b-chat-hf](https://huggingface.co/meta-llama/Llama-2-70b-chat-hf) - [meta-llama/Llama-2-13b-chat-hf](https://huggingface.co/meta-llama/Llama-2-13b-chat-hf) - [meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) @@ -394,20 +438,22 @@ You can specify any of the following Llama models via `openllm start`: Mistral + ### Quickstart + + > **Note:** Mistral requires to install with: -> > ```bash > pip install "openllm[mistral]" > ``` + Run the following command to quickly spin up a Mistral server: ```bash TRUST_REMOTE_CODE=True openllm start mistralai/Mistral-7B-Instruct-v0.1 ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -415,12 +461,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Mistral variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=mistral) to see more Mistral-compatible models. + + ### Supported models You can specify any of the following Mistral models via `openllm start`: + - [HuggingFaceH4/zephyr-7b-alpha](https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha) - [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) - [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) @@ -433,20 +483,22 @@ You can specify any of the following Mistral models via `openllm start`: Mixtral + ### Quickstart + + > **Note:** Mixtral requires to install with: -> > ```bash > pip install "openllm[mixtral]" > ``` + Run the following command to quickly spin up a Mixtral server: ```bash TRUST_REMOTE_CODE=True openllm start mistralai/Mixtral-8x7B-Instruct-v0.1 ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -454,12 +506,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Mixtral variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=mixtral) to see more Mixtral-compatible models. + + ### Supported models You can specify any of the following Mixtral models via `openllm start`: + - [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) - [mistralai/Mixtral-8x7B-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) @@ -469,20 +525,22 @@ You can specify any of the following Mixtral models via `openllm start`: MPT + ### Quickstart + + > **Note:** MPT requires to install with: -> > ```bash > pip install "openllm[mpt]" > ``` + Run the following command to quickly spin up a MPT server: ```bash TRUST_REMOTE_CODE=True openllm start mosaicml/mpt-7b-instruct ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -490,12 +548,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any MPT variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=mpt) to see more MPT-compatible models. + + ### Supported models You can specify any of the following MPT models via `openllm start`: + - [mosaicml/mpt-7b](https://huggingface.co/mosaicml/mpt-7b) - [mosaicml/mpt-7b-instruct](https://huggingface.co/mosaicml/mpt-7b-instruct) - [mosaicml/mpt-7b-chat](https://huggingface.co/mosaicml/mpt-7b-chat) @@ -510,20 +572,22 @@ You can specify any of the following MPT models via `openllm start`: OPT + ### Quickstart + + > **Note:** OPT requires to install with: -> > ```bash > pip install "openllm[opt]" > ``` + Run the following command to quickly spin up a OPT server: ```bash openllm start facebook/opt-1.3b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -531,12 +595,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any OPT variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=opt) to see more OPT-compatible models. + + ### Supported models You can specify any of the following OPT models via `openllm start`: + - [facebook/opt-125m](https://huggingface.co/facebook/opt-125m) - [facebook/opt-350m](https://huggingface.co/facebook/opt-350m) - [facebook/opt-1.3b](https://huggingface.co/facebook/opt-1.3b) @@ -550,20 +618,22 @@ You can specify any of the following OPT models via `openllm start`: Phi + ### Quickstart + + > **Note:** Phi requires to install with: -> > ```bash > pip install "openllm[phi]" > ``` + Run the following command to quickly spin up a Phi server: ```bash TRUST_REMOTE_CODE=True openllm start microsoft/Phi-3-mini-4k-instruct ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -571,12 +641,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Phi variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=phi) to see more Phi-compatible models. + + ### Supported models You can specify any of the following Phi models via `openllm start`: + - [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) - [microsoft/Phi-3-mini-128k-instruct](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct) - [microsoft/Phi-3-small-8k-instruct](https://huggingface.co/microsoft/Phi-3-small-8k-instruct) @@ -590,20 +664,22 @@ You can specify any of the following Phi models via `openllm start`: Qwen + ### Quickstart + + > **Note:** Qwen requires to install with: -> > ```bash > pip install "openllm[qwen]" > ``` + Run the following command to quickly spin up a Qwen server: ```bash TRUST_REMOTE_CODE=True openllm start qwen/Qwen-7B-Chat ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -611,12 +687,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Qwen variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=qwen) to see more Qwen-compatible models. + + ### Supported models You can specify any of the following Qwen models via `openllm start`: + - [qwen/Qwen-7B-Chat](https://huggingface.co/qwen/Qwen-7B-Chat) - [qwen/Qwen-7B-Chat-Int8](https://huggingface.co/qwen/Qwen-7B-Chat-Int8) - [qwen/Qwen-7B-Chat-Int4](https://huggingface.co/qwen/Qwen-7B-Chat-Int4) @@ -630,20 +710,22 @@ You can specify any of the following Qwen models via `openllm start`: StableLM + ### Quickstart + + > **Note:** StableLM requires to install with: -> > ```bash > pip install "openllm[stablelm]" > ``` + Run the following command to quickly spin up a StableLM server: ```bash TRUST_REMOTE_CODE=True openllm start stabilityai/stablelm-tuned-alpha-3b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -651,12 +733,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any StableLM variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=stablelm) to see more StableLM-compatible models. + + ### Supported models You can specify any of the following StableLM models via `openllm start`: + - [stabilityai/stablelm-tuned-alpha-3b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-3b) - [stabilityai/stablelm-tuned-alpha-7b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b) - [stabilityai/stablelm-base-alpha-3b](https://huggingface.co/stabilityai/stablelm-base-alpha-3b) @@ -668,20 +754,22 @@ You can specify any of the following StableLM models via `openllm start`: StarCoder + ### Quickstart + + > **Note:** StarCoder requires to install with: -> > ```bash > pip install "openllm[starcoder]" > ``` + Run the following command to quickly spin up a StarCoder server: ```bash TRUST_REMOTE_CODE=True openllm start bigcode/starcoder ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -689,12 +777,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any StarCoder variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=starcoder) to see more StarCoder-compatible models. + + ### Supported models You can specify any of the following StarCoder models via `openllm start`: + - [bigcode/starcoder](https://huggingface.co/bigcode/starcoder) - [bigcode/starcoderbase](https://huggingface.co/bigcode/starcoderbase) @@ -704,20 +796,22 @@ You can specify any of the following StarCoder models via `openllm start`: Yi + ### Quickstart + + > **Note:** Yi requires to install with: -> > ```bash > pip install "openllm[yi]" > ``` + Run the following command to quickly spin up a Yi server: ```bash TRUST_REMOTE_CODE=True openllm start 01-ai/Yi-6B ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -725,12 +819,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Yi variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=yi) to see more Yi-compatible models. + + ### Supported models You can specify any of the following Yi models via `openllm start`: + - [01-ai/Yi-6B](https://huggingface.co/01-ai/Yi-6B) - [01-ai/Yi-34B](https://huggingface.co/01-ai/Yi-34B) - [01-ai/Yi-6B-200K](https://huggingface.co/01-ai/Yi-6B-200K) diff --git a/openllm-python/README.md b/openllm-python/README.md index 9bc439ca..47b81e8a 100644 --- a/openllm-python/README.md +++ b/openllm-python/README.md @@ -98,20 +98,22 @@ OpenLLM currently supports the following models. By default, OpenLLM doesn't inc Baichuan + ### Quickstart + + > **Note:** Baichuan requires to install with: -> > ```bash > pip install "openllm[baichuan]" > ``` + Run the following command to quickly spin up a Baichuan server: ```bash TRUST_REMOTE_CODE=True openllm start baichuan-inc/baichuan-7b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -119,12 +121,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Baichuan variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=baichuan) to see more Baichuan-compatible models. + + ### Supported models You can specify any of the following Baichuan models via `openllm start`: + - [baichuan-inc/baichuan2-7b-base](https://huggingface.co/baichuan-inc/baichuan2-7b-base) - [baichuan-inc/baichuan2-7b-chat](https://huggingface.co/baichuan-inc/baichuan2-7b-chat) - [baichuan-inc/baichuan2-13b-base](https://huggingface.co/baichuan-inc/baichuan2-13b-base) @@ -136,20 +142,22 @@ You can specify any of the following Baichuan models via `openllm start`: ChatGLM + ### Quickstart + + > **Note:** ChatGLM requires to install with: -> > ```bash > pip install "openllm[chatglm]" > ``` + Run the following command to quickly spin up a ChatGLM server: ```bash TRUST_REMOTE_CODE=True openllm start thudm/chatglm-6b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -157,12 +165,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any ChatGLM variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=chatglm) to see more ChatGLM-compatible models. + + ### Supported models You can specify any of the following ChatGLM models via `openllm start`: + - [thudm/chatglm-6b](https://huggingface.co/thudm/chatglm-6b) - [thudm/chatglm-6b-int8](https://huggingface.co/thudm/chatglm-6b-int8) - [thudm/chatglm-6b-int4](https://huggingface.co/thudm/chatglm-6b-int4) @@ -176,20 +188,22 @@ You can specify any of the following ChatGLM models via `openllm start`: Dbrx + ### Quickstart + + > **Note:** Dbrx requires to install with: -> > ```bash > pip install "openllm[dbrx]" > ``` + Run the following command to quickly spin up a Dbrx server: ```bash TRUST_REMOTE_CODE=True openllm start databricks/dbrx-instruct ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -197,12 +211,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Dbrx variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=dbrx) to see more Dbrx-compatible models. + + ### Supported models You can specify any of the following Dbrx models via `openllm start`: + - [databricks/dbrx-instruct](https://huggingface.co/databricks/dbrx-instruct) - [databricks/dbrx-base](https://huggingface.co/databricks/dbrx-base) @@ -212,6 +230,7 @@ You can specify any of the following Dbrx models via `openllm start`: DollyV2 + ### Quickstart Run the following command to quickly spin up a DollyV2 server: @@ -219,7 +238,6 @@ Run the following command to quickly spin up a DollyV2 server: ```bash TRUST_REMOTE_CODE=True openllm start databricks/dolly-v2-3b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -227,12 +245,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any DollyV2 variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=dolly_v2) to see more DollyV2-compatible models. + + ### Supported models You can specify any of the following DollyV2 models via `openllm start`: + - [databricks/dolly-v2-3b](https://huggingface.co/databricks/dolly-v2-3b) - [databricks/dolly-v2-7b](https://huggingface.co/databricks/dolly-v2-7b) - [databricks/dolly-v2-12b](https://huggingface.co/databricks/dolly-v2-12b) @@ -243,20 +265,22 @@ You can specify any of the following DollyV2 models via `openllm start`: Falcon + ### Quickstart + + > **Note:** Falcon requires to install with: -> > ```bash > pip install "openllm[falcon]" > ``` + Run the following command to quickly spin up a Falcon server: ```bash TRUST_REMOTE_CODE=True openllm start tiiuae/falcon-7b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -264,12 +288,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Falcon variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=falcon) to see more Falcon-compatible models. + + ### Supported models You can specify any of the following Falcon models via `openllm start`: + - [tiiuae/falcon-7b](https://huggingface.co/tiiuae/falcon-7b) - [tiiuae/falcon-40b](https://huggingface.co/tiiuae/falcon-40b) - [tiiuae/falcon-7b-instruct](https://huggingface.co/tiiuae/falcon-7b-instruct) @@ -281,20 +309,22 @@ You can specify any of the following Falcon models via `openllm start`: Gemma + ### Quickstart + + > **Note:** Gemma requires to install with: -> > ```bash > pip install "openllm[gemma]" > ``` + Run the following command to quickly spin up a Gemma server: ```bash TRUST_REMOTE_CODE=True openllm start google/gemma-7b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -302,12 +332,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Gemma variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=gemma) to see more Gemma-compatible models. + + ### Supported models You can specify any of the following Gemma models via `openllm start`: + - [google/gemma-7b](https://huggingface.co/google/gemma-7b) - [google/gemma-7b-it](https://huggingface.co/google/gemma-7b-it) - [google/gemma-2b](https://huggingface.co/google/gemma-2b) @@ -319,6 +353,7 @@ You can specify any of the following Gemma models via `openllm start`: GPTNeoX + ### Quickstart Run the following command to quickly spin up a GPTNeoX server: @@ -326,7 +361,6 @@ Run the following command to quickly spin up a GPTNeoX server: ```bash TRUST_REMOTE_CODE=True openllm start eleutherai/gpt-neox-20b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -334,12 +368,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any GPTNeoX variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=gpt_neox) to see more GPTNeoX-compatible models. + + ### Supported models You can specify any of the following GPTNeoX models via `openllm start`: + - [eleutherai/gpt-neox-20b](https://huggingface.co/eleutherai/gpt-neox-20b) @@ -348,20 +386,22 @@ You can specify any of the following GPTNeoX models via `openllm start`: Llama + ### Quickstart + + > **Note:** Llama requires to install with: -> > ```bash > pip install "openllm[llama]" > ``` + Run the following command to quickly spin up a Llama server: ```bash TRUST_REMOTE_CODE=True openllm start NousResearch/llama-2-7b-hf ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -369,12 +409,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Llama variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=llama) to see more Llama-compatible models. + + ### Supported models You can specify any of the following Llama models via `openllm start`: + - [meta-llama/Llama-2-70b-chat-hf](https://huggingface.co/meta-llama/Llama-2-70b-chat-hf) - [meta-llama/Llama-2-13b-chat-hf](https://huggingface.co/meta-llama/Llama-2-13b-chat-hf) - [meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) @@ -394,20 +438,22 @@ You can specify any of the following Llama models via `openllm start`: Mistral + ### Quickstart + + > **Note:** Mistral requires to install with: -> > ```bash > pip install "openllm[mistral]" > ``` + Run the following command to quickly spin up a Mistral server: ```bash TRUST_REMOTE_CODE=True openllm start mistralai/Mistral-7B-Instruct-v0.1 ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -415,12 +461,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Mistral variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=mistral) to see more Mistral-compatible models. + + ### Supported models You can specify any of the following Mistral models via `openllm start`: + - [HuggingFaceH4/zephyr-7b-alpha](https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha) - [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) - [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) @@ -433,20 +483,22 @@ You can specify any of the following Mistral models via `openllm start`: Mixtral + ### Quickstart + + > **Note:** Mixtral requires to install with: -> > ```bash > pip install "openllm[mixtral]" > ``` + Run the following command to quickly spin up a Mixtral server: ```bash TRUST_REMOTE_CODE=True openllm start mistralai/Mixtral-8x7B-Instruct-v0.1 ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -454,12 +506,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Mixtral variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=mixtral) to see more Mixtral-compatible models. + + ### Supported models You can specify any of the following Mixtral models via `openllm start`: + - [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) - [mistralai/Mixtral-8x7B-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) @@ -469,20 +525,22 @@ You can specify any of the following Mixtral models via `openllm start`: MPT + ### Quickstart + + > **Note:** MPT requires to install with: -> > ```bash > pip install "openllm[mpt]" > ``` + Run the following command to quickly spin up a MPT server: ```bash TRUST_REMOTE_CODE=True openllm start mosaicml/mpt-7b-instruct ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -490,12 +548,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any MPT variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=mpt) to see more MPT-compatible models. + + ### Supported models You can specify any of the following MPT models via `openllm start`: + - [mosaicml/mpt-7b](https://huggingface.co/mosaicml/mpt-7b) - [mosaicml/mpt-7b-instruct](https://huggingface.co/mosaicml/mpt-7b-instruct) - [mosaicml/mpt-7b-chat](https://huggingface.co/mosaicml/mpt-7b-chat) @@ -510,20 +572,22 @@ You can specify any of the following MPT models via `openllm start`: OPT + ### Quickstart + + > **Note:** OPT requires to install with: -> > ```bash > pip install "openllm[opt]" > ``` + Run the following command to quickly spin up a OPT server: ```bash openllm start facebook/opt-1.3b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -531,12 +595,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any OPT variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=opt) to see more OPT-compatible models. + + ### Supported models You can specify any of the following OPT models via `openllm start`: + - [facebook/opt-125m](https://huggingface.co/facebook/opt-125m) - [facebook/opt-350m](https://huggingface.co/facebook/opt-350m) - [facebook/opt-1.3b](https://huggingface.co/facebook/opt-1.3b) @@ -550,20 +618,22 @@ You can specify any of the following OPT models via `openllm start`: Phi + ### Quickstart + + > **Note:** Phi requires to install with: -> > ```bash > pip install "openllm[phi]" > ``` + Run the following command to quickly spin up a Phi server: ```bash TRUST_REMOTE_CODE=True openllm start microsoft/Phi-3-mini-4k-instruct ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -571,12 +641,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Phi variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=phi) to see more Phi-compatible models. + + ### Supported models You can specify any of the following Phi models via `openllm start`: + - [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) - [microsoft/Phi-3-mini-128k-instruct](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct) - [microsoft/Phi-3-small-8k-instruct](https://huggingface.co/microsoft/Phi-3-small-8k-instruct) @@ -590,20 +664,22 @@ You can specify any of the following Phi models via `openllm start`: Qwen + ### Quickstart + + > **Note:** Qwen requires to install with: -> > ```bash > pip install "openllm[qwen]" > ``` + Run the following command to quickly spin up a Qwen server: ```bash TRUST_REMOTE_CODE=True openllm start qwen/Qwen-7B-Chat ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -611,12 +687,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Qwen variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=qwen) to see more Qwen-compatible models. + + ### Supported models You can specify any of the following Qwen models via `openllm start`: + - [qwen/Qwen-7B-Chat](https://huggingface.co/qwen/Qwen-7B-Chat) - [qwen/Qwen-7B-Chat-Int8](https://huggingface.co/qwen/Qwen-7B-Chat-Int8) - [qwen/Qwen-7B-Chat-Int4](https://huggingface.co/qwen/Qwen-7B-Chat-Int4) @@ -630,20 +710,22 @@ You can specify any of the following Qwen models via `openllm start`: StableLM + ### Quickstart + + > **Note:** StableLM requires to install with: -> > ```bash > pip install "openllm[stablelm]" > ``` + Run the following command to quickly spin up a StableLM server: ```bash TRUST_REMOTE_CODE=True openllm start stabilityai/stablelm-tuned-alpha-3b ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -651,12 +733,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any StableLM variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=stablelm) to see more StableLM-compatible models. + + ### Supported models You can specify any of the following StableLM models via `openllm start`: + - [stabilityai/stablelm-tuned-alpha-3b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-3b) - [stabilityai/stablelm-tuned-alpha-7b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b) - [stabilityai/stablelm-base-alpha-3b](https://huggingface.co/stabilityai/stablelm-base-alpha-3b) @@ -668,20 +754,22 @@ You can specify any of the following StableLM models via `openllm start`: StarCoder + ### Quickstart + + > **Note:** StarCoder requires to install with: -> > ```bash > pip install "openllm[starcoder]" > ``` + Run the following command to quickly spin up a StarCoder server: ```bash TRUST_REMOTE_CODE=True openllm start bigcode/starcoder ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -689,12 +777,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any StarCoder variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=starcoder) to see more StarCoder-compatible models. + + ### Supported models You can specify any of the following StarCoder models via `openllm start`: + - [bigcode/starcoder](https://huggingface.co/bigcode/starcoder) - [bigcode/starcoderbase](https://huggingface.co/bigcode/starcoderbase) @@ -704,20 +796,22 @@ You can specify any of the following StarCoder models via `openllm start`: Yi + ### Quickstart + + > **Note:** Yi requires to install with: -> > ```bash > pip install "openllm[yi]" > ``` + Run the following command to quickly spin up a Yi server: ```bash TRUST_REMOTE_CODE=True openllm start 01-ai/Yi-6B ``` - In a different terminal, run the following command to interact with the server: ```bash @@ -725,12 +819,16 @@ export OPENLLM_ENDPOINT=http://localhost:3000 openllm query 'What are large language models?' ``` + > **Note:** Any Yi variants can be deployed with OpenLLM. Visit the [HuggingFace Model Hub](https://huggingface.co/models?sort=trending&search=yi) to see more Yi-compatible models. + + ### Supported models You can specify any of the following Yi models via `openllm start`: + - [01-ai/Yi-6B](https://huggingface.co/01-ai/Yi-6B) - [01-ai/Yi-34B](https://huggingface.co/01-ai/Yi-34B) - [01-ai/Yi-6B-200K](https://huggingface.co/01-ai/Yi-6B-200K)