mirror of
https://github.com/bentoml/OpenLLM.git
synced 2026-06-11 18:09:52 -04:00
fddd0bf95e
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com> Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com> Signed-off-by: GutZuFusss <leon.ikinger@googlemail.com> Co-authored-by: GutZuFusss <leon.ikinger@googlemail.com>
454 B
454 B
LangChain + BentoML + OpenLLM
Run it locally:
export BENTOML_CONFIG_OPTIONS="api_server.traffic.timeout=900 runners.traffic.timeout=900"
bentoml serve
Build Bento:
bentoml build
Generate docker image:
bentoml containerize ...
docker run \
-e SERPAPI_API_KEY="__Your_SERP_API_key__" \
-e BENTOML_CONFIG_OPTIONS="api_server.traffic.timeout=900 runners.traffic.timeout=900" \
-p 3000:3000 \
..image_name