mirror of https://github.com/bentoml/OpenLLM.git synced 2026-03-11 11:39:52 -04:00

Files

Aaron Pham c7f4dc7bb2 feat(test): snapshot testing (#107 )

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

2023-07-10 17:23:19 -04:00

bentofile.yaml

peft(models): improve implementation (#60 )

2023-06-24 05:22:18 -04:00

README.md

feat: quantization (#27 )

2023-06-16 18:10:50 -04:00

requirements.txt

peft(models): improve implementation (#60 )

2023-06-24 05:22:18 -04:00

service.py

feat(test): snapshot testing (#107 )

2023-07-10 17:23:19 -04:00

README.md

LangChain + BentoML + OpenLLM

Run it locally:

export BENTOML_CONFIG_OPTIONS="api_server.traffic.timeout=900 runners.traffic.timeout=900"
bentoml serve

Build Bento:

bentoml build

Generate docker image:

bentoml containerize ...
docker run \
  -e SERPAPI_API_KEY="__Your_SERP_API_key__" \
  -e BENTOML_CONFIG_OPTIONS="api_server.traffic.timeout=900 runners.traffic.timeout=900" \
  -p 3000:3000 \
  ..image_name