mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-02-08 06:42:20 -05:00
Code Issues Packages Projects Releases Wiki Activity

fix(memory): correctly recommend instance types for cloud

Browse Source 
Signed-off-by: paperspace <29749331+aarnphm@users.noreply.github.com>
This commit is contained in:
paperspace
2024-05-23 14:42:39 +00:00
parent db523e2940
commit cec0aa5487
3 changed files with 25 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
openllm-python/src/_openllm_tiny/_entrypoint.py
View File

@@ -436,7 +436,7 @@ def build_command(
service_config = dict(
resources={
'gpu' if device else 'cpu': len(device) if device else '1',
'gpu_type': recommended_instance_type(model_id, bentomodel),
'gpu_type': recommended_instance_type(model_id, bentomodel, serialisation),
},
traffic=dict(timeout=timeout),
)