mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-05-03 13:22:43 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix(logprobs): correct check logprobs (#779)

Browse Source 
* fix(logprobs): correct check logprobs

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update changlog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
This commit is contained in:
Aaron Pham
2023-12-14 14:19:01 -05:00
committed by GitHub
parent ea272ba18c
commit 44383528b5
2 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
openllm-python/src/openllm/_runners.py
View File

@@ -162,7 +162,7 @@ class PyTorchRunnable(bentoml.Runnable):
input_len = len(prompt_token_ids)
if self.is_encoder_decoder:
if config['logprobs'] > 0: # FIXME: logprobs is not supported
if config['logprobs']: # FIXME: logprobs is not supported
raise NotImplementedError('Logprobs is yet to be supported with encoder-decoder models.')
encoder_output = self.model.encoder(input_ids=torch.as_tensor([prompt_token_ids], device=self.device))[0]
start_ids = torch.as_tensor(