mirror/OpenLLM
1
0
Fork 0
mirror of https://github.com/bentoml/OpenLLM.git synced 2026-02-19 07:06:02 -05:00
Code Issues Packages Projects Releases Wiki Activity

feat(openai): chat templates and complete control of prompt generation (#725)

Browse Source 
* feat(openai): chat templates and complete control of prompt generation

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* fix: correctly use base chat templates

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

* fix: remove symlink

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
This commit is contained in:
Aaron Pham
2023-11-22 06:49:14 -05:00
committed by GitHub
parent 7aa0918a6f
commit b28b5269b5
11 changed files with 146 additions and 316 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
openllm-python/src/openllm/protocol/openai.py
View File

@@ -62,6 +62,9 @@ class ChatCompletionRequest:
# supported by vLLM and us
top_k: t.Optional[int] = attr.field(default=None)
best_of: t.Optional[int] = attr.field(default=1)
# Additional features to support chat_template
chat_template: str = attr.field(default=None)
add_generation_prompt: bool = attr.field(default=True)
@attr.define