mirror/exo
1
0
Fork 0
mirror of https://github.com/exo-explore/exo.git synced 2025-12-23 22:27:50 -05:00
Code Issues Packages Projects Releases Wiki Activity

Demo

Browse Source 
Co-authored-by: Evan <evanev7@gmail.com>
Co-authored-by: Alex Cheema <alexcheema123@gmail.com>
This commit is contained in:
rltakashige
2025-11-20 20:03:51 +00:00
committed by GitHub
parent d793f5f96c
commit 28a91787e8
30 changed files with 645 additions and 332 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
tmp/run_llm.sh
View File

@@ -13,9 +13,9 @@ QUERY="$*"
curl -sN -X POST "http://$HOST:8000/v1/chat/completions" \
-H "Content-Type: application/json" \
-d "{
\"model\": \"mlx-community/Llama-3.3-70B-Instruct-8bit\",
\"model\": \"mlx-community/Kimi-K2-Thinking\",
\"stream\": true,
\"messages\": [{ \"role\": \"user\", \"content\": \"$QUERY\" }]
\"messages\": [{ \"role\": \"user\", \"content\": \"$QUERY\"}]
}" |
grep --line-buffered '^data:' |
grep --line-buffered -v 'data: \[DONE\]' |