mirror of https://github.com/ollama/ollama.git synced 2026-02-19 15:57:07 -05:00

Files

Patrick Devine 44bdd9a2ef Add MLX runner with GLM4-MoE-Lite model support (#14185 )

This change adds a new MLX based runner which includes:

  * Method-based MLX bindings
  * Subprocess-based MLX runner (x/mlxrunner)
  * KV cache with tree management
  * A basic sampler

The GLM4-MoE-Lite model has been ported to use the new bindings.

---------

Co-authored-by: Michael Yang <git@mxy.ng>

2026-02-10 14:57:57 -08:00

common

server: add logprobs and top_logprobs support to Ollama's API (#12899 )

2025-11-11 08:49:50 -08:00

llamarunner

flash attn: add auto mode for llama engine (#13052 )

2025-12-12 13:27:19 -08:00

ollamarunner

move tokenizers to separate package (#13825 )

2026-02-05 17:44:11 -08:00

README.md

Runner for Ollama engine

2025-02-13 17:09:26 -08:00

runner.go

Add MLX runner with GLM4-MoE-Lite model support (#14185 )

2026-02-10 14:57:57 -08:00

README.md

`runner`

Note: this is a work in progress

A minimial runner for loading a model and running inference via a http web server.

./runner -model <model binary>

Completion

curl -X POST -H "Content-Type: application/json" -d '{"prompt": "hi"}' http://localhost:8080/completion

Embeddings

curl -X POST -H "Content-Type: application/json" -d '{"prompt": "turn me into an embedding"}' http://localhost:8080/embedding