Daniel Hiltgen 855f4bf989 Report cached prompt tokens (#17943) ...

* Report cached prompt tokens

Add prompt_eval_cached_count to native responses and expose equivalent cached-token fields through the OpenAI- and Anthropic-compatible APIs. Keep prompt_eval_count as the logical input total while excluding cache hits from CLI and benchmark prefill rates. Surface processed and cached prompt counts in benchmark output.

Collect cache counts from llama-server and MLX, preserve coherent metrics across two-pass structured generation.

Fixes #8008

Related to #15758

* review comments

2026-09-02 09:30:44 -07:00

..

anthropic-compatibility.mdx

docs: format compat docs (#14678)

2026-03-06 14:53:17 -08:00

authentication.mdx

docs: add docs for docs.ollama.com (#12805)

2025-10-28 13:18:48 -07:00

errors.mdx

docs: update docs examples to use Gemma 4 instead of Gemma 3 (#16607)

2026-06-07 12:43:13 -07:00

introduction.mdx

docs: fix broken links (#17676)

2026-08-11 14:36:39 -07:00

openai-compatibility.mdx

imagegen: remove MLX image generation code (#16615)

2026-07-28 15:35:28 -07:00

streaming.mdx

docs: update docs examples to use Gemma 4 instead of Gemma 3 (#16607)

2026-06-07 12:43:13 -07:00

usage.mdx

Report cached prompt tokens (#17943)

2026-09-02 09:30:44 -07:00